 math right that makes August older than Bitcoin. I have no idea what that means. It's not as profitable as Bitcoin, but yeah, that was important to mention. Okay, so what's the problem that August tries to solve? Right, like why would you use it? Why does it even exist? So the core problem is that there's lots of situations where you want to edit configuration files programmatically. Meaning you're writing a script, be it a batch script or some Python code, Ruby code, Z code, what have you, and as part of that you need to fiddle some configuration settings somewhere. And as an example, if you look at this little problem here, in your SSHD config you want to turn off root logins and you're like, stand back, I know, regular expressions, I know said, and you come up with this, which works great until you realize that the number of spaces before the yes could be variable, right, some people put tabs there, some people put five spaces there. And so, yeah, you come up with a slightly more complicated set thing to catch that case, then you realize, oh, besides yes and no, you know, there's a bunch more options that people can set with the permit root login, you want to get rid of all of them. So 10 minutes later, you're at this beauty here, and you're like, wow, we're done, right? Everything works. You do a little grep just to check, and then this happens, right, like some, and that's sort of small, annoying problems that you encounter when you do that, and if you have to do enough of those, it gets really, really annoying. So in a way you can think of August as a repository of, yeah, all the knowledge that you have to have, all these stupid little things you need to know when you want to do something like, you know, changing the root login configuration file. At the core of the problem, what makes this really, really interesting is that there's a bazillion different file formats for configuration files. You can look at that as sort of, you know, pointless creativity, and at first I sort of looked at it that way too, but there's also, yeah, configuration files are user interface in a way, right, and they have a, I don't know, mouth field to them. You're not going to get confused whether you're editing the HGPD con for the NGNX config, right, like they give the whole thing a certain feel, and so, yeah, preserving these configuration files and just dealing with them as they are is really, really important to August. Like we're not going to go and say, oh, we'll just start over, yeah, everybody changed their demons or other programs that read configuration because we have this awesome store that makes it much easier to programmatically change things. We have to live in the world that exists, and, you know, that reality of all these different file formats is part of the reality. Another thing, yeah, when you, when you write code that needs to fill configuration files, an easy trap to fall into is you think about it for a moment and you're like, oh, this is really easy. I know exactly how I'm going to break this down and parse the file into the data structure that I can then change. The really hard part is actually making changes because, yeah, not just because of all the HG cases. I had an example, but also because you need to, you need to preserve a lot of unimportant detail. Like people get really grumpy if you strip out their comments just because, you know, you use, they used your tool to make some small change. People are really attached to the formatting of their files. That's something that turns up in bug reports when we get that wrong a lot of times. Okay, there's a long-standing debate in the config management world whether you should even try to modify configuration files or whether you should not just manage files, yeah, wholesale. Like, you know, whenever you need to make a change, you change a template or the golden copy you have in source control and then just copy it out. I'm not going to get into that debate. I just want to note that there's a lot of cases where that's just not feasible. One example is, for example, the Let's Encrypt folks. They have a little client that downloads an SSL certificate for your web server, but it also edits the configuration of your web server to have all your virtual hosts use that new certificate that you just downloaded. And so they can't go and just write over your HTTP deconflict. They need to go in there and find the exact places where you define virtual hosts and then switch their configuration to use the Let's Encrypt SSL certificate. And they actually use August for fiddling with the Apache for the Apache config. Okay, so how does it work? As I said, yeah, we handle configuration files the way they are. There's no need to have like special annotations in them. You don't have to have comments in them that say, don't change. This is managed or something like that. You can totally make changes with August, then go back, edit the file with BI, change with August again. That's totally fine. And yeah. One important point of August is that it parses these files into a data structure, and we use the same data structure for everything. That's somebody, earlier said August. That's the thing with the tree. Yeah, it's exactly that. That's the tree. When we parse a file, we parse it into a tree, and I'll talk a little bit more about what the tree looks like in a minute. I already talked about how people get really, really grumpy when you screw up sort of unimportant stuff. If you're just adding an alias in Etsy hosts, if Etsy hosts is all of a sudden reordered and has no comments anymore, people would like that very much. Okay, and so I hope you're enjoying this vintage 2009 slide. This is what you can do with open office from back then. Yeah, so the way August works is think of the blue box as the library that gives you an API. When you start using it, you say load a bunch of files, and what that does is that the library reads the files you want to manage, and it uses something called the lens. That's this green thing here to turn this file into a tree. This might be Etsy hosts here, and this might be Apache config, what have you. And then you use the API to make changes to this tree. You can find things in there. It's actually got a pretty powerful query language built in that's sort of close to XPath. I don't know how many fans of XPath we have in here, but for these things, it's actually useful. But yeah, so you make changes to the tree, and then you say, save my changes, and that's where the lenses get involved again. You take the part of the tree for Etsy hosts, send it through the lens, and that produces an updated file. And yeah, so these lenses are the thing that sort of encodes the knowledge we have about each file format. You have to have a different one for each file format. And that's one of the things that I've heard people say is like, oh, August would love to use it, but I don't really know how to write lenses, and I've heard it's really hard. In general, at this point, that's probably the 10 years on part of the talk, you don't have to write lenses. What you know about lenses after listening to me for the last two minutes is about as much as you need to know to use August. I did a little graph of the number of different file formats. Oh, I didn't change the title of the slide. This is really different file formats that August can deal with. Really early on, this is like me writing about 10 of them, and then Raphael Persant, I don't know if he's here, showed up, and he did a tremendous amount of work just on the lenses and really helped both wrote a ton of them and helped a lot of people write their own lenses and get them in and get them in the shape that we could ship them as part of the distribution. So right now, we're at a little over 200 lenses. We are doing 200 different file formats that August can deal with. I think the curve is tapering off exactly because, this isn't good, is it? Because we're at the point where August out of the box knows enough about most file formats. So this is going to be interesting now. I don't know, the video kept cycling in and out. I just have to talk really, really fast before the picture goes away again. How can you use August? What do you need to do to use it? First off, as I said, it's a C library. The writing in C was not fun, but sort of on purpose because I want to make August usable from as many languages as possible and see sort of the lingua franca there. We have language bindings for pretty much anything you can think of. The thing I'm really sad that's missing is intercal. So if anybody here feels like they want to contribute intercal bindings for August, totally do that. The other way you'll encounter August in the wild is in configuration management system, systems quite a few of them have some form of August support built in. And I hope you realize what I'm doing with the slide, like if your configuration management system does not include support for August, it's not top shelf, so you might want to complain about it. Okay, so the easiest way to get started is something called August tool. It's a little command line tool. And if you run help, it prints out all the commands it knows, the intent behind August tool is to really just let you explore how August deals with files, what the tree looks like, search things in the tree, make changes before you incorporate that into your programs. It's really mostly meant as a tool to get familiar with the API and get a feel for it. Although some people have started using that in scripts, it's not ideal for scripts because some of the output is not great for processing afterwards. And when somebody complained to me late last year about that fact that our tool is sort of not really useful in bash scripts, I started thinking about what could we do to make it better. I didn't go all the way to another command line tool to make changes, but I figured because August puts a lot of structure on the text files and you can query that structure, it would also be really useful to have a tool to just query the way August sees things without making changes. So I wrote this thing called Arc Match. If you wanna play with it, it's really new in 110.1 and I just released that maybe a couple of weeks ago so it hasn't really hit all the distros yet. But if you wanna play with it, there's a Docker container, Luder slash Arc Match that you can pull and that has everything in it and you can play around with it a little bit. And just to give you a feel for both how Arc Match works and how August sees the world, a simple example is sort of the NFS exports file to find things in there. It's not entirely trivial. Yeah, it's simple but not entirely trivial because you can have on each line, you can have any number of definitions for exports so that makes it a little awkward to deal with in grep or things like that. But it's still simple enough that I can talk about it here without killing myself. And so the first thing you can do is, yeah, if you just said Arc Match, it's the experts, it just prints out the tree. That's the tree that the lens has generated for that file. And you can think of this tree almost like a file system. It's very similar to how a file system is organized with a few important exceptions. One of them is that siblings in the tree can have the same name. So this DER1 and DER2, they're really two different nodes in the tree and they're both called DER. The one and two is just there to help you see that they're two different things. Similarly, like client has an option one and option two but they're really just called option and the one and two is just there to distinguish them. The other difference from a file system is that even directories can have content associated with them in August that's called the value, like each node in the tree consists of a label and a value. So DER both has some content associated with it but it also has children. And the export lens, how it structures things is like each of those DER entries is basically one line from NC exports and for each client definitions, you get one of these client things, right? There's a client one here and a client two. And then it breaks that into whatever, the network specification is for that and the options. So one simple thing you can do is just print out all the clients that you're exporting anything to. Just give me a list of the things that I'm exporting stuff to. And with AUG match, the important thing here is that you say I want to match DER slash client that means go through the tree, look at every DER node, then go to a client node and that's the thing I'm talking about. And then when we say exact, it just looks at that one node and not the stuff underneath it. AUG match by default would print everything that you're matching and underneath it. Exact says just that node. And then only value says, only print the value of the node. I don't care about the path to how I get there. And so that gives you a list of the things and it's the export. You can of course restrict that to just one export here. And that's the only change that you have to make is put this slash home into brackets here. Now you're saying what I want to match is a DER node that has slash home in it and then the client for that. So there's lots of, I could probably talk for an hour about these matches that are called path expressions and they're the XPath stuff I mentioned earlier. You could probably talk for an hour about lots of things, really cool things you can do with it. One of the things you might want to do is, is this legible in the back? I just realized that the red is probably not great. One thing you might want to do is just print me all the directories that I'm exporting where at least one client doesn't have root squash set. Right, so there's a sanity check to what I'm doing. So we want to print slash local because it's exporting something without root squash. We want to print slash home. The first client has root squash set but the second one doesn't. And temp is okay and pub is also not okay. We're only looking at root squash, not all squash. And that's the sort of the arc match expression you would use for that. I don't want to go too much into too much detail how that works. If you have questions, I'm more than happy to explain that later. Okay, so how can you get involved if you're looking at this and you're like cool and you've tried late with it a little bit and you want to do things. In my mind, the most important thing that August is missing right now is sort of reorganized documentation. There is a lot of documentation but it's sort of, it's grown. That's another side effect of it being around for 10 years. It's not well organized. It's sort of hard to find what you need. If anybody feels like writing and doing stuff like that, I would love, I've been looking at Gitbook to do that but haven't gotten very far. Would be really awesome to get some help there. Other things, I said we have a ton of language bindings. They all could use some love if it's just to make regular releases. In some cases, they could be changed to be a little more idiomatic for the target language rather than just being a straight up, here's a CAPI and you can call it. In the Ruby bindings we added, throwing exceptions and stuff like that which is just closer to how Ruby people want to use it. The other bindings could use some love with it too. Of course, there's always room to write more lenses. We're never going to run out of configuration file formats that somebody has come up and August can't cover yet. And then if you want a really big project, there's been over the years, there's been people who've tried and made some headway. The very first person that tried, that was one of my favorite problems, ran into that during the build while you compile August, it creates directly called AUX and on Windows you can do that. Windows just blows up because that's your, whatever, serial device or something like that. But there's more work to do there. I mean, August uses POSIX APIs and you need to find ways to round them on Windows. Or go to the bug tracker. There's tons of little bugs and stuff that. Here's the website. The website has a quick tour that I highly recommend if you want to learn how to use it better. It just walks you through some simple examples of making changes to files with August. And there's also links to the mailing list and the source code, the GitHub repo and so on. All right, and that's all I had. Are there questions? Do people have? Yep. The question is, is it able, except besides just showing the value, is it able to set the value? August underneath, yes. And with Octool, you can do that. You can change values and then save the file. I wrote Octmatch specifically just for reading things. Like, Octmatch right now doesn't give you a way to set things. But that might be an interesting addition to it. Yeah, so yeah, the question is about using something like Octmatch in script to make changes. I've loosely thought about. So the question is, how does August know which lens to use and what format to expect? And there's actually a lot less magic than you might think. It's just like, when you write a lens, you also say, oh, this applies to files that match the following globs, like shell globs. That's something you have with the lens. And so my example here only work because the lens actually has a little statement in it that says this lens works for the file slash etc slash exports. There's ways to, if you have an exports file, it's somewhere else. There's ways to say, use this exports lens even though you don't know about that. But sort of for the default locations, August just knows automatically what to use for. So yeah, this is the computer science part of the talk. YAML is outside the realm of what August can do because August is restricted to, so initially August was restricted to regular languages like any file format that you can describe with a regular language. At some point because Apache, I added support for context free languages, but YAML is at least context sensitive because of all the spacing thing. There's ways around that if you have a special lexer, you can turn YAML into a context free language, but none of that is in the audience today. And so there is a YAML lens, but it's really restricted to sort of one data, like it doesn't deal with indentation or nested things. Yeah, so the question is, what about files where the configurations spread across multiple files like a .d directory? And yeah, August very consciously focuses just on the file level. That there was another sort of design decision I made early on is that I don't want to add more than just syntax transformation to things. I don't want to get into any sort of semantic thing. There was actually somebody on the mailing list who had some, I've shied away from sort of include statements in a lot of configuration file formats, say you can include from files, and August doesn't even understand that yet, but I might add that. But yeah, it's very much focused on sort of individual files and everything beyond that is sort of application logic from August's point of view. Yes, yeah, so is it possible to write a Python port, like a pure Python port of August? Some people have also asked, what about a pure Go port of August? It's definitely possible, it's sort of a matter of time. There is, there's a lot of fairly complicated stuff going on behind the scenes, like I wouldn't be keen to do that. I think you're looking at at least six months to a year for doing that. That's like, I mean, for the language geeks here, there's an early power star in there and there's lots of fun stuff in there. So I would rather, for like the Ansible problem, I would look at whether you can statically link stuff and just copy that over, like just have a little binary stop or something like that, link August in statically and then just.