 Hello. This is going to be a short talk about a format for files called TestMark. It is a format which has the goal of bringing together documentation for your software projects, fixtures for your tests, any other kind of data you want, and doing this all within the code blocks of a format that we are all familiar with. Markdown. Hi. I'm Eric, mostly known as WarpFork on the Internet. If you want these slides, you'll find them at slides.com. And me on GitHub and Twitter as well. This is going to be a really simple talk because I'm mostly just going to show pictures of a syntax, and I'm going to hope that it's honestly self-explanatory. Here it is. This is TestMark. It's regular markdown. It's literally regular markdown. What the improvement that TestMark is going to offer is this data hunk here, this piece of JSON in this example. This is going to be legible through a TestMark library as a test fixture data. In fact, through a sort of like a pseudo file system abstraction. This format right here, this thing with the brackets and the pound sign and so on, this is a comment that you can sneak into the markdown format. You might have seen this from some other tooling. It doesn't render in markdown, so we can put more data in it. In TestMark, if you start with that word TestMark in brackets, that means you're going to give a hint to this kind of a library. And this string in the parentheses is going to be the label that gets attached to this data. Then you can use a library in a language of your choice to just parse out these pieces and use them as test data. Why would I want to do such a thing? So I write a lot of software, and that means I write a lot of documentation. And that documentation needs to contain examples. And a problem that I have just constantly is that these examples will get out of date if they're not part of the test suite. And maybe I can try to write like unit tests in one file and examples in another file and then remember to copy and paste the code from one to the other and hope that that stays in sync and that requires discipline. I am not a disciplined human being. I cannot be relied upon to do this manually. And so my documentation, my tests will invariably drift out of sync and that kills projects. So I wanted to solve this with automation. And I worked on TestMark to fill this burning hole in my heart. And I wanted it to be involving a format that is simple and something that people already use. Something that's easy to get into a website. So basically all static website generators know what Markdown is these days, right? And I want it to be easy even to just render it on GitHub and when I'm in a project that's too early, too small, or too lazy to have made a website yet I want it to render on GitHub. So Markdown. Building libraries to process a format like this turns out to be really easy. There are three of them already and they can all process if you want Golang or JavaScript or Rust. There's a TestMark library available which can take those data hunks and make them legible to you because parsing this format is easy. You don't need a full Markdown parser. We just need to get the key data out of the middle of the Markdown. And Markdown and co-blocks are pretty easy to find because they're just these triple backticks, right? So you can seek line by line through a file looking for those backticks. You can just remember the line number where you found them and then you just need to toggle modes that you don't find like frame shift problems. Every time you find a new open co-block you look for the standard comment format right above it and if you see it, you've got a TestMark data hunk. Go get the label out of the parentheses. Great, you've got a parsing library. This is really fast to implement. It's like less than 100 lines of code in almost any language and most of those are probably going to be comments. And it's also correct. You don't want a full Markdown parser for this because you don't need it. You can just keep the rest of the file exactly as it is. In fact you can patch TestMark files with that much information. You just want to remember where the start and the end of the co-blocks are and replace those lines in the middle. This is a good idea because people write extensions to Markdown all the time like some people have some other syntax that says oh, put a table of contents here or whatever and you don't want to screw with that. So just parse for ticks. Remember the line numbers. Go. If you want to look at how this works in the Golang implementation we built a feature where you can use the standard go test command and say like TestMark regen as a flag and it will trigger a library feature which does this sort of thing and you can use a series of callbacks to configure your test system so fixtures can be regenerated automatically. We use this in practice in projects that have adopted this library to... Do I echo? To make changes in our code bases and then some part of the TestMark file might be here's my fixture data like my world setup and then I have some code in my project that's doing the data transformation and then I have another hunk which is an output and I can have the series of callbacks in my test setup code regenerate that output and then I just review with GetDiff. Super high productivity. So yeah, this is quite simple. I built a series of extensions around this which can add even more features. The convention of having filename sorts of structures in here is a very simple and very useful one. You can have a library just preprocess these things prefixed by slashes and maybe like have a globbing function work on top of this. You can use this to then combine lots of files into one example file in your documentation. This makes it easier to edit a bunch of stuff at once and do like you the human review a bunch of files easily. Another extension that is available in some of the TestMark implementations in particular the Golang one is called TestExec. This is something that you have to opt into using by like calling some other library functions in order to make it happen. But when you call those library functions it can look at any directory and it will end up being the test name. And if it has the words input and script and output as some filenames within that directory it can automatically become a test. So this is an example from the TestMark repo itself that is like part of the self testing. And this input section will be piped into literally the shell script and it will do this thing to test you can run. You can imagine how you could use this to write tons of black box texting for any command that you want to test and it's fast. It's very easy to use. I'll show a couple of screenshots of other projects that have picked this up to give a little more impression of what it looks like. Another project that I work on is IPLD which I will not go into in this talk but we have here a screenshot of some of this source for some of the specifications of this fairly complex library and you can see we're using TestMark here you can see the comments in the middle of the source code and the rest of this is a normal markdown file so I've got regular markdown syntax highlighting in my editor which is fantastic. I've still got the ability to use the syntax hint like the word JSON after the triple back ticks here this is a GitHub flavored markdown extension of markdown TestMark is just like yeah okay we know what that is so you can have your syntax hints. This when it renders on GitHub is of course still just regular markdown rendering on GitHub you can see here the H3 tags coming over the comments does not show up and the rest of the code block renders like a code block so it's very easy to use bulleted lists all sorts of other rich text formatting and have your test data in the middle of this so you can build narratives this other screenshot off to the side here is the same thing again published through another static website generator so it's got different decorations a bunch of other rich stuff from the website set up here but it's still the markdown source a quick example of what the code in Golang looks like that runs this is there's a wee little function call here that says ah please build a directory index which means like group these things that I can range over them and the test code plucks out different hunks by these names you see data here data selector selector is it expect so and then the rest of the test code is of course up to you you still have to write whatever logic you want and whatever language you want to actually do stuff but this gives you a great place to put all of the data in between another project that's used it same story different example content JSON not JSON text you can put any format in here and it's going to be just fine test mark treats these as effectively binary so you can do anything it renders fine to github in this particular project we haven't got a website yet so there is no third screenshot here but when we want one it'll be easy the source code that's running this stuff looks about the same there's a one-liner to go parse the text test mark file you call some other please index me functions grab hunks by name it's pretty straightforward if you want to compare this to some other things that might accomplish the same goals you've got lots of options fixture files are not a new concept documentation is not a new concept there's many ways to go about it a couple of the other ones that are the most comparable you might have also encountered a tool called embed md from that's the like github username frances campoy i think his name is this is a tool that also works on markdown and test data but it kind of goes the opposite direction of where test mark goes with this embed md thing you write lots of individual files and you use the same markdown comment format in order to label code blocks where you'll put the other files in with this tool so with test mark we write one big file and a bunch of hunks inside of it and identify them as if they were smaller files so you edit one place embed md goes the other way you edit lots of files and it composes them for you this works it's a totally valid tool i used to use it my experience of it as a user was that it caused me a lot of headaches because i would keep looking at the markdown file and wanting to edit that because i was editing my pros my story around this data and then i would feel compelled to edit the code block in the middle of the story and then i would whack my forehead less later because i would run my make file and it would run the doc regenerator and it would nuke all of my changes in the markdown file and i would be like edit it again so it's a tool you can use but i found test mark to be a user experience that makes me personally happier you can also easily handle the need for fixture files with plain json for example i guess i should also mention that my story when i produced test mark is i was trying to solve problems by using people working in a bunch of different programming languages so like one team in golang, me another team in javascript and so on and needing to have implementations doing the same stuff so nobody was happy with anybody else's format there wasn't like we're a ruby team so obviously we'll use rspec or cucumber or something we have all these different languages what do we agree on so one of the other options on that table would have been plain json files and that would work okay and the series of teams of different languages that i was working with and did that for a while but the experience we had was that it still was quite painful to read if you're doing json data especially if you have like multi-line fixtures ouch json has no way to do that you just end up with lots of slash n the git diff is just agonizing if you want to review that and it's not really great for producing human readable documentation the real joy of test mark isn't just that you can do data storage and labeling in it it's that you can weave a story around it in the markdown prose sections if you write a bunch of data fixtures in json what do you do you can't the closest you can give is maybe you put a map key called comment and then you put some more prose in there it's not compelling and you can't produce a pretty picture or maybe you can but then you're writing a lot more tooling you're writing something that takes this particular json format and all the particular key value names you chose and built a website out of it you can do that but it's more work you're definitely not going to get that to work just like as a readme and github right another alternative that has come up at least to me because I've spent a lot of time in goliang is something called txtar I don't think any other language has heard of this it's kind of a goliang special it's easy to parse it's similarly like split on lines very easy pars you could implement it in any language quickly but I find this is a lot less fun I can't write a pretty narrative around this it's not pretty I can't render it to a website I can't render it to github so less power in general if you want to work in some way that encourages you to write a story write real examples, write good documentation and tests at the same time and keep them alive then testmark might be what you're looking for that is the end of the talk I hope indeed that this picture sells the thing if you'd like to check out the implementations these are the links to them there's also vastly more documentation especially in this first link please check it out if it helps you, I'm so happy if there are any questions go for it with microphones maybe thank you very much for this nice talk and nice tool that you developed so that old that I already know tools from the 80s and there was especially one kind of movement which was called literate programming which was started by Donald Knuth and perhaps some others you are aware of it that seemed to be so I was wondering how that relates into what is the difference so it seemed to me that with literate programming it was more the mixture of programming itself and documentation that you had fractions of your code and then the documentation for that or explanations and the next part of your code and so on and your test mark is more concentrating on tests and the implementation is separate or what would you say is the main difference and what are perhaps similarities it converges a lot literate programming is an idea that I appreciate more and more as I spend more time programming absolutely I do think about it a lot when I wrote this you're right I pitched this with more of a focus on test data because that's the context that I had mentally when I wrote this the first time around but if especially if you're writing Lisp for example yeah, you can put code in the middle of this why not, you can have a REPL here especially with the regen mode I've actually done that a whole bunch one of these other projects that was my example here this warp forge one I gave another talk on this earlier if anyone's interested in reproducible builds and sandbox things by the way this piece of JSON here is something that has executable so it goes very hard in that direction how how quickly you cross that gradient from like data to tests to wow this is a REPL depends on your choices so testmark itself doesn't do that, it's just like the parser for the data if you write this application code of yours so that it has like an eval function you have now got literate programming that sounds very nice so what would be a good starting point if I want to look at testmark this warp form project for example there are examples there, I would go to this repo here warp fork me no testmark this one got written first so a bunch of the documentation still lives in this repo I'm sure that it would be better if I cleaned it up and made like here's the spec and here's the implementation but the read me in this repo will be the good entry point I think okay thank you very much cool if there were no more questions thank you very much for your attention I hope again that this helps you do cool stuff