 All right. Yes, this is a give-your-program appeal. My name is Larry Hastings. I guess the description wasn't clear in the program. Someone just asked me what this talk is about. It's about a command-line argument processing library. That puts it in the same category as arc parse and click and get-opt. But appeal was designed for Python 3, and it takes advantage of a bunch of Python 3 expressive technologies like key word-only arguments and annotations. I feel like it represents a rethink of the whole problem. It has a new and very different approach. I see it as a little bit of a paradigm shift in how to approach this problem. A little bit of history about the project before we dig in. I wrote the first version of this library in 2012. To be honest, it had a different name. I'm not going to tell you what it is because it's kind of embarrassing. It's almost gone from the Internet, but not completely. It had a bit of a code smell. I released it, and I looked at it, and I was like, you know what? I don't like this. I rethought how the API should work. I tried to rewrite it as a 2.0, and then I gave up and just changed the name to appeal. That was about 2014. Honestly, it took me four or five different attempts. I would try and write appeal. I had a high-level vision of what I wanted the library to work like. I would get to a certain point and stop. I don't like how this is turning out, and I would just put it back on the shelf and see if I could try again in six months or a year or something when I had a better idea of how to approach it. Meanwhile, I would write a little command line utilities that needed to parse arguments. I didn't want to use arg parse or click or doc op, those things. I got pretty good at writing a little 50-line toy version of appeal. I would just code that up. There's one in Blurp, which is a tool that we use in CPython Core development. I happened to start writing one of those in 2021. Thank you. I tried some things a little different, and it turned out really well. That became the genesis of what became the real appeal library that I published a couple of months later. One last thing, if you ever wondered why I got interested in annotations and why I wrote PEP 649, which is going to change annotations in Python 3.13, appeal is why. I've had a use for annotations, and I wanted them to work the way that they have always worked. Let's start talking about how to work with appeal. Here's a command line, and in my slides, my command lines and usage information is always going to be with a blue background like that. I claim that a command line API is algebraically equivalent to a Python function call. Yes, in my slides, I'm going to have Python on white. The way that appeal works is you give Python your callable, and appeal infers what the command line should be from that callable. Here, in my app, it's kind of like get push and get add and get commit. It takes what's called a command. The command here add, well, that's just the name of your callable. A command line positional arguments like file.cpp, those map to positional parameters for your function, and because those are addressed by position, and keyword-only arguments map to options because those are addressed by name. So this is really straightforward. As it happens, appeal doesn't have very much in the way of API because you express yourself using Python, a language and an API you're very familiar with. You just write a Python function which has a rich API, as it turns out, and appeal just infers what the command line should be based on that. So there's very little you do with appeal itself. You just do all of your work in Python. So appeal is pretty easy to use. All you do is import it. You create your appeal object. By convention, I call that app. That's why it's called appeal. It's for your app. At the bottom of your script, you call app.main, which is if you don't pass in an array, it's going to pull out sys.argv, you know, slice one colon. You define your function, in this case add, and you just decorate it with app.command. You're done. This is already a working command line program with an interface. It has help. It has usage. So let's start going into examples of how you actually write your Python functions in order and what it turns into on the command line. So as we've already seen, as an example, let's say we have grep and it takes a pattern and a file name. Well, that's just a function that takes two positional parameters, pattern and file name. What if you wanted to take an arbitrary number of files? You just use star args. It's totally easy. What about optional arguments, position arguments on the command line? That just becomes a function with a positional argument that has a default value. In Unix dash is a convention that means read from standard in. So grep pattern, grep pattern dash do the same thing. They're supposed to read from standard in, and that's how you would do it with appeal. What if we wanted to take a parameter that was an integer? Again, making this up, let's say that the context argument for grep is an integer and it wants a number. You would use a default value that was an integer. Appeal would notice and say, oh, they want an integer for that. So the default value is zero, but if you pass in a string there, it's going to call int on it and pass the resulting int into you. But if you wanted to have something that was an integer and you didn't want to have a default value, just use an annotation. Now I'm starting to make up command. So let's say we have an add function that adds two numbers together and the second one is optional. Well, the first one, we just annotate it with int. The second one, we could annotate it with int, but we just use a default value of zero and we're done. Again, if you wanted to add an arbitrary number of integers, well, just annotate star args with a function, in this case int. I don't know if you've ever done that, but yeah, you can annotate star args and star star kwrs. You can't have a default value for those, though. And as mentioned before, keyword-only arguments map into options. So let's do a proper grep with a context argument. That looks like this. Again, pattern star args. So the context here is going to be a keyword-only parameter addressed by name and it has a default value of zero, so appeal says, okay, we need to take an op arg from the command line and op arg is an argument to an option and it's going to convert it into an integer calling in Donut for you. What about short options? A lot of people use dash v as a shorthand for dash dash verbose. Again, appeal gives this to you for free. The first keyword argument that uses a unique first letter automatically gets that letter and it's a short option. So if you say verbose equals false, you automatically get dash dash verbose and dash v for free. You can suppress it if you don't want it. Also, notice that we're four slides in and there has been virtually no appeal API calls. Everything has been just expressing yourself in Python and you get different command lines, you get what you want and you don't have to, you don't have a lot of API calls to explain to the library what you want it. Also, to point out here, verbose doesn't take an argument. If the default value is a Boolean or if it's annotated with bool, that's a little bit of a special case where it doesn't take an option and it just negates the default value. How do you write your documentation? You just write a doc string for your function. In this case, the first line is going to be used on a summary screen and the entire string is going to be printed out when they ask for help on your particular function. So it also outdents it for you. So if you ask for usage, you get the help command for free when you use appeal. It has computed usage based on the command line, the function signature that you specified and it just dumps out the doc string for you. And then there is a summary screen if you just ask for help that shows you all the commands that are available and it just prints the first line there. So let's talk a little bit more about these annotations. Appeal calls them the converter because it is converting a value from the command line to what it wanted it to be. Converters can get pretty fancy. So here is an option supported by the configure script that ships with Python. It is called with dbm lib order. It allows you to specify database libraries and a particular ordering on them. And the argument to it is a string of values separated by colons. Can appeal convert that for you? Absolutely. There is a built in thing called appeal.split. You pass in all of the delimiters that you want to specify. And that is going to pull the string off the command line. It supports this equal syntax, of course. It pulls the string off the command line and it splits by all the limiters you specified and it passes into you a list of the strings split by those delimiters. Now one thing to notice here, and again, by the way, since it is an option, it must take a default value. I think I forgot to say that earlier, but options are always optional. So keyword only parameters in appeal always have to have default values. That's not a big deal. I think everybody does that anyway. But the thing to notice about this, appeal.split is this is a function call that returns a function. This is not a class. This is not a type. This is a function. It's just a conventional callable. And it's being used as an annotation. That's perfectly legal in Python, but static type analyzers might not like it. So there's a way that appeal and static type analyzers can get along. It's called typing.annotated. Let's take our previous configure thing and make the static type analyzers happy. If you're using MyPy or something like that, then what you can do is you just throw typing.annotated around your annotations. Typing.annotated is just a function that takes an arbitrary number of annotation values. And MyPy will only ever look at the first one. Appeal will only ever look at the last one. So they are perfectly happy to get along like this. Now you can use static typing and appeal at the same time. So a converter is just a callable. It is a class. It is a type. It is a function. They're built into appeal. But you can write them yourself. It is not a big deal. Let's look at a couple more that appeal provides for you. What if you want to do data validation? Mercurial has a dash dash template argument to log that says, I want you to use this format. And it only supports about eight different formats. And if you give it one that doesn't exist, it complains at you. Can appeal do that for you? Yeah. There is a function called appeal.validate. You pass in all of the values that are valid as arguments to that function. And it will complain if the value on the command line is not one of those values. And if you put integers there instead of strings, it will notice it will actually look at the type of the first argument to validate and use that type and coerce the string from the command line into that type for you. What if you want to have an option that you can specify multiple times on the command line? All the ones that I've shown you so far, you can only specify once. If you specify it a second time, it's an error. So appeal has special support for this. This is one of the places that there's a little bit of a special code handling for it because it's a little weird. And I call that a multi-option. So as an example, there are a lot of programs that allow you to specify verbose or dash v multiple times in order to make it the program more verbose. Appeal supports this with something called appeal.counter. Again, it's a callable that returns a callable. The default value is zero here. And every time that you invoke dash v on the command line or dash dash verbose, it's going to increment that number and then the resulting number is going to get passed into you. Similarly, GCC has a dash capital D that allows you to find a preprocessor symbol. Appeal has support for this. Oops, my man, I thought I fixed my slides. Defined is where it's going to go and that should be an empty list down there. It's the default value. And I did this just to show you what it looks like. If you want to specify exactly what your option looks like instead of allowing to appeal to infer for you, you can decorate with app.option, as well as this would also be decorated with app.command. Defines is the name of the parameter that it maps to. Dash capital D, that's the option that we wanted to handle. And annotation is the annotation that it's going to use to convert that value. In this case, appeal.accumulator is just something that every time the string appears on the command line, it's going to add it to a list and then pass the list into you. You can actually use square brackets after appeal.accumulator to specify types kind of like typing does, the typing module does in a lot of places. And so you could say int, and it would accumulate a list of ints. You can write your own multi-options, of course. You just need to inherit from a base class called multi-option. And there are three things that you can overload. The first one's called init. This one's optional. It passes in the default value, too. You do whatever init set up you want. Then every time you're specified on the command line, it will call option. And whatever arguments you take there, it's going to pull those arguments off the command line for you. So if you have zero arguments after option, after self, then it won't take any parameters. And if you have one, it'll take one. If you have two, it'll take two. And if you specify an annotation of an int or something like that or a default value, default values, they won't be optional. Op-args are always required. But it'll automatically call the type for you. So it could be an integer, and it would call int and pass that into option for you right here. And then the third one's called render. That's called after all command line argument processing is done. And this should return the value that's going to be passed into the parameter that this was annotated with. So as I mentioned, options can take multiple op-args. Like I'm making up examples here. Imagine that the echo command allowed you to specify the coordinate to print the string at. So dash dash chord is going to take two integers as parameters. So look at what we're going to do here. Echo takes the message and then an option of chord, which is decorated with x underscore y. X underscore y is a callable. We've written our own converter here. And it takes x, which is an int, and y, which is an int. They're both required. And it's just returning them as a tuple. You could return whatever you want. X underscore y could be a class. And this already works. This is going to pull two parameters off the command line, convert them integers, pass them into x underscore y. The return result from x underscore y is going to get passed into the chord parameter. And if they don't specify dash dash chord on the command line, it'll, of course, get the default value of none. But I've snuck something in on you. Maybe you have not noticed I'm speaking kind of quickly. This is fully recursive. This isn't just one level deep. Echo is calling x, y. X, y is calling int twice. Instead of int, it could be another function, which could be another function. It nestles as deep as you want. It's fully recursive. So, and it doesn't just reply to options. What if we had a different version of echo that took the parameters right up front and we wanted to reuse our x, y function? Well, man, these, screwing up themselves. Assume it says message parenthesis colon. X, y, the first parameter here actually takes two strings off of the command line, creates the tuple and passes it in. So this first parameter to echo underscore x, y actually consumes two values off the command line, which are converted into ints for you. It gets weirder. Options can themselves have options. Here's a made-up example. Let's say that we have dash o or dash dash option. This takes one op arc, but it also has a dependent child option, which we called child. It's only valid if you use option. It only gets mapped in then. Can you do that with appeal? Yeah. We look at the bottom. Command has an option, a keyword-only argument called option, which is annotated with option underscore fn. Option underscore fn itself has a keyword-only option called child. That creates temporarily a child option. It's only mapped in for a little while. If it's a sort of a scoping thing, it'll get popped off a stack when you escape that scope. And here's something else to think about. All the stuff that I've shown you, like you're writing Python, and it's a nice, rich Python interface, it is really an API. It's an API that you're calling from the command line, but it's an API. And it turns out, if you look at it, you think, you know what? This is actually a pretty reasonable automation API as well. You use appeal. Appeal constructs your command line interface for you, but you also can just export the functions that you created with appeal as an automation API. And now Python functions can call you, too. And they have documentation. They have type annotations. It's really pretty good. But we're still not done. It's going to get even weirder. So a friend of mine mentioned to me, he was using click, and he had a problem that he had solved. He had a bunch of commands, and some of them he wanted to have logging options, and some of them he did not. And so he kind of wanted to mix in logging into a bunch of different functions. And he said, I had to write this 50-line function to do it for me, and it was real painful to do. Would you be able to do that in appeal? And I thought about it, and I was like, you know what? It's really not a big deal. So here is my, again, made-up example. Show and scrub both take logging options, dash v and dash dash verbose, dash l and dash dash log level, which takes an op-arg, which is the logging level. How do you do that in appeal? This time we're going to use a class as our callable. It's going to be a logging class, which takes verbose and log level. And log level, they both have to have defaults because they are keyword-only options. They are options. And you just use that, again, as a positional parameter. So now we have a positional parameter that doesn't take any positional arguments from the command line, instead it creates two options that can potentially be passed in. And, I mean, we're already done. So this is almost zero lines of configuration. You just say which functions you want to take the mix in, colon logging here as a positional parameter, and you define your logging class and we just infer what the options are from the signature of the init. We're done. This all works because there is just nothing magical about a converter. In appeal, what I'm calling the command function, the top-level thing, like add, that's just a converter. Converters are just converters. Annotations are just converters. A converter can have positional parameters, or it can have options. It can recurse them to other converters. They're all homogenous. They're all the same thing from appeal. And so you can kind of do anything with anything. You can have child options and app-in child options that take op bars. It's all conceptually the same. So I was thinking about this, and let's reflect, let's pull back for a minute and think about how appeal thinks of itself. Appeal's real job from appeal's perspective what it wants to do is you give it a function and it wants to call that function for you. The only way it can call that function is by introspecting the signature and figuring out the entire tree of functions that it has to call to satisfy the function you gave it. And all of the leaf nodes, all the leaf parameters are places where it can put strings. So this tree kind of turns into this thing here where the blue squares represent this is a place I can stuff a string from the command line. So I grab sys.rgv and I pull the strings off the command line and stuff them in those places. I can call those ints. Now I can call x underscore y. Now I can call echo underscore xy. Appeal has succeeded. It's called your function. But here is the question, could we use this technology for anything else? What if we don't get the strings from the command line? What if we got them from something else? So my initial mock-up for this was reading strings out of a CSV file because that's iterating over an iterable of strings. But I thought, what if we pulled them instead of out of a string, what if we pulled them out of a mapping? So now, appeal also, and I haven't released this yet. I just realized I was like, oh, yeah, I was going to release this today just before the talk. I'll release it right after the talk. I'm going to check it in. This is going to be appealed 0.6. It now has the ability to reconfig files. Here's how that works. So you use toml or jason or yaml or I have a library called perky which you've never heard of, I realize. But these guys all return a dictionary to you. You read out of a file and it gives you back a dictionary. So here's our dictionary. You can point appeal at it. You write another converter. It looks a little bit different, not vary. In this case, I'm using a data class. Data class maps really well here. So A is an integer. It's going to pull the integer out of a called A. B is called, it's annotated with the read B, which is a function. So it's actually going to say, oh, I'm expecting a nested dict here, but you can see up at the top. And I'm going to pull out strings out of that and pass them in. So this is done by name as opposed to by position. So appeal is going to, you point appeal at it. All you do is call read mapping. You pass in the callable you want called. You pass in the dictionary you want it to read from. It's going to compile, quote, unquote, config file and turn it into an appeal program internally that's going to pull out these parameters by name. If they aren't there and it has a default value, it's okay. So like in the case of config file, they're all required. In the case of read underscore B, they're optional because they have default values. So in this case, it's a very different paradigm than the command line. So positional parameters and keyword-only parameters actually work the same here. But this supports reading basically everything out of a configuration dict that you might get from YAML or Tom or Perky. It supports nested lists, by the way. You use a multi-option for that. You just decorate with a multi-option and it's going to iterate over the values and give you back a list containing those values. For fun, like I said, I also shipped the CSV file reader. So as a last thing, oh, golly, I'm going so quick. Oh, no, I have no idea how long I've spent. It says I've been talking for a minute. That's not right. I'm going to give you a quick overview of sort of a map of what the code looks like in appeal. So appeal is about 7,500 lines right now. This is segmented by color showing you the different subsystems. And the green area is all something I call charm. So charm is appeal's internal bytecode format. And the more I make it like a real computer, the better it works. So at this point I have a compiler that generates, that calls an assembler to create a program, which is a sequence of bytecode instructions. And then there are a couple of interpreters. There's one interpreter that actually processes the command line. There's another one that just generates usage. And there's a third one that's used for generating the documentation. As you can see, charm is easily more than half of the implementation of appeal. But this really helped me corral the complexity of the problem. So I view it as a really excellent approach here. So what can't appeal do for you? Right now, well, this will never be true. It cannot run code in between parameters. So there are some functions. There are some programs that, like, they process an argument and they handle it right now. And they look at the next one, they handle it, and they look at the next one, they handle it. And if you have an option that changes a default behavior, the ones before it are defected and the ones after it are. Appeal can't work like that. Appeal is going to consume the entire command line before it starts calling things, because it wants to make sure that it has all the data it needs. It doesn't want to call you and get part way and then have to throw an exception and we stop. So that mode of working just is never going to work in appeal. Appeal is going to consume everything for you and then start calling you. Appeal might work on Windows. I'm not sure. I haven't heard from anybody. So I don't support Windows myself. I'm hoping that somebody will contact me and say, oh, yeah, I worked fine, or send me a PR or something. We'll see. Traditionally, Windows has had a very different command line interface with slashes and things. But I hear that it's kind of converging on behaving more like POSIX these days anyway. And finally, I don't pass coverage. I got a religion about coverage this year, and so a lot of my other libraries now have 100% coverage with the unit S suite. Appeal is not there. It's about 75%. And I promise I'll get to 1.0 before I ship, excuse me, I'll get to 100% before I ship 1.0. And that is about it. You can install appeal right now. Like I said, later today, I'm going to ship 0.6 and you can play with reading config files. It's on GitHub, of course, and if you'd like a sticker, all this cool 60s stuff, I have some stickers with that. And I think I have time for questions. Thank you. Thanks for the amazing talk, Larry. By the way, if you don't know him, he is very famous in the community with the Gelectomy project. And really amazing library. Thank you. Thank you. Yeah? Hi. Well, it looks great, first of all. Thank you. But there are different philosophies in design of the software. And there are like Unix philosophy in doing one thing well. And this configuration example made me think of a different question. Are you going to make it into a web framework? So I was literally just telling somebody this. So the answer is I'm not a web guy. So I'm not the guy to make this into a web framework. I think that this technology would be wonderful for your web frameworks. Because again, there's no configuration. You just write Python and your web API, your REST API would pop out the other end. But I'm not the guy to do it. So go ahead and fork it and good luck. Okay. Thank you. Sure. Hi. Great talk. Is it possible to use environment variables instead of options? Sure. So I didn't make the slide. But part of the genesis of read config files was I was thinking about, how do you support for environment variables? So here is how you would do it. And I think actually, again, I have the documentation written for this config file reading stuff. It will be in the documentation when I ship 0.6 later today. But I have an entire example, which is a sort of over elaborate example of reading. Let's imagine we have a command line program like git commit that wants to pop up an editor for you. So there are a bunch of ways that you can configure it. Already with git commit, editor environment variable. And then visual takes precedence over that. And then there's a config file setting called git.commit. I think git.editor is something. I don't remember what it's called. And that takes precedence over those two. And so I, as an example of working with appeal, added a fourth one, which would be a command line argument that said what editor you wanted to use on top of that. And so all you do is you read in the config file and the function that you write that you're going to point at the mapping when you call appeal read mapping, that has a default value for editor, which defaults to os.environ visual or excuse me, os.environ.get visual and then the default value for that is os.environ.get editor and that has a default value of like user bin vi or something like that. And then that'll be overwritten by the value from the mapping if it is set in the config file and then you use the value out of that thing as the default value for your callable that takes the command line argument of what editor we want to use off the command line. So now we have support two environment variables and a config file and a command line argument all with taking proper precedence and again there was virtually no code to it because appeal did all the heavy lifting and it inferred what you wanted based on the configuration of your function. So very nice question. Thanks for taking that up for me. Okay. I release all of you to the coffee break. Thanks for coming.