 about how to make awesome command line apps with Ruby. I've heard a little bit about that before. So briefly, like I said, I'm Dave Kaufman, Dave Tron, 5,000 on Twitter. So I am currently an engineer at Living Social, and even though Living Social is a pretty big company, we still work in small teams. And if you've worked at a startup or work on small teams, you know that being able to write a command line app is very useful and you find yourself doing it a lot. So that's what this is about. Also, I'm writing this awesome book right here which you should totally buy right now. Use all of the internet to buy this book right now. It's okay. But I promise I will only show this one more time. Still working? Yeah, there we go. Okay, so slightly more history. I started off with Commodore 64, which is certainly a command line interface. Yes. Then in college, I did C and I had a computer where the cursor keys didn't work in VI. So I learned that HJKL, like really good. And so now when I have to leave HJKL, it really angers me. So that's why I love doing stuff on the command line because it's all right there. Of course I did Perl, and then I used this hot new language called Java for a while, and there was no IDEs at the time, so I wrote Java and VIM, and if I have to write Java now, I still do it in VIM and on the command line, and then started using Ruby. So that's what this is about. And just to be clear, right, there's no IDEs in there. I do not like them. I like it on the command line. I like glueing my tools together and all that details. Okay, so obviously I care about command line apps because I was born and bred and have no choice and that's just how I am. But the rest of you maybe didn't have that upbringing. So why should you care? So I'll demonstrate this with a tail that I've lived out many times and probably you have as well. So you're working on something. There's some sort of emergency that you must fix. Like you're gonna go to production with some new feature and there's some problem. So you make your little one-off script. You SSH through the gateway to production and you're making your script that's gonna save you today. You run it with, you know, you didn't know time to think of the command line since it's actually just run it and get it done. Done, all right. And of course, you know, you wanna save this for later. So you put in diversion control because you know, you might need it some other time and you know, you're humble. Say, I didn't save the day. I just automated a couple things, you know, done. So that's all finding good. You saved the day and you're a hero. Now, it is six months later and some ops guy comes up and he's like, hey man, you remember that script you read that one time? And of course you kind of maybe remember because it's been a while. So he says, there's a huge bug in it that you have to fix. I noticed your name is on version control and of course I don't wanna have to fix it right. I'm in the middle of doing something. Oh, and it's gonna have to parse XML. So you gotta add some new features to it while you're in, so this exact thing has happened to me at least twice, right? So you say, right, let's do it. Let's fix this thing. And so you run it and you get a random error that makes no sense. Maybe you gave yourself a help statement. You know, that would have been nice of you. No, you didn't do that. Maybe dash dash help, right? That's a pretty reasonable way to ask for help. Okay, that probably didn't do what you wanted it to do. But maybe you were in some alternate universe where you're like on Windows, right? And so you try that, no, of course that's no help. So to hang your head in shame and edit the source and figure out what's going on. So this kind of sucks. Because you're gonna have to write command line apps, right? Like there's no way to avoid it. You're gonna have to write them eventually. And they have a tendency to become somebody else's job. Someone else will be running your command line app unbeknownst to you and says, your name is on it, you're gonna have to fix it or add features to it when something is going wrong and it might not be the time that's most convenient to you. Right? And I'm not saying like save the whales. I'm saying future you, be, you know, help yourself later on. Future you would have appreciated a better help statement or a couple of statements in that script to be a little better. Also as we're pointing out, the note how to use a command line unlocks this infinite way of gluing systems together and makes you a better developer. So there is that too. And I guess that's also selfish because you can make one money be a little better. Okay, so what I'm saying is that even your one-off scripts, you should make them awesome. Right? Because those scripts is one-off. They have a tendency to live forever and you wanna be kind to future you. And fortunately for us, Ruby makes it really easy to make awesome command line scripts that are just as easy and fast to write as a Vash script. It's not better. So what do I mean by awesome? So an awesome app acts like it's a first-class app. It plays well with others and it is helpful. So we'll talk about what those mean. It's a first-class. Oh, there we go, okay. So a first-class app is not, you can't make one by just installing some gem or applying some technique. It's a kind of attitude that you have to have, right? So your first attitude is it's not a script. It's an application. So you should treat it the same as you would treat any other application. Don't hack it together. Don't treat it as this chore that you have to get through. Treat it as a real application that you have to write and do like you would any other. So how do you do that, right? Like I said, it's not some gem. It's a series of decisions you make before you start. You decide it's gonna have a real user interface. You decide you're gonna use some good design and you're gonna write a little bit of documentation. You're gonna distribute it through some real distribution system. And all this stuff is really easy. You know, you're deciding to care about this application the same as you would your real application. And of course, if you decide this ahead of time, that's the best time to decide these things and it's actually not that hard to do it. So number two, aspect of an app that is awesome. Playing well with others. So I would say an app that plays well with others or command line up embodies the Unix way, which is this. Expect the output of every program to become the input to another as yet unknown program. Don't clutter the output with extraneous information. Avoid stringently common or binary input formats and don't insist on interactive input. That's pretty awesome. If every command line app acted like that, our lives would be good. So this is kind of vague. What does this actually mean? Your output should be greppable. Like typical command line apps of process text like have some thing that they're doing. And so if you keep that thing on a line by itself, you can use tools like grep and said to manipulate each record, each logical record of what you're doing. You should make your output cuttable, which means that these records should, if they have fields, often they have fields you wanna delimit them with like a comma or a pipe or something like that. So you can use cut or ought to, again, break up and manipulate the output. And by you, I mean someone else, right? Because when someone else wants a feature, you can say, no dude, you gotta grep and cut. Just do it yourself. Exit codes, right? If things go well, if your app did what it's supposed to do, exit was zero. If your app did not, then exit with non-zero. This way your app could be using a pipeline or some other script and people can tell if your app did what it was asked to do. Finally, messaging. So if you are, the output of what your app is doing or the thing that it does should go to the standard output and any messages that aren't that go to the standard error because the user can then redirect them to the right place and have more control over what's going on. So you might think this is obvious, but there's certainly some apps that don't do this. So here's an example. It is called Maven. And I don't know if anyone's used Maven. It is how Java developers build their tools. This is the command line syntax to run one test. And here is the first of what will be three pages of output. You'll notice there's all kinds of sort of enterprisey log levels for our build tool. And except for here, who knows why. At least two developers realized this was unreadable because they put this like row of dashes and then another guy decided to row of stars but not the same number of stars as dashes. Who knows why? If we try to read this, none of this makes any sense, copying nine resources. Why am I copying, I don't know, who cares? The second page is more the same, horrible. There's an error in there if you're looking close, but don't worry, it happens on every Maven build and it's not actually an error. I don't know why. So then the third page we finally get to some results and again we've got more developers putting dashes in their unreadable output and some sort of quasi-readable thing that says our test failed and we ended at zero, right? Great for success. So this means that like, yeah. So Maven is very hard to put into another system. It's very hard to script and work with because it doesn't follow these basic rules. So an example of something that does play well with others. So we'll continue the theme of applications that I can't stand. We'll go subversion, which I don't like at all but it's a great command line app. It's really well written. So here's a situation you get in with subversion. You've got some conflicts to sort out. Now what I want to do is I want to edit the files that have conflicts and then in subversion, right, when you've done that, you have to tell it, hey, I fixed the conflict, so it's cool. So I don't want to copy and paste that because I'm lazy. So I'm going to do this, right? I'm going to say, give me the stat, which we just saw and then show me only things that start with a C, which means they're in conflict and then yank the second field of that, which is the file name, it said all that to VI to edit and fix, which we are now doing and then we'll do the same thing to say, hey, subversion, we're done. So if you worked with the developers of subversion and you were like, hey, man, can you add this feature to let me edit everything in VI and then automatically resolve it? They would tell you no, you don't, no, because you can do it like this because they've written an app that plays well with others and can be used and manipulated in ways that they haven't thought of. And that's what you want your apps to do so that when people want these features, you can just tell them to go do it themselves and you don't have to do it. Okay, third, be helpful. So essentially, right, you want to help your users or you understand how to use and understand your app. We saw earlier how terrible it was to not have a help system and you also don't want to punish the users when they do something wrong, you want to be helpful and be nice to them. So here's an example and so the theme of these examples are apps that I like a lot. So latex, I mean, I never would have gotten through grad school without latex, it produces beautiful output, terrible command line now. You type latex on the command line, which is a reasonable thing to do to figure out how it works. And then you can put into this weird mode where you're supposed to start typing your article. And you can't save it and I don't know why you, I don't know anyone that writes articles this way, but the only way to get out is control C. So if you try latex help, then it puts you into this crazy mode where it wants to find the file help.tech, which you can, and no matter how many times you get control C, it will insist on trying to reload that file that it knows is not there. Until you get control D, which is an emergency stop that produces a log file. So that's not helping me at all, it's pretty tough. The man page is also amusing and how it describes itself, but we'll talk about that later. So helpful. Git, I love Git, and Git catches a lot of shit for being hard to use and hard to understand, but it's actually got a really amazing help system. When you type Git, right, does it check your entire hard drive into your version control? Nope. Right, it gives you a nice help system. Yeah, that's very, that's very nice. And it shows you all the options that you can use and a big list of commands that I've snipped off of what you might do. And when you say, Git help command, it tells you help about that command. In fact, it gives you this whole huge, well-written man, well-written man page. Okay. Git check out of mic. How's that? No? How's that? Okay, got the magic again. Yeah, so if you want your app to be helpful, it kind of depends on what sort of app that you're writing, right? So if you're doing like these unixy, simple kind of one thing, one thing, only apps, you're gonna do it one way, which we'll see if you're doing, you know, like LS or Grap. If you're doing like a command suite, which Git is a command suite, there's all these different commands, you have to give it to do stuff, subversion, gem, they work the same way. We'll do it another way. So let's see those two ways. For a simple app, when you type it again, you don't want to do anything annoying or destructive. You just want to give the user a nice little help statement that says what the app does and maybe how to get more help. When you do dash H, which you should accept as the help thing, then you should show the fuller help statement, which should be every single option that you take, what the arguments mean, things like that. Dash dash help should do the same thing because users will expect both to work and so why not be nice and have both of them work. For a command suite like Git or gem, when you type it, it's gonna do essentially what Git does, which is just dump out a list of what the available commands are and what are the globally applicable options, you can use with every single command. And it should also have a command called help and when you run your command, giving it the help command, it should do the exact same thing. The help command though takes an optional argument, which is the name of another command and then it should show you the help of that command, including what it does and what all the options and arguments are that are available. So all this stuff is actually really easy with Ruby as we'll see. Also useful error messages are pretty helpful. Someone asked a question yesterday and this was kind of my response, like this is weak. You should never have this being output from your program, it's just, you know, just lazy. And I was checking earlier and you can use as many statements as you want, so just keep using them until your code is not crappy like that, you know, it's not that hard. So right, we can apply all that stuff to Perl or Python or C if we really wanted to. So what's so awesome about Ruby, I mean, I guess I'm sort of preaching to the choir, but it's worth pointing out, especially if you have a ordinary system like I had that doesn't like any language that is a bench. So I kind of touched on this later, so I'll go a little bit quickly, but we've got all these high level abstractions, right? You're used to writing OO code in your web apps probably. So seeing that in a command line app, it just makes it easier. You don't have to switch modes of how you're thinking. You don't have to go, oh, now I'm in procedural Perl, so how am I gonna do it? And you're so close to the metal, right? You can run all these system commands and I mentioned that yesterday. And I think we made the case it's fast to start up. It's certainly faster than running like a Java program or something like that. And RubyGems is really painfully easy to set up even for internal distribution to package your gems and get them distributed. So you don't have to be checking out a version control or anything like that. The ecosystem, I mean, command line is part of the culture. Like every library has a command line aspect to it pretty much. And I also noticed that a lot of Ruby command line apps tend to be rooted with some era of taste. You know, they're kind of nicely well done. So it kind of motivated me to like wanna make better command line apps because those who came before did such a good job. And of course there's tons of open source gems and we'll see a list of them at the end of some things that help you make command line apps. And of course there's lots of gems to do everything else. So whatever you think you need to do interfaces some web service, there's tons of gems for that. Okay, so that was the theory part. So now let's talk about like something real here. Let's see, so actually, so there's two signs of unawesome. One, oops, is, can I not do that? It is, being terrible at shelling out to the system. What's it called? Poor sub-process management. So here's a program that makes a backup of my SQL file in a really cheesy way. So hopefully nothing will ever go wrong because I'm not checking anything. I'm not checking the error codes. Who knows what's gonna happen here. It's not good. So this is how you should actually do it. You should check those exit codes. I mean, I spent all that time talking about it. Like you should check them. If you call out to something else, it's gonna tell you whether or not it succeeded and you probably wanna know that. You probably wanna know the command that you actually ran the exact command line syntax. I'm sure we all come into work and our cron job failed for inexplicable reasons and would be nice to know what cron tried to do that it couldn't do. Similarly, the output. The command outputs a bunch of stuff that might be useful. Don't throw it away. Like put it somewhere because you're gonna need to look at it. Again, future you. What would future you have appreciated having to figure out what went wrong? So Ruby 1.9 has this open free library which is dead simple. And so here's an example of using it. You call capture three on a command and it gives you the standard out of the command as a string, the standard error as a string and the status which is a process colon, colon status which will let you figure out the exit code and that sort of thing. So things are good. We don't have to even say exit code equals zero because we're gonna say success and we're good to go. Otherwise, we're printing things out and you'll notice that we're printing the standard error to our standard error because where else would it go? And we're printing the actual command that we ran to our standard error again. So the cool thing about this is if you ran, if you use this inside a app that you're running in cron and you have redirected your standard output to say log file, the standard error will be captured by cron and emailed to you only if there is any. So IE, you get an email from cron if things went wrong with all the information in it that you could need to figure it out. So that's cool. So right, so you throw this in some method. Oops, yeah. And then you get a nice script that with a couple of if statements is pretty bulletproof but still pretty readable in terms of what it's doing and what commands it's calling. So basically, you don't have to do it that way. Do it however you want. But customize it somehow. If though in some gem, you can use internally and then use that instead of percent acts or back tick, persistently. And then all your calls are bulletproof. They're all log-in. Everything available that you need and it's really the same lines of code. So that is pretty cool. Now we're gonna go to the second sign of unawesome which is crappy user interfaces. So this is much more endemic and problematic and I don't know why, but I will say in learning about this topic, it's very hard to find all these gems. Like try finding the main gem. I just learned about it a few months ago. It's impossible to find via Google. You have to just know that it's on GitHub somewhere and it's in Ruby and you can kind of find it if you do that. So I kind of understand at the same time, hopefully this will help you out a little bit. So here's how to make a crappy user interface for a simple app, right? Just start grabbing shit off of RGV. For a command suite, here's another way. Again, grab shit off of RGV and a case statement. That's not good. Because if you're new to RGV, that's not good. You're doing it wrong and you're actually making more work for yourself. And so what's the problem with this? Other than that it's cheesy, it's hard to maintain and enhance. So that means when someone else wants a new feature to your app that you're gonna have to do it. You know, it's hard to add complex features when you're just hacking RGV, right? Or for others to maintain as well. And it's actually more work as we'll see than using some common open source libraries. So for simple apps, there's OptionParser. It's built in, we talked about it yesterday. So here's an example of it. If you haven't seen it, it's super powerful. It's more powerful than I thought and it's really great. So basically you set up this block where the OptionParser is new and inside that block you're gonna have to clear your user interface. And the first thing you must do, I'm saying must, you don't actually have to, but you should, is make a banner which says what your app does and how it's used. Very simple, very nice because in the Ruby community we have very clever names for RGEMs and you might not remember what they are. So having this description is pretty helpful. Six months from now when you don't remember your clever name. So then at the end, we'll see how you specify the options and stuff. You just say ops.parser, it grabs everything off of RGV for you, does the parser. So what happens then? So inside this block here, you can do stuff like this. So you just call a series of calls to the on method and so this says that it'll accept the flag dash dash auto and its documentation is that it means auto regenerate and inside the block, the block gets called if the user specifies this on the command line. And here we're just setting an options name. You can do whatever you want in there, whatever makes sense. And so you can also do negatable switches. So if you do this square bracket no dash around there that means that dash dash auto if the user specifies it will cause true to be sent to this block and dash dash no dash auto will cause false to be sent to the block. So you can make this really fluid user interface which is really handy for cron jobs where you can see exactly what the app is supposed to do. To accept for flags, where you need to accept an argument, you just tell it, you just put the name in there and that will tell option parser that this thing takes an argument. And in this case, you can put any number of these things in here and so that means dash s takes an argument, dash s server takes an argument, here's the documentation and then the argument gets passed. And then here you can see an example doing some sanity checking. Again, you can do whatever you want in here and so we can just kind of barf if we don't like the value. Now, option parser can do a lot of other sanity checking for you. So if you pass a regex in then the argument must match that regex where it won't cause the block to get called. So that's pretty awesome. You can be guaranteed that IP is gonna match that regex. You can give it a list of allowed values. You can give it a map of your keys or the allowed values. Very cool. And for doing all of that very, really minimal work, you get this awesome help statement, right? Yes, right? Formating this is a huge pain in the ass and option parsers did it really nice. And dash dash help works, right? So you're good to go there. Now, option parser is easy for others to modify. When you see an app written with option parser it's very easy to modify it, which means someone else can take care of that for you while you do other things. And it's built with Ruby, so there's nothing to distribute that's really an excuse not to use this. Except if you're doing a command suite, option parser is not well suited. It's kind of a pain to get it to work for these command style interfaces, especially if you want something kind of complex. So there's several options. We'll talk about one that I wrote called GLI because I'm giving the talk so I can choose the gems. But I'll give you a list to be fair later of other ones. So it's basically me to make a command suite app just like the one that I was describing. So the idea is you want to have very polished UI without doing any work and also bootstrap things because I don't want to type either. So let's see what it looks like. Let's say we're going to make a to-do list app. So we'll install GLI and then we're going to make a to-do list app that takes three commands, new, list and complete. So we bootstrap it with the GLI init command and we have a pretty reasonable like Gemified project that we can start working in. So the first thing we'll do is we'll run the app and we can see that we get this nice help system already formatted for us. Obviously, it's boilerplate text but all the help stuff is there. It shows us all the commands. Very easy. And then we do help and a command. We get help specific to that command. Nice. So all the command and option parsing is done. The complex help for me is done and you get a boilerplate project. So here's what it looks like. I think I ripped this off from the GitHub gem. It's also sort of like reg I guess and that you kind of describe things and then declare them. So here we have a global option called dash F that has a default value. Here we have the command list and then inside the command block we declare the options that are specific to that command. So the list command takes a dash L but other commands don't. And then finally we have this action block here at the bottom and the action block will be called when the user executes this command. So the action block gets the global options that were specified, the command specific options that were specified and any other arguments that were on parse from the command line and then you can do whatever you want. So when we fill this all in, we get a nice looking app that has a nice looking help system without a whole lot of effort. So essentially we can just focus on what we need to do. Sons of other cool features, you can check it out there. So option parser and GLR is two examples of dead simple policy UIs that you get. You can spend time working all your app does and not formatting help text or as a case maybe not formatting help text because you skip that part when you don't have the tool to help. Then you can easily maintain the documentation. So future generations will know what your app does and how to use it. So there's a ton of other CLI gems. I'm gonna go through them really quickly here off aka command line app Safari. So for parsing we saw option parser, we saw GLI, there's Thor which is very popular which is a similar thing to GLI. Trollop which is used by Cucumber does both kind of apps which is pretty cool. Methodome which is something I wrote lets you use simple apps and also has support for testing and bootstrapping and for interacting with the user if you must. ReadLine is built into Ruby and it's just like Unix ReadLine so you can make a prompt with tab completion, very cool. And Highline is a sort of DSL for interacting with the user, asking them things, getting info from them, putting it back out. Terminal tables if you wanna make cool little formatted ASCII tables like if you wanna make your own SQL client that's really easy to do. Rainbow is one of about a thousand gems who color your output. It's I guess the first one I found when I searched so now I use that if you want cool colors. So testing and documentation. So Aruba is really awesome, right? It's you can test your command line app with Cucumber and Aruba provides a bunch of Cucumber steps that are tailor made for running a command line app like checking the status codes, checking the standard error, contain stuff, those sorts of things. Gemman lets you install man pages with your gem and then read them and then the user can read them which is super cool until you realize that MROF kinda sucks but then there's Ron which lets you write your man page and mark down instead of MROF so you can write like really nice man pages and have them totally distribute with your gem. So, right that's a list anyway this will be up later if you guys wanna check it out and please tell me if there's anything I'm missing cause I always like to hear about new command line stuff. So making an awesome command line app is really easy because Ruby makes it simple with its awesome language and awesome tools that it comes with. So yeah, some of those tools make it really fast to do what you wanna do and future you would really thank you if you use these things the next time you had to throw together a command line app. So if I'm missing something here or if I'm wrong which I doubt but it's possible please let me know as I said last show from my book it's in beta right now which means that you can totally buy it right now and tell me that things are wrong or messed up and I will fix them. So that's really helpful for me cause I wanna make this as good as possible. So that is it thank you and I guess we have about two minutes and 50 seconds for questions. Anybody has any? Yeah, yeah man I forgot to put that on the slide but main is pretty cool. You know, unbeknownst to my wife she has a full Ruby install on her laptop and I did try this on Windows. Most of this stuff does work like the exit codes actually work. This is a Windows 7. The exit codes work standard error standard out actually works redirecting actually works. I was pretty surprised. So the hope is that all the stuff I talk about in the book will work on Windows or we'll have a footnote that says well this doesn't quite work so next time she's out of town I'll test it out. So the question is have I reused stuff on command line gems for anything else? I haven't really, I mean I guess if I'm, it depends on what I command like because if I write like some sort of worker in my Rails app it's technically command line app and I might use it there but inside like a Rails app I haven't so far. All right cool well thank you guys.