 Okay, 623. Mohan was starting earlier though. Amjit, can you please share your screen, so I can put that up on the screen. And yeah, the question answers will be taken in the same way as before. If you can put, if all the participants can, attendees can put them on the screen, on the hop-in chat. Amjit will take them in the last five minutes if we have time or offline, if not. Now, a little bit about Amjit. He is the creator of DVCLI and open-source organization behind a few of the best-known command line database clients like PGCLI, MyCLI, LightCLI, MS EqualCLI, iRedis, etc. During his day job, he works at Netflix as a senior software engineer, focusing on reliability of distributed systems and recovery from failure. So the fact that we are watching this stream is from Netflix is somewhat of a blow to him, I'm sure. But he's here as well. So Amjit, the stage is yours. I think get muted, Amjit. You have to unmute yourself. Sorry about that. Let me start it again. Okay. My name is Amjit Rahmanajam and I'm here to talk about awesome command line tools. About five years ago, I started a site project called PGCLI, which is a command line client for Postgres databases. It had auto-completion and syntax highlighting from the terminal. And since then, it has grown into a GitHub organization called DVCLI with clients for a bunch of different databases such as MySQL, MS SQL Server, SQLite, and Redis. There are many more, actually. If you visit dbcli.com, you'll see them. And this talk is going to walk you through some of the design decisions that made these tools successful. And I obviously took inspiration from a lot of existing well-designed command line applications. And I'm going to try and summarize what those well-designed command lines tools did different than the others that didn't do as well. But before we get started into actually jumping into the various design decisions, I want to tell you about my own journey. So I started, I was introduced to computers in about 20 years ago. I was using MS-DOS. That was my first command line introduction. And during those days, we didn't have stack overflow. So I wasn't able to just simply copy and paste the command into the prompt to check out what it did. Instead, we used to thumb through a book or a manual. And whenever we saw a command that seemed interesting, like, you know, I don't know, format C colon or something, we would try it out on the prompt. The issue there is that I was not a very good typer. I would make typos and any time I made a mistake, I had to retype the entire command from the beginning. This took a very, you know, this took a lot of time. And at that time, my teacher took pity at me and he said that, you know, there is this thing called command history navigation. If you were to press the up arrow key, it would bring up the previous command that you have typed and you could fix the typo and then press enter to make that command run. And this was such a joyous occasion for me because I obviously was awful at typing and this saved me a bunch of time. And later when I was introduced to Linux in my grad school, I found out that bash also allowed me to navigate the history using the up arrow and the down arrow. But when I was pairing with a friend of mine, I noticed that he was able to complete file names and paths at a lightning speed, a speed that I've never seen before anyone typed. So I asked him about it and he said, and he introduced to me the concept called tab completion, which is when I type a prefix of a path or a file name and I hit the tab key, bash will autocomplete the rest of the file name for me. I didn't have to type the entire thing. Once again, this was a great day for me, even though I had now learned how to touch type and I was typing at a furious speed of 55 words per minute. Now, the point of the moral of the story is not that command lines have awesome features. They do and that's fantastic. The problem is that it is hard to discover them on your own unless a friend introduces to you or a mentor introduces to you. Or you go read about it in a man page, which is a different program than the program that you're trying to use. It is very hard to stumble upon them accidentally yourself. So discoverability, how do we solve this? Well, let's take a look at some of the GUI applications and how they solve this. Any new version of the existing GUI tool is released and all the new features typically tend to get an icon in the toolbar or an entry in the menu and being humans, we are curious and we look at a new button and we tend to go and poke at it. And that's a great way to discover these new features. But command line applications lack this ability. They don't have a way where you could automatically show an icon or some kind of a toolbar or anything like that because command lines are very spartan in their appearance. So what can we do? So let's take a look at PGCLI. This is the tool that I talked about which is a Postgres client from the command line. And how did we solve the discoverability issue in PGCLI? So I briefly talked to you about tab completion. I have just connected to a Postgres database using PGCLI. And when I start to type, so since this is connected to a database, I'm going to run a SQL query. And as soon as I start to type SE, I notice that there is a pop-up menu and then it gives me all the options that are available for that that start that happen to start with the SE prefix. So now I could I could simply go down the menu to select star from and all of these are available. So this is what I mean where I didn't have to press a tab key in order to trigger the completion that the auto completion was available to me as soon as I started typing them. This is a simple way in which discoverability was one of the discoverability issues was solved in PGCLI. For the next one, I want to show you fish shell. Now, if you have used bash or Z shell, you might already be aware of this little key binding, which is pressing control R allows you to search through your history. And what you do is you press control R and you start to type a little portion of the command that you may have typed some time in the past and it will search through the history and then it will try to find the command that has the little portion of the string that you have typed and it'll bring that up for you. So you don't have to type the entire thing. Once you have found this little key key binding, you will never type a command without control R because there is a good chance that you have typed a command already and you're simply trying to recover it from history. So you always type control R and then you will type the portion such as SSH into a host or you're making in your virtual environment or the seeding into a particularly long path or something. You could do all of those things using control R. Now, fish shell goes a step further. It surfaces this sort of history navigation and it's best if I showed it to you in a quick demo. Now, this is what fish shell looks like. It's just an alternative shell for bash or Z shell. And if I start to type, let's say I do CD and you'll see that it automatically fills in a folder name for me because it knows that this is a command that I have typed previously and it automatically pops it up for me. And I could press the right arrow key to fill the rest of it for me. This is particularly useful if you SSH into multiple hosts in your day-to-day job. For instance, earlier today I SSHed into a particular machine and I would like to SSH back into it. Instead of trying to find the host name, I could just type SSH and it auto fills the name for me. And if this is not the host that I'm interested in, I could press the up arrow key and then it will search through the various SSH commands that I have typed in the past and it will automatically provide them for me and I could choose the one that I need to SSH into. This is another clever way that fish was able to solve the issue of history navigation rather than binding it to a key binding. It was able to automatically surface that when it was needed and right at the time when we would need that. So the point of discoverability is that we want to make it easy for the user to find the features, the fantastic features that command line tools have built in. How can we do that? Well, there are two things you could do. One is be more forthcoming. What I mean by that is as soon as a feature becomes applicable to a particular scenario, surface that feature right away rather than waiting for the user to press a trigger using a tap key or some kind of a special key. So do not hide your features behind some kind of a special key but instead make it more forthcoming and make it available as soon as it makes sense or it's appropriate for that situation. That's how we solve discoverability in some of the tools that we have built. Now the second feature that I would like to talk about for command line applications is user focus. Now this is not just for command line tools. This should be for any kind of a user interface at all but the point behind user focus is that users come first. Usually when you are trying to implement a program, you would automatically try to optimize in a way where it is easier for you to develop the code. It is easier for you to iterate on it and so forth but I think that should always take a backseat to how it should feel for the user. So build the interface that is most powerful for the user, most intuitive for the user and then worry about how to implement it, how about going to implement it. I want to give you a quick example, a demonstration of what that would look like. So I'm going to take a second application called MyCLI which is a MySQL command line client which is very similar to PGCLI that talks to the MySQL database. Now before I show you MyCLI, I want to show you the default MySQL client, MySQL and I'm connecting to a particular database called world. I want to log into the database. I type SEL and I hit the tab key. In fact, I am hitting the tab key. You may not be able to see this but there is no autocompletion which is surprising. All right, let's try it again with uppercase and now I hit the tab key and there is autocompletion available. And this is what I mean by user focus which is the user, the programmer for MySQL could have made a simple choice where to make the autocompletion case insensitive but it was more convenient for them to have to change it from uppercase to lowercase or lowercase to uppercase in order to show the completion so instead they are forcing the user to type it all in uppercase which is a very poor choice that was made. And further that, if I do select star from and then I hit tab again, it's asking me if I would like to see 805 possibilities which is very surprising because this is a small database that I created just for this demo which had three tables at most. And so 805 is a bit surprising because after a from keyword you always have a table name so I press yes to see all the available options and it's giving me every single keyword that MySQL has in its repository instead of providing me just the table names which is again a very unfortunate way to interact with the user. Now let's contrast that to MyCLI. I'm connecting to MyCLI and I'm doing the connection to the exact same database and when I do that and I start to type SEL you will notice that it immediately pops up pops up the auto completion pop up for me and it also seems to be providing the fish style auto suggestion towards the end but we'll get that end in just a minute. Now if the user were to use all caps what happens let's see SEL and it still gives them gives them the option to auto complete it for them and this time the auto completion is presented with all caps. It knows that the user prefers to to use uppercase and so it provides auto completion in uppercase rather than providing it in the lower case. And then if I go further and then I do select start from and then I hit the tab key I notice that it is only giving me the three tables that are in this database because after a from keyword you can only fill in a table name and so I do that and it goes even further if I wanted to add a where clause it gives me just the column names inside of that particular table. It doesn't give me everything under the sum and then I could do any population over 500 and here are all the all the cities. Now this is a clear use of how the user focus comes first in the implementation second because when I first implemented PG CLI in fact I took the same approach that my sequel was taking which was I would complete I would show completion for every single keyword every single table every single column in that database and the user has to type in in order to narrow down that completion menu to find out what they are actually looking for. When I showed it to a couple of early users some of my friends they told me how inconvenient it was it gave them the illusion that it was going to do something smart but then instead it gave them the list of everything and they had to type in the whole name in order to figure this out so it took me another month or so to just write the completion engine for SQL so that I could make this thing work where it provided this context sensitive auto completion. Now that is a large undertaking you might think but it is totally worth it because that's what made these projects actually a success and so that's what I mean by user focus. User always comes first and implementation later do not focus on the implementation do not try to make it easy for you to implement some feature make it easy for the user to make it more intuitive and make it make it more powerful. And the next tool that I would like to showcase for you is called B Python this is a alternative Python interpreter that comes without a completion again and let me show you what that does. So before again before I show B Python I want to show you just Python REPL and this is the standard default Python REPL I type imp and I hit tab and it gives me import and I'd like to import requests module I type req and then I hit the tab key and it does not give me the module names but instead it's providing me with the the functions the the built-in functions that are available. It's it's kind of it suffers from the same problem that my SQL was having which was it is not context sensitive completion but let's take it even further so I do request and then I want to call requests get and I hit the tab key and it inserts a tab character into the into the code obviously Python this there is no use for tab key in a Python code so this makes absolutely no sense. Let's contrast that with B Python and see how that does. Here again I do imp it gives me import in a little pop-up menu I fill that in and I do re and it automatically gives me just the modules and I see requests as one of the available options and so I import requests and then I go in and I do request dot and it gives me a list of all the all the methods that I can call on this module I do get and I open a bracket and it not only gives me the arguments for that function but it also pulls up the doc string for that particular method which means I don't ever have to leave my Python REPL to find out how to use this use this particular method I don't have to open a separate browser window to read about the documentation or even to call help on this particular method to see the doc string it is available right when I need it it doesn't pop it up any any time before that but right when when I'm when I'm about to give it the arguments it tells me URL what does URL mean and what are the params and what are the return type and so forth. So that some of you might argue that this type of auto completion or this type of documentation information can be made available in the in the default Python interpreter by adding a little Python RC file. Now my argument to that is configurability is the root of all you this might seem like a bit of an exaggeration but whenever your your program introduces a new configuration option it is simply because the program is not smart enough to figure out what is the right thing to do for the user by default it should ship with sensible defaults it should not require configurations for the simplest things where it can figure out what is the best available option for the users. Now I'm not arguing that configuration all configuration is bad there are some configuration that is absolutely required for instance choosing a color scheme this is a very subjective choice this is based on someone's taste somebody likes dark configuration dark color schemes some some other folks like light color schemes and that should absolutely be a configuration option but having the auto completion and or having the history navigation using the fish styles and history navigation those things should not be configuration options but instead they should be defaults that start up with that. Okay so what that means is whenever you are building an application and you want to build a config file make the config file only have the options that are subjective taste based options. We talked about discoverability user focus and configurability and you and I showed demos from fish shell b python pgcl and myclite at this time you might be thinking I found a bunch of new tools that I could use in my day-to-day workflow which is great and these seem to be you know good replacements for the existing default tools and I'll go and use them but if I were to write my own command line client implementing all these auto completion or these history navigations fish styles suggestion seems a bit of an overkill or it's going to take more time than than I could I could spend on my on my projects so when I if I have to write something at work or as a site product for myself I probably won't use this use these things to to build it in my own. Now you might think that these are hard features to implement but what if I told you that you could implement all of these in mere 10 minutes and in simply 10 lines of python code in fact I'm going to do that for you right now in live the this is a checklist that I typically use the the checklist that I use has is that a any command line application should have a persistent history what what I mean by this is any commands that I type into a particular session after I quit this and then I go back to the application the history should be preserved so I should be able to press the up arrow to see what the last command that I typed or the previous command and so forth and I should be able to search through this history either using the ctrl r key because that is now kind of ubiquitous and everybody knows about it or using the fish style auto suggestion where it suggests things from the history itself and in addition to that it should also have emacs key bindings which is if you press ctrl a in your bash prompt it takes you to the beginning of the line ctrl e takes you to the end of the line and ctrl b takes you one character at a time just takes you back one character at a time and ctrl f takes you forward one character at a time and so forth but this has become a a common key binding that is available in most command line applications and I think any command line application that you build should build this in as as the users have come to expect this and auto completion and syntax highlighting are what give joy to a user when they are using a particular application so let's build all let's build all of this in about in about 10 minutes now I'm going to use a library called prompt toolkit it is written by jonathan slenders and this is the library that is used in pgcli mycla and in fact all of the tools that I that are under the dbcli organization and I'm going to use that library to build a repel a repel stands for read eval print and loop and what this means is you read the user input you evaluate the input and you print the results of that evaluation and then you go back and ask the user for the next input that's what a repel is so python when you type python and you hit enter and you go into a little interpreter that is a repel because it's asking for the user input evaluates whatever you have typed and then prints the results if there is any result and then goes back to asking you for the next input now we're going to do the exact same thing here I have a split window and I'm going to create a brand new file called repel.py is absolutely blank and on the right side I'm going to try and run this program every time I change something on the left side now if it was a regular standard python repel I would I would use if I'm using just default python that I would use input store that into a variable whatever the user has typed give it a little prompt perhaps like that and that will ask the user for an input and store whatever value comes back as the int into the int variable now we're not building any any kind of a regular repel we are building an awesome repel so in order to do that we're going to import prompt toolkit and use that prompt toolkit import prompt and instead of using input I'm going to use prompt and I mentioned that it has to continue to ask the user for multiple inputs so I'm going to put this inside a while loop and repel has to evaluate it and print the results in this case I'm going to skip the evaluation and then print right back to the user whatever they have typed so I'm it is let us call it a echo repel which is whatever the user has typed in it will print it back to the user and with about four lines of code we were able to build a simple repel now it's asking us for an input we could type hello world I can press the up arrow nothing happens because there is no history built into it I can type a select command and if I press control a and control e that seems to work control b is working control f is working and so forth so without much effort we have added key bindings by simply replacing the input with a prompt if I press control d it quits but it quits within traceback and the reason there is a traceback is because all I have to do is catch that exception and quit gracefully we can do that in a later time but for the sake of simplicity of the code I'm not going to add the tripe tripe except clots here now the second thing we want to do is add a persistent history once again we're going to use prompt toolkit to do that I mentioned that we want to have a persistent history and so I would like to use the file based history so that when I quit the application it saves the history in a separate file the way to do that is I add a new history keyword and I give it a file history and I give it a file name let's call it history here now if I were to run this again I do hello world and I press the up arrow and I have history and I can do the select star from abc and now if I press control r I have reverse search that's available so I could type in world and I get world or I could press select I get select and if I quit and if I come back to the rebel one more time and I still have the history available so this is what a persistent history is supposed to do because it persists between program restarts now that's history but using using control r is nice and all but we would like to have fish style completion because we talk so much about it so let's do that as well toolkit autosuggest import autosuggest from history here I do autosuggest equals autosuggest from history now I quit and I relaunch it again and now if I start to type the select statement I see the the auto completion auto suggestion automatically giving me that I have already typed a select statement in the history and so I could just press the right arrow key and get it filled in for me and that is how we add auto suggestion with a single line that we added to the code now let's add a completer so that it can pop up without having to press a tab key what possible completions are available so again the prompt hook it has a completion engine that is built in and we can do word completer and what word completer does is if we give it a list of words and we start to type any kind of letter in the in the prompt it will try to look through that list of words that we have and then it will automatically try to match whichever word matches with whatever we have typed so far so that's a word completer and since I'm pressing I'm typing select statements let us try and use use build this as an SQL completer so we give it a list of SQL keywords so let's give it select show from where tables and since I made such a big deal about might equal not being a case intensive completer I'm going to set the ignore case to true so that ours it can be smarter completion engine so I do a completer equals SQL completer and let's try this again relaunching this and I do s and immediately a completer pops up for me I can do show tables and there we go and so this this now gives me select start from ABC where the F equals GHI so we've already built ourselves a nice little ripple that provides us with auto completion auto suggestion persistent history one other thing that we had in the list was syntax highlighting let's see if we could add that now syntax highlighting is going to be provided by pigments which is another third-party library that has built-in auto syntax highlighting for a bunch of different bit of different languages out there so I'm going to take one of the pigments auto completer and I'm going to use that with prompt token so prompt token has the ability to take in multiple lexers which can auto which can syntax highlight and it has built-in ability to do the the pigments highlighting so from pigments import pigments dot lexers SQL import SQL lexer and then again I add the next one where I say lexer equals pigments lexer of SQL lexer and now if I go back and relaunch this and if I do the SEL and I'm going to use the auto suggestions to automatically fill it for me and there it is it is um syntax highlighted for me where the where the the coded string is uh is clearly syntax highlighted if I remove it it doesn't have syntax highlighted for me and so I can I can do it again and so forth so that is all it took for us to build a auto completing uh auto completing uh REPL I did say that we would be able to do this in 10 lines of code I did take 18 lines but let's uh let's make this all right let's make the code slightly ugly here we go voila 10 lines of code so if I did not uh bother with white spaces then I could build a uh a REPL in 10 lines of python code and let's uh this is just the code that I just typed in case I I choked on the live demo so in the checklist we were able to implement persistent history history search emax key binding auto completion uh in fact we did not have any configuration at all so that qualifies as minimum config um and we added syntax highlighting at the very end so the resources that I covered today are a dbcli which is the github organization and that has multiple clients inside of it we have pgcli uh mycli uh lightcli for esculide and there's even a redis client called iredis um and I talked about fish shell which has a fantastic design philosophy document which I take inspiration from quite a bit uh that you should check out and I mentioned the python which is my favorite python REPL that I use on a daily basis and I also talked about prompts toolkit which is a library that allows us to build these rich command line applications with auto completion and auto suggestion and so forth thank you very much and you want to come that's all I have thank you so much um that was very well timed again I mean a lot of value in a very short amount of time which is I think extremely cool um I think and we do have time for questions as well let's see if I have any questions over here yeah so Solomon asks again there's a user specific thing but what is your view on zsh uh zshell is uh I was a zshell user before I switched over to fish shell the one thing that I noticed was when I started adding new features to it uh it became slightly sluggish meaning whenever I launched a new terminal a zshell would take a few seconds for it to actually show me the prompt and that was annoying enough that I started to look into something else and fish shell provided me with the ability to you know configure it to my liking but at the same time there is absolutely minimal configuration it just works right out of the box without me having to muck with the you know adding completions for git or other things it has all of that built in um so that's one of the reasons why I switched over to fish shell uh from zshell actually right um and um I think one question was can we add b python functionality to an editor like sublime so I think there are other uh so there are completion engines that are available for python um such as uh for uh vs code I know that it uses a language server protocol and um uh vim itself um uses like the jedi completer which is a python library that allows you to do completion I am not exactly sure if uh taking the b python codes or the taking the b python's completion engine and trying to make it work with sublime is uh going to be successful or not I haven't personally tried that um it I feel like there are other editor focused completion engines that are available that it might be easier to manipulate them to make it work with sublime right all right um I think we are fine and in terms of time and questions questions everything is sorted unless anyone else had more questions if not I'm just going to throw up that screen where you had any contact information on some language it would be reached yeah absolutely and as with all uh speakers all other speakers um I'll uh create a separate topic for you on Zulip so it can be um I know it's pretty early for you it's around six ish in the morning yes it's uh 6 30 I will hang around and uh Zulip I did close that just a second ago during the talk but I'll reopen that in a second um reach out to me on on twitter uh my twitter handle is um jith are um I'm happy to answer questions either publicly or my private DMs are open so you're welcome to ask me questions in the private messages as well as well excellent I don't think we can ask for anything more and uh should suffice uh again uh thank you again and um hope to see you soon sometime yep thank you bye bye