 One of my favorite command line utilities is a program called AUK. AUK is a text processing utility that means you give it some text and it can grab certain columns, certain rows, certain fields from that text for you. You can tell it to go search for certain string patterns in the text and even replace those string patterns with other strings. It's a really powerful program and it's one I use all the time. I probably overuse AUK. I use AUK everywhere especially in my shell scripting because I'm really comfortable with it. It's one of those programs that once you learn AUK, you wonder why you didn't learn it sooner because it's such a powerful program. Let me switch over to a terminal emulator and let's play around a little bit with AUK. I'm going to run the PS command just because this command gives us some output that we can manipulate the text a little bit with. It's nice and in columns and that's kind of what we want. Columned information because then AUK can go in and grab certain columns from this output for us. If I take the output from PS and then run it through AUK and then single quotes and then give it inside braces and action, the action I want it to perform is I want it to print and then space dollar symbol one. That stands for the very first field, the first column. So print the first column from each line. So I should get the output of PID and then these numbers here and if I hit enter, that's exactly what we get. Now if instead of the first column you wanted the second column, you know, you just change dollar symbol one to dollar symbol two. And how it determines what the columns are, spaces are columns in AUK. That's what it is by default. You can change the field separator from spaces to any character you like, but by default, if you don't change it, it treats a space as what separates the columns. For the print command dollar symbol zero, if you do print dollar symbol zero, that means print everything essentially. That's essentially like catting a file and or in this case, we're just doing the same thing as the PS itself. It's like we're not manipulating anything. We're just printing the output exactly how it came to us. If I did print without anything at all, that's essentially print dollar symbol zero again. It's kind of like a cat. So one of the files that people love to use AUK on on GNU slash Linux systems is the slash Etsy slash past WD file. That is a file that lists all the users on your Linux system. So if I did a cat on slash Etsy slash past WD, this is what the output looks like. There's not a lot of spaces to it. There are columns. I mean, it is separated into columns with the columns. They are separated by colons here. Remember, AUK treats spaces as the column delineator. So what what you would have to do here is you would have to do something like AUK dash capital F for field separator. And then inside double quotes, let's tell it instead of using spaces as the field separator, let's use the colon as the field separator. And then let's do inside single quotes and then the braces print is the action and dollar symbol one that should give us a list of users on the system. And then we of course we need to specify where we're getting all of this from we're getting this from the slash Etsy slash past WD file. And that is a very convenient way to get every single user on the system on a Linux system. So you use that all the time. You'll use it just you know, system administration or sometimes through scripting. Sometimes you don't want to get a list of all the users in a script. Now let's take that same command. So I'll do an up arrow. And instead of just printing the very first column, what if I wanted, you know, several columns? Well, you can do that you can print the first column and then I could do dollar symbol six for the sixth column dollar symbol seven for the seventh column. And what this is saying, I want you to print columns one, six and seven from the slash Etsy slash past WD file using colons as the separator. And that's what the output of that would look like. Now it's not very readable, because we didn't tell it to separate the columns with spaces or colons or anything we told it a print columns one, six and seven basically all jumbled up together is what that is actually telling it to do. But we can tell it hey, you know what put some spaces in here so I can do double quotes and then space and then double quotes inside the double quotes a space in between the columns here. If I run that we added spaces to that. Now that still doesn't look very readable, probably better than spaces inside the double quotes would be a forward slash T. And what that will do is that will give us a proper tab in between the columns and hopefully that lines up a little better and it does it it's much more readable that way. Now other than specifying a field separator to search for and use you know to determine what the columns are, you can actually specify all to actually print out the field separator as well and you can tell it to change the field separator to a different character as part of the output. So if I ran this command right here, what this is is inside the single quotes. Now I have two different actions. So the very first action is begin and then FS equals colon and then semi colon OFS equals dash. So this is saying hey, find this the colons treat them as the field separator to perform these actions but as part of the output print a field separator that is a dash and what I want you to print I want you to print the first six and seventh columns on slash it's a slash past WD. And if I run this, you can see the output. Now I have columns one, six and seven and the separator is dashes and that's actually printed out this time. It actually prints out the separator where by default all typically doesn't print out separators. Now one cool thing that you can do with all as far as printing columns, there's a very easy way to print the last column of each line. It's with print dollar symbol capital N capital F. Let me show you this what I'll do is let's cat the slash it's a slash shells file here. And this list all the shells that are available on my system right now. And what I could do is if I wanted to print out all the shells on my system but I didn't want the full path name. You know, I just want sh bash get shells the sh fish, you know, the proper names without the full path. What I could do is I could pipe this through all or I could just use the all command I don't need to cat and then pipe it into all I can just do this all dash capital F and then inside the double quotes here I'm going to tell it use the slash as the field separator. Then what I want you to do is then I want you to search for a particular string here. So I'm going to do slash slash. And what this is inside these slashes here, all is going to search for a pattern. What is the pattern I want you to search for? I want you to find the beginning of every line. So that's the carrot symbol that begins with a slash. Now you have to escape that slash in between the two outer slashes. So it's going to search for basically every line that begins with a slash. That is what we're looking for that way it ignores these first three lines which are, you know, not information I'm interested in. And then after that what we want to do is we want to actually print and dollar symbol capital N, capital F, print the last field. And if we did this right, and of course we didn't do this right because I get no output because obviously you have to specify slash at C slash shells at the end, my bad. And then I get a list of all the available shells on my system. But you see there are some duplicates. I have two ZSH. I have two fish. So what I could also do is other than, you know, just running the alt command, I could then take the output from that alt command and then pipe it into unique, you know, and then it removes the duplicated entries. So every entry now has to be unique. And that is actually a list of all the available shells on my system right now. And of course through the magic of piping, if I wanted to properly sort this alphabetically, you know, to make this even more readable, I could do that as well. And let me clear the screen. And let's run the df command because this is another common command that has nice columned information that people love performing alt on. So let's do df and I'm going to run this through alt. I'm going to go ahead and pipe it into alt. And let's look for a particular string of information. So what I'm going to do is inside these slashes, what do I want to search for? I only want to return the lines that have slash div. Actually, let's narrow it down even further. I want the lines that include slash div and slash loop. So that particular string slash div slash loop. And then what I want you to do is I want you to take those lines that have that string and then I want you to print columns one. Let's just do one. Okay. And then let's print a few other lines. And let's do, again, let's just do tabs here. So let's print the second field and then another tab and then print the third field. So we're going to take the first second and third column on every line that includes the string slash div slash loop. And there we go on that. Now one neat thing you could do here is alt is a proper scripting language in itself and it does do simple math if I instead of printing columns two and three, maybe I wanted to perform some kind of arithmetic, you know, some mathematics on these numbers here. So what I could do is I could do something like print column one and then a tab and then print two plus three. And there you get column one and then you get the sum of columns two and three. The numbers, you know, added together. And of course, if I instead of plus wanted to do minus, I could, but on the loops those columns are actually the same number. So it should be zeros across the board. Now one of the cool things you can do with alt is you can filter results by the length of the line itself. So let me cat slash etsy slash shills one more time because it's such a simple file to work with here. And what I could do is I could alt and then single quotes again and this time what I want you to do is run this function length and then inside parentheses dollar symbol zero. So I want the the length of the line and I want it to be greater than seven characters and then give it a file to actually run this on slash etsy slash shills and it's only going to return the lines that were greater than seven characters because there was one line that was less than seven characters slash ben slash sh was only six characters. Let me see, I filtered that out by using the length function. If you wanted to see just slash ben slash sh, what I could do is rerun that command itself instead of length greater than seven length less than seven. And now the only line that is returned actually was the blank line. There was one empty line. Let me rerun that command and do less than eight and now I have two lines. Now I have the empty line and then I also have the slash ben slash sh line. Another example of alt, you know, as an almost scripting language in and of itself. Let's do the ps command again for the processes. I'm going to do ps dash ef. This is a list of all the processes that are running on my machine right now. What I could do is I could pipe this into alt and I'm going to do inside the single quotes this time and then a brace pair here. I'm going to do an if statement. So let's do if and then inside parentheses I'm going to do dollar symbol in f. So if the last field because remember dollar symbol in f is the last field space equals equals and then if the last field equals slash ben slash fish because fish is my default shell. So there's going to be several instances of fish running on the system right now probably at least I'm assuming this will not work obviously if fish isn't running but I'm going to print dollar symbol zero. So every line where the last field is slash ben slash fish and right now there are four processes running on my system that had slash ben slash fish as the name of the process. Another cool scripting example that you could do with alt you could do something like this. Let's do alt and then let's do begin and then let's go ahead and get the squirrely braces out and instead of an if statement this time why don't we do a four statement. I'm going to do four and then let's go ahead and declare a value of an integer so I'm going to do i equals one and then semicolon. I'm also going to declare that i is less than or equal to ten and then semicolon and then i plus plus. So I want you to incrementally add one to i until you get to ten. Right we're going to take i from one to ten and then what I want you to do is let's do a print statement after that and then I'm going to do double quotes and inside the double quotes I'm going to do the square root of and then comma space i comma space the word is comma space and then i times i and then semicolon at the end. So can you guys see what that's going to do? We're going to take every number one through ten and get the square root of it right when it's going to print out the square root of i is the actual square root i misspelled square though. Let's go ahead and actually make that the correct spelling and there you go the square root of one through ten. Another thing you can do with alt is you can have it print out every line of a document that begins with a certain character or a certain string. So here's an example of that from my bash rc. So what I'm doing in this is I'm telling alt and then in the side the single quotes I'm saying the first column needs to be uh it needs to match this search pattern and it needs to match the beginning of the line needs to be either a b or a c and then what I want you to do is print the entire line. So print every line where the very first character is either a b or a c and those are all the lines from my bash rc that meet that criteria. Now let me clear the screen. I'm going to cat out this file that I created just as an example file here because you often have this kind of stuff on your system or you know sometimes you're dealing with these column layout kind of files especially where you have line numbers. So you know one column is just numbering one through whatever and then you have a second column and you know sometimes you want to print out just the second column. Now of course with all what I could do is I could tell it just to print out the second column right because spaces spaces are the field separator but you know sometimes you know sometimes you may want to do it a different way depending on the layout of the file. So let me show you an alternative method we could use. I could do something like awk and then inside single quotes print and this time I'm going to use this function here this substring function and then inside parentheses we're going to tell it to print the dollar symbol zero comma and then four. So what I'm saying in this statement here is I want you to print every line but I don't want you to print until the fourth character. So ignore the first three characters of every line and if run that you see no longer do I have the line numbers you know that very first column now I just have the second column which was a list of window managers. Now another neat thing we could do to that number dot text file is I could run a command like this here. What this is going to do is it's going to find every line what I want to do is I want to find every window manager in that list that had the letter o as part of the name. So that is what the match function is doing is saying match on every line the letter o and then every line that has the letter o in it I want you to run print which I want you to do is print of that line and then follow it with this string has o character at and then the function r start that is the index location where the letter o actually appeared on the line. I hope that makes sense let me show you this in action. So this is the output line one x monad has o as part of the name so x monad has o character at the index position is six meaning o is actually the sixth character on the line. Now sometimes you want to print a range of lines so there's the df command but you know I don't want all of this information maybe I want only a section of the output from the df command well what I could do is I could df I could pipe that into all let's get into the single quotes and I'm going to tell it nr which is the number of records so you think of the number of records as the line numbers essentially nr equals equals seven so you need two equals for it to actually mean equals so nr equals equals seven and then I want to do a range from line seven to line let's do line 11 and then space and then we want to run this action here print nr the number of records dollar symbol zero so if I run that what we should get is line six through 11 print it out from the df command and there you go and we even get the line numbers as part of the command because that's what the in the print nr means now if I didn't want the line numbers as part of the output I just delete the nr part and now we get the output without the line numbers a common thing you can also do with awk that I find myself doing occasionally is getting a line count with awk so what you could do is let's do this on the slash etsy slash shells file that we were working with earlier so I'm going to do awk and then inside the single quotes I'm going to do end space and then let's do the print action again print nr again that's the number of records that's the number of lines on slash etsy slash shells and we get 13 return that's how many lines are in slash etsy slash shells now what's really cool is you can actually do this on multiple files and it will actually add the number of lines together so if I wanted the number of lines all together in my slash etsy slash shells file and also my slash etsy slash passwd file I get 55 returned but that's just a little bit of awk there I'm not a master at awk by any means but I know a lot of the the awk commands I know enough to where it's one of my go-to things especially with shell scripting if I need to manipulate text in some way you know typically I do it with awk even if it may make sense to do it with other tools you know you have other things like grep and said and a bunch of other command line utilities I'm always awking like I said in the beginning of this video I probably overuse awk a little bit but but it's such a cool command that I think everybody needs to know about especially if you really want to start getting serious as far as shell scripting you really need to know how awk works now before I go I need to thank a few special people I need to thank the producers of this episode Ebsi Gaye James Mitchell Akami Allen Chuck David Dylan Gregory Euryon Paul Polytech Scott Stevens Finn Wesson Willie these guys they're my high steered patrons over on patreon without these guys this episode about the awk command would not have been possible the show is also brought to you by each and every one of these ladies and gentlemen as well all these names you're seeing on the screen all these fine ladies and gentlemen they help support my work over on patreon because I don't have any corporate sponsors it's just me and you guys the community if you'd like to support my work I'd appreciate it look for distro tube over on patreon all right guys peace