 One of the things I like to do on this channel, every few weeks or so, I like to do some simple bash scripting and do some of this on camera. I like to come up with a problem that I want to solve and solve it through the use of bash scripting. And what I want to do today is I want to create this little demon use script that searches the ArchWiki for various topics and then opens the appropriate ArchWiki page in a web browser. But here's the deal. If I don't want to actually search the ArchWiki as it exists on the web, I want to have a stored local copy of the ArchWiki, an offline copy of the ArchWiki on my machine that I can search through a demon use script and open the locally stored wiki articles on my machine. And this is something that I think I can do in a half an hour or so. This will be a really simple script. It's only going to involve some really basic bash scripting and a few of the basic GNU core utils, some of the standard command line utils we use on Linux. Let me switch over to my desktop here. And this, of course, is the ArchWiki main page here. And what I want to do is I want to basically download the entire ArchWiki. So it's stored locally on my machine. Now luckily, I don't have to do this because the ArchGuys make this really easy. They already have a package in the ArchRepos, the standard ArchRepos. Look for Arch-Wiki-Docs. And here is Arch-Wiki-Docs. So that is the name of the program. So all you need to do is open a terminal. And let me zoom in here and do a sudo pacman-sarch-wiki-docs, I'm not actually sure if it's installed on this machine or not. And it looks like it's already installed, so I'll decline reinstalling. Now that I have ArchWiki-Docs installed on my machine, I should actually tell you ArchWiki-Docs is not a program. ArchWiki-Docs, as the name suggests, is just the Wiki articles themselves, the documents. It does nothing. It's not a program you can execute. It's not some command line tool you can search with or anything like that. You have to come up with your own way of searching the docs, right? You have to use the other GNU core utils, the command line utilities to actually do stuff with those docs that are now placed on your machine. Documents on GNU slash Linux systems should be placed in slash user slash share slash doc. So I'm going to CD into slash user slash share slash doc slash ArchWiki is where it places those. And then, if I actually just CD into that and do an LS, you see HTML and text. So there's two different versions of the docs. There's plain text and then there's HTML. Plain text would be nice if you're just catting it out at the command line. But what I'm going to do is I'm actually going to have it open as proper HTML in a web browser. So I'm actually going to keep on cd-ing into HTML and now if I do an LS, that's a lot of image files, but there are subdirectories and the subdirectories are of course languages and I want en for English of course, right? So let's go ahead and cd into slash en. And now if I do an LS, these are the actual English Arch Linux wiki pages for the various topics. So those are the files and what we want to do is we want to somehow pipe these files into a D menu that we can then choose the individual files and have those files open in a web browser. Now, of course, to open these files in a web browser, I only want files that are actual HTML files. I don't know if there's other stuff here that is non HTML files. One simple way is to actually just look in a graphical file manager. So if I do PC man FM here, it should just open PC man FM in this directory that we're currently in. So it opens user share doc arch dash wiki slash HTML slash en. And you can see that there are other directories in this directory. So there's obviously these directories are not HTML files, but it does look like all the files themselves. Everything in here is a HTML file. So I'm going to close the graphical file manager. I just wanted to open that just to get a good visual representation of exactly what was in that directory. So let's think how we're going to start this script. First we need to go ahead and grab all the HTML files out of this directory. We also need to grab all the HTML files in the sub directories within this directory. So for this, I would use the standard GNU find command. I've done a video about the find command in the past. Check out that video if you're unfamiliar with how the find command works. But basically the find command works this way. You do find and then the name of a directory or the path to a directory. If you don't give it any path, it's just going to search this directory or you could be really specific and just do a period, which also means this directory. And I want to do an insensitive name search. So I'm going to do dash I name for insensitive name search, meaning I don't care whether the letters are uppercase or lowercase. And then I'm going to do inside double quotes, asterisk, asterisk is a wild card character. It basically represents any standard character dot HTML. So I want any file that it doesn't matter what it begins with, as long as it ends in dot HTML. And if I hit enter, you know, it spits all of that out in this nice little format here. Now I will say looking at the format, we get the dot slash as part of the file name. I will say that's going to give us some problems when we try to start opening things in a web browser. I probably want the full path. So to get the full path instead of the period, which is an alias for this directory, let's actually do the full directory path. So I'm actually going to specify slash user slash share slash doc slash arch dash wiki slash HTML slash en slash. And if I do this, it'll do the same search, but instead of the period, it's going to substitute the period for the actual full path user share doc arch wiki HTML English. And now that is something we can work with. We can pipe this into D menu. And then when we choose something, you know, we already have the full path, which would get then piped into our web browser. And it would open that particular HTML page in our web browser. So let me go back to the web browser here and I'm going to close out the arch wiki pages here. I'm going to go to my get lab because I have a project on my get lab called DM scripts, which is a collection of D menu scripts that I've found useful. If I click on it and I go into DM scripts and then the scripts sub directory here, there are a collection of a, I don't know, 20, 25 various D menu scripts that are part of the DM scripts package. DM scripts is packaged up and available on DTOS. DM scripts is also packaged and available in the AUR. So anybody that's using a arch based Linux distribution can install DM scripts through the AUR. You'll find it as packaged as DM scripts dash get. So I'm going to create a new script. You can see the naming format here for all of the DM scripts are DM dash name of script. So let me open up Emacs and I'm going to navigate to where I have my DM scripts repository here. And there is all the scripts and what I'm going to do is I'm just going to create a new one. So let me go ahead and create DM dash wiki. And of course, this is an empty file. All I'm going to do now is when you create a bash script, of course you need a shebang at the top. So what I'm going to do is I'm going to do crunch bang, right hash symbol exclamation. So crunch bang is typically how we refer to that slash user slash bin slash ENV space bash. So that specifies that this is in fact a bash script. And let me write that and zoom in so you guys can see everything that I'm going to do here. And since this is going to be part of my DM scripts project, there's a certain format that we follow for the DM scripts. So what I'm going to do is I'm going to just briefly go into a different DM script because the header for the DM scripts is always the same. So what I'm just going to do is I'm going to do a copy and then go back to this buffer here and do a paste. Let me zoom out a little bit because I was zoomed in way far, but you can see what this will be is some comments here. So the very first comment here and comments are not necessary, but I'm doing this because this will eventually be pushed to the DM scripts get lab. So this script name is DM wiki and then the description choose from a list of configuration files to edit. We are going to say search an offline copy of the arch wiki and then the dependencies. There really shouldn't be any dependencies other than a D menu of course and then arch wiki docs. Something else that we're going to use in the script is just going to be part of the standard GNU core utils, which should be installed on every GNU slash Linux distribution out of the box anyway. So no need to specify that. And then the location of the get lab and the license should all be the same. The contributors right now it's just going to be me. And then the next section that was part of the copy and paste. If you were doing this script yourself as your own individual project, you really wouldn't need a lot of this. So up here I have this line here set dash EUO pipe fail and what this does, I made a lengthy comment here to let you know exactly what set EUO pipe fail does. It calls us the script to fail if certain things happen, which is a good thing. Otherwise you can get hidden bugs that are hard to discover. The next part here is path equals and then the statement here and then the if statement behind it. This has to do with there's a helper script that is part of the DM scripts project. So all the individual scripts within DM scripts can use this DM helper shell script. There's also a specific config file that is used by all the various DM scripts and that gets sourced here. So there's a config file that has a bunch of variables and arrays and things that get sourced by all of the DM scripts. That way, you know, you're not setting the same variables for every single DM script since there's so many scripts that are part of the project. But again, those of you that are following along, you know, you're going to write this script with me, you actually didn't need any of this. All you needed was the shebang at the top to make sure it's a bash script. The rest of this is mainly specific to me for this particular project that I'm eventually going to push this to. Now let's actually get to the actual project. So let me zoom back in here. The first thing we want to do is we want to create an array of all of those HTML files that were in that particular directory we were searching. So I'm going to create a new variable and we're going to call this wiki docs. So this is going to be our array and the array is going to be whatever happens to be a member of the following list. So we're going to run the find command and we're going to run the find command on slash user share doc arch dash wiki slash HTML slash in for English. You know, now that I think about that, not everybody that will eventually use this script will want the English version of this. So what I'm going to do actually is I'm going to delete that and I'm going to make that a variable. So I'm going to have a new variable here called wiki dear and inside quotes will paste that and now the wiki docs array will be find wiki dear. That way, it'll just be a simple enough change. For example, if you wanted to change from English to Spanish, for example, you could change, you know, in here to ES, for example, you just make that one minor change to the script instead of having to go in and change this variable here in the string here. And since DM scripts have its own config file that gets sourced, I'll probably add this variable eventually to the DM scripts source config file here. But for now, I'm just going to leave it right here. So find in the wiki directory, remember the flags we used with the find command in the terminal dash I name for an insensitive name search. And then we're going to search for asterix dot HTML. So everything that ends in dot HTML. And that should give us an array to pipe into D menu. Now let's go ahead and create a function here. We'll call it main because this will be our main function. So I'm going to do main opening closing parentheses. And then let's do a opening and closing braces here inside the braces is going to be the meat of our function here. And I'm going to create a variable here. I'm going to call it choice because this will be the D menu list and whatever we choose of course will be our choice. And choice needs to equal dollar sign and then in parentheses dollar sign parentheses signifies that everything inside this parentheses is going to be a command to run. The command to run is going to be printf and printf we're going to do inside single quotes percent s and then backslash in. So it's going to take everything and put everything on its own new line that's backslash in is the new line. And then what is it printing? It's going to print everything that is part of the wiki docs array. So do inside the curly braces wiki docs and then opening closing bracket and inside the bracket the at symbol. So this signifies that we're printing every single member of the wiki docs array. And then of course we want to pipe all of that into D menu. I'm going to give D menu the following flags dash L for line count. So we're going to do a line count of 20. So it's going to do a vertical listing of 20 lines that gives us a preview of 20 lines. So it'll be a vertical listing rather than the default horizontal listing that D menu typically uses. I'm also going to give it dash P for a prompt. This allows us to specify a D menu prompt. I'm going to call the prompt arch wiki docs colon space. And just in case there's any kind of problems with this printf command and the wiki docs array I'm also going to go ahead and do a or exit 1. Let me do truncated lines, turn truncated lines off here in Emacs so that we get the line right up here. So choice is going to equal this command here. For some reason this command here we're trying to run fills, it's just going to do an exit. So now that we have choice defined, what happens when we actually choose something? Well, let's go ahead and do an if statement. So I'm going to do if and then inside quotes dollar sign choice. So if I actually make a choice then, well if I can type, this will be an if then else statement. So let's make sure we have all the components here first. So if then else and then of course at the end of if should be fi, that's the ending if statement. So that signifies that the if statement is completed. It's done. And I like doing all of this before filling it in just to make sure I have all of those components here. And then space over about four times is typical in bash scripting. And then let's go ahead and fill out the if then part. And this is going to be really simple. If I make a choice, then I want you to open my browser. For example, my browser is brave. I want you to open brave. And I want you to open it with the following file path right choice because choice is going to be the full path to that HTML file that I chose. That would work. But you know to make this more portable for everyone, I could do xdg dash open, which is the command on your GNU slash Linux systems xdg open means whatever file type I give you open it with the default program for that file type. So on your system, if your default web browser is Firefox, then your HTML files will open in Firefox. If your default browser is chromium, it'll open with chromium on my system. It's going to open it with brave yada yada yada. So that's a nice way to do this, I think. Now let's go ahead and finish the if then else statement. So if I make a choice, obviously open our choice in our browser else if I don't make a choice, what do I want you to do? Well, I want you simply to echo program terminated because if I don't make a choice, that means I've probably escaped out of the D menu script, which if you escape out of it, it really means I'm just closing the program anyway. So just echo to standard output program terminated and I'll also do an and exit zero there. And then the only other thing we need to do with the script would of course be to actually run the main function because this just defines it. It doesn't actually tell it to run. So at the end, you probably want to just write main and let's actually test this out because I'm not sure if this is going to work or not. So let me go to my terminal here and if I do an LS, you can see we're still in this directory. There is DM wiki. And let's go ahead and try to run the script and see if it runs and it runs just fine. So it does define command, it gets all those HTML files and it pipes them into D menu. We get a line count of 20. So it previews 20 at a time here. And we've get our prompt arch wiki docs colon. And if I choose one of these HTML files, it should open that path to that HTML file using the default web browser. And if I hit enter, it does not look like it's using the default web browser. It actually opened up my library desktop app. So maybe the problem with that is for some reason is my library app, the XDG Open default program for HTML files on my system. It could be. Let me actually just specify brave. Let me write that and go back to the browser. Let's try that again. We get to our D menu script. I'll pick a different HTML file. Yeah. And now it opens in Brave. So XDG Open is what I would like to do. But for whatever reason on my system, I don't know why my default browser is the library desktop app. That's kind of weird here. Now there's some things I would like to clean up, you know, even though already it works and I could live with it. There's some things I don't like. For example, I've got the full file path to everything. User shared doc arch wiki HTML EN slash and then the name of the file. That is a lot of extra file path to each one of these. And that does create a problem when you're searching. You know, if I'm searching for grub, for example, you know, instead of actually getting what I'm searching for, I'm going to get a lot of crap because G U R B. That's a lot of letters that are part of the file path. The file path, you know, is really not what I want to be searching for, right? It's really it's the name of the file that I want to search for, not that full file path. You know, let's clean this up and let's get rid of some of that file path stuff in the D menu choices. So let's use some of the standard GNU core utils. Many of these I've done videos on. First of all, this is a long line. I'm going to split it up. I'm going to do a backslash here. That way I can break that line right there. I'll line it up here. We're going to take printf wiki docs, pipe it into D menu, but actually we're going to do something in between here. First, let's do a cut. So we're going to cut dash D dash D signifies a delimiter. The delimiter is going to be a slash because what we're going to do is these, of course, are file paths, right? And we're going to pipe them into cut and based on the slash is we're going to cut certain fields and looking at wiki dear here. We have one, two, three, four, five, six fields. I don't want these fields as part of the file path. And actually we're going to have seven fields, really, because I think it counts the one between the slash as well. So I think what I need to do is cut dash D for delimiter slash and then dash F8. So I want you to cut all the way to the eighth field dash. So starting with the eighth field, I want you to print the eighth field and everything after it. Everything before it, I don't want you to print. And let's see how this looks. So I'm going to do a pipe symbol, slash again, because we're going to keep the line breaks. So every new pipe, I'll do a new line just to keep things easier to read here. And let me go ahead, colon W to write that, get back to the terminal. Let's rerun DM wiki. And that's a lot easier to read, you see? Instead of the full path, now we just have the file names. And now when I search for stuff, I'm much more likely to get what I'm looking for. It does look like it is case sensitive for the search. But I can go back here and make this insensitive name search here inside D menu by adding dash I for insensitive, which is going to be very important for this. We definitely want to, we don't care about capitalization, right? When we're searching for something like grub and there you go grub, capitalize. And if I click on one of them, it does not open it because it still wants the full file path. And we messed up the full file path by doing the cut command, right? So how are we going to solve this problem to where we get not the full file path printed to the D menu. But when I make a choice, the browser actually needs the full file path. This will be interesting. Well, there may be better ways to do it. But the most obvious way I can see is the choice array here. We cut, you know, the first seven fields, which was this here, the wiki deer variable. So when brave opens choice, really what brave needs to do, it needs to open dollar sign wiki deer. And I don't need to add a slash because it's already going to the slash and the slash is part of the variable here. Let me write that and actually see if that works. Because if that works, that would be an easy fix. So once again, I'm going to search for the grub file that I opened before. That was a problem because it didn't open the right path. And now, yeah, that corrected the problem with the path because now it adds user shared doc arch wiki html en to the path when you make the selection and it opens with the browser. So that solves that problem there. I have a new problem. Let me go back to the browser because one thing I just noticed, they're not sorted alphabetically. The other thing I noticed is the file names all have instead of spaces, of course, they have underscores, which makes sense. And when you're naming files on a Unix like operating system, typically you don't want spaces in your file names. You want to replace the spaces with either dashes or underscores. And you want to keep it consistent. If you choose dashes, use dashes. If you choose underscores, always choose underscores. They're doing the underscores. I want to replace those with spaces just to make it easier to read. I also want to get rid of the dot html at the end of the file. And I want it sorted alphabetically. So we've got some stuff to do here. But again, the standard GNU core utils we've already used cut. And that's really helped us out. And I think we could use the sed command to get rid of all those underscores and replace them with spaces. So let's do sed. If I did a sed substitution, and the way sed substitutions work is you do sed and then inside single quotes, s, the pattern that you want to search for. And then the pattern you're replacing that with. And then typically slash g for global, meaning every time you find this pattern, change it. If I don't add the g, it's going to find the first instance of it and change that one instance. But I want to do this, of course, probably thousands of times with these underscores. So I'm going to do s slash underscore. So we're going to find the underscores, slash, and then space slash. So change the underscores to spaces slash g for global. And then once again, add the pipe, and then the backslash for the line break there. Now, I also said I wanted to get rid of the .html at the end of the file. We could do that with sed as well. What I need is multiple sed patterns here. So I'm going to do sed dash e, and then that first pattern there, and then space dash e again for a new pattern. And we're going to do a substitution. So s slash .html slash, and then slash with nothing in between it, meaning we're going to find .html and we're going to replace it with nothing, meaning just delete it. Now, let me write that and see if that works. Let's get back to the terminal, rerun dm wiki. And it works just fine. You see, we now have spaces in the names. We no longer have .html at the end of the file. Of course, now when I decide to read an article, for example, maybe I choose lxq here to read the wiki page for that. Of course, it's not going to find that because lxq no longer has .html at the end of it. Also, if it was a file name that had underscores in it, well, we just boogered them up with that sed substitution. So again, we're going to have to be creative with how we clean up this array, how it gets displayed in the menu. But when I actually try to open it with the browser, I want the URL to actually be the way it originally was. With the underscores and with .html. So I think the obvious thing to do here is just add .html back to the URL here that Brave opens. But I think to clean this up a little bit, let's actually, if choice, then let's change choice to something else. Let's create a new variable here. I'm going to call this new variable article. So it'll be the article. And let's go ahead and do another printf command. And then inside the single quotes, we're going to do %s and then backslash in. And then let's go ahead and do double quotes, dollar sign, braces, wiki, dear. And then behind wiki, dear, we're going to do dollar sign, curly braces, choice. And then behind choice, of course we're going to add the .html. And now down here, Brave, all that's going to do is launch the article. Now we're still going to have spaces. We want to take those spaces and make those underscores again. So we're just going to reverse what we did up here. So with the said substitution, this time we're going to search for spaces and put the underscores back the way we found them. And let's go ahead and write that and go back to the terminal and see if this actually works. I'm going to search for that same grub article that I know didn't work before because it's got spaces grub slash tips, space and tricks. And now it works, right? So that, again, just some basic problem solving, right? We had some issues. One thing we still didn't do, we didn't sort everything alphabetically in the D menu. So after doing cut and said, the last thing I want to do, let's go ahead and add sort and then pipe and then we'll do a line break there as well. So now let's write that, go back to the terminal and see if we get an alphabetical sorting we do, you can see the first listings are all numerical and then we get the A's, yada, yada, yada. If I pick one of these, for example, allow our program to continue after log off. Let's read the article, there it is. Very cool, so I'm going to go ahead and clean this script up a little bit and I will eventually, of course, push it to my DM scripts repository on my GitLab. And it'll be an official part of DM scripts for those of you that already have DM scripts installed either through the AUR or through the DTOS core repository, you'll have a new DM script. The next time you update, you'll see a new version of DM scripts and it will be available for you guys. So just a little bit of what you can do with some simple shell scripting and using a tool light D menu, you can create these little graphical menus where you can make a choice and it actually does something and everything I did on camera today, I mean, I've done plenty of videos about bash scripting, about D menu scripting, I've done plenty of videos on the various core utilities that we use today, things like the find command, the cut command, the said command, the sort command. So check out those past videos if you are new to this stuff or maybe you just need a refresher course. Now, before I go, I need to thank a few special people. I need to thank the producers of this episode. Devon Dustin Gabe, James Maxim, Matt Michael, Mitchell Paul, Scott West, Wanya Bald, Homie Allen, Armored Dragon, Chuck Commander Angry, Yokei, Dylan George, Lee, Linux Ninja, Maastrum, Micah, Yon Alexander, Peace Arch, Madore, Polytech, Realiteats for less red profit, Steven and Willie, these guys. My highest tiered patrons over on Patreon without these guys. This quick little bash script session, it wouldn't have been possible. The show's also brought to you by each and every one of these fine ladies and gentlemen as well all these names you're seeing on the screen. These are all my supporters over on Patreon because I don't have any corporate sponsors. I'm just sponsored by you guys, the community. Do you like my work? I wanna see more videos about Linux, free and open source software scripting with bash and D menu. Subscribe to DistroTube over on Patreon. All right guys, peace. This offline arch wiki script can replace my man pages.