 So now recording is on, let's get the executive summary of the previous ones. So we went already the previous days or the day yesterday there was the loops and conditionals. That was already done. And we went pretty much lots of details and technical nicely things like matching operators, like working with the different events for the arithmetic. We went through the how to do the loops with the for loop and with the while loop. And now the next step is to proceed through the several practical parts. So my thought was that this day we should be able to cover how to work properly with the input and then how to make the traps. What's the here docs and here stream operators. Then I was thinking about tell you and show you several techniques. Well, at least one technique how to make the script run in parallel in bash, even though it's not the real parallelization, but at least that's kind of emulation of the parallelization. And then if we have left time, we will cover also how to use the arrays. Let's see how far we will pass this time. So working with the input. It's actually an executive summary because we have already touched this subject several times. You know somehow how to use with the arguments. Then we use already the read command. So you know how to get the output from the prompt from the command line. And then additional to that will be how to read from the standard input. Let me start it to do it one by one. So I have prepared several things. I'm in my beam directory once again, and I have prepared the several things for the input things. So input parameters. You remember already that everything what comes to the input parameter as the argument is available to the script for several variables. One of these variables is this dollar sign and the number sign. It will give you the amount of the line arguments. Then if you want to just list all of them, you will have to address this dollar sign and the add. So that's the whole list of the arguments which are given to you as options to your script. Then in case you want to just address them one by one, you're working with the arguments like dollar one, dollar two, dollar three, etc. So this you already know, this you already remember. And here I'm making you some one single file which makes a summary of these arguments given to your script. The first one is using the arithmetic expression to check out the number of the line arguments. Then I do the counter. I go through every single element and output it on the line. And here I just increment the counter. And this is my demo. So if you see amount of arguments is zero, list of commands is empty, which is correct. But let me see that I can do something better and I give some arguments. And here you see that I'm listening. So now these are just some methods how to get those arguments and how to get started with them and how to use them. At the end of this slot, I will also mention you some advanced techniques how to use it. But before that, let's go back to something which is more simple. So read command. Read command is actually pretty straightforward. You can use it within your file. You can use it from the command line as well. Pretty much like that. We have had already one example with that. And so here I have my read variable. So these are which the variable name can be any. The point is that we just assign it my text, whatever it is. And so if I want to have access to that variable, I have access to that variable that way. So we have had already one example and one exercise. So you remember how to use that. But let me surprise you. Actually, the read command, which we are using right here and which we are using to give the input from the command line. The originally hung is designed not to read from the command line, but to read from the standard output. And so what happens with the read command? What happens if you, for instance, do some piping? So essential example. So if I want to count, you remember, the amount of files and directories in my current directory. So this is what I do. I re-address the output of one command to another with the piping. So what happens over here in between those comments is that the standard output of this first command goes as a standard input to the second command. So that's being said how to do next. So if I want, for instance, execute some command and pass it to my script, what will happen? How to make sure that script will be capable of actually doing something with that input that comes out? So let me show you several approaches. And I will start with the read command. So by default, if I just say read all, that's what will happen. Read by default will grab everything that comes to the standard input of my script. And put it to the all variable. So to proof of concept, let me do it this way. I put read all and echo all. So everything else was commented for the future explanation, but this one is just two commands. So let's see what will happen. If I produce, if I give no input, then essentially it will be waiting for the input, but then nothing happens. But if I keep, if I give some kind of some input, okay, and you will see that actually, exactly what I gave to that command to my script will be has been read by read command. And it's now part of the all variable. Another way to give so piping is just one thing to redirect something to the pipe. Then you can also do it like this. So if you have a file, for instance, you can redirect as a file. Or if you don't have a file, but you have some, some text. Yeah. So I never told you about the process substitution. So it's probably the right moment to tell you right now. So what's the process substitution is. It's a bit, it's a bit advanced technique. But in case I have a comment and result of the comment is not saved in any file. It just dynamically generated. But I still want to use it as a standard input. Okay. So what can be done over here? And I will tell you then later then why it's in some situation, the must to go method, even though most of the kind of things you can do it with the pipe, but sometimes you can't really do it with the pipe. So what I expect here, for instance, if I'm doing something like echo some input, what will happen over here? Echo will generate some inputs and these construct the redirection sign and then the, then the brackets around what they will give me, they will generate a dummy file descriptor and this file descriptor will be forwarded to my script. And so now let's see what will happen. So we will get exactly this standard input in the same way. If you have a file, for instance, I have a file like students remember from the previous sessions. So I can just redirect the file if I have it. If I don't have a file, I can use the process substitution. So the just to take away message that process substitution will generate a dummy file on your behalf and it will be deleted right after the command is executed. So you don't need to do the only savings all the time. Okay. Now you know this, you can put it on your demo. So there was also some additional information in the material if you want to. But here I just, I show you what will happen if I, if I forward this student CSV. And what has happened over here is that my read command, which I did over there actually has read only the very first line of the file, which is correct behavior. So we, it has read the very first and then they ended and then it has been assigned to a variable. That's it. Okay. But let me see. I will come back to this just in a second. But I still want to continue with the read command in general. So if I want actually to put something into a number of the variables or even into the array, I will tell you later about arrays. If you will have, if you will up to this, but now let's just look at this with the several variables. So if I put to read and after the read, I will put a number of variables. What will happen? Read will use this default IFS, uh, telemeter. So by default, it's just the, uh, space on new line. So what will happen over here? Uh, every single set of characters, uh, delimited by the space will be assigned to the first, will be assigned to the first variable, second to the second, third to the third, et cetera. And then for instance, if there is not enough variables, then the rest of the line will be assigned to the very last variable. But if variables are too much, then those remaining variables will just stay empty. So let's see what will happen here. So for instance, if I want to do the proper check, yeah, I want to put it like this. Just to see that if we got anything empty. Okay. So now let's see. I'm getting back to my input parameter. So by default, if I run it as is, everything is empty. Uh, actually, I can even do the timing so that you know that it's the time out. So if nothing is given for, say, for four seconds, then just, just die. Or it can be even one second because I'm expecting the pipe. It's not interactive mode. So in this case, it will be just like that. Okay. If I provide some input and let it be a, a, b, b, c, c. For instance, these are three, five variables or three, five kind of different input snippets of the, of the stream. So you can see that actually what has happened, it has been interpreted as one single one because it's in characters. Uh, no, no, because pipe is missing here because my pipe is missing my bad. Uh, so here you go. Oh, this input dot s steady dot sh has got this line as the standard input. And what it read has done for me. So it has assigned the variable F one as a, a, it has assigned a variable F two as b, b, F three as c, c. And since there is no other variables except the rest. So the rest has got the value of the rest of that line. Okay. So far, so good. But let's see another thing that if variables is not enough, then actually the rest will remain just empty. It still will be assigned, but it will be empty. So that's the way input is being given to the file and read by the red command. Okay. But now we were working with one single stream. In most part of the situation, you're not working with the strings. You're working with the whole bunch of text, the blocks, then how to do the blocks. I comment out this part and I go next one. And here where it comes into use the while loop and the approach that we already tried last session. And now I will go in the same way, but I just read line by line. So what's happening here? Read is again in action. Read is trying to read everything what comes to the standard input by default. And so what I'm saying here minus R is just the, it can be basically omitted as well, but it's good to have. Just to avoid this all kind of special characters and make sure that the, all the lines not in one. So we're making sure that IFS is empty. So because we are, we want really the line by line. So line in this case will be the variable. It's of your choice. It can be any and just remember that exactly the same variable you will be using over here. And for a sake of proof of concept, I could do whatever I want with the line, but I'm just to the new mirration. So I put the counter over here and increment this counter every single iteration. So let's see what will happen. Now it will expect some bunch of text, some bunch of text. And so here this is where I can use already my file. You can use your own, but here is the one which I have already in the ready to go ready to play with and see that actually proof of concept. So I went through all these files and they have been enumerated so that the kind of proof to you that actually every single line was taken separately. It's not the kind of just taking the whole bunch, the whole block of the information and output it to the back to the standard output. No, it's actually working one by line. And in the same way you remember already from the previous session when we were trying to make the summation of the total number of students. So there were also possibility that you can split them into the variables and actually work with every single line separately and with every single field of that line separately. So that's the standard input, but that's not yet it. So I can come back to this once again. There is another methodology quite commonly in use is that when you are checking when you're using minus p operator. So you remember that this double square brackets, they have had all this minus D. If you want to check that director X minus F if you want to check that the file exists, et cetera, minus N minus set. So it has also another one operator, which is stands for the minus P. And you can check whether standard input pipe is existing. So what happens when you run a script when you run a script this and bash will automatically it's actually not actually the bash but the Linux in general will automatically detect whether something comes to the standard input. And if something comes to the standard input, this standard input blocks pipe will be assigned. Okay, now what we can do, we can easily check it and use it. So one way to use it, it could be as well while like here, it could be read like here, or it could be just normal cat. So many of comments, not all of them, but many of the comments that you are used normally, they're actually expecting some kind of standard input. And if within the script, you run some command without any argument and the one which is expecting standard input, it will grab the one which comes to the script itself. Okay, so in this case, I can of course say deaf as steady, it's for a sake of clarity. So this way we get kind of clear notation that we are getting the standard input that comes to my script. This is unique. I mean, it's this kind of virtual device, so this is unique for every single program. But in general, you can simply omit this and just use it without. So let's see what's happening when we use this kind of approach. So if I'm taking, if I do nothing, it will check that actually no pipe is existing and so nothing will happen. And now if I'm giving something to the standard input, so in my case, it's going to be actually like that echo. So it will go through the pipe. So this technique is expecting pipe. So the previous, the while one was the kind of redirection of the file, but this one is expecting the pipe. So you can see that actually some standard output has been generated. In the same way, we can do something else, something huge, I don't know, LS output. So that will be also going as a standard input. Okay, saying that, I must admit that I am pretty much done with the input parameters. Now I want to go and make a demo for you, make a demo, which will give you some impression. Actually, I can make this bigger demo later on, but now we can go to the first exercise. So it's already time for the exercise. So my very first exercise would be this handler inputs. And my idea of this exercise was that you pick up the function, which has been early added to this function command file. And I want you to actually, you know, here we grab the dollar sign, dollar sign one, which means that the first argument which is given to the function given from the command line. But now I want you to do something that some kind of piping would work as well. So try to use that minus P and then a steady and try to do that in the same way, but with the with the pipe. So I think 15 minutes should be good enough because subjects is interesting, even if the task is only one, but I mean it's a little bit more complex than simple running simple LS, etc. So that's a bit of programming. So we will come back at 30, not at 40, 30, no, 30, 39. So now it's your time, time for the exercise. If any question as usually just ask. I will ready to go. So let me take a look at the up and path. So I will just copy paste the ready to go solution so they were probably you may have it implemented in some other way. So the way you want it. But here is the one of the example of working implementation. Let me copy paste it over here to the right place. So it's over here so to take a look at. So what I'm doing here. So I'm checking that the pipe is existing. So now even if it's function, it's not anymore in the kind of compact way. It's just the kind of if else, etc. So more readability. So I'm checking that out. Then I'm reading it into the path. So that's the way to read the ester there as that into the to the variable. And then I need to also check whether this path has been already added previously. This is from the original script. And this is from the original script too. And then it's just the check and if nothing is missing, then we put the error message. So the whole thing was over here here and here. And so now you should be ready to go. If you want to play with that so you can already actually put both parts together. And see what comes from the standard inputs and see also what comes from the command line. Okay, this is being said. So now I want you to actually do some real stuff with me. So I have spent my evening developing the RIT further. So what has happened over here? You remember we were playing with the loops. You remember we were playing already with all these if conditionals. And from now on, we were also looking at the standard input and all these parts together. So now my paths together would say that I want to maybe be able to tar. Not only current directory or not only a single directory that I provide to the RIT, but I should be able to make a list of directories and make the archive of multiple directories. I mean separate one by one but still multiple within one run. And then my target over here was also okay. I can also get the names of the directories from the standard input. Essential example over here would be that if I run find and I find some directories within and I just need to pass this list of these directories to my new script. Everything is here. It's pretty much implemented. I've got the working version of that. But now I have done several kind of mistakes over here. And my next exam for you and we will spend on this another 15 minutes from now. Now you should be able to read this code. Now you should be able to analyze this code. And now I expect that you should be able to find several syntax errors and several logical errors in the code and fix them and get these codes to work. So now another 15 minutes is yours. We can give you even more just because this is very interesting. So this is once again, so you grab my I.T. The one which I have on my homepage. This is this one and try to work with this. So let's say that you have another 15 minutes to go. I hope it's enough. If not enough, we can quit because I think that's actually quite an important topic. So the floor is yours. I can give you several hints over here for the sake of simple syntax check. You can use bash minus n and then the script name. And for the sake of logical one, I can give you another one hint. There is one way to see how this script is being implemented line by line. It's the bash minus X. So go ahead and check out what's going on and why some things do not work as they supposed to work. So I'm muting myself. And now the floor is black yours for the next 15 minutes. And we will be back to the back online at 13. And actually let us join it with the break. So let us see 15 minutes for the exercise, 10 minutes for the break. And then we are back on 1310. I'll put it over here. And I do hope you have had fun. So it's actually, I hate debugging someone's code. Not sure about you, but on the other hand, that's the perfect way to learn the things. So analyze the code. That was actually quite complicated. I'm pretty sure you are not done most of you. But I'm pretty sure you have spent at least those time with having fun. So I need to find the errors. So probably I should have told you how many errors. I think I have done about four or five, but no more. But these are, we will find them out. So my first touch would be let's see the syntax error. So because the logical errors, they are somehow somewhat logical errors. They are somehow less intuitive, but at least the syntax error, they can be found right away. So the first error that I have found, that's on line 21. Let's see what's there, something with the then 21 then. Yeah, essentially that's the one of the errors which is quite often being done. So when you are forgetting this semicolon and when you put in the if operator and then operator on the same stream, on the same role. So let's go and see next, are we done? Yeah, the syntax errors are there, but I'm pretty sure there are some others. So let me see what's going on. What's the, what's happening when I execute the code. So I can try to execute it just like that. So this, remember my expectation from the code analysis, if I run it without anything, I should be, should get the copy of my current directory, archived copy of my current directory. So comment not found on the line 13. So one more error to catch. So let's see what's there on the line 13. On the line 13, we have this construct. And what I would say is wrong here is that this dollar sign. Let me explain why. So this is the arithmetic expression. And as I've told you previously, the arithmetic expression, we use just like this, just the round brackets. It will return you only the exit code. And this is what you need. So you only need the exit code because after that comes the logical end. So it expects nothing else. No other output with the dollar sign over here. You actually you prior producing the output. So what you are doing here, what, what this code is doing here. So it's actually producing putting here just the number of the number of the arguments. In this case, that was empty. It was zero. So that's still not considered to be a right output for the command line. So that's the one. And then let's see further what's happening. Does not exist. Skipping PVD, blah, blah, blah, does not exist. There is no line anymore. Okay. So my next step is that I will try to bash minus X. So I want to see the execution of the code line by line. So I will have better impression what's going on and at which point this PVD does not exist. Skipping. Okay, echo PVD does not exist. And here at some point is checking the PVD, which is ridiculous. So that should be the directory name. Let's get back to the code and see that directory name. So the only PVD we are using is this line. And you can see the error is here the quotes. So the quotation is wrong. And it does matter as I already stressed here on the very first lecture. So if you are using the single quotes, you will get the content of this, what is behind, what is in between those quotes without any substitution. But what we expect here, we expect here a substituted command. And actually VIM is also was able to make your highlights properly this way. So we've got already how many free errors I think fixed. Yes. So let's think the next one. So what's going on? RGZ exists. Skipping. This is already correct. Something looks like no, it's actually doesn't exist, but it's still skipping. So what's going on? Let's try bash minus six once again. At which point we are skipping. So we are skipping it at the point where we already generate the archive name. And we are checking that the archive name existence. And it thinks at some point it exists. Why it doesn't. Okay, let's take a look at the logic of this. This is already not anymore a syntax error, but that's the logical error. And the logical error says here that minus F operator actually returns success. So it returns success if the archive exists. So what we actually need here, the negation. So we want to check that if archive does not exist. So the exclamation mark over here tells us that this archive should not exist. Only after that we should proceed. Or we also can just swap else and if over here. But in this case, this fixes way faster. So now let's try next one. RIT. So now these are these just errors that come from the tar. They're harmless. And now we actually got the b. So at least this part is fixed. But now let's check me. So I remove it to clean it up. So I go up and I want to make several directories. So let me do it first from the command line. So my bean and my store. Okay, let's see how it works. Can it take two directories at once? Okay, bean store does not exist skipping. Well, at least this part does not work. One directory works the current one. But when it comes to the, when it comes to the list of the directories, it's already stopped over here. So let's see at which step we are. So we are minus D bean store. So at some point when we are checking the existence of the directory, we are checking the, not the one directory at once, but both directories. So this is definitely a mistake. Okay, and now I can see mistakes right from here. But let me show you where it's here. It's also a common theme. So that's the bean. It's a common thing when you're working with the poor loops, people keep putting this list of items in quotes. I do it time to time also. So it's not surprisingly, actually here there should be just the list with the, with the separator. So separator is usually space or it can be any, but usually it's space. If you put the something like quotes around, so that means that it's a one single item with all these spaces included into it. So that's another one error. It's actually both syntax and logical. So, and then if you use the other ones, it's already correct. So variable in case of conditionals, that should be quoted. But for the loop list, that should not be quoted either. So exception over here, if you have a list of some items, which are, which have special characters, and then you have to put into quotes every single item, but then gay. So between the items, there should be a concrete separator. So there should be space from now on. Let's try. Are we good? Okay. Something already exists. Archiving bean. Archiving what's why this dot is out there. So let me see what we've got. We've got bean story. Okay. It's not exactly what we expect. We expecting two of them. So let's see what's going on. Bash minus X. Echo exists skipping. So somehow the archive name was not generated properly. So somehow the archive name was just that dot and without really anything else. So let's see what's going on over here. So this is already over here. Four deer in deers. So we don't know yet the directors. So we expected that something comes would come from here. Yeah. We expected that this one should go to this one. Dear deer and archive. And this one was missing. So what I suggest, I suggest that we actually put one echo command over here to see that what's the status of that variable. So there is no good debugging tool for the bar scripts. You are not in the stage where you're programming on CEO Fortran, not even the Python one. So you cannot really trace the variables one by one. So we are tracing is just to put in this print and echoes. And so let's see what's going on further. So my deer is story is correct. Okay, but story somehow is not given correctly over here. Okay, I see the problem. So the problem is my the problem is my construct over here. So I was trying to use this magical magic of this variables and was trying to use this curly brackets operator to remove everything which comes before the slash. Okay. So what that says to me that actually I was planning to remove the path only, but I have actually removed the whole name. Why? Because I was giving the by default directory name with the slash. So my script does not accept this one. And if I remove it, see what's going on. Okay, now it's much better. Now you should have got already two directories archived been and store both are there. And then if I try it once again, what will happen? I should get that they're existing. Okay. And now let me do one thing. I want to make sure that it's also works when I provided for the pipe. So let's say that I will provide being from the pipe and store from the command line. Let's see. Okay. So surprisingly, this one works to right away. So I haven't done more, more, more errors in this code. So, okay. So now we can remove both. But now we have one trouble over here so that we have found, but let's see that's going to be a feature. So we need to get rid of, when we get this arguments, we need to get rid of this leading trails. And at some point we will do this, but let that, let it be the exercise for the next one. For now, we've got pretty much work entire it and so you can copy and use it as an example. Or the other script writes. So now I will drop myself. So in using, so if you still have questions or comments, don't hesitate yet to ask. I don't know is there anywhere going anything so, but just put them online if you want to. Then my next subject, my next subject will close the time. So my next subject would be here dogs and the placeholders. So what I was about to tell you is that here dogs paradigm is pretty. It's not even bash. It's Linux in general. So it's in all the other, in all the other, this concept exists in all the other shells as well. So it's not kind of new, nothing surprising. But the thing is that you can actually redirect the output of whole block to a command. So why it's, it can be useful. For instance, if within the script, you need to generate an email or just a pretty long message to the and output it to the screen. You can of course do the echo and just shift the lines from one to another and use this trailer and slash. But you can also use this here dog. So let me show how to do that. So I have prepared you one while one script, which actually generates this exactly what we are doing. So couple of dummy, dummy input requests, so asking something from the, from the command line. And now I'm what I'm doing, I'm doing here the here dogs. So these are two back redirect signs. I call them back redirects, bug directs or by redirects. So you can of course could call them less than greaser than etc. But I prefer to call them redirect and the back redirect signs. So it's just my attitude. So what we are doing here, when we are redirecting a block of information, we are putting just the command. And we are putting the back redirects signs over here, these two operators. And then we're putting the magic word, magic word is kind of stop word. So from now on, bash knows that everything what comes from the stop word to the end of this stop word will be piped to that command. The benefit over here is that actually on the fly everything would be piped. It will be examined and then the variables common substitutions and whatever you guess here to come, they will be placed over here. So let me see, let me show you how this particular example will work. So if I run it, it will ask me for my name. Ivan. Then the Rwanto. And then here the new generated text based on that template which was there. So you have see now at least three replacements. So both variables have been replaced with the existing ones. And then also the date substitution, command substitution has been replaced. Take a look once again at this one. So if you need any email, if you need any large text, if you need to redirect this text to the, let's say to the file, you will work with pretty much the same thing. So let me show you now the example once again here, document some name, some surname, whatever it's called. And now we've got the file with exactly the same thing. So you can use it as a template. Okay. So I don't need that one, but let me get back to this one once again. So there is one particular feature is that if you don't really care about all the substitutions, you can avoid them. You can tell to Bash that please do not don't make any substitution. So whatever those variables will be, whatever the dollar sign, so any special character will come into the text, they will not be touched. So trying again, some name, whatever it is, it doesn't really matter. So now because we expected that there will be no substitution, there are no substitution. It's not a surprise. So probably not what you want, but sometimes it's useful. Another one thing that's maybe useful and how you may think to use it is to make the comments. Usual comments, they go like this, this number sign, number sign. And there is no any special syntax in Bash where you can make the long comments. But there is this work around that you can use with the here doc. So now I'm introducing your, well, this stop what it can be any. I'm just putting the comment here. I'm introducing you one thing which is called no operation command. So this column, column operation, it does nothing. It literally does nothing. So if you output something to that operator, so nothing happens. And you can use that to use this as a comment. So now what will happen, this read will not be executed. Here you go. So that was just skipped. And then if you ever want some comment to run, but do nothing, just do it like that. In Bash, it's perfectly fine. So it's part of the show. Okay. I think that's what I have to say about this much. What I wanted you, I wanted you to play a little bit with the here doc. So that something would have left in your memory. And say that let's put something to do. Yeah, but just another one addition to that one. Another one addition to that one. So there is also along with the here doc, along with this double back redirect, there is the triple redirect that stands for the return stream. So it will be sent in the stream. So for instance, if you, I don't know, if you want to see the, how the variables looking like one particular example, IFS, you can redirect like this. So that's, I don't, well, I haven't found too much usage of this kind of operator, except that for IFS or some other variables. But somehow, for instance, in many cases, you can use pipe. So you can use IFS. But for instance, in this case, IFS is not really printed properly. Never actually. Better to take, to find the why it's this way. What's the echo is doing wrong. But at least here, if you want to avoid anything else, and you want to put the stream as a role standard input output input for the command, that's the way to go with the free ones. All right. So you can use IFS, but for instance, in this case, IFS is not really printed properly. Never actually bothered to take to find the why it's this way. I think I will not go any deeply with these ones. So now I have them mentioned. And now let's, let give you 10 minutes just to play with that and just to see that how, how this kind of things are being done. So if you are, whenever you are done with this one, but and if you still have some time and I want to play. So at the end of the material, there is example how to use the templates. Play with that too. That's interesting. So this is something where you can use as a template. And then if you will be later implementing some kind of script, which will be working for you as a template, this also possible to be done. Okay. So now let's say that we have 10 minutes to do that. And that means that we are back at 44. And I guess, so let's continue. So the here docs was already there. Hopefully you also had another part of your fun playing with that. So let me show what I was expecting from you. So let me say here or just end or whatever you call it and forward it to file. And then let's see what happens if I do it from the command line from the command line. Bash will continue this, this continue asking me what I want to. And so my point was here that I type some text type some text. Okay. Then type more text. And then I need to finish it at some point and finishing it. I need to use this magic stop board. So now the cat should have generated this file for me, which is there. And not the one thing. So if I want to play with the variables, for instance, so and I want to say that variables like home, like shell should be here. What else was requested? Common substitution like pvd, pwd. And then is that it? Okay. And, okay. I was trying to do it in the wrong way. Somehow I have duplicated those comments twice. So now let's try it again. Home, shell and the common substitution pvd. And then that's the end. So that should be correct. File name. And here you see that all this information that I have expected. Okay. That was there. That was the task for you to play with. My next is traps. Oh, nowadays it's quite often that you will start running something. Well, nowadays the hardware is so powerful so that the codes are so well optimized. Quoted of course. And then, but sometimes you have to just try to kill the existing script. So in our particular case, so we were playing with the tar id. And the tar id was making the archives. And then at some point you realize that actually you're archiving the wrong directory. Or you are realizing that actually you're archiving the right directory, but this is too big and it's taken too long or it's taken too much of the IOs of your computer and you want to kill it. What will happen after? So killing the process in Linux in general, the concept is very simple. You are sending the signals and then the programs supposed to actually react somehow on the signals. Bunch of the signals which come from the operating system or from the user, they are catchable. And so the command and the script can catch the signal and can react this and can do something. There is one signal, maybe more, I don't know exactly. I don't remember, but at least one signal is not catchable at all. So it doesn't make sense to try to react this. I mean, this Linux is trying to kill it with the signal number nine. The process will be simply killed. But all the other cases, common cases, they can be still catched and somehow reacted. So let me show how to do the reaction. So I don't have any script ready, but I can come up with the trap.sh and see bin bash as usually. So we always start all the scripts with this one. Then some comment, trap testing, and then actually let's do this. So what trap wants? Trap wants some command to be executed. Echo, for instance, say we are interrupted. And then the signal, which is expected when we are kind of killed. So the signal, in our case, I would say it could be one. So this hub, one session is disconnected, then two. For instance, it's just the interruptions. So it's when you're trying to kill something with the control C, that's exactly what you send. And let it be the just termination signal. So that's the first signal. When you are trying to kill something, the program will get this signal number 15. And then if you try to kill something concretely, you send the signal number nine. Okay, so let's try this out and let's see what will happen with the script. We also need some kind of body. So trap is a trap, but we need to be some kind of body which will be executing. So I introduce you the command called sleep. So what the sleep is doing? Sleep is sleeping, essentially. So when you are sleeping this, so that means that you can say that how many seconds you want to be sleep. So let it be that we can sleep for 20 seconds just that good enough for us. And so that should be enough just to play with that and see what's happening. So let me change model plus X and let's see what's going on. So we are sleeping. If I press control C, you see what has happened after control C. This trap has been executed. Okay, it's already something for us, but best practice would be actually not to have a comment over here, but to define some kind of function. So I can define a function which I called whatever I want. So let it be end or let it be even interrupt it. And within this function, I can already start doing something and then I will put it over here. And so that would make my code way easier to read and understand. And so that I know that every time once happens something when my program is getting one of these signals, this function will be executed. So once again, it could be some text, it could be if you are in the middle of something, you can for instance remove some temporary files and directories. Okay, it could be that you are you some other processes because you know that you have to clean up after yourself. For instance, you know that you are running several processes which are dependable on this one. And you want to say to those of them, okay, something is going on. Please kill yourself or just clean up after yourself. Then what else can be done here? These are at least something which is very usual. So when you need to clean up some temporary files and then you need to kill some other processes which are dependent on you over here. But this is one of the best practices how to work with the traps and how to make your code even more robust. So let me say it once again trap. If I kill it, so interrupted command node found. Let's see what's going on. Okay, it's the it's the misprint interrupt it. Yes. Yeah, now that should work. You know what's the bushes doing? Even if there is this kind of issues, it will never tell you before it actually comes to that point after that it will generate error. So that's a little bit tricky. Some text. So at the moment we are just killing it and so and getting the text and nothing else. Practical example, practical example would be our tar it. So let's say that we copy our tar it. So the what was the latest version we have developed that was version three. We are copying it to version four and all of them are for a sake of clarity named like that so that you would know will see the development of the stuff that has been done. And now I can try to do it more. So let's say that I will be developing this version number five, and I need to define the trap trap is usually being defined. That very beginning of the code. So that's so that's the code would know that exist. And so I define the function and over here over here. So my functions looking like so what I want to do here. For instance, I'm in the middle of archiving something and tar what is doing when it starts to archive something. It will create the file and starts to fill it up. And if it has been interrupted this semi or even just some partially filled archive file will just remain there. So this is definitely what I don't want to. So I want to keep everything what has been done, but what is ongoing, I want to kill. And so what I'm doing here is I'm saying precisely we have been interrupted. Okay. And then I'm saying that echo cleaning up the ongoing archive. So here I have to put the double quotes just to make sure that archive will be. And then I actually do the cleaning. I can also check if it exists. So I mean just in case if it's in the process and make it even more robust than it is. And then just simple clean up archive after myself. And then I would say that our code is becoming more and more clean and robust. So I'm not really going to check it because there is no any huge territory or actually let me check it with. Should they try to play with my home? Okay, I can try it. So first of all, let me check the syntax without running bash. So the syntax is correct. And let me say that I want to go with my home. Oh, I can do it even this way. So it will definitely take some time. So it will try to create some directory over here. You can see removing blah, blah, blah, archiving over here. I don't want this. I decided that I went wrong. So I keep control C and what has happened, it has been interrupted and cleaning up the ongoing stuff. And so there is nothing over here. Essentially, yes, robust. Yes, easy to implement. Of course, yes. Now you know one technique how to make a how to surprise and impress your future employee when you keep doing these things. So my think I was, I was actually thinking to give you this sleep stuff to do, but I happened that I haven't done it already myself. Let me, let me catch for you and slightly do the slightly difference. So now the traps. So catch signals, let's put it this way. I create a file, create a file, any, you can use touch something. It can touch will do you some empty file. You decide what you want. File name and then catch any of those signals. One, two, five, one, two, fifteen with trap and remove the file name. Also put some text when interrupted. Okay. So now you have, I don't think you even need 10 minutes, but let's say that we have 15 minutes altogether, the break and the plane with the traps. And so you share your time up to you how you do this, but just to make sure that we are in time. And so we will have still two sections to go. Okay. 15 minutes again yours. And now we are coming back to 14 to 15 years back online should be ready to go. Hopefully the trap was okay. And I don't think I need to demonstrate anything. It's just simply should be ready to go as is with this sleep command. Or let me instead of this sleep, what I was expecting from you. This is that you touch a file. I can even make it more professionally file name, file name. Let it be kind of less professional my file. And then I touch file name. Then I don't need this. Let it be sleep over there so that it takes some time before we actually do something. And then I want it to remove that file name. Remove that file name. Okay. So I guess that would be looking like this. Interrupted. Yeah. That should work out. I will copy paste it to the notes as well. Okay. Now we go with the trap analysis. And now my next step will be I will debug in profile. Let's say that we have somewhat covered it already with the code analysis when they did this. But I don't really think there is any sense to tell you more about what can be done except this bash minus x, etc. So the profiling really sucks and the debugging really sucks with the bash. So it's where the programming abilities actually ends pretty much. Or at least I have not heard about any kind of good debugging tools. Maybe already exists or because I'm an old school guy need to check it out. But at least nothing that I would have heard about essentially. So what I wanted to tell you and show one technique is how to proceed with the parallelization. I'm not talking about the any parallelization that you would think from the HPC point of view. It's not the HPC way. So it's not the MPI. It's nothing to do with the shared memory models if you know what I'm talking about. But it's rather how to make the command executed within one script. So consider this script just normal one, whatever. So let's even get this star ID back to the screen. So consider the script that every single commands is executed serially. So it waits for the result of the previous command and then will be executed one by one, one by one. For most part of the code, it's completely okay. But then when you come to the body, for instance, here you decided, okay, we are running the archive. Archiving procedure. The archive number two will run after archive number one has ended. Sometimes it's okay. Sometimes when you don't really care about the amount of time it's spending, or maybe it's the amount of time it spends on to execute this thing is so negligible so that you don't really care about this. But sometimes you do actually care. And in this case, you have an option to send your command to the background. What that means. So the bash has a concept of foreground and background processes. Everything what is in your screen right now running right now. For instance, I'm running VIM. VIM is taking my foreground. It's taking my terminal. So this is what I'm interacting with. And VIM is now the process which is running in the foreground. But there is a bunch of huge list of the programs which are running in the background and you can add to them everything what you want. So I can send the VIM to the background. I can send whatever I want to the background. You probably already know about this. Or if you don't, then I just send you to the section from the first part of the course which is called processes where this concept has been introduced for a ground and background processes. You can go through it on your own. But then I come back to the parallelization technique. So what will happen to my tar-IT script if I will send every single tar command to the background? Well, it happens exactly this. This command will go to the background and the script will continue with the new one. And this way all the programs, all the tar instances, they will be running in parallel in the background and the script will be just waiting for them all to end. And when all they have done, so the script will end as well. So this is kind of emulation of the parallelization. So basically in this situation we are using the multiprocessing of the ability of the operating system. Before I actually do any change to the tar-IT, let me demonstrate you with the sleep command. So I think I can call it parallel. I would put quotes around, but Bash doesn't really like it. So let me see that it's over here. So I still need to put my sheet back. Bash once again. I think I said it faster than it was. So if the job is optional, you don't need it, but I just for the best practice I kill doing this, I will explain what I'm doing here. So you see, if I go, if I do the loop over a number of iterations, so I'm using here just the seconds, one, three, five. So every time I run a command, so I'm using here the grouping curly brackets in this situation they are used as a group. I mean, I group the command such a way that ampersand. So sending ampersand at the end of the command is sending this command to the background. That's the, that's the rule. And so in order to sense the whole bunch of this command to the ampersand to the background, I need to group them. And so I'm here having this curly brackets and grouping this command inside the, inside the for loop. So what happens over here? Sleep, then one of these seconds is substituted. It will be one process, which will be sleeping for one second. Another process, which will be sleeping for three seconds. And then another one for sleeping for five seconds. Let me first see what will happen if I do not send into the background. I will need to, I will need to take a time. Let's see what's going on if I don't send anything to the background. So my parallel code change mode plus six. And I do the parallel shell and see how long it takes. There is this utility we have used it already with the Gaussian implementation. Let's use it once more. So time will tell me the total time of execution. So you can see that it's executed one by one serially. Altogether it took us nine seconds. So it's okay. One plus three plus five is nine. That's what we expected. But then let's see what will happen if we will send it every single process to the background. Let's see the execution time. Once again, time parallel. And now you can see that actually they will be sending to the background all the time. So basically what we are getting the execution time went down to the five seconds. So what that means, that's the longest chain in this for loop. So basically one, three and five were executed in parallel. This has ended early. This has ended after, but this was the last one. So the whole cycle took me only five seconds, which is more or less in some situation, maybe the life saver. So if your code really is taken a lot. And one key thing over here is that by default, if you just send the stuff to the background, it will be sent to the background. And then from the bash perspective, that would mean that the comment came to the end. And that means that, okay, it came to the end and it will be just ended as is. So what's happening right now, what we really want to, we really want to tell to the bash, please, you have something in the background and check it out. And try to wait till they are done. And only when these kind of background processes, which this script has generated, so they will be kind of the child processes of the script. Only of one, only when your child processes are done, only at that stage, please end the whole thing. And so this is exactly what wait comment is doing over here. So we need this. So just remember when you will start a plan, this concept of the foreground and background jobs, you will have to use the wait as well. And then another one approach which I think is reasonable over here is to kill all. So basically what I'm saying here that find me everything with this comment. This comment jobs minus P will return me everything which has been created by this script. If by some reason, by some reason I want to interrupt the script without waiting for the end. And if some reason this one of the comment is just taking so long or one of the comment processes hanging became zombie or something. Well, it's became something that's other difference. It's just hanging. It just doesn't do anything. I want to make sure that I clean up after myself. And so that's these jobs what is doing. So it will show you a list of the processes which has been created by this script. And with the command kill all will kill them all. Okay. And so I'm saying that every time it's the when the signal exit is giving. So please do this for me. So that's that easy. And now we can also see her it what can be done over here. So if I would like to implement this within the archive. So let's me see that I actually copy this one to the it's already version five. Okay. And now I'm doing this changes once again. And I'm saying that the most time consuming part of the script should be executed in parallel. So I put the ampersand over here. So every time when my script comes to this point, it will submit the tar to the background. Okay. And then I need to make the same thing that I would have done in probably so I make to wait it wait till. And so I put the comment where till all our tasks are done. Okay. So we can also put the comment over here just to know. Yeah, we put it into the background. We can try to execute it. And then, yes, I would say that this trap also makes sense to put it over there. So the traps can be many. That's another one thing for you. So here I catch these signals, but here I can catch some other signals. So the trap comments can be many. It's not a big deal over here. I can just copy paste it or you can just incorporate it into this one. So it's just up to you how you feel it. Check my syntax and run it for the very good. Okay, it's correct. So again, those ones are harmless. It's just simply because I am creating the file and putting the file into the same directory. So the tar complains that something has happened in this directory, but this is completely fine. So Arrim bin. And now I'm a bit, because this story, et cetera, they're actually quite small. But just trust me that, yes, we are running a number of the tars in the background. So if I do it like, for instance, I want to have story. Yeah, I want to have been and I want to have my home. Home will take time. So let's see what's going on. Story bin home, run it. So now we are kind of in story parallel bin. And then when parallel also are kind of in my home directory, which is what I wanted. I send it to the background, the whole thing. And now let me see with the command jobs. I can actually see what's going on. So I have several processes running over here. So tar it tar it. And then actually they have been stopped because I killed them and send to the background. The original file and there was a trap, which was okay. But now I just want to get it back. Well, again, I don't really want to interrupt you much with that because that's a concept which has been already very well explained in the first part. So if you are curious what's that, then just go to the first part and take a look at the material over there. Okay, so this part I didn't come up with any good example how to do that. So this is why I was thinking to why I had better demonstrated on myself because it's somewhat complicated and not really covered by this course. But hopefully you have found something useful out of this. What I want so well, I can clean up with first for myself being this can be killed. Then this another one which has been created occasional as well and final name. I don't need it either. So now it's more more or less clear. So now the time to ask the questions. Otherwise, I have the last section. If you need some specific and you want to yet ask me, then please do because that's the last half an hour that we are running the course. Otherwise, I will just jump to the arrays and seems that I have enough time to go through there. From this button, we even have time for the exercise or at least this exercise will be with you as a homework. So arrays, arrays. This is something within the bash which has been introduced not recently, but I think many people don't know that the arrays are existing in bash. So most of the people are working with the variables even the I've seen even the quite advanced users, which were really surprised when they heard for the first time about the arrays in the bash. Not that often used, but if you know how to use this, you can definitely make a use of them. So array is nothing else than a list of items, which under the same name. So there are two types of arrays in bash. One is known to you is that normal indexed array. And then another one, which is less known, but to those who is running, for instance, Python or PHP codes, they may have heard about the dictionaries and the in case of PHP, that's the associative arrays. So this is what is implemented also in bash. In bash, it's called as well as in PHP associative arrays. So they are one-dimensional both. And in case of indexed arrays, so it's where every single item has an index and you can address it by index. Sorry, somebody knocking the door, but it's not really open. So in this case, you just open the, in this case, you adjust what you have to do is that you have to really make a list of elements of this array. So for instance, in my case, let me clear first the screen just to not really override with this one. So you, you do the array and you put the list within this rounded brackets over here. And every single element of this one will be the will be the index. Okay. So let me see what's exactly is going on. So there is this part. Okay, there is something that's running. It's there. Let me check that is there some jobs still running. Nope. Okay, no, I mean, back to the back to the arrays. So now I have once again, I have defined this one. And now, for instance, if I want to address some elements of the array, what do I do? So here where comes the curly brackets back to use once again. So my array elements, they're just go now name of array within the curly brackets, and then the number of the index. So you do echo array, and the index elements. So in my case, are your member. This one was looking like this. So my first element, it's actually marked as zero. So the indexing starts with a zero is this one. So my second one is this one. And my, for instance, I don't know my fourth one. There's no fourth. Okay, strange. My third one. Okay. Yeah, of course, it's starting from the zero. So basically, these are elements like that. So if I want to assign the new elements. So assigning new element is pretty much in the same way you can do this are, for instance. And now I know that the element number four is missing. So I'm assigning it like this. So new. Now this element has got also the element number new. So the easiest way to declare the easiest way to print it out. There's a dirty way. There's a common declare, which we will use just in a second. You can declare and say, print me the archive. Archive, sorry. So much of the tiring print me the array. So here you can get kind of very rough overview what elements of the array I existing. Okay. And then the next one is that, for instance, you are. What you're. Honestly saying, so the arrays, these functionality is pretty much limited. You can assign lots of stuff, but it's not, it's nothing to do with the real languages. So all these kinds of libraries to work with the areas like what you have in the. In the different real programming languages, like sorting the arrays finding the smallest element or going through the. Finding kind of all possible combinations, et cetera. So this everything what you think about is not implemented by default and it's not part of the really bash. So you will, if you even want to sort made the sort of the array. So you will have to Google and you will have to find out how to do that properly with the sort command. So it's not even built in functionality. It's the external functional external command that can be used. But again, something is still can be done over here. So now this is the indexed array, which has been created. You can change every single, you can change, for instance, every single element. So I can put it new tool. Yeah. And so that you can see that actually this became that this became this has gotten you value. You can assign some values which are somewhere not on the list. And you can see that actually all the other values in between four and 12 are missing, but 12 is here. So basically, you can do pretty much the same things with area elements as you would do with the variables, but they just get the index. So couple of things, how to address them properly. So here is the simple construct, which I can explain you. Let me put it to the beam as well. I'm missing my explanation mark. So what's happening over here? I'm printing it out. So what I'm saying here that I'm saying here that please pick up this array and go through the indexes. So the indexes, this curly brackets in this case, it's the mark for the explanation mark. So that says that actually I'm going through the indexes and not through the values itself. Otherwise, that will give you the values of the array. And so what I'm saying here that every index is as this value. So this is the way to print the array nicely. So but the only thing that I have to what I have forgotten to say here, so I have forgotten to define it. So let me just copy paste some dummy example from here. The address of the universe team working at. Okay. And then let it be even more like this. So let's see the only thing that I have to change mode once again. Oh, here you go. So my small poll loop just went through every single element of that array and kept it over here. Most probably cases where you end up using this, it's the assigning of the array elements. From some comment output. That's the way it's been done. So again, you put the list and then inside of the list, you just expect some common substitution. And then that means that you expect actually some some list of some items over here. So they will be automatically assigned to one, two, three, etc. So if you need to append elements, so this arithmetic stuff will work also for the array elements. If you want to kill them somehow. So for instance, you want just empty. That's enough. If you just assigned the empty list to the array, or if you want to unset it completely. So in the same way as you would do it with the variable, you can do it with the name of array. So that should be just there. And then if you want to unset, for instance, just one particular element is over here. And remember, I told you that how to make the sorting, it's not straightforward at all. So you have to be really, so you are using, you are, this is where you are using, here is string operator. And so you are using also the, in general, sort command and then the common substitution. So it's not, it's not that cool at all. Same to say, if the normal array, indexed array, what I'm saying about, let me check it something, something is all going on. But I don't really, I didn't kill one of the process. Let me see which one I didn't kill and why it's still trying to do something. No, there should be nothing strange. Okay. I'll take care about this later just to be sure. Okay. So let me get back to this. So as I told you, the indexed array doesn't need any declarations. So whenever we have this R equals something, that's enough. Yeah. But if I thinking about associative array, what it looks like, associative array, it's supposed to be when instead of index, numerical index, you actually have some real name. And the key is a real name. So this is why sometimes they're called it like dictionaries like in Python. So basically you're saying here, okay, my E is university, but the value of this element is the name of the university. So it's kind of meta information and then the subject itself. And this is what is called in the IT world the associative arrays. And this what is in the bash world that needs to be declared. So you need to say, I mean, if I will try to do this right from the command line and do this associative array right away, it will be the error message. So what I have to do, I have to put the capital A and say that associative array will be this one. Now Bash knows that I should be able to assign the to assign the array items this way. Okay. Syntax broken is address something is broken. Okay, it's probably my life has gone. So the system thinks that I'm out of this. I'm pretty sure it's about the what? No, it's actually always correct. Okay, so let me back. So now if I want to read it, declare minus P in the same way as you would do it previously. So now I have it on the Espo. Interesting. Why? So let's try to get it back again. Syntax error and expression error talking is address. That's becoming interesting. It's probably some misprint somewhere which I don't really see. Street address error and expression error talking is address. There should be somewhere misprint which I don't see. Well, anyway, let me delete at least this part. It has already been. Yeah, it's probably didn't like that part at all. Well, there is only one element Espo. Why? And now I'm in depth. What's going on? So that should be pretty much correct. So that has been declared as the associative array. Okay, no, it hasn't been declared as associative array. So I did it in the wrong way. So the name for the which has been declared is this one and the name that I've been using to assign it is this one. So what I'm doing right here so that I will declare the right one. It's even doing the convert and run the fly. Then I do the assignment, proper one. So you see that actually misprints once again as I declare. It says it cannot convert. Okay, I want to kill it first. Like that. I want to declare it once again clearly. It's okay. And now I want to put it like that. Now I should be able to print it as array. Okay, now everything is perfect. So you see that sometimes mistyping with you to this kind of unpleasant surprises. But anyway, here you go. And the way you work with them is pretty much the same. So let me demonstrate you on exactly the same beam, beam array. And so if I do the assignment, we're here that way. Yeah. And I do the declaration right away. And now I do the commandee. So you can see that actually I do the declaration and then assigning off this stuff on the same line. Which is correct. It can be done as well this way. And now I should be able to print associative array in the same way as I would do it with the index. But instead of index, I would use keys. So let's see how it goes. So it goes perfectly as expected. So you can see that the element, the key is here and the element is here. And so the only thing that I've lost somehow on the way this treat in the address, which is not what I want. So this is probably, it could be something like this, or it's could be better something like this. I think that was there, why we didn't get it properly. Streeties, addresses, why they're missing. Okay. This is a demo effect. I didn't try, I didn't test this script. So we'll not go deeply into that. Let me spend my last seven minutes better saying you something more useful. So this is one of the examples which I wanted to tell you, and this is copy pasted just from my own bin directory. So this is one of the things which I'm doing more or less automation for my backups. So and in a sense in parallel, let's say demonstration how to use the associative arrays. So look at this. So I do the declaration of the associative very cold years. And then I defined for every several items by every single directory which I want to make a backup. I define it like this. You can define it some other way. I'm just for a sake of free the ability to do it this way, but you can put it everything in one line. And that will be also pretty much working solution. Now I do something with my rsync. So I defined what I want to do with the where I want to put what kind of options will be given to the rsync. And now what I'm happening over here so that I go through all the indexes in that array, all the keys. Sorry for that array and syncing them in a way that I just substitute the value of that one. And then everything what comes from here. That's that very simple. But yet working solution. So nothing would stop you from making a single list. But as soon as you learn how to use arrays, I'm pretty sure you will not use the lists anymore. You will start using the arrays. So that's kind of advanced techniques and let's leave those normal lists to newbies. And an advanced users of bash. So I have five minutes left actually four and 50. So my my task was that we would have done already one more exercise. But I think it's just doesn't make sense to keep being in the rush. I put the exercise over here. I can say this is this will remain as a homework if you want to. If you're really keen on trying something. And if you have any trouble, please send me an email. I can help you with the code. Then what I would definitely ask you to do is to go at the very end of this document and give us some feedback. So please, I will leave you four minutes of your valuable time to write a couple of words for us, which are important in the sense that we are adapting the material all the time. We are adapting the course, the length of the course and the type of the material which we are getting hands on tutorial, etc. So otherwise I thank you for the participation of this course. And then also I already mentioned to Enrico if you're up for the credit, if you want to go for the credit, then just let me know or let Enrico know. We have some special assignments really to know. These are quite complicated, but I can still provide you. And if you really want to spend more time, so I can make sure that you will spend at least one week doing some other work for the BUSH program and learning some stuff. And then I can also promise for myself that I can help with the special assignment if there will be a need. But anyway, that's an option. Thank you from my side.