 All right, welcome back to operating systems. So today we're going to do the last lecture that will help you in lab two. So if you haven't started lab two already, you should start lab two because you will encounter some issues and you will probably need to sleep on it. If you leave it to the last day, you will be screwed, trust me on that. So I'll try and show you some more ways so this won't be anything really that new. So we'll just play with processes more today and it will be fun. So here is our task and only task for today. So we want to send and receive data from a process. We want to communicate with it two ways. So we want to create a new process that launches a command line argument while this should probably mean it needs to exec something. So we'll see a nicer way of using exec because it was kind of annoying before. I had to do like slash user slash bin LS. It was kind of a pain in the butt. So we'll see a way to make that a bit easier. And then what we want to do with this process is send the string testing to it. Then we also want to receive any data it writes to standard output. So that's probably the easiest thing to do after doing the exec part. So we only need a few new functions. So a more convenient API to exec VE is called exec LP. So it looks like this. So its first argument doesn't need to be a whole path to a file. It just needs to be the name of a program. And this function will automatically search for that program in some default directories specified by this path environment variable. So this would be like things like user slash bin, some other directories, that's where it will look for this file for you. And it is much more convenient. So then the other convenient thing with this function is it doesn't need a large array of arguments. We can just give it C strings and it's a var arg so we can give it any number of C strings we want to represent the arguments and it just has to end in a null. And that's it. We don't have to create an array of C strings so this is less annoying. But otherwise, aside from those two convenient things, it functions exactly the same. Negative one on failure, set error no, all that good stuff. Otherwise, if it succeeds, it starts executing a different program and replaces this process. So our other final APIs that you need for the lab, which you could have looked up, but here we'll actually use them today just in case you don't wanna read. So there is dup and dup2. So what this does is essentially create a new file descriptor that refers to the same thing that old fd is pointing to. So in this case, whatever old fd is pointing to, it will create a new file descriptor that also points to the same thing and it will return you a new file descriptor number that you can also access that by. dup2 does the same thing except it lets you pick what file descriptor you want to replace. So if I want to, if I have file descriptor five or something like that and I also want to make whatever five point to, I also want file descriptor zero to point to the same thing, I could give zero as this argument and this function will go ahead and do some fun stuff for us. So it will close zero if it is a valid file descriptor and then it will make old fd, whatever it points to also point to zero. So it's a bit of an easier way to target what file descriptor we want to replace. So it just makes our life a bit easier and we'll see how to use that. That's it. So now we do the coding example. So how this is set up is you have a skeleton that's given to you and the idea behind this is I don't give you the solution. We come up with the solution and then later if you want it again, hey, it's good practice if you just figure out how to do this again. Or you can watch the recording but that's a whole lot less fun. So what do we have here? So we have our check error helper function that we've been using so far that just checks for negative one and then does that fun exit stuff for us if we need it to. Otherwise in main, it uses command line arguments. So I'm expecting there to be two arguments. So remember with this, you get command line arguments through argv which is a array of C strings and argc tells you how many elements are in that array. So I'm expecting this to be called with two arguments. So the program name by default is like the zeroth argument and then I want the name of a program to run. So if I don't have two names, I return this einval which just means some invalid input. I'll set aside some space for some file descriptors for pipes because if we wanna communicate with a child process and we wanna capture its input and then give it some data, well, we should probably use something like pipes to do that. So after that, I will call fork and I have these little helper functions just to make things a bit more readable. So if it's the parent, it calls this parent function that takes those two arrays and also takes the process ID of the child for us. And then in the child process, it takes those two pipe file descriptor arrays and also the argument that's supposed to represent what program we need to run. And then otherwise it just exits out of main. So right now the parent does nothing and then the child does exec LP for that program. Cool? Cool. So if I go ahead and I just run your name by itself, this is the name of a valid program. If I run it and I don't give it any arguments, it'll tell me what operating system I'm running. So right now it tells me Linux because that's my virtual machine. So you've used this before, right? In lab zero, you did dash R and that told you the version number of the kernel you're using. So hopefully that number is familiar but if you just do uname, you get the name of the operating system. So if I run something like this, what should I expect to see? Yeah, I should just see Linux, right? So I run it, my parent, my fork, I create a child process. That child process does an exec LP and it replaces the child process by uname and then it just prints out Linux and uname would have printed out Linux to file descriptor what? One, hopefully. One, yeah, so it would have printed out to file descriptor one. I didn't change it or anything from before the fork or after the fork. So it would have just, by default, file descriptor zero, one, and two. We've all just been my terminal so I see the same output, right? So if I want to capture this output and have it available in my parent, what should I do? Yep. Yeah, I should make it right to a pipe instead of just the terminal. So I will go ahead and I will use this out pipe FD. So I will call pipe out pipe FD and then say what I'm doing and then check for errors. So after I do this, well, what file descriptor should I open? So I should have file descriptor one, which is standard in, then file descriptor, sorry, file descriptor zero, standard in, file descriptor one is standard out, file descriptor two is standard error. Then after the pipe, well, file descriptor three is going to be out pipe FD zero. Whoops, pipe FD zero. And is this the read or the right end of the pipe? Read and of pipe. Whoa, okay. And then we would also have file descriptor four open, which is the right end of the pipe. All right, so first question to you. So if I do a fork, then both the processes would have those five file descriptors open, right? What would happen if I instead did something, oops, like this? Yeah, so if I did something like this, at the time of the fork, I would have only had file descriptor zero, one and two open. And then after they fork, they become independent and then each of them would call pipe. So each process would be making its own pipe and there's no way to share those file descriptors, right? Happened after the fork. We only get to copy it at the time of the fork. So that is why we have to call it beforehand. So with these pipes, we can't, you know, like there's no name to access them. So the only way to share these pipe file descriptors is through forking, right? So I have to do this ahead of time. So now I have these file descriptors open, great. So if I want to capture the output of the child, what should I do? Read the read end of the pipe. So here I want to just read the read end of the pipe. Oops. Yeah. Oh, if I'm in the child, I want to write to the right end of the pipe. So I want to do something like this. What do I want to write? No, so I want to, I want to write the string testing to this, but first I'm going to capture the output of this. So I want to capture Linux, standard out. Yeah, so well, so U name is writing to standard out, so if I want to capture its output, what should I do? Yeah, read the output of U name first and send it back to the parent. Yeah, I have to call exec LP first and get the result out of that. Like this returns something. So I have to read standard out. So I'm just, okay, let's create a magical buffer of some magical size. So I just want to read to it like that. So use the dupe to make zero the same as five. So zero is standard in and five doesn't four. So I want to make, I want to make zero what? I want to make zero the right end of the pipe. So I make the right end of the, so, so that program is writing to standard out, right? Yeah, so I want, so I want one to point to the same thing as four, right? So I want it to probably look like, what do you say, four? So I want to look like this. Yeah, so I could do check error, oops, dupe two. So what I want to copy is out pipe FD, oops, out pipe FD one, right? And I want to replace file descriptor one with it. So that is that. And then check for errors, all that fun stuff. So that better? Yes, so this read, this would never happen, right? So if execlp is successful, it starts executing another program. So these lines would never run. So I can't do anything with it, right? I lose complete control over it, although this will save me some typing later. So I'll just shoot that up here. All right, so is that it? That's all I have to do. All right, we can run it. That's like kind of promising, right? Yeah, that's pretty good. Yeah, yeah, so what I did here, because of this dupe two, I changed what file descriptor one refers to. So I made file descriptor one point to the same thing as out pipe fd one, which is the right end of the pipe. So I set that up so that by the time execlp happens, right? Whenever I'm executing this program, you're not allowed to change this program. It just does whatever it does, right? And whatever it does is it writes Linux to file descriptor one, whatever the hell file descriptor one is. So in this case, I wanted to get its output. So I'm going to force it to write, whatever it writes to file descriptor one, it writes to the right end of the pipe, and that means I should be able to read it, right? So if I want to read it, I should be able to read it in the parent, right? So I can uncomment this. So now where do I want to read from? Do I want to read from file descriptor one? I could read from four or one, the read end of the pipe, right? So the read end of the pipe, what is the read end of the pipe? Zero, right? So I read from that, I should probably check for errors and all that stuff, but it's already, it's harder to stand up in code. All right, so check for errors, cool. So I could probably print off whatever I got. So I'll print off, so dot star S, because, oh, I'll print out got to, so because whatever I read is not going to be, I'm not guaranteed it's going to be a C string because it just outputs some bytes, it's not going to end in an all terminator, it just is whatever it is. So I can do star dot string, and that way I can give it another argument to say, how many characters are you actually going to print in the string? So I'll tell it, I've bytes read characters to print, and it is filled up in my buffer. So if I do that, type int, yeah, yeah, okay, I will quiet it down. So I'm allowed to do that because ints are smaller than size T's. So if I do that, hopefully what should happen now if I run this? Yeah, I should see print, I should print got Linux, right? Cool, so our parent process, just read some information from the child process by just monkeying with the file descriptors. So was I a upstanding citizen with these file descriptors? No, right? So it is considered very impolite to when you create a new, or when you run a new process, or sorry, when you run a new program to have file descriptors open that are not zero, one, and two. So this is actually quite rude that I have file descriptor three and four still open. So ideally I would want something like that. So what should I do? Yeah, just close both ends of the pipe, right? After the dupe, while I just replace file descriptor one with the right end of the pipe, and I don't need any other file descriptors for the pipe anymore. So I can just close pipefd zero and then close pipefd one. So now I clean them all up, everything is nice and tidy for me. So, yeah, so everything's nice and tidy. If I go ahead and run it again, well, whatever. I get the same thing, but now I feel better about myself. Yeah, so the child is only going to write to file descriptor one right now, right? So it's not going to do anything with file descriptor three and four because so when it reads from the output of the pipe, so when it reads, that process is reading from what file descriptor? Out pipefd zero, no, yeah. Oh, no, this is the child. Oh, so the question is, if the child closed pipefd zero, how can the parent use it? So remember, file descriptors are independent in each process. So if the child closes, any file descriptors the child closes, doesn't matter, the parent still has it open, right? So, nope, I didn't close in the parent. So here, all I did was created the pipe here. I created the pipe and then I called parent. So I didn't screw with the file descriptors at all. So when the parent starts running here, it looks like this. Yeah, so if I'm being polite, I should probably close them also in the parent, right? So should I just, you want me to just do something, what? Like this, after everything, I can close the right, but not the read. All right, does that make us feel better? So in this case, we don't have to, but we'll see quickly that my advice is something you should listen to. So this is good practice, so this will save us probably, hopefully. So I close them as soon as I don't need them anymore. So the parent doesn't need them, right? Cool, so now if I run something like this, it should be all good. Okay, cool. So the other part is we want to send information to that process, how do I do that? Yeah, a signal. What about if I did like this? Yeah, rewrite, holy crap. Well, now I know how to get your attention. Yeah, rewrite standard in, right? So I would ideally like this process to whenever it writes the standard, or sorry, whenever it reads from standard in, it is reading from probably a pipe that I can send data through, right? So I could send it a signal, but like cat, all it does is read from file descriptor zero over and over again, right? It probably won't do anything with the signal if I send it, except like die. So I can probably kill it, but I can't get any information out of it. Cause right now, whatever I type here, if I hit enter, well, I'll see it because I capture the output of it, right? So if I want to capture the input of it, probably need another good old pipe. So I will do this. Yeah, sorry for hitting the microphone. All right, so if I do something like that, is that all I need to do, set up another pipe? Yeah, I also need to read from it, read from it. Who needs to read from it? Yeah, this process needs to be reading from it, right? The child process. So if it wants to be reading from it, what should I do? Another dupe two to standard in. So what do I want, whoops, not nine. What do I want to be standard in? Let's see, I guess that doesn't help. So there's in pipe FD. So do I want one or zero? So this is in the child right now. So we're replacing its file descriptor zero and it's just going to read from zero over and over again, right? So zero, yeah. So we want to make it read from a different pipe. Yep, how do you see the output? So when I ran it, like, why did I see that? It still worked. Why did it work? So why did this work? Anyone want to explain why this worked? Yeah, exactly. So because I didn't change standard input, it was still the terminal. So when cat ran and I typed something, it read from the terminal and then it outputted that. Remember, cat's really stupid. It just reads from zero, writes to one. So it would have written to one, which I replaced by the right end of the pipe. So it filled that pipe up, which is just like memory managed by the kernel. And then my parent process read from that. So then I got, my sound effect was less good, but you get the idea. So the parent process read whatever I wrote. So now, so this looks good. Am I done here with replacing this? Close, right? I should probably close because it's considered rude. So I can do this much faster by hacking. There we go. So now this is all cleaned up, right? So before this exec LP, only these file descriptors are open and it looks like this. So zero is the read end of the in pipe and one is the write end of the out pipe. And that's it. Every other file descriptor is now closed. So that's good. Yep. Do I close one? So here I just made file descriptor zero, also point to the read end of this pipe and then I don't need any of the other file descriptors open because these are like file descriptors three and four. I don't need them anymore. Yeah, so just by convention, it's considered rude not to keep them open, right? Because by convention only file descriptors zero, one and two are supposed to be open. Otherwise it's considered weird. And remember we had that weird situation where we left a right end of the pipe open and the kernel considered you could still write to it so it would never return from read and we got stuck forever. That's also a reason why. You should just close file descriptors if you're not gonna use them. Just like memory, except unlike memory, all you get is like a segfault, which isn't that bad. You can see what happens. In this case, if you forget to close, well, what did we discover happen last time? It just kind of hung forever, right? It just got blocked. It looks like an infinite loop, although it's not an infinite loop. It's harder to debug than an infinite loop. So that's fun. Okay, so are we done? So that looks good for our child, right? So the child is currently, we can't change whatever this program does, or it's going to always read from zero and write to one. So now we did the correct replacements, so that's good. And we also read from this, we also read from the out pipe to get its output. So if we want to give it some input, what should we do? Probably write to the in pipe, right? So in the parent, if I want to give it some information, let's say I want, so our goal was to write the string, oops, const, car, star, message. So I want to write the string testing. So I want to send that string testing to the child process. What should I do? Yeah, write to the right end of the in pipe. So that would be in pipe FD1, right? So I write to pipe FD1, which is the right end of the pipe, and then my child process is reading from the read end. So it should read any information I write to it, right? Cool, so I will give it this message and say, fill it up with, have our main characters are in that message. And let's check for errors, because that is what we should do. Check error, but it's written. All right, so now we check for errors, are we done? I didn't close, yes, yes, what should I close? In pipe one, all right, is that it? Yeah, I could close in pipe zero before, right? So I close any file descriptors as soon as I'm done using them. So the parent does not read from the read end of the in pipe, and it also doesn't write to the right end of the out pipe. So I should close those immediately, and then the other ones I use, I should just close them whenever I'm done using them. So, in this case, we compile it. I still haven't used that, which is fine, but now if I run build sub-process cat, what should I see? Testing, right? Which should be cool, got testing. What happened there? Well, I wrote the string testing to the in pipe, then the cat process read it, and then it outputted that same thing to its file descriptor one, which was the right end of the out pipe, and then I read that information. So we communicated with that process, we gave it input, and we got output from it. Crazy, eh? So, this is, if you, since using this, there's a Python module called sub-process, which basically does this stuff for you. Great for calling programs. Now you know how it actually works. So you can, some of the arguments you can give it, you can say SDN equals pipe, and then you can send input to it. You could say standard out equals pipe, and you can get output from it. Guess what? This is like how the grading scripts work. So it just uses Python sub-processes to call your program, get some output from it, read it, do all that fun stuff. So cool, am I done? No? I need a what? I need to kill my child? Let's say I was a responsible parent. I should acknowledge it, yes. Yes, it's a bit violent to kill my child immediately. All right, we're, so if I wanna acknowledge it, I need to call weight. Where should I call weight? At the very end. So I can go ahead and call it here. Declare W status, call weight with the address of W status. Oh, I should also check for errors. Oops, check error. Oh, I wrote this in the wrong order. All right, so I will check for errors. And then I'm going to be lazy here and just write asserts. I'm going to assume that W, if exited, is true. So I'll just assert that that's true. And again, I'm writing asserts because if that is not the case, it's going to blow up and then tell me that that was not true. So I'm also going to assert that W exit status is equal to zero. That means my child process exited successfully and I didn't violently kill it. So if I violently killed it, that wouldn't be true. So there, am I done? Yeah, so the other thing, exit zero. So what will happen is in the parent, whenever parent returns, it will return from this, go outside then return zero from main, which is the same as exit zero. All right, yep, yep. Yeah, we closed all the file descriptors. All right, so if we compile this now, in fact, to get rid of this error, while I know the process ID of my child, instead of wait, I could have just called wait PID and told it that, hey, I want to wait on my child PID and then I can get rid of that stupid warning. So no other arguments. I'll do something like that because I get rid of that warning, yep. So wait will wait for your first child to terminate and wait PID, you can also do that behavior if you give the PID argument as negative one, otherwise you can wait for a specific child. So in this point, I'm just gonna, I only have one child, I can just wait for it directly. It doesn't matter, yep. So the asserts here are like just a C thing. So whatever you write in there, if it is not true, your program will immediately crash and say that wasn't true and tell you what line that wasn't true. So I put these in whenever I'm making some assumption because I could write more code to handle more cases if it didn't exit, but I'm going to be lazy and just put an assert there instead. So whenever I see an assert, then I know I have to write some more stuff instead of it like failing in some weird way. So it's just a common thing I do when I develop stuff. I'll just write a simple version, write asserts and then as long as it works, hey, it's good. And then if some assert triggers, I know about it and then I have to write more code and have more fun, I guess. All right, so I'm done now. I think so. So if I run this, let's go ahead and what was it called? Subprocess. Yeah, so now I got testing. If I do PID of subprocess, well, I don't have any orphans or anything like that. Everything is all cool. So could I move this weight up here? I will move this down. So is that cool now? Yeah, I got one thumbs up at least. Two thumbs up, three, four, five, no more. All right, we got five. So if I run this, if I go ahead and compile and run this, no problem. Thumbs up, right? What about if I moved it, whoops. What about if I moved it up here? Thumbs down. Why is that going to be a thumbs down? I killed the child, violent. So let's go ahead, let's run it, run it. So let's see. So I'm waiting for the child to exit. When's the child gonna exit? Yeah, what? Specifically, we saw the program, right? It's going to exit when it knows it can't get any more input. So if I wait for it right here, the parent process still has the right end of the pipe open. So it thinks, hey, it could get some input. So read will not return zero, and it will just go on living. So I'm waiting for it to terminate. It's not gonna terminate because it's waiting for me to send input. Catch, double e-duce. So that is not good. So this was good. What's another way to cause that to hang infinitely by just commenting out a single line? If I comment out the right line here, if I want, if I comment out this. So if everyone agree that if I comment this out, that will be a major mistake, believe so. So let's see. Yeah, so now it hangs, right? Because I still, the parent still has the right end of the pipe open. The child thinks it's possible to get some more input. So read will never return zero to signify it can't get any output. And then you will start messaging on Discord and being like, oh my God, my program's broken. I have an infinite loop or something. But really, you messed up your file descriptors. And in lab two, probably nine times out of 10, this is going to be your problem. So that is a warning. So remember what I said, close your file descriptors as soon as you don't need them. Yeah, so yeah, why commenting this out causes bad things to happen? So here, so the child process is just reading from the read end of this in pipe, right? Right? Yeah, so the child is reading from the read end of the in pipe. In the child, we were nice. So it only has access to the read end of the in pipe. So it doesn't have any access to the right end of the in pipe. So the child is clean. Okay, no, not gonna make that joke. So the child is clean. And in here, we close the read end of the in pipe but we still have the right end of the in pipe open. So because this process has the right end of the in pipe open, the child would think like read zero would not return zero to signify that there's no possible way to get input because this process, the kernel will keep track of that and know that this process has access to the right end of the pipe and it could write data to that pipe. So I can't say that no data will ever come into that pipe anymore if I forget this close. As soon as I do this close, then at this point, no process has the right end of this pipe open. And if you try and read from it, well, it would just return zero signifying it's closed, can't get any more input. So the child cat process can just return. All right, yep. If I, so if what took stuff from the terminal? Yeah, so what if the child process was reading from the terminal? So what if, so what if the, yeah. So it was like, you know, you read something. I'm not gonna write this, but you read from like FD zero and then you just write to in pipe FD one, something like that. Yeah, so if you did this, if you just have one right, then that's fine, but it would, you would probably have to write this so that it reads from FD zero and then as soon as that input is done, then it does the same thing where it closes the right end of the pipe because it knows no more input is coming. So you can write that it should behave just like cat did when we wrote it, right? So yeah, you, you, we could do that if we wanted. So then any input we type goes directly to that process. So we could do that and that would work fine. We just have to be careful that eventually we do close this, right? We would only close this whenever we read from FD zero and it returned zero to say no more input. And you know, you can't imagine we could have a long chain of processes that do this. So if one of those processes are screwed up in the chain, one of them is going to wait on some output and just stay there forever. Cool, so we got this all good. Well, don't, don't come crying to me after. So yeah, if you see, hey, my program stuck, you know, I wrote an infant loop. Guess what? You did that. You forgot, you messed up a file descriptor. So you say you don't want to, but you should. All right, let's remember pulling for you. We're on this together.