 All right, welcome back. So I have to hunt. OK, everyone else has pretty much done Lab 1. So I think everyone here has pretty much got 100 on Lab 1. So I just have to hunt down one more person. So Lab 2 is out. Lab 2, you should not expect to just get 100 right away. Lab 2 will be much, much, much more difficult. So it should be OK to get 80 on Lab 2 if you don't want to spend that much time. But it won't take, like, 30 hours, like past labs. And this lecture is designed to help you with that, because Lab 2, we're dealing with sub-processes. And they do a bit more. And in this lecture, we'll go through a whole sub-process example. So here's our task today. So today is going to be a whole programming thing. We'll go through, we'll make a bunch of mistakes, we'll have a bunch of fun. So we're going to create a new process. So our process will create a new process. And it launches whatever matches the command line argument. And then we're going to do some inter-process communication. So we're going to send the string testing with the new line to that process. Again, this is like you're sending the actual string, not a C string. So no null terminator or anything like that. So we're sending eight bytes to a process. And then we're going to see what that process outputs. And whatever that process outputs, we're going to output it to standard out. So we'll go into a bit more convenient API, because execve is a bit of a pain. For Lab 2, I essentially gave you execvp, but you never have to send any arguments yourself. So you don't have to worry about that. So our more convenient API is there's another version. There's execlp. And that just takes a list of or just a lot of strings. And you don't have to do an array of strings or anything like that. So the first one is just the command name, which it would look up to see where that binary is. And then you just give it string arguments. And when you're done, you give it a null argument. And then that's it. So again, this is the same idea as execve. It doesn't return on success. And it'll return negative one on failure, set error no, and all that good stuff we know and love. So as part of the searching, it will search an environment variable called path. So that's just a list of directories. If you've ever wondered what happens on your terminal when you just type a command and how it gets translated to a program, well, all it does is search through those directories, looking for an executable with that matching name, and we'll just execute the first one that gets found. So there's no real mystery to it. So final APIs we'll need to accomplish our tasks that you will definitely need in Lab 2 are dup and dup2. And they just return a new file descriptor on success and essentially copy whatever that file descriptor is. So for just plain old dup, if you give it a file descriptor, it gives you a new file descriptor that represents the exact same thing. So if I dup, I don't know, standard out, well, I can either write to standard, or sorry, if I dup standard in, I can either read from standard in or read from whatever this returns because it'll represent the same thing. So that might seem a little weird why you would do that, but we're going to use dup2 to replace file descriptors. So what dup2 will do is make that old file descriptor actually point to the same thing as whatever new file descriptor. So if you give new file descriptor the argument two and give old fd whatever value, it will make file descriptor two point to the same thing as old fd. So you can start using that. And if two was a valid file descriptor before, it would go ahead and close it for you because it's nice like that. So yeah, so it does that atomically. We don't know what that means yet. That's okay, but it'll essentially make a file descriptor, whatever number you decide, which will prove useful. So now we'll just get into the coding example. So this is set up. The examples is plain and doesn't have any solution. So the goal is we go through the solution today. So let's go ahead and look at the code because we'll be stuck here. So let's start reading main. So in main, all I'm going to do is check that I have two command line arguments. One by convention is the name of the executable I run. And then the second I'm going to use as the name of the sub process to run. So I'll just make sure there's two arguments. If not, do I need to make this bigger? Probably, yeah. So index zero would be like the program name, and then is our first argument. So that's just two arguments. So it doesn't need to be three because we only want two things. Really we only want one thing, the argument, but by convention we get the name of the program. Okay, so we'll do that. Oh, I should just hide this so we can actually read the code better. Okay, do I need to make the text bigger? No, okay. So we're just going to set up some room for file descriptors and array of two ints for something called in FD, and then in pipe FD, and then one for two ints for out FD. So we just initialize them to zero because we're not going to use them yet. Now we're going to fork, so that creates a new process, and which is the exact copy of whatever's currently running, and the only difference is the return value of fork. So the original parent process that created a new process, well, fork would return the process ID of the child. So PID would get assigned to the process ID of the child, and then in the parent, this if statement would be true. It'd be greater than zero as long as there's no error, and then we just call the parent process which passes in the pipe file descriptors and then the PID of the child. And then in the else branch, the newly created process, well, we'll just assume that there's no error and go ahead and just call child which passes in the same file descriptors arrays, and the name of the thing we want to execute. And then both of them, whenever they run that, will eventually just return zero if they don't call exact or something like that. So in the child, it will call exact LP, and that will just search for, it just copies the same argument twice. So if I give it the name LS, well, it would try and find the file LS, and then the first argument would be LS, which we know is just like for help messages, it's just a convention that we set it, but we'll set it and follow some conventions, and then null. So right now, this will create a new process and just try and execute whatever command line argument we give it. So if we do something like, if we do LS, well, this would create a child process, it would become LS, and then we get whatever LS. Oops, I didn't recompile, that's the solution. You didn't see that. So we run LS, we get whatever LS happens to output. We could give it different commands like uname and the new child process would go ahead, be created, exec LP that, we get whatever it's shown there because we're so far not monkeying with the file descriptors are doing anything. So if I just run uname, it's the same thing. I'm just creating a new child process and it's becoming whatever. So last thing is, oh, come on, key binds. So last little helper function we'll have is this check error function because we're gonna be calling a lot of functions. So we'll just see, they all just return negative one if there's an error and set error no. So we'll just capture it, print out the error message, which this by default prints out to standard error or file descriptor two. And that could also set error no. So that's why we save it beforehand. And then we just exit with that save value of error no because we wanna exit with the first error we encountered just to make our lives a bit easier. But we're gonna be perfect today and not write any bad code. So in order to do that, we saw a pipe before. So I wanna communicate between two processes. Yep. Oh, okay. So there's a question about static. Does anyone know what static means here? Yep. Okay, what does static mean in C? Cause don't know? Yeah. Yeah, so were you about to say the same thing? That's const, yeah. Okay, so static in C, if first year let you down, which it did for the ECE students. So all static does in this case is make it so you, yeah, make it so you can't use it in any other C file. So it basically makes that function private. So like, okay, private's overloaded. It basically makes that function only visible in that C file. So no other C file can see it. I can't link to it. I can't use it in anything else. So it just, in this case, I only have one C file. So it just kind of cleans things up a little bit and gives the compiler a bit more freedom. Funnily enough, if we're doing that, anyone want to tell me if I do something silly like this? What does that do? Yeah. Okay, so then what's the difference between the static int and then a global variable? Yeah, so if you, so static's overloaded, they mean different things. So if you've static on a variable there, if I've static in X, it basically means that X is now a global variable, but I can't access X from anywhere except that function. So it's like a slightly cleaner global variable. Why they need static to represent multiple things, I don't know, but if you do a lot of C, you get in the habit of like, if I only use this function in my one C file, I just make it static so it doesn't, I don't have any conflicts if I happen to use the same function name again later in another C file, while I don't get any weird errors where like you define this function two times because it's only available in this .c file, but that doesn't pertain to our little problem here. Okay, so let's go through this. So we want to communicate between processes. One of our goals is we want to read some data that the process gives us. So we'll go ahead and use a pipe. So the pipe, we just need a pipe system call. So pipe, we will create a new pipe and give it pipe FD. And we'll go ahead and use this check error. So that will make sure we don't have any errors with that. So now we did that before the fork. So we probably would create, you know, if we have these file descriptors set up when we start executing. So by convention, whenever we start executing our program, while we're gonna have three file descriptors open and the convention is zero is standard in, one is standard out, and then two is standard error. So in this case, we create pipe. So we'd have those three file descriptors and very likely pipe will create two new file descriptors. So we'll get out pipe FD zero and out pipe FD one. And this would likely be file descriptor three and then file descriptor four because it just allocates them in order. And we saw pipes before, what end of the pipe is this? If anyone remembers that from memory. So this would be the read end of the pipe. Right, end of the pipe. All right, everyone remembers what pipe is, hopefully. Any questions about what pipe is? Okay, so now we created a pipe. The goal of this pipe is we want to capture all the output of the child process essentially, that is doing the exec. So because we did a pipe before fork, both processes have all the file descriptors. So they have the exact copy of all the file descriptors and only past the point of fork are they independent. So what should we do in the parent? Does the parent so far need to use the file descriptors or we can go to the child? Okay, so let's go to the child first. So what should I do with the child's file descriptors? So I want, essentially I want this out pipe. I want it to write information into that and then I want to read information from that. So any guesses as to what I should do with the file descriptors? Yeah, so if I replaced file descriptor zero, I would change its standard in. So right now, if I just did like print high, so right now if I did print high, well print F just writes to file descriptor one and so far I haven't monkeyed with file descriptor one. So whenever I started main, I had zero, one and two. I created a pipe which created some new file descriptors forked so both processes have a copy of everything. So in the child and parent, they have file descriptor one which is unmodified. So if I do print F here, it'll just print out to the terminal, no harm no foul. But I want this child process now. I want to capture anything that that process prints basically. So I don't want to go to the default place, I want to see it. Okay, well I'll write something. So what I want it to do is make that exec LP, whatever process, whatever program starts executing, I want whatever that is when it writes to standard out, I want to be able to see that data. So I want to take advantage of my pipe. So I probably want that process to write to the right end of the pipe. So I should replace its standard out with the right end of the pipe. So what is the right end of my pipe? If we remember. So let's scroll four, yeah four or I could just take the number. So the right end of the pipe is out pipe FD one. So I can do something like this. So they give you a define just to make sure you don't have to remember zero, one or two, but you could just put one here if you wanted. So what this will do is make file descriptor one, essentially make this, the right end of the pipe B represent the same file descriptor as file descriptor one. So if I access out pipe FD index one, which would be like the value four, if I access four or zero, either one of those are gonna be the right end of the pipe now after this call. So I'm essentially making them point to the same thing. So let's go ahead and check for errors, dip two. Okay, so any questions about what that should do? Yep, if I pipe them together, so which out pipe to standard out? So this would make the right end of out pipe equal to standard out. So that's what this does. So you can only just replace file descriptor. So I'm just replacing file descriptor, standard out with the right end of the pipe. So if that process tries to do like a print after anything like that, instead of writing to a terminal that I can see, it's going to write to the pipe. And then because we know how pipe works now, if it writes to the pipe, well, I should be able to read from the other end of the pipe. And that's one way we can communicate together. Yep, how would I read from standard? So we wanna see whatever this does, right? How would we read from standard out directly? Yeah, yeah. So we can't read from standard out directly or anything because if we exec LP, we just start executing that program. We don't have any control anymore. So we have to set things up so that when we exec LP, everything's set up the way we want. And we know that we can't change what it does. If it does like a print up for anything, it just writes to file descriptor one. Yeah, yeah. So here I'll show. So before my call to dupe two, my file descriptors look like this, right? So I've, what, five file descriptors open and that's what they do. Then after the call to dupe two, it will atomically close original. So it would take this argument, the second argument there, which is just one and it would close it. So now, if it tried to use one, it has no meaning. One isn't connected anywhere, it doesn't go anywhere. And then as part of the dupe two is it's going to copy four and make file descriptor one, essentially point to the same thing. So it would just do, whoops. So after dupe two, it looks like that. Does that make sense? Yeah. No, they can, so I just changed it. So one is now this. So if it writes the standard out now, it writes to the right end of the pipe. Yeah, yeah, because the goal of this is we wanna capture its output. So we don't want it to go to the terminal because if it goes to the terminal, that means we don't know what the hell it is. So what we should do, oh, yep. So it's gonna run a new program. Yep, yeah, so our goal here, remember we are doing some type of inter-process communication. So we want our parent process to read what's going on in the child. So we're making the child because fork copies all the file descriptors, well, we're going to use the pipe to communicate between two processes. So we set up the pipe that we fork, so both of them have all the file descriptors available. In the child, we monkey around with it so that it writes to the right end of the pipe and then our goal is in the parent, we read from the read end of the pipe. Yep, yep. So in this case, all the file descriptors, exec LP doesn't monkey with file descriptors. There's like some caveat there, but for our purposes, unless you tell it to monkey with it, they stay the same. So if we set it up so that we have one is the right end of the pipe and then we exec, if that process now writes to one, it's our end of the pipe. And then there's another one there, yeah. Yeah, so we actually set up the pipe in main right here before the fork. Yeah, the get one is completely empty. Yeah, yeah, so the pipe system call, you give it an array of two ints and it populates them for you. So it would have written, like it would have written the value three and four here. Okay, so what, so if I want to clean up the child, while the convention is you only have three file descriptors open zero, one and two. So what should I do to file descriptor three and four? Do I need them anymore? Let's see, like some, no. So if I don't need them, what should I do? Did someone just say crash the program? No, I close them, I'm done with them. I don't need them anymore in the child. So crash the program, come on. So we'll make sure, we'll check our errors. So I don't need any of them. So at the point after I close everything right before exec VE, while my file descriptors probably look something like this, which is kind of what I want. So let's go ahead and compile. So now if I do build sub process and say I run uname, so before I saw it print out Linux, what am I gonna see now? Nothing, because what's that uname process going to write to? Yeah, writes to the right end of the pipe, which I never read from. So it kind of just goes to the void. So if we wrote correctly, we don't see anything. So it doesn't matter what we write, LS, whoops, that's not a real program, LS, whatever, just kind of goes to the void. So what we wanna do if we wanna get some output from it, whoops. So the value of out pipe FD index one is three. So we just made FD one, essentially, you can think of it like a pointer, we essentially made it point to the same thing as FD three. So you can think of file descriptors as like pointers to have something else. Yeah. So yeah, so that's a good question. I accidentally typed something silly, like I mean maybe I have a process called LW, but I also, so why is that print nothing? You expect like an error or something like that? So why wouldn't I get an error from that? Yeah, close. So let's argue. So in the child process, this would also run it would set up file descriptors, then exec LP would happen. And the first argument I give is some trash. So would this call succeed? Probably not. I'd probably get an error from it, but I never check errors cause I don't care. So it would return negative one set error no and I happily ignore it. It's like Java programming. And then it would return from child after failing exec LP and this returned zero and then it's done. So it never would write anything. It would just kind of silently die. Just like everyone does in Java. Yeah, I worked on the JVM for a little bit and it's like the worst piece of code ever. So I hate Java. All right, yeah. Yeah, it's, I never write to it. It just gets an error from that exec VE call and I never check it. So who would write to standard error? Yeah, I don't write to standard error. So let's see. So in our parent, we want to get that data back out of the pipe. So let's create a little buffer. And so if I want to actually read whatever that process writes, what should I read from? Yeah, out pipe 50 at index zero. Yeah, so this is the read end of the pipe. I want to read from the read end of the pipe because it would have written to it. So I'll go ahead, fill up my buffer. So the other arguments, I just give it that buffer, which is that special size that we'll learn next week. And that's it. So I assume it succeeds. I can go ahead, check error that's not negative one. And then we'll do a bit of fun. So I'll print F what I actually read from it. So, if you didn't use this in lab one, this is a weird format. So this is a bit of a weird format. It's just the star here means that, hey, I'm not going to give you like a width specifier. I'm going to give you the width specifier as a number I passed you. So it makes it so that just writes a string as just a bunch of bytes and doesn't expect a C string. So because if I read from the read end of the pipe, it's not a C string, it's not going to end in a null character. So if I just do percent sign S, doesn't an all character, so it will print the string and then probably keep on going until it just happens to find a zero, which probably isn't good. So I only print out the number of bytes I read. So now if I run it, let's not run that one. I can see that, hey, in my parent process, I can now read whatever that output and in this point it outputted Linux and then a new line. Yep, smart. Okay, yeah, that was going to be my next point. So now the question was, why don't we close the pipe in the parent because we should close them whenever we don't need them. So that would be a good idea. So in the parent, well, before I even do anything, I don't use out pipe FD, whoops. So I don't use pipe FD1 because that's the right end of the pipe, the parent shouldn't use it. So I can immediately just close it. It's a good idea to clean it up as soon as you don't need it anymore. And then I can go ahead and after I read, I can go ahead and just close the read end of the pipe. Yep, so yeah. So if I don't close the pipes in this case, it doesn't matter, we'll see a case where it does matter, but in this case, if I don't close them, well, it's just going to print F and then return from parent and then return zero, which would exit the process. And then when the process exits, all its file descriptors get closed, so they get cleaned up. So in this case, I'm just being proactive about it. We'll see that that's a smart idea, but we don't have to if we absolutely don't need to. Okay, any questions about where we are so far? So we create a new process, we monkey around with some file descriptors, so it writes to a pipe and we read from it. So now our next task is we want to actually send information to the pipe. So let's fast forward through this. So we'll set up an in pipe and then in the child, it will pretty much be a similar process, except we're not going to write it all out. So in that case, we do the old Stack Overflow special, copy paste, and we'll just go ahead and do the smart thing, rename in to out, cool. And the last thing we do is set up. The read end of the pipe should be connected to standard in. So that's the only thing we do. So now that new process, that new child process, if it tries to read from standard input, it's actually going to read from the read end of the pipe. Any questions about that? Hopefully my copy paste worked. Copy paste never lets anyone down ever. And then here I'm going to write a string. Consecure. So I said I was going to write the string testing. So, whoops, so I have the string testing and if I want to write it, well, in, it's written. I'll just do a write system call. So what file descriptor should I actually write to? Yeah, in pipefd at one. Yeah, so that is the right end of the second pipe that I'm going to use to connect the other way. So if I write to that end of the pipe, while my child process is reading from that end of the pipe so it would read any data I write. So I'm going to write my message and then I'm going to write however many bytes are in that message. And then of course I'm going to check for errors just in case I do something silly. But I never do anything silly, except for today when I accidentally sat on my lunch, that was cool. Yeah, so we just write some bytes, don't do anything. So now we'll probably have some fun debugging anything else I should do. Yeah, close the file descriptors. So first off, well, I can move this one up because I don't need it immediately and I should probably close my other file descriptors. Oh, let's have some fun before that. So if I don't listen to advice, well, now if I do something like sub-process uname, well that puts the same thing because that process doesn't even read from standard end so it doesn't care. I could write any old garbage to it. It's not going to read it, it doesn't care. But the odd thing is, hey, what if I run cat? So if I run cat, it actually kind of works but by a bit of a fluke. So I wrote out the string testing to it, it read it and cat's just dumb, it just reads from its standard output, output to its standard output. So it read the string I gave it and then just spit it back out again and then I read it again and then spit it back out again. So that actually passed through a process, went in one end, out the other, went through two buffers. So any questions about that until, yeah, any questions about that? Okay, so one last thing we need to do with sub-processes are what did we forget to do? Reviewing back, are we a good parent? Yeah. Yeah, we need to wait on it, we're a terrible parent, we're not acknowledging it, we're not doing anything, we're essentially just shouting instructions at it and then just kind of dying, which seems rude. So where should I wait for it? Before I write to it, after I write to it, but what about when I read to it or read from it? So it's a number of them. All right, who thinks we should wait at one? Few? Who thinks we should wait at two? Oh, 50, 50. All right, well, I'll just do whatever. So, whoops, I just deleted a bunch of stuff. So I'll just wait right after I write to it, which is probably more confusing, maybe, because I'm waiting for it to die and then after it's dead, then I'm going to read from it. Read whatever it spits out, but seems maybe a bit weird because it's already dead, but let's go with it. So I'll wait for the PID. I have to declare an integer because it needs to write to one. So I'm just gonna wait for that process to die and then I will check error. Okay, so I'll check error and then I'm going to write some asserts just to make sure that everything happens as I think. So sometimes writing asserts will help you as you're developing stuff, probably especially help you in lab one. So I'd be like super conservative in lab, sorry, lab two, and just write asserts everywhere if you think something's going to happen. So then you'll discover that your asserts probably aren't true and then you'll have to fix it. So I'll just assume that process is exit and then if a process exits, I assert that its exit status is equal to zero, which basically is the way it's supposed to tell me that, hey, I exit normally, everything's okay. So if I compile that, well, it's a good sign I got rid of all the warnings so I'm actually using all the variables now. So any guess is what happens when I run this and this is like help, my program is stuck, please help me. So yeah, that the child is waiting to be read, so its output is waiting to be read, close. So remember a cat, when essentially did our cat program we wrote exit if we remember. Yeah, so what happened when we hit control D on cat? So we can see, so let's go ahead, we can go to our buddy Strace. So we can Strace cat, just run it. We can see, hey, it's stuck on a read system call, file in or a standard in. So if I hit control D, we can see what happens. So when I hit control D, well, it knows that it is exit because the read of zero returns zero. Zero bytes read is a way to communicate that, hey, no data is possible, like no future data is possible, so you can go ahead and exit now, no data is possible. So with a pipe, a pipe will return zero from read if all of the file descriptors pointing to the right end of the pipe are closed, which means that pipe can no longer receive any data. So did I close the right end of the pipe in this case? No, because I didn't listen to my own advice of closing file descriptors whenever I'm done with them. So let's go ahead. So I would go ahead, I can close the, I'd close the read end of the pipe, which it uses, but that's not the problem. The problem is I'm actually done with the right end of the pipe here. So I should close it as soon as I'm done with the right. And then that means that, okay, this should be if we've set up things properly, that's the only right end of the pipe available. And if we close it, that means we can no longer write, like that pipe can no longer receive data. So it would actually see a read. If you try and read from it, it would just return zero after it reads everything and say, hey, no more data's possible. So if we go ahead and do that, compile. So now it works. So now our cat program is actually exiting correctly. So it passes through all our certs of everything. So it would have read from file descriptor zero, got a zero, which only happened because we closed it and we can verify that for sure, that's this line. So if I comment out this line again, where I don't close my right end of the pipe, which is the last thing up, stuck again. I mean, I can kill it with control C, but it doesn't exit nicely. So any questions about that? Yeah, okay, so let's go ahead. So other suggestion is to just put weight at the end. All right, any guesses as to like something, is something catastrophic going to happen? Well, we can just execute it. So nothing catastrophic happens if I wait at the end. Anyone want to hazard a guess as to why. So in this case, the only difference is, well, after weight, that process is definitely dead, terminated, everything gets deleted. So in this case, I read and then I write and that process might still be alive while I'm reading from it. But it doesn't matter how long that process lives because it's just writing to the right end of the pipe, which is essentially a buffer managed by the kernel. So that buffer is going to stay alive no matter if that process is alive or dead. So in this case, that process might be alive when I actually read whatever it spit out at me, but doesn't have to be. If I moved it back where it was originally, well, I write some data to it and then I wait for it to die. So it would go ahead right to write, fill up that buffer, I wait for it. It is now dead. Everything associated with that is now deleted, gone. Except, well, it wrote to a buffer, which is managed by the kernel, which wasn't necessarily associated with that process. So that buffer still exists. So my program can read from it and still get data from it, even though that process is long dead. Well, not long dead, well, maybe long dead. So I can still read from it, do all that fun stuff. Okay, any, yep. Oh yeah, so if we just did this and say we still forgot this. All right, so if we do this where we write some data to it and then read some data from it and then wait for it to die, well, this is where our behavior would actually change. So in this case, if we run it, well, we get the testing string back because we wrote to it, it's alive, so it spit out some output for us. We didn't wait for it to die before we read from it. So we would still read from it, but it's still alive. So it's still waiting for more input. So we'll get a little bit of input out of it, but still waiting for more. So we have the same problem, except this time we see output, but we still get that weird hanging behavior. But yeah, that was a good one. All right, any other questions with that? Yep, sorry? Yeah, so the question is, how does cat get the testing string? So the way we have everything set up, oh crap, I deleted it. So the way we set it up was we made, before we exact, we set up our file descriptors like this where zero was the read end of the in pipe and out was the right end of out pipe. So the way pipe works is anything you write to it, the kernel guarantees it comes out of the read end of the pipe. So what happened here is this program would be reading from zero, which is the read end of the pipe. So the data it's gonna read from that is anything we write to it. And since we created the pipe, we're the only ones with the pipe. So it's only gonna read whatever we write to it. So we would write to it, and then as part of what the curl guarantees, anything we write to it, it would read from it. So we set it up so that we're writing to in pipe and that's reading from it and it would read it. So yeah, this process, and this process is the only one that uses the read end of the in pipe and we're the only one that uses the right end of the in pipe. Does that make sense? Yeah, when I put, so this does work now if I do uname. Yeah, uname works. So yeah, don't worry about, yeah. Yeah, so the question is, well, like in that cat process, if I want cat to write to the terminal again. So in this case, whatever file descriptors that process gets, it won't get them back unless it like explicitly opens them and then guesses what they originally were. So there's essentially no way to go back. Yeah, okay, so this is a kind of simplified version of lab two, so I suggest you start lab two because remember that bug where it just kind of hung there for a while? Yeah, you're gonna encounter that. So if you leave it to the last minute, you're gonna be put delicately, you're gonna be screwed. So start early and this gives you a good framework for doing this, so like you should be able to pass a few test cases. So just remember, I'm pulling for you. We're all in this together.