 All right, good afternoon. So we are doing the lab four primer, which will also help us prepare for quiz three because we'll be messing with processes. So this will essentially be able to write some useful program that will essentially be lab four. Hopefully we can do it by the end today and that should well prepare you for lab four. So this is mostly you guys programming because I'll show you the tools and you guys will come up with a solution and it'll be lots of fun. So here's our task for today. We want to send and receive data to a process. So we want to actually do some IPC. We wanna communicate with some processes. We don't wanna do the lame thing of we like write to a file and then we read from a file. We're gonna do some actual communication while both processes are alive. So our task for today, we want to create a new process that launches whatever program we give it as a command line argument and then we want to send a string testing with a new line to it and then we want to read any output it gives and that's it. So we want to send data to a process, get data from the process and turn it into a let it execute a different program. So we're gonna need some APIs we'll whirl around through them. Our first new API is called pipe. It is kind of a one-way communication stream. So here int pipe and it takes an array of ints that are just two file descriptors. So it's gonna create two file descriptors. So it's a C wrapper for a system call. So it returns zero on success, minus one on failure, sets error no, kind of what we're used to. So it's a one-way communication channel using file descriptors. So again, I said it gives you two file descriptors. So it gives you file descriptor at index zero, which is the read end of the pipe and a file descriptor with index one, which is the write end of the pipe. So the way this works is any data you write to the right end of the pipe pops out of the read end of the pipe and you use the write and read system calls just on any old file descriptors. So you can think of this pipe as a kernel managed buffer. Again, any data you write to the right end is gonna pop out of the read end and we'll be able to use this to communicate. Yep. So if you want to communicate, they're gonna have to both have access to it and that's what we're gonna solve. So if we fork, it's a copy, right? Yeah, but if you create it and then fork, it's a copy at the fork, right? Oh, it's gonna shake all day because they're jack hammering. Yeah, so anything you write, it doesn't matter what process has it, as long as they have that file descriptor, if you write to the one end of the pipe, you can read that data from the other end of the pipe. Yeah, but the kernel manages the buffer for you so you can use it to communicate between processes and that's what we'll do. Yeah, that's the same question. Okay, we'll see how you use this. Yep, the kernel gets the, the kernel does it, don't worry about it. So it's a kernel managed buffer, there's probably a limit to it, but yeah. Processes, so it's a file, it gives you two files descriptors, you write from one side, pops out the other end. That's all it does. You can use it in a single process, but at that point you are communicating with yourselves, which is pretty lame and we can do something better. Yeah, so one way meaning that if you write into it, it just comes out the other end, but you can't write and go the other way. So if you wanna communicate in both directions, you'd need two pipes. Yep, they could, but if you wanna communicate with them, one should be doing the writes, that's sending data, the other should be doing the reads that is receiving data. Yeah, you can't see the change after the fork though, right? No, they're independent. Yeah, so file descriptors are kinda like pointers, so they're pointers to something that, whatever that file descriptor represents. So when it copies, it's gonna copy all the file descriptors as well in the kernel. Yeah, but the kernel will allocate that for you and manage it. Yeah, that's an array of two ints. Yeah, so that's the argument, but I mean everything's zero index, so if you give it an array of two ints, you have index zero and index one. You don't have index two. So we're gonna do this. So we have a task today. Okay, second one, and then we'll throw it all together and have more questions. So we saw exec v e, it's kind of a pain in the ass because you need an array, blah, blah, blah, and then you have to give the whole path to file. You can use exec lp to make your life a bit easier, so instead of giving a full path that starts with the slash, you just give it the name of a file, and then instead of an array, you just give it a bunch of strings for the arguments and when you're done, you just throw it all at the end. So this behaves the same way as exec v e, except it's a little more convenient under the hood, lib, or yeah, the standard C library is gonna transform it to exec v e, so it's just to make it easier to write for you. So if it's successful, it doesn't return because it transforms this process and starts executing whatever the program is and it's going to search for executables given some path environment variable that's default set for you and this is how your shell would work if you just typed ls. So I'll also find wherever that executable is. So it just gives us a little bit more convenience. All right, next API we're gonna need in our collection is close. You close a file descriptor, so it returns zero on success minus what? Yeah, some aliens are attacking, cool. So close, it's another C function, zero on success, minus one on failure, so that's error no. Close is the file descriptor for the process, so that's no longer usable and the file descriptor is just a number, you don't know what it refers to. So it frees up the file descriptor and it can be reused because the way the links kernel works is whatever it creates a new file descriptor, it's going to use the lowest number possible and we kind of saw that way back in the day. Yeah, we're razor fast. I mean, you can look up the kernel implementation if you want, but it's probably gonna be, just zoom through or razor quick and you don't have that many file descriptors anyways. Yeah, yeah, but your web server's just gonna read any old data, right? So up to you to make sure the data's valid. So this is direct, I'm going direct from one process to another. Yeah, but if I'm reading a file in my system, I have to do an open call and it makes it a file descriptor anyways. So, and then my program, my file descriptor could represent network connection too, it doesn't, it's just file descriptors are anything you can read and write from. Doesn't matter. No, it's just called a file descriptor. File descriptor is just a number that represents something you can read and write bytes to. So we've seen you're used to it being a file, but it can now be a pipe. It could also be a network socket, which you'll probably see-ish, but everything we learn, it's all the same. Yeah, so it's just making our lives easier. So it searches for that file in your path instead of doing like slash user bin LS, you just put LS there and it'll find it for you. So it's just making our lives easier. No, so this is instead of exact VE, remember we had exact VE before and we did slash user slash bin slash LS. So instead of that, with this version, I can just put LS in this argument and it will find wherever that executable is for you. So it's, no, so it searches for everything in this path environment variable, which that's like the default thing here we can see. If we echo path, that's how your shell works and how everything works. So it's gonna search it in that order. Yeah, first folder it finds. So that's why they put local before the system one, so it would take any local changes before. Like your UG whatever system's gonna do the same thing. Yeah, but in my shell if I do that, right, I don't give it the full path. So when I do this, my shell is essentially doing that, searching path to find LS for me. The LS program, like the executable file, then it'll launch it. Okay, so that was closed. All right, so our final API is our dupes. So we can screw around with the file descriptors. So dupes takes an old file descriptor and a new file descriptor and makes both of them point to the same thing. So in dupes case, it just returns a new file descriptor on success, minus one on failure, sets error no. And then since it returns a file descriptor, whatever file descriptor it returns points to the exact same thing as the file descriptor argument you pass as old file descriptor. So and in dupes too, you get to pick what the new file descriptor is. So you get to reuse new file descriptor. So if it's open and valid, it will go ahead and atomically close it and then replace it with, and then make old file descriptor points to the same thing as new file descriptor. And this should, and this is why it's a different system call because it needs to be atomic because it's gonna close and then it has to reuse that same number. So if it's not atomic, you might have another process come swoop in and steal that file descriptor number. So that's what they do. And yeah, any questions about that? It returns a new file descriptor. So both of these return the new file descriptor. And in the case of dupes too, you would, yeah, it makes a duplicate of old. And so in the dupes case, it will return new. Well, in both cases, they return new, but in the first case of dupes, you don't pick what the new number is. And in the case of dupes too, you get to pick what the new number is. So it's gonna close it first if it's valid. Is there a question at the back? So we are going to use them to play around with the file descriptors. So we're gonna do that now. That's everything we need to know to do our tasks for today. So we'll see. Oh no, okay. Let's get to it. All right, so again, here's our task. We want to create a new process. We want to receive any data it writes. And it's just gonna write data as standard input. And since we can play around with file descriptors, we should have an idea what to do. And then last one, we want to send a string testing to that process. So what would our first step be here? Do we know how to create a new process? Fork and then execute something? Exact feed, okay. Let's start with that and then we'll work with the communication. We'll use LP. So here I have some placeholder code and then I fork. So at the point of the fork, they're exact copies of each other. The only difference being the value of PID. So just to make things more readable, if it is greater than zero, it calls a function called parent and then gives a PID of the child. And if it's the child, it calls the child function and passes in the first command line argument, which should be the program to run. So if we want to just run the program in the child, we can just do exec LP and give that argument to it and it should go ahead and find it for us. So if we go ahead and compile that and do you name, so you name just outputs like the name of the system it is. So that's a good one to test. If I do that, print out Linux, which is exactly what you would expect if you just ran it. So why does it do that? Maybe what's going on? So yeah. So here the child changes to you name right after the fork and the parent does nothing and it just exits, right? We're all good. So why do I see Linux on my terminal? Yeah, your name is running, but so when I start my program here, I have some file descriptors already. That's just by convention. There's three file descriptors whenever a process starts. So anytime you print the three file descriptors, so in the parent, it calls parent, which does nothing exits with the return of zero. And then in the child, it goes ahead and just exec LP's you name without messing with the file descriptors at all. So it'll be the exact same terminal as what we originally started with. So it would just print out the results to standard out and we see it on the same terminal. Yeah. Yeah, but we haven't done anything with them yet. So there's this place holder so I don't have to write code to make this faster. So we haven't done anything with pipes yet. Yeah. So they just refer to something. So they're kind of like pointers. No, just by convention, it's just zero is standard in, one standard out, two is standard error and what they actually are connected to will change depending. No, it just depends what you wanna do with it because we're gonna screw with them right now. Right, so what's working, you name is just outputting its answer to, it's just writing to file descriptor one like we saw in the beginning. So if I wanna capture its output, what should I reach for? So I'm trying to communicate between two processes. So I probably need a pipe to communicate between them. Yeah, wow, we're not awake today. Okay, so what is, so far we just forked. So ignore all the declarations, I just forked. We know what fork does, creates a new process. So now after the fork, I have two processes. One's gonna call parent, one's going to call child. If I am the parent one, it just does nothing and then it would just return zero. It's done so, it exited zero. If I'm the child, it's gonna exec lp the argument which in this case was uname and then that's it, the child just becomes uname. But now I wanna capture the output of the child. And just read it in the parent process that created it. Put into file, yeah, but put, okay, yeah. But first I need to create a pipe. No, I just allocated space for them. There's no call to pipe. Okay, so first, so this is just an array declaration that just zeroes it out. If no one's read this syntax before. But so I have these check error functions that essentially check error just returns if the checks if the value is negative one. If it's not negative one, it means it's fine. And if it is, it's just gonna print error message and then quit out of the process. So this is just the sanity check because it might save our ass later. So when check error, we're gonna call pipe. So we'll have a pipe that represents the output that we wanna capture. So we just give it the array and after it will set it for us. So we have this. So now after pipe, it's gonna create two more file descriptors if it's successful. So file descriptor, it's gonna use the lowest number. So we have the default file descriptors up here. And then we have, it will create two file descriptors. So we'll have out pipe FD, which will be file descriptor three. And that is the read end. And then we'll have file descriptor four, which is out pipe FD, whoops. Pipe FD, pipe FD. And this is the write end, okay? So what also, yeah. Yeah, and the kernel create an internal buffer. So if you write to one, you can read it out of the other. It's just a number. If you can come up with the numbers yourself. Yeah, so the process could, so standard processes open like this with three file descriptors, but someone might be a jerk and whatever they forked and then exact you, they had like 10 open file descriptors. So I know because I'm launching it through my terminal and I know nothing screw is going on, but you won't always know this. Okay, so that's my pipe. Why don't I just pipe here? Close, close. Anyone want to correct? That was so close. So after the fork, they are the same. Yeah, and then after that, they're independent. They both call pipe. So each new process creates its own pipe and they would get the same file descriptor numbers, but they would be different pipes. So the child would create a pipe, it would get three and four, and then the parent would create a pipe, it would get three and four. Yes, they would be different pipes. So this way I have two pipes that I can't share anymore because I should have done it before the fork. And then if I move it up here, I have one pipe and both processes have references to it. Yeah, mm-hmm. Yeah, it has like a file descriptor table. Yeah, it has a file descriptor table and all of them are essentially, the file descriptor table just has a bunch of pointers to stuff. It's just a table, you- People would just put that. Yeah, yeah, so if I, P and E. How many are you using? So for example, I think I showed this in another lecture, like you can see all the open file descriptors for a process just by looking at this directory. So for example, my shell had, at one point it opened 103 file descriptors. So it messed with a lot of stuff because they're broken. So they're different colors depending on what they actually refer to. So this one is like some file. This one's a socket. These are all shared memory that have to do with Chrome for some reason. And then it has its own status but you can screw around with file descriptors. I think I showed it before, but that's neither here nor there. Yeah, that launches your kernel. You can, but generally, if you like to BIOS updates, it's like a EEPROM chip and it takes a long time and it's generally the BIOS just boots your kernel and then tries to hide itself. But you can interact a bit with it depending but that's much more advanced. Okay, so now we have a pipe. So out pipe FD is correct. So if I want to capture the output for this process, it's writing to file descriptor one. So if I want to capture it, I should probably make it right into the pipe instead of just whatever file descriptor one is. So how would I do that? So here's a hint. We want to use dupe two. Oh, this calls as much more confused than the other one. Okay, here, I will write it and I will proceed if someone explains it to me. Or here, I'll make a pretty number. So what would that do? Which is just a file descriptor that's one. Yes, so it's the right end of the pipe. So I'm trying to, in the child, I'm going to copy the right end of the pipe and replace and make file descriptor one point to the same thing as the right end of the pipe. So now I just overwrote because file descriptor zero originally was copied from the parent and it was to the terminal and now I just replaced it. Well, you only have three file descriptors so we're just replacing it. Well, so standard error still connected to the terminal but in general, you'd have to, you lose it. Yeah, all right, first. So dupe two should return one because that's the new file descriptor it gave. Yeah, which now points to the right end of the pipe after. Yeah, right. Yeah, but I only want three file descriptors when I launch this process just to be nice. No, it replaces. The only way to make a new process is fork. Everything else gets replaced. So like if it's already has like the libc open and it uses libc, that won't get touched and like some other things like that. Yeah, the right end of the pipe. So here I'll go ahead, check for errors quick because we should always check for errors. So if I do that, it makes, it's fine. And now if I run it, what do I expect to see? Nothing, why? Yeah, so I'm not reading from the pipe so the data's going somewhere, I can't see it anymore. Yeah, that's up to the kernel. I think it would just give you an error eventually that you filled up the buffer. So it doesn't lose data. Okay, so everyone good with that? It's getting captured. It's writing to the right end of the pipe. Let's see what happens if we go to the wrong end. So if we connect standard out to the read end of the pipe something very bad should happen. And there you go, bad file descriptor. So be trying to write to the read only end of the pipe which you can't do, that's an error. So we can see that we're actually doing the right thing. So you should only write to file, like write to file descriptor zero and only read from file descriptor. Only read from zero and write to one. Yeah, that's not weird, that's cool, right? This is how things work. Yeah, so it has permissions that the kernel keeps track of. So write only, read only. All right, so let's be good citizens here and close our file descriptors just to be nice and clean. Close this. So I'm just closing, so I'm copying, making essentially, if I give them numbers I'm making four and zero points the same thing. And then I'm gonna close three and then close four because I don't need them anymore. Yeah, in different processes too. Because the parent still has file descriptors open. I've closed a pipe that the parent has but to close a pipe completely every reference to it has to be closed. So the parent still has a reference to it. Okay, so this takes care of the child. It's writing to the read end of the pipe and it's not going to use the, sorry, it's writing to the right end of the pipe. So it's not going to use the read end of the pipe and we already cloned it so it's pointing to what we want it to point to so we can close it too, we don't need it anymore. So whenever we exec LP, there's only three file, there's only like this, there's only the three file descriptors. So there's still only these file descriptors when it launches, except one, we replaced it with what was originally out pipe FD1. And that's why our terminal worked. So we cloned it and then it printed to the terminal because we were sharing the same file descriptor. Okay, so it's writing to the pipe, how do I get data out of the pipe? And I probably want to do it in the parent. So how do I get data out of the pipe? Such a responsible parent. Is read blocking or not? So remember we wrote cat that just read and write? It's just stay there waiting for data if we called read. So we can call read, that's cool. So let's call read. So what does read need? Read needs a file descriptor, a buffer and some bytes. Who wants to tell me how to create a buffer? Come on, I hit my head this morning, I don't know how to program. Help, a bunch of bytes, what's a byte in C? Character. Character, create a character array. Buffer, I'll just give you, I'll give you a good size. That size is really good, you'll see why later. So I'll create a buffer this size. Now I can do read, right, I think. So what file descriptor do I want to read from? Okay, so I want to read the data that the child wrote. So the child wrote to file to out pipe fd one. So it should pop out of the zero side. Out pipe fd zero. And then I'll give it my buffer and then size of buffer. So that should come, I'm missing a bracket, thank you. Missing a bracket. So what does read return? Perfect. Yeah, now number bytes at read or negative one if it failed. Bites read equals this. And then I can check error because it's negative one. So I'll check error for my read. All right, cool. So my parent process should have read from that. So if I execute it, what should I see now? Nothing again. But hopefully I read some data. So let's print it out. So this format specifier is gonna be weird. Just trust me, it works. So how do I print off the data? So did I read a C string? No, I just wrote a bunch of bytes. So anyone who's advanced enough to tell me why I should put here? Oh, everyone should know this off the top of their head. There you go. So that means, so this is a string, so there's still percent sign string, but there's a dot star that's saying how many characters to print and it gets the first argument. So bytes read is the number of characters to print and then that's the array. So I don't have to give it, it doesn't have to be null terminated, I'm just printing that many bytes. No one's taught you this? No one knows this off the top of their head? Come on now. Yikes. All right. So if we do that, hey, cool, it outputs. And it outputs because in the print F, we didn't touch our standard out, so we're writing to our original standard out, which means we can see it on the terminal, but we wrote out the data we read from the process. So we communicated, we got data. Yeah, but what about if that process did something cool and you wanted to capture its output and do stuff? So for instance, what if I had to grade your programs? I'm gonna run your program and then maybe I parse some output from it so I could just read output from it and do something interesting. So hey, that's how our grading works in the Python sub-process module. All right, any questions on that? Yeah, yeah, because we're setting up the file descriptors. You can only create a new process from a fork. They're gonna be created from a fork. So it's specifying how many bytes to actually print and it takes the first argument. So instead of just a string, it takes a length and then the string. Yeah, so if I just say, hey, print the first two characters, all right, that should just print li. What? Oh, that, oh, at the bottom. So the terminal will tell me if it doesn't end in a new line. Yeah. So yeah, in the child, I overwrote it so that it went to the pipe instead and in the parent, I never touched it. I mean, it would just, you'd write into that and write data to yourself and- Write it into the buffer. Yeah, which is kind of silly. Okay, let's do the other half. And we only have, oh, God, we only have six minutes. All right, speed run. So let's create another pipe in pipe. Okay. So now we have two pipes. So we want one that we send data to and we already have one that we get data from. So our child's probably gonna look more or less the same. Oops. So I will just replace out in. How'd I do that? I'm cheating, send it in file. Okay. So now I'm replacing this. So we sped run this. So now I'm replacing the standard in with the read end of the pipe. So now whenever it reads data, it's gonna be reading from the read end of the pipe. So I need to put something into the pipe. So in the parent, the right end of the pipe is going to be in pipe FD one. That's what I should write. So I said I wanted to write a message. So I wanted to write testing new line. So I want to write message Stirland message. So it's just gonna wait for some data. Okay. So now check error by certain totally we're speed running now, right? All right. So that should write some data to that process. So I think that's all we need. Yeah. So if we execute that, if we do you name, we should expect to see the same thing because it doesn't read anything from standard in. All right. Just does that. So what reads something from standard in cat? So cat, if I do that, if I hooked up everything correctly, which maybe I didn't, I should see testing. Hey. So there's the data I wrote into it. Yep. So let's see. Let's quickly be a very good parent. So you told me I should wait. Let's wait three minutes. Jesus. So, so this is how fast you program your labs. Right? Check error. Pid. Oops. All right. So we want to wait on it. Let's just write some asserts. W if exited. New status. Let's just assert. And then in exit status, we should get its exit status because that's a good thing to do. Do you exit status? This is what we had before. And we'll just assert that exit status. What, what's good? Good zero. All right. That's what everyone was going to write. All right. Let's run it now and see what happens. So what's happening? So we're waiting indefinitely which means our cat process isn't exiting. Why would cat not be exiting? We wrote testing. So I wrote testing to it, but it's still waiting. The child's just executing the program. So what's happening? I'll give you another like few seconds. So I wrote to, I wrote to its standard in. I wrote the string testing. It worked before. It was fine. I got it. I printed it out when it was done. That was fine. But now I'm waiting for it to die. It's not dying. So why wouldn't it be dying? Well, so essentially cats waiting to read more input and more input is possible because we even closed the right end of the pipe. So it's still, it's just waiting. It's possible it could get data. It's possible I slept for like half a century and then I'm going to call right again. So what I should actually do is after I write to that pipe, I should be a good citizen. This is why you should be a good citizen because now I'm done with it. So I can close it. And that way cat will know that no other output's possible because no one has that file descriptor open. So you don't. Because it's closed, it means it's done no more output. That's the only way cat, you can't. So I'm waiting for it to die. So I'm waiting for the process to end. How can't communicate with a dead process? All right, so if I do that works properly, continue this on Discord or next class because everyone kind of looks confused. All right, just remember I'm pulling for you. We're on this together.