 So, you guys probably noticed that the weather is cooperating with assignment one, so this is great, let's be inside and write synchronization primitive sort of weather, so I would encourage you to take advantage of it before it warms up next week, and then you won't want to be inside writing synchronization primitive. So what we're going to do today is we're going to finish up with the example that we left off with last time, which is the producer-consumer buffer. We'll look at how to safely allow multiple threads to access that shared state using a synchronization primitive, and then we're going to kind of pop up from this unit we've been doing on synchronization, which was time to help you with assignment one, and we're going to go back to talking about the system call interface and get through exec and wait and exit, so that's the goal for today. So we're kind of assignment one, we're pretty much done with the announcements, I think everything's working, for the couple of you that decided to wait until like yesterday to sign up for discourse, hopefully you're getting new emails from Auth0 with new tokens in them, but please make sure you can access that stuff. The other thing I'll just say, given the weather today, is just to remind you of the cancellation policy, so if Buffalo Public Schools are closed then we will not have class, and probably one of you guys should remind me about that, because I probably won't notice, I'll just be here and then you won't be, but you know, if you're wondering whether to come to campus and there's two feet of snow outside, if the Buffalo Public Schools are closed we will not have class. Alright, okay so let's go back to our example, so the producers and consumers are sharing a fixed size buffer, we want to make sure that happens safely, and the sort of invariance that we wanted to establish in this problem, were that if there's data in the buffer, and there's a consumer that's available to consume it, that consumer is busy, if there's space in the buffer and there's a producer with an item to put into the buffer, that producer can put it in. Otherwise the producers and consumers should wait, they should essentially sleep waiting for work to do. Okay, so here's where we started, and it's not a bad idea when you write a code like this, that you know needs to be multi-threaded to start with some skeleton code that doesn't need to be multi-threaded, so other than the while at the top and the bottom, this is pretty much what we want. The producer puts an item into the buffer, the consumer removes an item from the buffer, why am I maintaining a count here? That's one thing that if I was doing this in a single-threaded context, I wouldn't need to do, why, where is that going to come in handy in a minute? You want to guess? What is the count, first of all? What is the count supposed to, what is the count measuring? Yeah. The number of items in the buffer. So I pound to find some full constant that defines the size of the buffer, and so if the producer tries to put an item into the buffer and it's full, it needs to do something, if the consumer tries to withdraw an item from the buffer and it's empty, it needs to do something. Okay, what do you guys think is the right primitive here? There's a couple that could work, what's a good primitive, yeah? A condition variable, who agrees that that's a good primitive? Me too, what else could you use? You could use a semaphore. And the way to do this with a semaphore is not as intuitive as you might think, but you can, I think you would need maybe two. But to be honest, semaphores are not particularly useful when solving problems like this, frequently you'll find that a condition variable works better in most cases where you might think that you want to use a semaphore. There's a couple where it's not true, but I would encourage you, if you were thinking about using a semaphore, try to solve the problem with a condition variable. Condition variables in some way are a more general form of a semaphore, right? You could think of a semaphore as a special type of condition variable that forces threads to wait when the count is zero, whereas a condition variable can be very general. Okay, so the conditions here that require waiting, so there's three conditions the buffer can be in. If it's full, who has to wait? The writers or producers. If it's empty, who has to wait? The readers or consumers. And if it's in between, nobody has to wait, right? If it's in between, all they need to do is make sure that access to the count in particular is atomic. And in reality, well, I probably also want to make sure that access to the buffer is atomic. I'm not showing you the code for get and put, but those would have to do some synchronization on the internal buffer state itself to make sure that two gets don't return the same object or something like that. Okay, so let's try this. So I made modification use in a CV. So now what's going to happen, remember with the condition variable comes a lock. A lock is what protects the condition in between me checking it and going to sleep in this case. So I've created a lock and a condition variable. What I'm going to do, what the producer is going to do is when the count is full, it's going to wait on the condition variable. When the consumer, same thing. When the buffer is empty, it's going to wait on the condition variable. Why is it? So, okay, so I kind of understand this code, right? I acquire the lock here in the producer, I check the count, I call wait. Why is it safe to wake up and check the count again in this while? Yeah, so what does CV weight do to the lock? Why am I passing a lock to CV weight? What is CV weight going to do with the lock? Nothing, poop on it, I don't know. Yeah, what's that? Okay, it makes it atomic to get to the weight channel, but what's the semantics of CV weight? Well, let's not worry about the internals of the CV weight, but CV weight is going to drop the lock, right? If CV weight didn't drop the lock, what would happen to other producers or other consumers? Let's say that CV weight didn't drop the lock, what would happen to this system? As soon as the buffer became full or empty, everything would just deadlock, because somebody would grab the lock and get in here, and then they would wait, and they're holding the lock. So if I'm holding the lock, can the condition change? No, because nobody can acquire the lock to produce or consume. So this is bad. So again, what does CV weight do? It releases the lock. When it wakes up, what does it do? Why is it safe to recheck the condition here at the top of the loop? Why don't I need another call to lock the choir here? Yeah. Because CV weight internally reacquires the lock. So CV weight returns with the lock held. CV weight has to be called with the lock held, and it returns with the lock held. So it's completely safe to sit in this loop, just CV weighting, checking the count, and also when I drop out of the loop, I still had the lock. And so I need to release it down here. Okay. So does this work? No, why not? Yeah, so there's no signal. When you're using condition variables, whether they're semaphores, locks, CVs, reader-writer locks, whatever, very often you have pairs of operations, and CVs are no different. What's the pair of operations you have to be cognizant of with a lock? Yeah, you guys thought I was going to say CV, but I tricked you, right? This is like jeopardy you buzzed into early. I said lock, what's the pair with a lock? Every time I acquire a lock, I need to release it. And this is great coding practice, right? So please, when you're writing your code for this assignment, when you write lock, acquire, you know, hopefully you're programming with your partner and your partner is pointing at the screen and saying, where are we going to release the lock? Because those should always go together. With the CV, the paired operations are, there's really two groups. One group consists of what operation? There are three operations in a CV. You have some non-trivial chance of getting this right, even if you're just guessing. Yeah, for Josh. Okay, so weight is in one camp. Weight stops me. What are the two things that can wake me up? Signal or broadcast, whatever. Wake one or wake all, okay? So here I've got a weight, but I have no signal. Ah, you guys, I should have to give you a dollar for every time that happens. Of course it does this too. So in this case, there is, sweet, okay? I don't know where my phone went though. Okay, so in this case, there's a weight, but no signal. Let me fix the problem. Good. Now, where do I want to put the signal? When does the condition change? In a way that's meaningful, right? So I need to call CV signal or CV broadcast. Where should I do that? I hear mumble, mumble, mumble. Not the correct answer, yeah. After the count changes, okay? So I can put it here and this will work. This is working code. What's the problem with it? This isn't ideal. So this would be the point in your Google interview where the interviewer says, okay, this works, but can you improve it? For Josh, I'm not calling on you again. Yeah. Right. So remember, we actually had three conditions. There is full, empty, and other. And the consumers only care when the condition changes from empty to other. And the producers only care when it changes from full to other. So this will work. But what will happen is that there will be cases where all, you know, essentially if I'm in between, then nobody should be in here, right? So for example, if the buffer is neither full nor empty and my change, my decrement of the count does not change it from full to empty, then nobody should be sleeping up here, right? Because everybody's falling through this check and putting their items into the buffer, okay? So let's try this, okay? So this is better. This avoids those spurious wake-ups. So what is the producer doing? The producer is saying, I'm only going to broadcast if I know that the condition changed. So if I'm the guy who put an item into the buffer and it was empty, I need to wake up people who are waiting to consume from it. Same thing at the bottom here. So if I'm the person who created a spot and a full buffer, I need to wake up the producers. So there's an interesting question here, and this came up when I was looking at the semaphore code today actually in some of the tests we wrote. So why not use CV signal? Would it be safe to use CV signal? So is it safe to use CV signal here? So what's the difference between CV broadcast and CV signal? Yeah, Rajesh. Oh, I said I wasn't calling in you anymore. Sorry, stop raising your hand. I'm fooling you. I only am caching about 10 seconds worth of state here. Yeah. Right. So in this case, I'm notifying everybody. And I'll say I have 10 producers that are all waiting there and the buffer is full. And now a consumer comes in and consumes one item out of that buffer, how many of those producers are going to get to proceed? One. Okay. I'm waking up all of them. So is this a good, do I want to do this? This is a good thing to do. Okay. So it turns out you can use CV signal here. I don't work. But there's this interesting interaction between the synchronization primitives and the scheduler that I want to point out here. So, and this is something, you know, you see and you want to kind of be cognizant of certain situations. If I use CV signal here, what's going to happen is that the weight channel code is going to pick a thread from the weight channel and choose it and make it runnable. The problem with that is let's say that there's 10 threads on the weight channel, but one of them is more important than the other threads. The weight channel code probably doesn't know that. What knows that is a piece of the system that we're going to talk about in a week or two, which is called the scheduler. But if I don't make all the threads runnable, then the scheduler doesn't even know that they were able to run. So if I take all 10 threads and I make them all runnable, what happens is that the scheduler then gets to say, aha, you're important, you're going to run, you're going to be able to produce, everybody else goes back to sleep. So this is a little bit of a trade-off here. The price of waking up fewer threads is that the weight channel code has to be trusted to make the right decision, it probably won't. The benefit of waking up all the threads is that I allow the scheduler, which knows more about the threads than I do, and we'll talk about how schedulers make scheduling decisions in a little while to make this a system, just a comment. Yeah, right. Yeah, you probably could. You could probably have some function that the scheduler exported was like, choose between these threads. Yeah, that would be possible. Any questions about this code before we go on? This is the end of synchronization. All right, so just let me make a couple of comments about how to approach these types of problems, because this assignment is not the last time that you will have to solve synchronization problems as part of these assignments. You'll have to solve them as part of assignment two and definitely as part of assignment three. So as we pointed out at the beginning of this example, there's a lot of cases in which you can use, even among the four primitives that you guys will have working by the end of assignment one, there's still cases where you can choose multiple primitives to use for a particular problem. And there's usually one primitive that just feels better. In some cases, the primitive won't work at all. There's certain problems that you just cannot solve with the lock because of the semantics of locks, where you can solve it with a CV or with a semaphore. So you're going to do this for assignment one, obviously. And again, so before you start writing a lot of code, my suggestion is kind of sit down, maybe sketch out things a little bit. And if here's a sign that you've used the wrong synchronization primitive, you're having to use other synchronization primitives. You said, oh, I'm going to solve this with a semaphore. Oh, wait, I also need a lock. Oh, wait, I also need a condition variable. And if you have three synchronization primitives trying to solve the same problem, the likelihood is one of them is just free writing, right? It's the others are doing the heavy lifting. One of them is just not helping. So back up and see if you can solve the problem in another way. Okay. In general, when solving this kind of problem, you know, the first thing to figure out is what do you want to be true? So we did this with the producer-consumer buffer example. We said, here are the constraints. Here's the things we want to be true about the world. Not only do I want there to be coherence in the buffer itself, but I want to make sure that threads go to sleep if they can't do something and wake up when they can. That's important. You could write that code. So another way to write the producer-consumer buffer is just to have the thread sit there spinning, you know, banging the buffer over and over again. Is there space? Is there space? Is there space? That's not what we wanted. The way we set up the problem, we don't want that to happen. We want threads to go to sleep when they don't have any work to do. Identify shared state. So this is actually really critical. If you don't know the things that are being shared between the threads that you're trying to synchronize, you really have no idea where to start, and you have no idea when threads are accessing things that are potentially unsafe. Now, there's times when even global state does not need to be synchronized because there's some other constraint that applies that means that it's only going to ever be touched by one thread. But frequently, any sort of global state that you introduce into your kernel, you need some kind of synchronization. So people that have been, you know, something of an example, people that have been booting their OS 161 kernel have probably noticed a synchronization problem. Can anybody describe what it is? What is shared state that is currently not properly synchronized until you finish parts of assignment one in your kernel? Yeah. What's that? Yeah, like the output from the screen. So you guys, if you've booted a kernel with multiple cores, you've seen that terribly interlaced output. It's kind of amazing. So the current scheduler is round robin. So what happens is the threads actually run 1, 2, 3, 4, 5, 6, 7, 8 over and over again, and so it's beautifully interleaved, right? I mean, you would always have a hard time getting that to work by, if you wrote it by hand. But that's not what you want to see. So even on a system as simple as the one that you're already booting, there is shared state, the console driver. And bad access to that causes that pattern, you see. Okay, choose a primitive. Maybe choose multiple primitives. Sort of work through a solution with a couple and see how clean it is. Pair your operations. So, Waking, Sleeping, Signal, Weight, Release, Lock, Unlock, PV. Same thing with a semaphore. If you don't have the right number of calls to P and V, the count is either going to go this way, in which case probably things won't sleep when they're supposed to, or it's going to go and everything is going to deadlock. So you always have to have paired calls to these two, to two ends of most of these primitives. And also watch out for cases where you need to access multiple resources. So anytime you have a lock and you haven't released it and you're grabbing some other lock or you're peeing, and you can have a deadlock between a lock and a semaphore too. So just these are things to be aware of. And try to convince yourself, you know, and this is something that we recommend when you think about the synchronization problems, come up with some of the corner cases first and convince yourself what should happen. So for a whale mating example that we use as part of assignment one, what happens if I start 10 males and 10 females, but no matchmakers, what should happen? And maybe try that with your code to make sure that it works properly. It does what you expect. Okay, I need questions about synchronization. Before we go on, this is the end of synchronization, but of course there'll be more fun this week and next. Questions? Yeah? In assignment one, it says to be careful about building synchronization primitives off of one another. Can you elaborate on that? Yeah, don't, in general. Other than spin locks. I mean, spin locks are kind of the basis of everything else, right? And then reader-writer locks. You probably can get away with it. Reader-writer locks are probably the case where you can layer on a couple of other primitives. I think it's because the spin lock is essential. Other way around, yeah. Yeah, yeah, yeah. Spin locks are useful as a building block for the other primitives. But again, don't build your locks using semaphores. Don't build your CVs using... I mean, CVs are trivial, right? CVs are very easy once you understand the weight channels. That's a good question. All right, so let me go back and we'll do a few slides of review here just to kind of like help everybody go back in time with me. So when we started talking about synchronization, we got here by talking about fork. And that was the first in a series of process-related system calls that constitute the process life cycle. So fork, exec, weight, and exit. So fork returns twice. You remember, returns zero to the child. And what does it return to the parent? The child's process ID. The child thread, and again, there are new variants of this to other things, but in original fork, the child's and the fork he will write. The child begins to execute it at the same point as the parent did. So this is kind of like the ultimate case where you have to start thinking about multithreading because once you call fork, you have two processes that are executing the same code. So there are two processes hitting this loop. One does one thing, the other does the other. But that's because we wrote it this way. If we didn't, you'd have two processes executing the same code. All contents of memory are identical initially, but that memory should not be shared between the child and parent. So the child and parent can go off in their separate ways and they don't affect each other anymore through shared memory. And they start by default with the same files open at the same position. And this allows them, if they want to, to set up sharing through things like pipes. So we talked about the fact that fork is expensive. Now today we're gonna put that in context because we're gonna talk about the next thing that frequently happens after fork and the reason why fork can seem so futile because it's copying all this state around. And what we're about to talk about is how processes change. We talked about some solutions to this. Let's just kind of review, okay? And then because processes are created through fork, every process other than one, which is in it has a parent. And so you can rebuild all the processes back into a tree and here's what that looks like on a typical system. Any questions about this? We go on. Any questions about fork? Somebody remembers fork well. Okay. So the next thing, the next couple things we're gonna talk about are the rest of the process life. Like it was a little faster than fork, I hope. That includes how processes change and then how processes exit and a little bit of communication that goes on around exit. So obviously if you just had fork, this is one of my favorite slides all semester, it's still stupid, right? You just have like a lot of copies, I should do more, right? More. You just have a lot of copies of the same process and that wouldn't be a very interesting system. So the way processes change, the way a process gets to execute some other piece of code is by through a system called exec. And exec is actually a family of system calls. If you look at man exec, there's like exec with 10 different endings to it. Exec v e, exec, and that has to do with how the environment gets passed from the old process to the new process at exec. So this family of system calls, it replaces, it is a process saying, I want to be different. I want to replace my contents based on a blueprint that's laid out in this file. And if you've ever wondered what's in a binary, has anyone ever accidentally opened a binary file on their computer or something? Has anyone actually intentionally inspected binary files using the tools that are designed to do this so it just doesn't look like a bunch of gobbledygook and like break vim? Oh, you should try it. That's kind of fun. So there are actually great tools and there are tools that we give you that are part of those 161 tool chain that are designed to allow you to inspect the contents of an executable. As you would expect, the executable is a highly, highly structured file. The best way, the way I like to think about exec is that, and the binaries that exec uses is that these binaries contain a blueprint. So it's like someone gave you a blueprint to build a house and that blueprint needs to describe exactly what the house looks like, down to where the furniture is and everything. And the operating system is responsible for taking that and creating a process that replaces the process that called exec. So some of it is all sorts of things that are in the ELF binary, but mainly the ELF binary contains a couple of things. One of them is instructions on the contents of memory. So the binary says, here's a chunk of data and I want you to put that at exactly this location in my address space. So when we come back and talk about virtual memory, this will make a little bit more sense, but you can imagine that when the process starts executing, it's gonna refer to contents of its own address space using these, you know, addresses, memory addresses, and it wants to make sure that those things are what it expects. So for example, a lot of the contents of the processes memory initially are its code. And if the code isn't copied correctly, then the process doesn't execute properly. The ELF also tells the system where is the entry point? So when I actually start executing the first thread in this process, what's the first instruction it should load? And there are standard layouts that a lot of processes use by default. So if you compile something using GCC, GCC kind of puts things in standard places, but really you can pretty much layout things however you want. And as long as the file is a valid ELF executable, the operating system will follow your instructions as it sets things up. So ELF, everyone's heard of ELF as the executable and linkable format. And ELF describes not only binaries, which are things that are actually executable, but also libraries, including the shared libraries that a lot of programs rely on to function. I wish there was time in this class to do like a week on linking and loading. Does anyone actually understand how that works? It's pretty cool actually. When I took this class we actually talked about, it gets really gnarly and it's kind of neat. You can imagine like how do I take a library, binary code, and load it into a processes address space given that process A, B, and C may want to put it in different places. So that's the real challenge, particularly for building shared libraries is how do you make them relocatable so they can kind of go anywhere. So that's neat. But anyway, we're not gonna talk about it. Okay. So here's a helpful tool that's available on your system. It probably isn't available on your system, maybe. If it isn't you can install it. It's called Read ELF. And let's look at something like Bin True. So I'm asking Read ELF to tell me about this particular ELF binary. And here's some of the things that we talked about. So the ELF file type isn't executable. What would be another type? We just talked about this. Like a library, like a loadable library or shared library. What's that? DLN? Yeah, Windows, yes, yeah. So on Windows, I think Windows, well, I shouldn't say anything about Windows because I know little about Windows, but on Windows I think a DLL is pretty much equivalent to a loadable shared library on Linux. This is something that's running on Unix on our loading site system. So the entry point, what is this? So, well, okay, so this stuff down here is really more describing the contents of memory. But again, what's the entry point? It's a single address. What is that? What's that? So that's the first instruction that the first thread will execute when this process starts. So one of the last steps, when I set up everything, I copy all the contents in to memory for the process. And then one of the last things I do is create a thread and start it in that exact location. So that's the first thread that's gonna start to execute when the program runs. So been true, as you remember, is pretty simple, but it still has eight program headers. So this is, oh man, oh boy, I have to kind of remember. Okay, so, and this kind of, remember when we looked at PMAP? So PMAP was telling us about the address space of the process that was executing. This is telling us about the information in the ELF file. So for example, the ELF file is not gonna contain dynamic things that are allocated at runtime, like dynamic structures or whatever stacks or things like that. The ELF contains the stuff that gets loaded in when the process is initialized. So as you have here, these are program headers. You notice that these are marked as readable and executable, so that's program code. And you'll see that the ELF file indicates the offset. So this is the offset in the file, right? So there's a little bit of header in the file and then that says, starting from this address, I want you to load this much data and I want you to put it right here. I don't know exactly why there's both a virtual and physical address here. This may be for systems that do weird things with memory, but you can think of this as being, this is where I want you to put, this is where I want you to put the contents of the file that start from here with this size. So it says, take this much stuff from the file and put it in my memory right here. You'll notice that the entry point for the file is inside this region. So it's actually a little bit above where this region starts, right? And so that's inside the code section. Okay, so this has a bunch of other information about sort of various things that I want to load. Now, this is interesting. So this says, requesting program interpreter. So the L file actually tells the system how it wants its loadable libraries to be loaded. And when I was putting these slides together, I went ahead, I thought that was kind of interesting and I tried to run it. So here's what happens when you try to run that. It's kind of cool. What I love is that it spews this long error message that concludes with, you probably did not mean to run this program, right? Cause this is a program that's used by the system to load shared libraries on behalf of a program, right? And again, unless we were gonna spend a week talking about linking to loaders, I just thought this was funny, right? I did intend to run this program, actually. And it was nice that it printed such a helpful error message. Okay, so one of the things, so the process calling exec, one of the things that it provides is the path to this binary that it wants. But exec also allows the parent to pass arguments to the, sorry, keep saying parent. It allows the process that calls exec to pass arguments to the new process it's about to become. So when I call exec, like if I was gonna run a program that had some arguments, I can provide arguments to that program. And the tricky thing is the kernel, and this will be you, your job in assignment two, has to get those arguments out of the process that called exec, save them somewhere, completely replace the address space of the process that called exec with the new blueprint that I got from the Elf file, and then load those saved arguments in. And I'll have a slide about this in a minute, but you will think about this quite a bit when you do assignment two. So if you've ever wondered like where does, where do the arguments to main come from? Has anyone ever wondered that? Never, just outer space, right? I mean, main receives arguments, they come from somewhere, if main didn't receive arguments that came from outside the program, the world would be pretty bored, right? You'd have to run, you'd have to create lots of different versions of every program with all the arguments sort of built into them. So that would be bad. But this is where main, this is where they come from. And there's a little bit of extra work that C-Library does to set things up for main, but essentially it's the kernel that provides arguments to main when main starts to run. And then exec is kind of an interesting dual of fork. Remember fork returns twice, exec does not return. There's nowhere to return to. The code that called exec is gone. It's history, it's been replaced by this new process. The new process has no idea what just happened. It's like, just woke up, here I am, here are some arguments that came from somewhere, outer space apparently. And I'm gonna start running, that's what happens. Okay, so here's my fancy diagram showing how that happens. Imagine this is the caller of exec and it's trying to run bin true. And imagine this is bash, this is my shell. And I'll come to this in a minute. We're gonna actually build a little very, very simple shell in class through a combination of fork and second week. So imagine that my shell, you asked it to run bin true. So actually what would have happened is my shell would have forked a copy of itself. That copy is what's gonna run bin true. Otherwise if I ran bin true directly from my shell what would happen? Let's say that the way your shell worked is it took the arguments that you typed in and it just called exec. What would happen? There'd be no shell anymore, right? Shell is gone. The bin true is like, well I'm not gonna draw a prompt, I'm just gonna exit and then you'd be sitting there and there'd be a terminal in front of you not doing anything. So that would be sad. But the copy that the shell has forked of itself calls bin true. So the first thing I need to do is get these arguments that it wants to pass bin true out of the caller into the kernel. And the reason is because I'm about to completely destroy the caller's address space. Using the address space I've created from this blueprint that I found in the L file. Now bin true doesn't take arguments but just bear with me. I mean it probably, actually bin true probably happily takes arguments, it just doesn't do anything with them. So then I replace the address space of the caller and I copy the arguments into the new process in a place where the process is going to expect them. So this is really critical. Because when the process starts running, what you need to guarantee is that argv actually points to the argument array that was provided by the caller. And then I'm back and I'm running bin true and I hit the entry point and off I go. Any questions about this? Yeah, Ron. Of course you're getting a copy of the file table in the new process in the previous one. Yep, I can. Wait, when you run exactly, the arguments to be passed over presumably they already exist, why not just not delete them when you clear out the rest of the file table? Yes, so okay, so the arguments don't really have anything to do with the file table. The arguments are just in memory, right? But you're right, I mean the question is, I've already got these arguments in the caller, somewhere in caller's memory. Why not just leave them there? And you know, I mean you could try to do that. The problem is the new program may have, so there are a couple of places, the problems with that. One is that those arguments could be on the heap, for example, the new process has no heap yet. It hasn't started to run, right? So that would be interesting. And the arguments are probably either on my heap or on my stack, in which case I can't really leave them where they are, right? But you're right, I mean if they were in a memory location that was okay for the new process to use, I could just make sure that argument be pointed to them, right, but that's not what's typically done. It's a good point. Any other questions about exec? Okay, so now again by convention here, exec and there's a gazillion options to exec, but you can have exec run without changing the file table. And this is intentional so that I can go back to what we did with fork and pipes. Remember fork did all this work to preserve the file table so that the child and parent could communicate with each other. If exec undid all that, then that would be bad. So it turns out the default behavior would be exec and you should remember this for assignment too because people get super confused about this, write this down. Exec does not do anything to the file table, nothing. Leave it alone, there's no need to do anything to it. The new process has the same files open as the process that called exec. Now again there are exact variants that allow you to do all sorts of things like re-initialize the file table, whatever, but the exec that we're asking you guys to write does not have to do anything to the file. Okay, so let's go back to this simple shell or let's produce a simple shell. And this is, I'm not pretending this will run. You can look, so there is, if you try to read the code for bash, I suspect that you will get sad because I think it's complicated. I've never looked at it myself, but I can imagine that bash is a fairly mature shell is probably fairly complicated by now. However, there is a simple shell that you can read the code for, where is it? It's in your OS 161 distribution. Even that simple shell is, David added some features to it so it supports things like backgrounding now. I don't know why, but it's kind of nice. So, but this is, and this is way simpler than that, but fundamentally the core loop of a shell consists of just a couple of commands. I get some input, so I would probably draw the prompt and then you would type something. Then what I'm going to do is I'm gonna call fork because remember, if I just exec that input, I'm gone. There's no shell no more. So what I wanna do is I wanna create a copy of myself and if I'm the child, I'm going to call exec and try to run the command that you gave me. So this is almost what we need for a shell. Any questions about this? Just a couple of things that we're missing here though, but we'll come back to that, okay? So the hardest part about exec is making sure that exec can fail and the exec can fail correctly. So for example, if I get, let's say, you can think about exec as like, you say, here's a blueprint for a new house. I want you to replace my old house. So I'm gonna go away for a week and when I come back, I expect there to be a new house. Now, you might not be able to build the house I gave you. There might be a problem, but what I don't want to come back to is an empty hole in the ground because you said, well, I got done tearing down your old house but I couldn't build the new one, right? So when I come back, I either want there to be a new house or the old house that I left there and exec pretty much has the same challenge. Of course, anyway, I mean, exec has memory and it's a computer so it's not quite the, the metaphor is not quite perfect. But what I want to make sure is that if exec fails at any point for any reason, I can return to the caller and that's the one case where exec will return, and success, exec does not return on failure, it returns to the caller. Okay, so now we've seen how processes are born and how they change and the last component of this is exit, death. How do I get rid of a process? And processes signal to the kernel that they are finished using another system call called exit, pretty simple. The only thing that's interesting about exit is that a process gets to, and this is quite useful, it's a very small amount of information but it's quite useful, a process gets to report something about how it died. Did it die of a natural cause or did it die of some violent death because you gave it terrible arguments or something went wrong? And the only information that gets reported by exit is this one value, just a return code or an exit code and those exit codes, the typical semantics of exit codes that are very well established at least on Unix-like systems are that zero means what? Zero means I died a natural death and everything was fine. Anything other than zero means that something bad happened or interesting and how that maps on to the actual meaning is totally depends on the program. So you can certainly run mad to find out exactly what the exit code returned by your program means. So the question is where does this exit code go? I call exit, I tell exit allows the process to tell the kernel why it died who does the kernel tell? Nobody just says, oh, that's too bad. I'm sorry, I'll just get rid of that exit code. Who does the kernel tell? Or who is allowed to find out? So remember we talked about the parent and child having a special relationship. And one aspect of that is that the parent process is guaranteed to be able to find out why one of its child, or to find its child exit. Try that again. The parent is guaranteed to be able to receive the exit code generated by its child. Now there's a bunch of other processes on the system and some of them may or may not be interested in why a particular process executed and that's fine. And they may or may not be able to find out why that process executed, but there is exit. But there's one process that's guaranteed to be able to find out and that's the parent. And how the parent finds out is the last piece of this and that's wait. So this is like, you know, has anyone ever been a Lilydale? Does anyone even know about Lilydale? So there's a spirit, you guys are gonna think I'm nuts. There's a spiritual community that's like, I don't know, maybe half an hour from here. So it's a town, it's a very cute town actually. There's all these cute little buildings, but it's full of medians. So there's like 50 people there who for a small fee will help you communicate with your dead ancestors or whoever you want to, right? If you haven't been there while you're in Buffalo, I would encourage you to visit, it's pretty cool. So Mike Buckley, he used to teach in this department, he insisted that it was very funny. For years he was telling my wife and I about this, oh we've gotta go to Lilydale, you guys are really gonna like it. And then we got there and he was totally freaked out and wanted to leave right away. So it was really strange. And it was pretty creepy I have to say. But go there. So this is kind of how the parent can get in touch with the child through the afterlife, right? So the child is dead, the child is gone. Everything about the child has been cleaned up. Everything, the address space is gone, the file tables are gone, whatever. There's just one little piece of information that's left there kind of swirling in the ether. And that's this exocode. So when a process calls exit, the kernel maintains the exocode until the parent calls wait. Or in certain cases until the parent exits, in which case I figure if the parent exits without calling wait, then the parent is no longer interested in its child. That happens, it's sad, but okay. So the parent retrieves this exocode by calling wait. Now remember, what did fork return to the parent? The process ID of the child. So this is where this gets used. Because a parent can have a bunch of children, right? I mean, if you, you can do this with your shell. You can start off 10 different things in the background. All of those processes are children of the shell. So when I call wait, I need to be able to identify my child. And the way I do that is by using its process ID. So when I call wait, what'll happen is by default, wait will actually do what it indicates. It'll wait for the child to die. Now we're getting morbid. And then return the exocode that was returned by the child. If the child has already died, already exited, then it will return immediately and it will return the exocode. Now, what's one interesting consequence of this? So remember I have these process IDs. And process IDs, you know, you might have thought, well, what are the semantics of a process ID? Clearly while a process is running, its IDs shouldn't change. But can I remove a process's ID and hand out the same ID to a new process immediately after it exits? Why not? Someone wanna venture a guess? Yeah. Yeah, so it turns out I actually have to wait until its exit code is retrieved. And the reason is, let's say that I'm a parent and I create child four. And then the kernel, child four exits. And the kernel says, you know what? I like child four. I have fond memories of child four. I'm gonna create a new process for. And that process is the child of the same parent. So now when the parent goes to call wait, it says wait for. Actually I should have choose five or something. Anyway, so it calls wait for. And the kernel goes, I have no idea what you want. Because I have exit codes for two processes four here, which one do you want? So I can't reclaim process IDs until the exit code is gone. Because otherwise the parent can't uniquely identify the child. So these two system calls I'm almost done are typically sort of considered together because when you combine them together, they remove all traces of the process from the system, executes sort of most of the stuff, and wait does the rest, okay? Okay, I'll finish this next time. So I'll see you guys on Friday. Good luck with assignment one.