 Good morning, everybody. All right, let's do the Wednesday thing, where everybody stands up. And people have started to realize that I'm actually serious about halfway, well, not quite halfway through the week, but we're getting close. All right, so today we're going to talk about, we're going to finish talking about context switches, the actual mechanism that we use to switch between threads, so that we can produce the illusion of concurrency. Next, we're going to talk about threads, right? We talked a little bit about this at the beginning of class. We said a thread is one of the abstractions that the operating system creates that allows processes to use the CPU more efficiently. So we'll go back to that abstraction. We'll talk about what is a thread and what pieces of state are associated with a thread. We'll briefly discuss differences between threading implementations, right? So we'll try to justify why would you write a multi-threaded program? We've talked a little bit about this before, but then we'll discuss different ways of implementing user space threats. So historically, user space threads have been implemented both in user space libraries, and we'll show you a little bit about how that's possible, and also in the kernel itself. And there are pluses and minuses to either one of those. And then finally, we'll talk about the kernel's view of threads, different thread transitions that go on within the kernel as the kernel moves threads back and forth on and off the CPU. So the AutoGrader is alive again, and it will start grading some of your scripts today. And once that happens, then we'll be able to grade your scripts and assignment zero implementations. And then pretty much at that point, everything else should just start to work again. So you guys should be able to submit. For assignments one, two, and three, there aren't any scripts to submit. It's just implementations and code reading questions. And then for assignments two and three, there are design documents that need to be submitted. Those are clearly not graded by the AutoGrader. I wish we could, but those are graded by Humanize. So those will be graded by the staff. All right. Any questions about assignment logistics? I know some people have been having struggles saving answers, Sarah. Yeah, I saw that. OK, yeah. That sucks. I'm sorry about that. I'll. Oh, interesting. OK, well, OK. So saving should post something to the server. If it does it, then it's not saving. So I do think it has something to do with the JavaScript. And if you guys, are you using Chrome? Yeah, it's a VM Firefox. We have the same problem. Yeah, Jeremy. Yeah, OK. So there are also some issues with resubmissions that we'll try to fix. But the saving thing I will look at, and apparently this is broken in the VM. So that gives me a good place to start with reproducing it. But I'm sorry about these sorts of issues. But if you can get them submitted and into the system, then we will grade them. And maybe as a backup, it would be nice to keep them at a text file somewhere, so you don't have to. I know it's a pain to put them back and forth into the browser. But I will definitely look at the saving issue today. All right, so let's review. Well, first of all, do we have any questions about the material that we covered Monday and Friday? So Monday we talked about context switching. We gave you a historical overview of limitations with the CPU and how operating systems have tried to address those. So any questions about that material before we do a little review? OK. So when I transition between two threads, what do I call that, Robert? Context switch, all right? That's what it is. So what were some of the problems with the CPU? We talked about limitations with the CPU that the operating system is trying to correct or overcome or improve upon? Manish? What's one of them? Time? That might be a move in the right direction. Let's see, Kevin. Sure. Jim? Well, OK, so that's a good point. So how would you express that problem? The problem is that what? OK, but what creates this issue of usage in the first place? Yeah, Tom? Yeah, so one problem is that the CPU is much, much, much faster than other parts of the system. And part of what we do to try to use the CPU efficiently is identify delays in other parts of the system and try to hide those delays from the process. There's no point having a process sitting there, a thread sitting there waiting for the CPU to do something when I could use that time more efficiently to let something else run. So while I'm waiting for something slow to happen, I might as well grab the CPU and give it to something else. If I didn't grab the CPU and give it to something else, what would I be doing? We talked about this as a certain type of scheduling approach. What is the kind of scheduling where I don't switch between threads? Just give the one process the whole machine and let it run until it's done? Yeah, Sarah? Yeah, we call that batch scheduling. So that meant I give the entire machine to one process, one application, and just let it run until completion. And one of the problems with that was the mismatch between the speed of the CPU and the speed of other components on the system. But what's the other, I would argue, even more fundamental limitation with the CPU that we're trying to address? Is it Satish? Yeah, basic, basic, basics. Yeah? There's only one CPU. There's only one, right? Or some small number, right? Maybe not one anymore, maybe four or eight, but there's a smaller number of CPUs than there are available things that the system is trying to do. And again, the CPU is also far faster than other parts of the system. So these are limitations. To some degree, the second one is an opportunity, as well as a limitation. It gives us a chance to use those delays to do other things. We just talked about this. What is batch scheduling? Anybody? This is a remembering what I said two minutes ago question, Wembley. Yeah, I allow a job or a processor and applications to just essentially run with full access to the entire machine until it completes. And this was a form of scheduling that was done on early computer systems to allow them to do multiple things, but without requiring that I do any more fine-grained resource sharing. So I just give you the entire machine, I run your thing until it's done, it produces some output, and then it gives the machine to somebody else. And what's the problem with this? I wonder who came in late. What's your name? Yeah? What's your name? Ravi. So what's the problem with batch scheduling? Why wouldn't I just hand over the machine to your process and let it run until it's finished? Well, but that's OK anyway. Your job will probably run faster if it has access to the entire machine. Help him out. Harish? Why is it item? That's true. Yashas? Yeah. Why is the machine idle most of the time if I'm doing batch schedule? And what parts of the machine are idle most of the time? I liked that answer. And then you took it in a bad direction. The CPU is idle most of the time. While the disk is fetching data, while the network is moving data around, while even to some degree, while memory is loading and storing information, the CPU is idle. And so this is the reason that every slow device, anything that takes longer than the CPU and everything takes longer than the CPU, will cause the CPU to stall and just sit there. And this is, again, this is a problem with batch scheduling, but it also creates the opportunity to do things more intelligently. So I have a uniprocessor system. One processor, how does the operating system create this illusion of concurrency? Is it? What's your name? Mike. How do I do this? I have one processor. I want to look like the machine is doing multiple things at once. Yeah, I'm going to rapidly switch back and forth between different processes, so it looks to the user as if things are happening simultaneously. And what do I exploit here to do this? Does anybody remember? What fundamental limitation of a computer system is the CPU able to exploit by doing this? Whose wait time? Yeah, user, your limitations. If suddenly human beings, if suddenly we all took some sort of pill, maybe some of you guys take these pills already, but if suddenly we all took a pill that caused us to be able to detect, what is it? I think dogs, dogs can actually see much higher frame rates that humans can, which is, I guess, one reason why some dogs aren't interested in TV, because it just looks like a series of pictures to them. It doesn't look like something moving. It would look if I showed you a very slow flip book. So if suddenly we all took this pill and you guys were able to see things much, much, your reaction taps are much faster, then we'd have to redesign computer systems. We'd have to change all these scheduling quota, because you would notice. You would see Firefox repainting the screen. Luckily for us and for the computers, that pill doesn't exist, or at least it's not legal. And people aren't taking it that off. So yeah, what we're doing is we're exploiting. Uh-oh, it's happened again. One of these days, I'm just going to see if I can power through this slide without fixing it. But clearly it needs to be fixed. Sorry about this. OK. So now finally, how does the operating system make sure it stays in control? Now we talked about this problem where if a process is running, I've given it control of the CPU. And if for some reason there's no hardware activity and the process doesn't explicitly request help from the operating system by making a system call, it could potentially maintain control of the CPU indefinitely. So how do I make sure that that doesn't happen? Yeah, definitely. Yeah, I use an interrupt. So that's the mechanism. But how do I make sure that periodically I get this chance? Yeah. Yeah, I use a timer. It's Akshay. Yeah, so I use a timer interrupt. So every system, every modern system that implements this type of, the name is escaping me, but any system where the kernel is going to periodically rip the CPU away from one thread and give it to another, has to have a mechanism to do this. And it's all done by using these timer and episode are generated by hardware. So I just attach a timer to my CPU and I just run it. And every time that timer fires, I enter the operating system and I look and I say, do I want to keep running this thread or should I start something else? OK. Any questions about this sort of stuff before we keep going? Yeah, I'll show you. Is there a way of measuring the context of switching timing within two threads? Measuring what about the context? Sure. Yeah, absolutely. There's overhead to this. We're about to talk about that. It's a good segue. Any other questions? So as Anshil pointed out there is, OK, we just talked about this. Where? Hold on. We're going to come back to that in a second. So remember, providing the solution concurrency means that I have to stop threads. And if I am not willing to allow threads to help me figure out when to stop them, then I'm essentially going to stop them roughly at any point. So at any point your thread can stop running. Another thread might run. And when your thread starts running, some things will be the same and other things will be different. We'll talk today about what the operating system will guarantee will be the same and what things can potentially change. But one way of thinking about it is the processor state should look the same. In between instructions, the thread should not be able to tell that it was stopped, another thread was started, ran for a while, and when the thread starts running again, it shouldn't be able to notice. I mean, it might notice that if it was sitting there grabbing the real time, it might be able to say, OK, a little bit of time, little bit of time, little bit of time, oh, big, big gap of time. Clearly, I was stopped and somebody else ran. But it shouldn't be able to tell by looking at the CPU state. On the other hand, the memory state that it's exposed to can change quite a bit. And that creates a lot of the synchronization problems that you guys are going to be hopefully finding and fixing and thinking about when you're writing your operating system. And certainly, things you have to think about when you write multi-threaded user programs, right? That's certainly as important as not more important in the operating system. Yeah, auction. Is the time fixed for the time? Yeah, Jeremy asked a question about this last time after class. And normally, it's a fixed rate timer. That time frame doesn't complete the execution that you wanted to do. Sure. Switch back to the other process. Get back to date. Yeah, we'll talk about that today. But the idea is, if I'm in the middle of doing something, then I'm in the middle of doing it when I come back. Firefox may be trying to repaint, or trying to send a network packet or something. And it's in the middle of constructing a request, and it gets context-pitched. And when it comes back, it just keeps going. Hopefully, and it doesn't, I don't get much control over this as a user program. This is what we're talking about. At any point, I can be stopped, other stuff can happen, and I can start to run it. And if I'm in the middle of doing something, I just go back to it. As a programmer, you're not really exposed to this very much. You write your sequential code. In reality, your program is constantly being stopped and started. But because of the context switch mechanism we're talking about today, your threads don't notice. Unless they're trying to share state and other things are happening, we'll talk more about that quite a bit in the next few lectures. Jim. OK, this is a good question. A little bit of a review. So I set up this hardware interrupt to fire periodically. What happens when the interrupt fires? Three things. What happens when the interrupt fires? This will definitely be on an exam. We've covered it like six times now. Yeah. It goes into kernel mode. It goes to CPU, goes into privileged mode, right? What's the second thing? Sheldra. Three things. Jenny, you want to help? What's that? Yeah, it records some state about what happened. And then what else? Yeah. Yeah, well, the operating system gets control, but how does it happen, Jeremy? Yeah, I jump into my interrupt service routine. So what happens is every time that interrupt fires, this happens. I'm going to show you a little bit of this today. How many people have been in recitation this week? Then you saw a little bit of this as well. You saw some of this happening on your own system. You saw the interrupt service routines. You saw exactly what happens. I'm going to show you a little bit of that code again. So when we talk about thread state, again, a thread is this abstraction that is designed to encompass the state that the processor was in. So what and some other state that's associated with the thread. So when you think about a thread and the state that I would have to save, so again, I'm going to stop one thread. I want to preserve as much information about what that thread was doing. So that I can restart it, what do I need to save? Bam, timer interrupt happens. I'm in the interrupt service routine. What information about the processor I need to save immediately so that I can restart it? Yeah, it doesn't. Yeah, so I need to save. Who knows what this is called? How would I save this information? Yeah, Robert. Yeah, the program counter. I need to save the program counter. Program counter is a special case of another larger class of things that I need to save all of, Satish. I need to save the registers. Like, that's what the CPU is. The CPU is a bunch of registers and instructions that allow me to manipulate those registers and potentially memory. So if I'm going to put everything back, I mean, think about it. This is like, I hate using this house analogy over and over again, but this is like, someone came into your house and they were going to clear it out so Beyonce could come in there and stay. And they are going to sort out the M&M issues. But they also need to make sure that when you come back, they want you to not notice at all that she's been there. So they need to carefully look at everything in your house exactly where it was. Take detailed measurements so that when they bring all your stuff back, they can put it back in the exact same spot. The goal is that the thread has no idea what has just happened to it. Registers, and then what else? What's the other? So the registers encompass the CPU state, and then what's with one other piece of information? What else do threads? Where do threads store sort of local state that's not in registers? The page table is actually not a per thread, so that's actually shared between threads. But you're getting closer. Yeah, the stack. So in C and other languages, threads have a private memory area called a stack that they use to store local variables and other things that aren't stored in registers. So when I stop a thread, I need to save two things. I need to make sure that I save the registers, and I need to preserve the stack. OK? Essentially, what happens in the kernel is that we rely on memory protection to keep the stack unchanged. We do need, however, we do need to make sure that we create a new stack. We adjust the stack pointer properly when we switch threads back and forth, because otherwise threads would be using each other stacks. But basically, the way that we preserve the stack is we just make sure that whatever happens next doesn't use the same memory, and we rely on the memory protection mechanisms to keep the stack unchanged. So here's another good question. How does a thread know where its stack is? Yeah, Amit. There's a stack pointer, and the stack pointer is a what? Tim? He's looking confused. It's a register, right? So when I save the register state, oh, I got both of you guys. When I save the register state, and you guys both had the right answer. When I save the register state, I'm also saving the stack pointer. So I will also record the exact point at which the thread had was using its stack. And when I restart that thread, assuming that I can keep that memory from being altered, the stack contents will be unchanged. So if you remember, when we enter the kernel in OS 161, I'll show you a little bit of code here that Aditya has already looked at. The thread state is the first thing that we save. Maybe not exactly. I mean, it's like 10 instructions later. We go into this big block of code that's saving all of the thread state. Why is that? Why would that have to be the first thing that we would do when we enter the kernel? I don't know. Yeah, but it's not just a stack or memory, right? The instructions that I execute on the processor are modifying the registers on the processor, right? So as I'm running, if I don't quickly make sure I save all this state, I'm going to start altering it, right? And then I won't know what it was, right? So for example, I need to know exactly what value the process had in register A3. Because if I don't, then eventually the compiler will have outputted some instructions that use A3 for something else, and then that register will be gone, right? If you look in OS 161, what we call this saved register state is a trap, right? And the first thing that the kernel does or one of the very, very first things, right, is it allocates space for the trap frame, and it stores all the registers into that structure, right? So there's a trapframe.h, I think it's called trapframe.h. It's somewhere in the machine dependent code. And one of the first things we do, again, is just we enter this block of code, which goes on and on. And again, there's a big block there with maybe 20 or 30 of these instructions. And what this is doing, this is saving all of these. This is saying, take the value in T9 and store it in this memory address, right? And what we've done is we've allocated space on the stack for my trapframe. So what's wrong with this, right? What do I need to make sure of before I can start saving the values of the registers on the stack? Tim, I wouldn't want them to be cleared, right? Because I want to know the exact state that the process had them in, a grim. Remember, there were a couple of things we needed to preserve about the state of the process, right? Ravi. OK, but interrupts are off here, right? So this is a nice big comment, right? One of the things you guys should appreciate, just as an aside about the OS 161 source code, is that all this is for your benefit, right? So when you guys are looking and browsing through here, all of these long comments, you'll never find this stuff in Linux, right? In Linux, there'll just be thousands of assembly instructions, and it's like, good luck trying to figure out what they do, right? Because they're written by someone who is really smart and who looks at that stuff and just writes assembly and is asleep, right? And the reason this stuff is so well-commented is so you guys can understand, right? So it's really kind of a nice thing that David has done here along with this code, right? Again, the state for the thread consisted of registers and what else? Daniel. Stack. So I'm saying I'm storing the registers on the stack. What's wrong with that? If I'm still using the same stack, then I'm potentially modifying the stack that the process was using, right? Why is that okay here? There's an answer in the comment. Yeah, because I've already saved the old stack pointer, right? So up here in code that I didn't show you, I already saved the stack pointer that the process was using, and what I've done is I've repointed it to a new kernel stack that's a page of kernel memory that the kernel uses as a stack when it's executed, right? So even before I start saving this trap frame, what I've done is I've switched stacks, right? So I said I'm not gonna use this old stack, I'm gonna use this new stack over here, and now I can save my registers into a data structure on that stack without worrying about modifying the stack that was there, right? And then again, here's this trap frame, right? This is the trapframe.hStructure, and you can see it has space for all the contents of all the registers, right? And yeah, I just answered this question, but there's two issues here, right? The first thing is I don't wanna pollute the user stack, but when we start talking about memory management, I also don't trust the user stack, right? I don't believe that the user stack points to memory that is potentially not bogus, right? And so I don't wanna touch it, right? Because it's possible that the user program is trying to mess with me and it loaded some bogus value into its stack pointer register right before it started, right before it made a system call and it's trying to give me to overwrite that memory and to do something nasty, right? So I don't believe this fact, right? All right, so now I've entered the kernel, I've saved the state associated with the thread, I've done whatever it is I'm going to do, right? Maybe I've just started to start a new thread, maybe I'm returning to the old thread. What do I need to do? Oh God, I can't abandon that thing. Wait over there. What do I need to do when I return, right? So again, I've gone, the people came into your house, they made detailed measurements, they took photographs, they recorded everything, we're at the location of every piece of lint, every piece of spare change, right? And then they put it all into this big box and they shipped the box away, right? Now I'm about to send you back to your house, what do I need to do? Yeah. What's your name? John. Yeah, so what I need to do is I need to reload all those registers and take this trap frame that I saved and reload all the registers, right? And then what's the last thing? I don't know if I did you, it got you all the way through the return, right? But the last thing that happens, right? On that exit path out of the kernel is I call this special MIPS instruction called RFE, right? And RFE is the dual of syscall, right? RFE kind of undoes the exception state that syscall created, right? And RFE will essentially put you back into user mode and allow the process to continue running as well, right? But it's really important, so, and Adich is doing a great job of this in recitation, but as we go on for the next few weeks and we start talking about this gooey stuff, I don't do too many examples like this in lecture because I like to leave this for recitation, but it's really important you guys go to recitation and allow Adich to walk through this stuff, but it's also important to remember that, like, you guys have access to all this code, right? So this doesn't have to stay abstract, right? If you're confused about how something works, go read about it in the OS 161 source code, right? At some level, like, that is the text for this class, right? And you guys can see exactly, right? What happens when I handle an exception, every instruction that's in there and the different paths for interrupts and system calls and all sorts of things, right? And again, the code is not only there, but it's very nicely set up and it's commented in a way that's designed to aid your understanding, okay? All right, so, somebody, Anshil, I think, brought this up before, yeah? So the... Yeah. Yeah, yeah, yeah, this saved state, right? So remember, I had this on my stack somewhere, right? And this structure called a trap frame, right? And on my way out, I still have this kernel stack that I set up, right? That I'm using as scratch base. So on my way out, I reload all the registers from the kernel stack and then when the last things I do is I have to switch the stack point, right? So I get the registers all back and then when the last instructions I do is I switch the stack pointer back to the user stack and then I call this RFE instruction, right? And the RFE will put me back into user mode. So it lowers the privilege. So, and the mode, say RFE is not called for some time. Well, I mean, RFE has to be called, right? So there's, yeah. Like, we are still in the kernel mode, right? Yep. But I'm not, so what I said before is I don't trust the user memory, right? But I've never touched user memory, right? I've loaded the registers from the CPU into kernel memory that I trust and then I've loaded them out of kernel memory and back into the registers, right? Then I do point the stack pointer back at a user address, right? But I don't do any reads or writes to it, right? It's like one of the very, very last things that happens is I do that and then I jump back to whatever, I have to set the PC back to what, sorry, the program counter, back to what the program counter was that transfers instruction back to wherever the user space code was actually, right? Yeah, great, get a question. Yeah, yeah, exactly. And you guys will see places, if you walk through the system called code, the system called handler is actually take values out of the trapper, right? So for example, the arguments of system calls are passed in specific registers, right? So what happens is when you make a system call, the trap frame saves all that register state and then your system calls will actually process it, right? And in the case of a system call, you'll modify the registers, right? Because that's how you return about, right? So the calling convention is that, you know, you pass arguments and certain registers and I think on the MIPS it's A3 is the register that holds the value of the system call return, right? So your system call will set that value to indicate what happened, right? So usually zero if everything was okay, right? And then there's different values indicating different types of error. And then, so when you come back, so again, I mean, this is a good point, right? So there's a difference when it comes to system calls versus context, if there's a context switch, right? A system call will send you down the same path, but the system call handling code will modify the trap frame, right? And it will modify the user's view of the world, but that's because the user asked it to, right? The user said, hey, do something for me. And I did it, and of course it's world changed, right? Because it needed to. When I do a context switch between threads, right? When I stop a thread that's running, that didn't make a system call, right? Or I process a hardware interrupt, I need to set up that thread so it looks like nothing happened, right? So there's a small difference between a system call where the thread expects something different to happen, right? Because it asks me to do something on its behalf and, you know, a timer interrupt or some sort of other kind of hardware interrupt where the thread was just going along, minding its own business, didn't ask me to do anything and then suddenly this happened, right? So there is a difference in the semantics between those two cases, yeah? So what you'll see is that usually when you're processing some of these interrupts, you disable the timer, right? Because in that case, you know, if I'm already in the interrupt handler and I'm already processing, then I don't necessarily want to handle timers, right? And there's cases, you'll see cases in the kernel where I disable interrupts and that's usually because I'm doing something and I don't want to be bothered by the time, right? Why did interrupt handlers that are thread safe is difficult, right? So normally interrupt handlers in certain places want to make sure that nobody will interrupt them, right? And the kernel can do that by changing the interrupt mask on the processor, right? Processes can't do that, right? So a process can't stop the timer from firing, right? But when I'm running in kernel mode, I can't, right? And sometimes I do because I want to make sure that I'm not stopped, right? That's a good question. All right, so let's see here. So as Anshiel pointed out, these are great questions, right? As Anshiel pointed out, there is this overhead, right? To getting in and out of the kernel, right? To context switching to interrupt handling to making system calls, right? And that overhead consists of, you know, basically saving all the state, right? Saving all the state of the trap frame, right? Running in the kernel, doing something, right? Whether it's if I hadn't have to handle the interrupt, maybe I have to look around and figure out what to do, maybe I have to read some of the data from a device or whatever. And then transitioning back to the thread that was running, and this overhead limits the rate at which we want to perform these context switches, right? So, again, there's this nice balance here, right? If I wait too long between context switches, then I start to violate these user perceptual limits, right? Then you start to notice, right? Then the computer starts flickering at it, right? If I go too fast, right? Then what happens is the amount of time I spend in the kernel figuring out what to do starts to dominate and I don't get as much useful work done, right? So modern systems, you know, choose some scheduling quantum that is a balance between these two desires, right? And depending on the type of system you're running, you know, maybe if you're running a server workload, you might want a longer scheduling quantum because there's not this interactive response necessary, right? You want threads to be able to do more work before they're stopped and I have to think about what to do again, right? So again, too fast, I spend too much time in the kernel figuring out what to do, saving state, restoring state, right? So for example, if I take a hard, if I set my hardware timer to fire really fast, right? And I don't have very many threads on the CPU, what might happen is the hardware timer fires, I enter the kernel, I run my scheduler and I decided to just keep running the same thread that was already running, right? And then I have to exit the kernel, restore the thread state and let it go, right? So now what I've done is I've taken this completely spurious context switch, right? Accomplished nothing, except give me a chance to think about what I wanted to be doing and it's a waste, right? So all those instructions are wasted, right? All I did was put the machine back exactly the way I found it and let the thread keep running, right? And if I do that too often, then I'm wasting a lot of time and things slow down. Yeah, AJ, I don't think so. This is like usually like a kernel compile option and Jeremy was asking about, you know, would you want cases where this could be dynamic, right? So I could have dynamic and I think people have looked into this and they've just found that it doesn't help that much, right? So normally there's some value that's a good balance and tuning in at runtime is, you know, kernel designers and you guys maybe will pick up some of this as you start to do some of the harder assignments in this class, kernel designers have this fetish or preference for simplicity, right? Particularly, you know, if you tell someone who's hacking on Linux, I can improve performance by, you know, 2% in exchange for this huge mass of new code and new complexity they will always say no way. Like no way, it doesn't matter, right? 2% in exchange for this huge pile of code that has to run all the time but I don't really know what it does and it might be buggy. They just don't care, right? Like if you have a simple solution that improves performance, great, right? Simple changes, simple fixes but all these little tunings that don't actually end up doing much good but introduce a great deal of extra complexity are typically just discarded and filtered out by people who do this for a living because they don't want to maintain that stuff, right? They don't want to think about, oh, well, let's see here my timer and up was normally firing at 10 milliseconds and now it's seven and what does that mean or whatever? They just, they don't care, right? They keep things, yeah. Question? Okay, yeah, it's the keep it simple, stupid sort of principle, right? All right, so now we have this idea of a threat abstraction, right, where you have registers and stack and I think I just, I want to stop here and kind of sort out what belongs in the threat abstraction and what belongs in a process, right? So registers are clearly this thread private state, right? Thread has registers and when I start and stop it's important to save that register state, right? Threads also have their own stack. So when we start talking about multi-threaded user programs, every thread in the process has its own stack, right? And those stacks have to live in memory somewhere and they're not shared by threads, right? Memory and the file descriptor table are shared by all the threads within the process, right? So this is kind of an important thing to keep in mind, right? Stack registers thread private, right? You know, you can share variables on your stack with other threads, but it's typically very dodgy, right? If you want to share variables, you know, malloc of variable and send an address to it to the other thread and that will be in the heap, right? Anything on your stack, who knows if it's still going to be there in a couple of seconds, right? Because you may have gotten rid of it. So registers and stack thread private, right? Typically thought of as private to each thread. The rest of the address space, right? So my heap and all the code and any other things that are in my address space are typically thought of as shared between threads, right? And this is where all the GUI problems start to come up, right? With some of the shared states, right? So the file descriptor table and memory are shared between threads in a process, right? Let's see what time it is. So we talked a little bit about this before, right? And part of what's going to make your life miserable for the next couple of weeks is you guys work on assignment one is going to be dealing with threads, right? So in a user space perspective, right? Let's not talk about the kernel for now. Let's just talk about you're writing a user space program. Why use threads, right? Or I should say, why use more than one thread, right? If you guys have written multi-threaded programs, why are they designed that way? Yeah, yeah, so it can be trying to accomplish multiple things at the same time. Of course, frequently, you know, let's say again we're talking about a uniprocessor system, we know the concurrency is an illusion, right? We know that. There's only one thing happening at once, right? So anybody, yeah. So there's some notion of multiple things, but it's not necessarily happening at a time, right? What do threads allow me to do maybe as I'm programming? Yeah, I agree. Yeah, okay, okay, so, right, okay, so that's a good point. So rather, let's say I wanted to run, I'm running a web server, and I have a bunch of different requests coming in, and I want them to be able to be handled in parallel, right? I could fork off a bunch of processes to do this work, right? So I could just take Apache, and I could start 50 Apache processes, right? But as the grimace pointed out, there's this overhead to switching between processes, right? I have to go into the kernel, I have to, you know, save all the state for the thread that was executing. It turns out when we switch between processes, there's memory state that also has to be updated, which we'll talk about in a few weeks. So there's this overhead between switching between multiple processes, and maybe it's lower if I'm switching between multiple threads maybe, Jeremy. Yeah, and yeah, so, so on some level, one of the reasons we use threads is conceptual, right? I mean, your program, again, your program might not actually ever be doing multiple things at the same time for real, but it's a nice way to think about it, right? So for example, you could write a web browser that had one huge like for loop, right? Actually wouldn't be a for loop, probably like one huge while true, right? And you could have, you know, 6,000 different K statements in there, right? Like if the user clicked the mouse, right? And if like a socket IO request completed and boom, boom, boom, boom, boom, like all the different things that could happen that would cause me to wanna change the state of my process, right? You can write things that way. Like you could write a web browser that way, and if you could wrap your mind around it, right? It might not work that badly, right? Especially if you were doing sort of asynchronous IO. But again, what's hard about that, right? Why do we write a web browser and have like one thread that just sits there waiting for mouse events, right? Looked at. This kinda comes back to what Jeremy said, right? So the thread that's sitting there waiting for mouse events may be blocked a lot, right? Cause there might be times when I'm not moving the mouse or I'm not clicking on things, right? While that thread is blocked conceptually, that part of my code is not executing, right? And the nice thing about writing in a separate thread is that I can write a little piece of code that just reads whatever the event was and then updates some state, right? And it doesn't have to do everything else, right? So in my big case statement, let's say the user opened a new tab, right? Well now, first of all, I have to process that mouse event and then I have to redraw the entire screen, right? So there's all this interdependency between these things and threads can be a nice way of trying to separate those out as a program. Yeah, right? Yeah, oh, yeah, yeah, yeah. Don't worry, we'll get back to that, right? Yeah, so threading introduces this gooey problem of synchronization that we're gonna spend quite a bit of time on. You guys will spend time on that as I'm at one, right? Yeah, and then again, threads may naturally, there's these natural separations of concerns within many applications that allow me to write them using multiple threads, right? Usually there's some data, for example, about a particular tab, right? Or request that I'm making or a particular page. I'm using web browsers as an example here, right? But similarly, for something like a web server, right? It's kind of like here's a request and then there's all this stuff that has to happen and to some degree, again, that stuff happens very independently of anything else that's happened, right? So I can give that to a thread and that thread could just take care of it and all the latencies are sort of sunk into that thread and while that thread is waiting for IO, some other thread is running, right? And this is the other thing, right? So slow devices in a multi-threaded program if one thread blocks, right? This goes back to the argument we made before about the CPU, right? In your Firefox browser, if the thread waiting for network IO blocks, the whole process doesn't stop running, right? It can still paint. It can still respond to your mouse clicks, right? Other things will happen. If you had a single-threaded process and you weren't careful about it, right, and you used blocking IO, every time anything blocked, the whole thing would just sit there, right? And that would produce a very, very ugly experience. Brian? What you said doesn't that depend on... Yeah, okay, so what I'm talking about here is user-level threading, right? And we will get to probably Friday differences between where that can happen, right? But it actually turns out that I can do user-level threading quite easily without any support from the kernel, right? But now many modern kernels have support natively for multiple user-level threads, and we'll talk about the differences in trade-offs between user-level threading libraries and kernel-level user-threading libraries, right? All right, so this is one of my favorite analogies, and I think this is a good one, for threads, right? If any of you guys have been to a nice restaurant, that there's like 30 cooks in the kitchen, right? And for any given dish or meal or thing that you order, many of them will be involved, right? Like, they have to cooperate, like, at any given point in time, right? They're all doing one thing, right? There's no real concurrency when you map all the things that have to be done down to the cooks in the kitchen, right? Unless they're like chopping with two hands or something, right, but that doesn't sound safe. So they're doing one thing at a time. They're frequently doing different things. So there's frequently a separation of concerns, right? I might have one person who's handling like one stage of a particular meal, right? Or one task, like chopping onions all day long or whatever, that sounds pretty miserable as well. They all have to cooperate, right? They're all sharing some recipe, right? And this to some degree is kind of like thread sharing the same code in an address space, right? Like, they're all sharing the same set of instructions. They have private state, right? So they have the stuff that's right in front of them, whatever they're thinking about, but it's very easy for them to communicate, right? And one of the differences between threads and processes are multiple threads in the same process can communicate very easily in shared memory, right? Whereas multiple processes have to communicate using these sort of gawky IPC mechanisms that we discussed at the beginning of the semester, right, which exist partly because processes don't trust each other, whereas multiple threads in the same process implicitly do trust each other, right? The operating system plays no role in defending multiple threads when this is in the same process from each other, right? It does play a role in defending multiple processes from each other, right? And then finally, these guys have to coordinate to get anything done, right? And that's the other big thing, right? All right, so I think this is a good place to stop. On Friday, we will talk about, you know, more multi-threaded applications and then discuss the pros and cons of user versus kernel level threading approaches. Hey.