 It's thinned out a bit today. All right, it's Friday. Everybody's excited about Friday? I'm excited about Friday. OK, so today we're going to talk about threads and how threads are implemented. And at the end, we'll talk a little bit about thread states. I just was noticing, I think that is my favorite feature of this room by far. The periodic chart of the elements that's up on the wall, can anybody read it? I mean, I can't read it from here. So I guess if you take a chemistry class, they give you some binoculars so that you can see it during lecture. But I mean, it's a nice idea. It just needs to be like eight times that big. All right, so as you guys probably know, the design doc and code reading questions for assignment two are due today at five. And the goal here, again, is that once you've done the design, and once you've done the code reading questions, you must be just itching to get on to building assignment two, implementing these system calls and getting your system to actually run real user programs, which is actually super exciting. On the other hand, I mean, the thing I always have to remind people is that, and the website tries to do this, but with a limited degree of success potentially, assignment two is a lot harder than assignment one. Like a lot harder. Once you get through assignment two, you have some idea, you can imagine at least what assignment three is like. Assignment three is harder than assignment two, but they're more similar to each other. So please, you guys have three weeks, I think. Is that true? Three weeks from today to complete the implementation? That's plenty of time to do this. However, it's not plenty of time if you spend 20 of those 21 days thinking, eh, my partner and I, we knocked out assignment one in an afternoon. We're cool. Then you guys will hit a big train wreck. So I'm giving you guys a lot of time, but I'm not giving you the time to just sort of hang out. Like please start working on assignment two. Look, if you start working on it and you get it done in a day, great, then you can chill out and start studying for the final or midterm. But please use some of this momentum to get started. At least sit down and try to write some of the file system system calls and get a sense for what you're in for. Because again, assignment is not easy. And it's a lot harder than assignment one. Any questions about logistics before we go on? All right. You look like you wanted to ask a question, but you didn't. Do you have a question? Or is your hand just in the question position? OK. All right. So let's talk a little bit about thread transition. So when I stop one thread and start another thread, what does that call? It's called a context switch. What were some of the CPU limitations that we were trying to address? We've built up all this machinery now. We can stop a thread at any point and restart another thread. Why are we doing this? What are the limitations of the CPU? And this is something we'll come back to. In every case, when we do a unit in this class, there's some limitations with hardware that the operating system is trying to address. We'll do that when we talk about memory. We'll do that when we talk about the disk. With the CPU, what are the problems with the CPU? What are the limitations with the CPU that the OS is trying to make go away? Why am I doing this in the first place? So there's only one processor or N processors. So I don't have very many because they're expensive. And it's also a lot faster than the rest of the system. So the first maybe is a limitation. I wouldn't call it speed a limitation, but the fact that the processor is so much faster than the rest of the system is what gives me the opportunity to play these games with how threads are scheduled. So I'm trying to keep the system as busy as possible. So what is batch scheduling? Let's say I have a series of things I need to do. How would a batch scheduling system process those tasks? What is batch scheduling? I can use my thing. Oh, no pictures, apparently. Alex here? No, Alex? How about Jason? I need a photo. I can't see the people. All right, batch scheduling. Mojit. Now I have pictures. I need to start using this to take attendance clearly. How about four? Ananda. Batch scheduling. You guys have a few minutes to think about it. Look for the notes. Yeah. Right, so I run a job or a task until it's done. And then I run something else. While it's running, it has full access to all the system resources. Nothing else is running at the same time. So I'm running things one at a time. This is not anything I'd ever want to do in an interactive system. It's not a bad idea, though. And this is done in some data centers for various reasons, particularly when I have jobs where no one's eyeballs are waiting for the result. But what's the problem with batch scheduling? Why don't I do that? Yeah. Right, remember, the CPU is a lot faster than the rest of the system. So while the job is running, it's likely that the CPU or other parts of the system. Memory 2 is faster than the disk. So it's unlikely, unless I'm very, very careful about the job that I've created, that batch scheduling is going to lead to good use of system resources. OK, so now, again, how do I, you guys, when you interact with your computer, you think that multiple things are happening at once. What's actually happening? Pretend that you have one processor. Maybe some of your smartphones still only have a single processor. What's happening? Yeah. The periodic table will answer my question. Exactly. So I am switching back and forth between tasks really quickly. I'm writing a little bit of one thing, and then I'm writing a little bit of another thing. And what you experience that as is, for example, the mouse is moving, and at the same time, Firefox is rendering the page behind it. You guys are probably most conscious of this when you're using your GUI, when you're acting with the graphical user interface, because there's a lot of stuff going on. There's things are spinning, and the YouTube video is playing, and some banner ad is scrolling by, and your mouse is moving, and so all that stuff's not happening at the same time. One of those things is doing a little bit of work, stop it. Somebody else runs, and I continue this process. OK. So how does the operating system make sure that it retains control over the processor? I want to make decisions about how to allocate the CPU on the operating system. I'm in charge of multiplexing system resources. This is my job. How do I ensure that I get a chance to do it at a regular basis? Yeah. Yeah, I use a periodic timer to generate hardware interrupts. Those interrupts cause the operating system to run and get control over the device. Every one of those interrupts is a potential point in which I can make a scheduling decision and decide that I'm going to stop the process that was running right before the scheduling, the timer interrupt took place. And we know this stuff. The timer interrupts, however, mean that I'm going to schedule the threads preemptively. Cooperative scheduling is this idea that threads are only descheduled at moments when they would expect to be descheduled. I made a system call. I know that I'm going to trap in the operating system, and it's possible I'm not going to keep running. In the kernel, you guys have a mechanism for cooperative scheduling. What is it called? Maybe something that you guys saw in the Assignment 1 testing code. In your OS 161 kernel, there was a function that I would have to use if I was going to try to cooperatively schedule threads. What is that function called? Thread yield. Thread yield says, hey, by the way, you can deschedule me right now. I'm cool with that. I'm right here. I'm just telling the operating system it's possible that you could deschedule me. We don't rely. Now, this is something that's potentially useful in the kernel. Remember, the kernel is in charge. The kernel is the boss. The kernel is written very carefully. But the kernel does not trust user programs to do this. One way of doing scheduling would be to require unprivileged code, unprivileged user applications to call thread yield. And that would be the only time that the OS had a chance to reschedule the CPU. That's not how it works. And when we start the thread up again, we want it to appear that nothing has happened. And we pointed out that there's limitations of that. Obviously, time has gone by. The device is in a different state than it was. But there are certain invariants that we want to preserve. We want, for example, all the registers to be the same as they were when I stopped the thread. What is the thread state that I need to save in order to perform a context switch? Well, I just told you one thing. I need to save the registers so I can reload them when the thread restarts. What's the other piece of state? Yeah? Stack. Stack. So these are my two pieces of private thread state. And keep in mind, we talked about this right at the end of class. Somebody brought this up. It was a great point. There's an overhead to performing a context switch. I have to enter the kernel. I have to save all this state. And what this means is that there's a cost to transition into the kernel. And we're going to come back to that today when we talk about different ways that you can implement multi-threading. Because we'll do a little bit of a design exercise at the end of class today, where we talk about two different approaches to implementing multi-threading. One in user libraries, one in the kernel. And we'll talk about some pros and cons of each one. And just a hint for a later question. One of the cons with doing multi-threading in the kernel is that there's a cost to switch between threads. And the cost of the kernel is actually potentially higher than the cost to switch between threads if those threads are implemented in a user space library. This is one of the trade-offs. And we also pointed out that I can't run the timer that drives scheduling too fast. Because if I do that, then the overhead of doing all these context switches, which, again, is totally useless. Useless except to achieve the illusion of concurrency. If I'm doing this too often, I have a lot of wasted cycles. And the machine is going to start to slow down. Luckily, you guys are so slow to perceive things that the kernel has. I can run the timer at reasonable rates and still get very, very good interactive performance. I wish Guru was here. Maybe somebody else knows this, what the default scheduling quantum is on Linux these days. Guru knows the answer to this question. It's one of the things about him that's so awesome. But I don't remember what it is. But I think it's on the order of milliseconds. It's actually pretty long. OK. Questions about this stuff before we talk about threads? OK, great. I did such an awesome job of explaining it. OK, so at this point, you guys sort of know what this is. So what's a thread? We talked about switching between threads. We've talked about saving the state of a thread. What is a thread? Yeah. Yeah, so it's the way that we abstract the CPU. That's a better answer to the question than I have upon the slides. What does a thread consist of? What's that? I heard a wrong answer over here. From the periodic table, clearly. Stop it. You're a chemistry table. You don't even know anything about computer science. OK. Two things that a thread consists of. You just did this, registers, and a stack. That's really what a thread consists of. This is the state that's private to a thread. Anything else is part of the process, right? The address space, the file handles, things like this. This is what we consider to be thread private state. The registers, which represent the CPU. And the stack, which maintains some history of what the thread was doing. The stack is also an important way that threads save state. Yeah. Yeah, we'll come back to that. That's a great segue. I actually have a slide about this in two slides. So can we do that? And then you can tell me if it's still. Yeah, so a lot of times, particularly for your OS 116, when we kind of talk about threads and processes interchangeably, which is a totally OK thing to do as long as there's a one-to-one mapping between them, right? You guys are not expected to do anything else than implement a one-to-one mapping between threads and processes. So people for assignment two will come to me and be like, well, do I need to have a separate process structure? And I always say, well, I mean, you're going to have your thread structure. It's going to have a pointer to your process structure. Your process structure is going to have a pointer back to your thread structure. So you are totally welcome to create a completely new, unnecessary structure and two more pointers that you have to maintain and ensure don't get corrupted, but that's up to you. If I was doing it, I would probably put that stuff inside the thread structure and call it a day. Fewer pointers means more beers in general. So yeah, when you're talking one-to-one mapping, but we'll get back to this later today, and there are certainly times, and it's very common, obviously, to have multiple threads inside a single process. How the OS views that is a good question. So actually, let's talk about that right now. So are registers private to a thread or part of a process? Top part of the slide. So registers are considered private to a thread. Stack, also private to a thread. What about memory? Shared between multiple threads. So this is now part of the process. If a thread writes to memory, another thread can see that that right. Now, this is a little weird, though, right? So I just got done telling you that the stack is private to each thread. Where is the stack located? In memory. So it turns out that if you are careful about how you do it, you can actually access variables on other thread stacks. The caveat is if you are careful about how you do it, because it's very, very hard to do right. But inside a traditional UNIX process, all the threads that are part of that can access all the memory that's been allocated to that process. So if I want to, I can poke around on your stack. Again, usually not a very intelligent thing to do, right. Better to allocate an object on the heap, and then we can share that object together, right. But there is usually not necessarily any protection provided in user space to thread stacks. They're just typically considered to be private state for that. Does that make sense? So technically, they're stored in memory, which is part of the process. In reality, they're usually not shared, right. That's not where things that are shared by multiple threads live. File to script or table. Process or thread? Process. I should have put these in a different order, because it's kind of obvious. So now we have a diagram, really fun diagram. Here's my thread, right. That thread has registers in a stack. The address space, now I wish I had another thread here, but if I had another thread here, I'd have another one of these guys with multiple registers and its own stack, right. So the stacks when I have multiple threads are usually put up sort of high in the address space and usually separated from each other far enough so that if I make a bunch of recursive function calls or something, my stack doesn't overwrite the guy next to me. OK, and the file table's proposed. Any questions about this before we go on? Gila, did I get your question answered? Cool. Are registers physical things? That's a great question. What's that? No, no, no. So you would not share registers, right? Are registers physical things? I mean, registers are provided by the CPU as a way to hold a value, right? That's the fastest form of storage that the machine has available, right? Can they be shared between threads? No, not that I know of, right? It's hard to say no about stuff like this because people will build all sorts of weird stuff, right? So who knows? There's probably like some machine that was like, this would be an awesome idea. I'll let thread share registers, right? Like when a thread's context switches up, I'll just leave stuff in there for the next thread to find, right? I don't know why that would be useful, right? But that doesn't stop people from making it, right? Just because it's fun. And sometimes you build things just to see what happens, OK? Good question. Any other questions? All right. OK, so here's the thing, right? I mean, you guys are, to some degree, I think a lot of you are comfortable with threads, but why do we use threads? Why don't we think about threads? Because what I'm going to try to do in the next few slides is convince you that threads are sort of the ultimate abstraction because they're not, they provide you with a nice way of thinking about how the system works. But they're not actually required to achieve concurrency. We'll come back to that in a minute. But why do you guys, why do we use threads? Why does the idea of a thread make sense to us? What's sort of intuitive about it? Let's say I have a web browser, and my browser's got a bunch of threads. How do I usually divide work between those threads? Give me one example of something that a thread in a web browser might be doing. Yeah. Oh, I like that answer. Blocking ads. Yes, sweet. Yeah, definitely, right? Definitely on my web browser. So yeah, but the answer to that question is really, I think, really indicative of how we think about threads, right? Threads represent one of multiple things that a particular process is doing quote unquote simultaneously, because while I'm blocking those ads, I'm also rendering the page and responding to clicks in various menus and stuff like that. So again, I mean, there's multiple things that are going on. And we know at the low level, what's happening is things are constantly getting started and stopped in order to make it look like all this stuff's happening at once. But in reality, the way we like to think about it is I have one thread that's rendering my page and one thread, like Firefox, for example, might have, I should stop talking about Firefox. I don't use it anymore, and I'm proud of that. Chrome might have one thread per tab. Makes sense. You have a tab. That tab has something that needs to get done. I have a thread that's allocated to that. So this is really a mental model for us to think about how we structure concurrent programs. Threads also frequently and naturally encapsulate some data. So there's data that's private to the thread that's blocking the ads. So there's things that it cares about that other threads don't really care about. Like it might have a list of all the ad servers on Earth, and it's using that to filter out content and make sure that I don't see ads for herbal biographies. And the other thing, and this is really important, right? So threads can block. As programmers, we really like this illusion that I can be doing a bunch of stuff and then I can make some sort of blocking call, like I might read a file. And my thread will block until that finishes. And that's OK. I'm allowed to do that. And essentially what's going to happen is, to me, it looks like this one call just happened, and now I'm underneath it, and I keep going. To the machine, what happened is that you did that, and then I got a bunch of other work done. And I restarted you sequentially. We're going to come back to this in a minute, because there's other ways of doing things. So here's my favorite example of threads. This is a picture from my wife and my favorite restaurant in Cambridge, Massachusetts. And this is a nice example of Wikipedia getting something right. So threads, you can think of threads as multiple cooks in the same kitchen that are trying to prepare a meal together. So normally, if you look at well-run, really efficient kitchens, there's a separation of tasks. It's not just like anyone's doing anything. You've got the one guy who's in charge of chopping vegetables or whatever, and the other guy who's watching the soup or the other guy who's cooking stuff on the stove. So each one of them, they've broken the task down. They're each doing their own thing. They're probably doing different things. No, they may not be. If you went to some huge restaurant, if you were on a cruise ship, I bet they have a lot of people chopping vegetables and cooking french fries. Apparently, french fries are one of the number one things to get eaten on cruise ships. And now, if they're working together on a recipe or to handle a whole bunch of diners, they're all trying to follow the same plan, but they're working on different parts of it. So that recipe is sort of like the source code of the program. They all have access to all of them, but each one of them is probably confined to one particular area. They have private state. So what's the equivalent in this cook's example of private thread state? Yeah. Yeah, like whatever's going on in my little mind. All the stuff that's up here. I know stuff. And I gather data about my own environment that's important for me to process. And other people really don't care. Like no one really cares exactly what percentage of the vegetables that I'm chopping I'm done with. But that's for me to know. On the other hand, they have to coordinate. So they also have to communicate with each other. They have to talk. They have to say things. There's nonverbal communication, whatever. So this is, again, sort of a nice analogy to think about the difference between processes and threads and also some of the, and maybe this sort of thing is why threads are such a natural abstraction for us human programmers to think about. And if they don't, then you can have stuff like this. There are so many cake fail pictures on the internet. You kind of wonder if people are doing this on purpose. OK, so anyway, I want to talk a little bit about event-based programming. And I think this is a good time to do it. How many people have ever heard that term or done any event-based programming? Cool. I know it's taking over. It's really exciting. So threads are a reasonable way of thinking about how things work and how to program concurrent systems. But I don't want you guys to get the sense that they are the only way of doing. And sometimes they're not even the best way. So going back 10, 15 years, my PhD advisor wrote a dissertation about how to write high-performance web servers. And one of the ways to do that was not use threads. So these big, multi-threaded web servers, it turns out that when they get under really, really heavy load, like it's 10 AM on the morning of 9-11, they just shut down completely. They can't make any forward progress. So if you look at the load curve, it's like it goes up like this. And then once they get to a certain load, they're just not satisfying any requests anymore. You want your web server to keep plotting along. You want it to keep making progress, even if it's really busy. You don't want it to just go to the point where it can't serve a single request. But this was what was happening to a lot of web servers in certain cases. Like after an earthquake, everybody suddenly finds the web server of this poor guy at the National Geological Survey, whose web server has been getting like one hit a day. Suddenly it's getting like 1,000 hits per second. And it just goes, phew. OK, so another way to do this is something called event-driven programming. And some of you guys have done some of this. Some of you have been forced to do some of this. How many people have ever written a JavaScript and a web browser? Well, you've done event-based programming, because a lot of that's heavily event-driven. What do you do in JavaScript? You write little event handlers that happen when certain things on the page take place. On click, do this thing. That's an event hand. That is not. And so that's a little bit of a taste of what event-driven programming is like. And has anyone worked with Node before? That's cool kids. You guys should check this out. This stuff's awesome. It's essentially like all these people. So again, how many people have written client-side JavaScript? There was a larger number of hands up. So all you guys got really frustrated that you couldn't run anything that ran on a server. So you decided, hey, let's run JavaScript on the server as well. And now you have all these people that all they did for years is write these stupid little interactive web pages. And now they're taking over the rest of the internet. It's a little frightening. So here's an oversimplification of how this works. Threads can block. So what we think of doing with the thread is doing, potentially, some long-running task that can have delays at any point in time. And the way we think about those delays is you did something that took a while, and you stopped running, and something else ran, and then you started running at the next state. In a lot of cases, the right way to do event-based programming is to write event handlers that are so short that they don't ever block. They do a little bit of work, and if they need to block, they produce another event that somebody else has to hand. So this is a totally different way of thinking about how to achieve concurrency on systems. So if I have multiple threads, getting good concurrency sort of relies on switching between them. When I do event-based programming, getting good concurrency relies on writing those events in a way that I can process a lot of them really quickly in a row and keep the system busy. Anyway, I just didn't want you guys to get the sense that threading was the only way to do this. So let's think about some. We did this at the beginning of class. It was probably good to come back to it. Give me some examples of applications that are naturally multi-threaded. Yeah. It controls what? You guys feel like this is like a great example day. We had to add block. Now we have drones. Yeah, like a drone, because it's up there. It's got to fly around. It has to spy on people and has to fire missiles at them and kill them. It's got a lot of things to do. Or if you're running one of those drones, maybe it's just hovering over someone's wedding being annoying. Who knows? But it has, at minimum, even just to control the thing. It's got like four different, again, I wish Guru was here. Guru has this little drone he's been building in her lap. I have no idea what he's going to do with it. But when he attacks the White House, I had nothing to do with it. I should actually really be careful. Anyway, so you've got four little motors on that thing. They're all pushing in different directions. So yeah, a drone is a good one. What else? We started with drones. We're going to get somewhere good with this conversation. We just go on long. That's another naturally piece of natural multi-threaded stuff. Web servers. So that's a great example. A lot of times, web servers use a separate thread to handle in each incoming connection. The connection comes in. I fork off a thread to handle it. That thread will block potentially multiple times. And eventually, that thread is responsible for taking that request, doing any of the processing on the server side. Maybe I have some Python to run. Maybe I have some Node to run, whatever. Well, Node wouldn't do it this way. They would do it differently. But anyway, doing all the stuff I need to do. And eventually, at the end of the day, that thread hands off a web page to a waiting client. So this is how a lot of web servers work. Well, usually, I don't fork a new thread. Usually, I have a bunch of threads that are sitting there waiting. But as requests come in, I hand them off to a thread. And that thread goes into the desktop. Web browsers. So again, I have multiple threads for every open tab. Frequently, I have, hopefully, you guys have noticed this. I mean, probably I haven't noticed it because you probably were born after it started to happen. Web pages used to just load all in one big clunk. It was like you went to some website and you sat there and it spun for a minute. And then suddenly it was like, whomp. That's not what happens anymore. You go to the page and the text comes in really quickly. And then there are these ads that are being filled in and images are loading. And this is because there's been a huge amount of work on figuring out how to process web pages more efficiently. Frequently, just to get stuff in front of your eyeballs more quickly. I want to start reading the article. I don't really care if the stupid little banner ad doesn't load for 10 seconds. In fact, I probably don't want it to load at all. So scientific applications. There are some types of data sets that can be easily split up so that multiple threads can work on them together. Sometimes this is also done with multiple processes, but you can certainly do it with multiple threads if you're lucky enough to have something where you can communicate with memory because then you're doing it better. So go back to the data sets example. So when I have something that I want to break up and process, why not use processes? So you guys are probably going to have to do this at some point in your lives. You're going to have something that's going to, you're going to figure, I need some degree of concurrency to solve this problem. And you're going to have to make this design choice. Do I do it with threads? Do I do it with multiple processes? What's the downfall of using processes? So remember, IPC, getting processes to communicate is a lot more difficult than getting threads to communicate. There's a lot more overhead potentially. There's more tricks you have to do to set things up properly. Inside a process, you want two parts of it to communicate. Just spawn up a thread and off you go. It's super easy. Getting IPC to work properly is usually more complicated. I think at some point earlier in the semester, someone came down and we were looking at some of the output from one of the PS dumps when we were looking at Apache. A lot of applications today, though, do a mixture of these things. So I think modern versions of Apache have this do this weird thing where they both fork and clone. So they create some threads, but they also create some processes. And they do this for performance reasons that I don't want to get into in depth. But in certain cases, having multiple processes can be difficult. IPC overhead design. And there's some state associated with processes that doesn't scale particularly well, like the address space, for example. OK, so now let's talk about two different ways to actually implement multithreading. And this we're going to get away from the one thread per process model. You can implement threads in user space using unprivileged libraries. In fact, on Linux for a long period of time, this was the only way to get multiple threads in your process. There's a library called pthreads. And there are other libraries that do similar things. Or you can actually implement threads in user space. I think this is one of those things that really starts to hit home, that threads are an abstraction. Because if you could implement the abstraction in user space, then it's not really part of the kernel's multiplexing responsibilities. I can't multiplex things all the time in user space, but I can't implement some of these abstractions. And threads are one of them. So we refer to this as the M to 1 threading model. Because as a result, what happens is the threads are implemented in user space. The operating system doesn't know about them. The operating system thinks, from its perspective, it's like there's one thread in that process. And that's all I know about it. It turns out, behind the scenes, that process is using multi-threading internally to get more performance. But the OS has no ID. And part of the pros and cons of these approaches have to do with the fact that the operating system doesn't know what's happening. And there are certain things you can't take advantage of. You can also implement user threads in the kernel directory. So the kernel could provide a way for user processes to create multiple threads. This is pretty much completely ubiquitous by now. User space-studying libraries are still in use, but I think most modern operating systems provide this feature. This we refer to as the one-to-one threading model. So every user thread is visible to the kernel. The kernel knows exactly how many threads are actually present in a particular program. So again, this is the M-to-1 model. From the kernel's perspective, all the kernel sees is one thread, despite the fact that the user program has used a library to create multiple threads. We'll talk a little bit about how this is done in just a minute. This is the M-to-M model. I do sort of a gratuitous diagram, but just forgive me. So implementing threads in the user space, how do I do this? It seems like something that the operating system would have to do. But the first thing is I've hinted at is that this doesn't really involve any multiplexing, so it is something that I can do. And let's think about how I do a couple of the things that I need to do to implement threads. So how do I save and restore context? How do I save context? Yeah? It's just the registers for the thread, right? When you're a thread, you can copy the registers out of the CPU, no one's stopping you. They're your registers, you're running, right? You just say, OK, I'm going to grab the value for A0 and put it in this structure, right? There's nothing privileged about that code. That code's not using any special instructions. It's just copies, right? Pretty simple. So I can take that block of code that we saw in your kernel. I can move that to user space, and it works exactly the same. It just takes all the registers that I care about saving, stuffs them in a structure somewhere. OK? The C library actually, so we'll get to this in a minute. This is, I mean, of all the things you can do with C that are bewildering and weird, this is probably up there at the top of the list, right? And to be honest with you, most of the things that you can do with C that are bewildering and weird are like causing errors, right? This is not causing an error. This is actually a function. Has anyone ever used this before? I hope not. I don't even know why these are provided. These are terrible, right? I'm about to show you some of the most un... You could use this. This is probably something that people use for those like, obfuscated code competitions, you know? Have you guys ever tried to read some of those? It's like a single line of pearl that decodes in FB3, you know? It's terrible. So that's the only reason to use this. I'm gonna show it to you because I think it's awesome, but you guys should please don't use this in your code because as you will see, you could use this to write some of the least readable code on Earth, okay? How do I, but here's another question. I mean, implementing multithreading means that I have to be able to stop, potentially if I wanna do preemptive multithreading, I need to be able to stop one of the threads that's running and run another thread. And this is probably important for a bunch of performance reasons as well, right? So this isn't just, you know, it's probably true that a user space library that couldn't do this would have pretty bad performance in a variety of settings, or it would require programmers to be really good about making sure they gave up control of the CPU. So I have to like put thread yields all over the place anywhere where I think that it might be time for another thread to run, which is really, really a bad thing to do, right? But how do I do this? Anyone know? Yeah, yeah, see now that the slides are online, everybody knows all the answers. Yeah, so I use periodic signals, right? So the idea here is that the user space threading library installs a signal handler that's kind of almost like the OS scheduler. That signal handler is attached to a timer that the OS delivers to the process periodically. When that timer is delivered, that signal handler runs and essentially performs the equivalent of thread scheduling, which we'll talk about next week. So this is pretty wild. Now keep in mind, if you're running in user space, this is all cooperative, right? If I have like a bad thread, I don't like the other threads, I could just overwrite the signal handler with my own signal handler and then there goes your threading library, right? So you have to, but again, we're kind of assuming that the threads within a process are willing to play along with each other if they're trying to use this like this. But you could certainly, there's nothing, unlike the kernel, which is always gonna get control back, right? There's nothing guaranteeing that the threading library will be able to continue to operate if one of the threads is probably not malicious, but just badly written or stupid or something like that. All right, so now we gotta look at, we gotta look at set up long, okay? So here's my little piece of code, right? I'm gonna go through a loop and here's what's gonna happen. When the value is five, I'm going to save my state and then break out of the loop, right? When I get down here, I'm going to, and I have this variable restored, which I set to zero. If I'm not restored, so I haven't done this already, I'm gonna call long jump, using the state that I saved into this thing that's called a jump buffer. Well, I'll ask you guys, what's in that jump buffer in a minute? And then what long jump does is it's gonna essentially return to the point that set jump, that set jump left. And the return value from set jump is going to be whatever I pass into long jump. So long jump gives it a one. So when I get here, set jump saved is gonna return one. When I go back, does this make sense? This code is, again, this function is a little bit difficult to understand how it works, but who can predict what the output of this is going to be? So clearly for the first few values, I'm just hanging out, I'm printing stuff, I'm gonna print value of I is zero through five. Now I get into the fifth iteration of the loop and what's gonna happen? Anyone wanna guess? Yeah. So here's how the flow of this code is gonna work. I'm gonna go through this loop five times. And the fifth iteration, I'm gonna break out of the loop, but not forever. Now again, you guys thought there was no way to do this and see, and please don't use these new powers for evil. What's gonna happen? I'm gonna get down here. Long jump's that gonna take me right back into the loop. I'm going back in time. And then I'm gonna continue executing the loop, printing off the rest of the values until I exit. So this is the actual output of this code. I'm not sure you could quite compile this. I think I left off a few variable identifiers, but whatever, it's close up. So again, I go through till I get to five. Here I saved my state. Now I saved CPU state and I break out of the loop. So now I should have a printf down here to say I'm out of the loop, haha. But then when I call long jump, I'm back inside the loop again. And I'm gonna continue executing the loop. So now that you've seen this, what are such up and long jump actually doing? What's in the jump buffer? I guess? Yeah, I mean, you could rename the jump buffer the trap frame, right? It's the state of the thread at that point in time. That's what allows me to work back into the middle of the loop, right? And you can do this with, like this is a pretty simple example, but you can do this with arbitrary code, right? I could save the state in the middle loop and then go off and do a bunch of other stuff and then suddenly transport myself right back into that point. Again, terrible way to write code, right? If you find yourself solving a problem this way, then I would suggest that you find another way to solve the problem. All right, any questions about this? I mean, this is a clear indication that we can do this in user space. I can save state in users. This is how I implement this mysterious function, yeah. Saved is this jump buffer, right? That's a great question. That would be on the stack, right? But I could put it anywhere. I could put it in the heap, right? And then the fun thing is I could return from the function, I could do all sorts of other stuff and I'd still have the state, right? That would allow me to get right back to where I was. So yeah, in this particular case, it's on the stack. That's okay because I still have my stack right here. But I'm assuming if you put this on the heap and put it in a more permanent place, then I could store it across multiple function costs, right? I could trade it between threats, right? I could be like, hey buddy, you wanna have my life experience? You wanna like eliminate some ads today? Here's a jump buffer, now you're running my code. How weird is that? Wow, anyway, it's pretty cool. Again, do not, you can use these two tricks to impress your new friends. You may lose a few on the way, but, and this is the meme I came up with. I don't remember what her problem was. It was the vaulting thing, right? That's where she fell down? Anyway, I don't know. It probably should have been nailed to set jump, forgot the long jump, but that's not it. Okay, any questions about set jump long jump? So let's come back to our comparison here and this is how we'll close out today, right? So what's potentially good about doing threading in user space? Why would I wanna do it this way? Yeah, yeah, it's less overhead, right? I don't have to cross the kernel thread, kernel user boundary. There is, now look, there's still some overhead to running the saving state in user space, but it turns out to be smaller than getting in and out of the kernel, right? So the overhead to switch between threads is lower and the thread state's actually quite a bit smaller because the kernel has a bunch of other state about the thread that is necessary for the kernel to work, that a user space threading library doesn't necessarily care, right? What's a, so, again, there are no, I would hope that there are few to no systems left on Earth that require you to use user space threading libraries. Why is that? What can I never do with a user space threading library? Remember, the kernel thinks there's one thread. What does that mean? Why would that make you very sad today? Like very, very sad, yeah. I can never use multicore, sorry. I've got parallelization inside my application that's never getting out, right? I've got four cores or eight or 64 or whatever, right? But it doesn't matter. I'm only ever gonna be able to use one because the OS is like, you only got one thread. What do you want me to do? I'm not gonna thread your, I'm not gonna schedule your one thread on multiple cores, I don't do that. That's weird, right? You only get one core at a time. So I can't use multiple cores. The other problem here is that, well, there's something else that can happen, right? So the, so there are scheduling issues because I don't know if there's one thread. What's one thing, can you guys think about perils? If you guys have used P threads or some of those libraries, what can a thread and P threads do that can potentially bring this whole charade just like crashing down, right? What could I do in P threads that could just really, really cause serious problems in my application, right? I'm running along. I've got this fake pseudo thread that P threads has created for me. How could I cause the process to lose control in a way that could be really problematic? What if I make like a blocking IO call, right? So my little thread tries to do like a read or write. So what's gonna happen? There is no way for the threading library to intercept that call. I'm gonna make a system call. The whole process gets put to sleep until that read completes. The problem is it's possible that a bunch of the other threads inside that application could have actually kept going and done useful work. So to use threading libraries like this effectively in DOIO, you have to use these wrappers that the threading library provides that allow you to do reads and writes that ensure that the threading library gets control when you do that, right? So what's gonna happen is I'm gonna call, I don't know, P threads read or whatever, and it's going to say, okay, I'm gonna issue what's called an asynchronous read to the kernel, right? That's a different way of doing read that doesn't block the process. I'm gonna put you to sleep on my own little thread queue here. I'm gonna run a different thread, right? But if I make a blocking IO call, you know, that's a way to sort of ruin the opportunity to use user-space threading libraries to achieve concurrency, right? The whole process will block even if there's lots of other useful work that could get done, right? So there's some careful programming I need to do in order to get this to work, right? I just can't take a big blob of code that somebody else wrote and drop it into my, into mine. Although maybe P threads is far enough to like rewrite the C library wrappers of read and write to do the right thing. I don't know. Okay, there we go. So what's good about kernel, now we've talked about, you know, the pros and cons are clearly kind of related. What's good about kernel multi-threading? I mean, obviously I can use multiple cores. That makes me happy. What else? I can do IO in my usual way, that makes me happy. Any other things you guys think could be good about other than correcting the things that are bad about user-space multi-threading? One thing, and we'll come back to this next week, is that given more visibility in the process, it's possible that I can do better thread scheduling than the internal thread scheduler that runs in the user-space thread library does, the kernel knows more about the machine, right? It has more visibility into how the resources is actually being allocated. It might be able to do a better job, right? And of course the cons are, again, these are highly related. I have a context switch into the kernel anytime I switch between threads. So it's a little bit more overhead for using the multi-threads on the machine. All right, so, oh, thread states. Okay, I'll leave this online. I think you guys can, now that the slides are online, I don't have to get through everything every day. It's awesome. I'll just leave it as homework. You guys can look through this. This stuff's pretty intuitive. We'll start with it on Monday. What we're gonna do next week is start talking about scheduling, right? So we've built the mechanisms at this point. I have full control over the CPU. I can start and stop threads whenever I want to. I have this nice abstraction that I'm providing to programs for how to think about how to use the system effectively. So now, and again, we'll come back to this when we do memory and files and other things. Now we need policies that are gonna effectively make use of the system resources. So that's the topic of thread scheduling and CPU scheduling, and we will talk about that next week. So good luck with your site and two design documents. I'll see you guys on Monday.