 So today, last time we talked about why we were going to schedule threads and what scheduling is. Today we're going to talk about some different algorithms that we can use to accomplish that. We'll talk about some extremely simple schedule in algorithms that don't really require any additional information. And then we'll introduce a pretty common systems design principle, and use that to design a more effective scheduler. OK, all right, I don't think I have any announcements today. Any questions about course logistics? You guys have about two and a half weeks left to do assignment two. So do assignment two? Any other questions? All right, so what is thread schedule? Quick review. What is thread scheduling? Yeah, OK, it's simpler than that. Priorities we're going to talk about later. It's an abstraction that we use potentially when we're scheduling threads. But what is thread scheduling? Yeah, right, I've got a decision to make right now. What thread should be running on the core or course? It's a process of choosing the next thread to run, or the thread that's going to run right now, or deciding whether the thread that's running should be able to continue running. So ordering threads and establishing which thread runs next and which thread is allowed to use this particular CPU resource. Why is scheduling so important? Why is it something that the kernel has to do? Why do we need to schedule threads? So that would be why to schedule threads well. But remember, we need to create the illusion of concurrency. We have more threads than we have cores. Therefore, we need to establish an order in which those threads can use the cores. If we have fewer threads than we have cores, we do not have to schedule threads. We just let them run whenever they're ready to run. The idea of scheduling, just to go back to this briefly, is that there are more threads that are ready, that are in the ready state, than I have cores. If I always have fewer threads that are in the ready state, it doesn't even matter how many threads I have. As long as I have fewer threads that are able to run, that are ready to run, than I have cores, I don't have a scheduling decision to make. So I may have 10,000 threads. But if they're all blocked, waiting for stuff to happen, and there's only three threads, three threads that I need to schedule, and I have four cores, I don't have a scheduling problem. I just run all of them until I have a scheduling problem to solve. And of course, the kernel is in charge of doing this because the kernel is in charge of multiplexing resources, and the kernel is what runs every time I enter up fires. It's the root of kernel privilege. When do I have a chance to make scheduling decisions? When does the schedule have a chance to run? What's that? Yeah, so essentially whenever the kernel runs, it's an opportunity for me to make a scheduling decision. I don't have to make one every time. But if a thread calls thread yield, if a thread makes a blocking system call, when a thread exits, and when I've decided that the thread has run long enough and a timer fires, which gives me control back so that I can stop that thread and start up another thread. So we talked a little bit about what you guys expect from your computer. You expect it to respond when you do something. You expect it to be able to continue tasks at a rate that's sufficient to produce what appears to you to be smooth performance. And you also expect certain things to happen in the background into complete after a period of time. These expectations are sort of what drive our scheduling goals. So when we start thinking about schedulers today, we're going to think about some of these things. How well does it even know anything about deadlines? Is the thread even able to distinguish, for example? Sorry, is the scheduling algorithm able to distinguish between a thread that has a deadline and a thread that doesn't? Between the thread that's about to redraw the mouse pointer and a thread that's running some sort of index scan in the background, can the algorithm even distinguish between those two at all? Some can't. Some can't. And also, how completely do I allocate system resources? And the CPU is one of them. But remember, access to all of the underlying resources requires use of the CPU. In general, this is a little bit of a tricky thing to evaluate. But one way, one proxy for this, is to think about when a thread is ready to run, how long does it have to wait before it gets a chance to do something? Imagine I have a thread that's repeatedly using the disk. Let's say it's copying files back and forth between one file and another file. It's copying files back and forth. It's copying data from one file to another. When one of those threads, all that thread is doing is doing reads and writes. All it's doing is doing file IO. Every time one of those reads and writes completes, the efficiency that we can measure for that particular thread is how long it spends waiting for the CPU. Because all it's going to do once it gets the CPU is initiate another IO. So it does a little bit of IO, sleeps for a while waiting for that to finish, wakes up again. If an algorithm forces that thread to wait a long time before it gets to use the CPU again, then you're slowing down that file transfer. And this is not a completely uncommon pattern. There are certain threads or tasks that end up waiting on other system resources to complete. But in order to tell whatever underlying device I need to use what to do, I have to use the CPU. Remember, I can't normally do this without running. So this is something else to think about. We also pointed out that you guys don't like having your time wasted, which is a good thing. And so deadlines on interactive systems and interactivity in general are usually something that drives scheduling algorithms for human-facing systems. Don't really care about how efficient your computer is running. You care about how efficiently it's able to respond and interact with you. You don't want to spend more time interacting with it than you would. Not if it's for something dumb, because it's like painting the screen. I mean, if it's like chatting with your friends on Snapchat or whatever, that's fine. And finally, the last thing I want to throw in here today is for any, and this is something that for when you guys are studying computer systems, operating systems, distributed systems, any kind of system, this is always a consideration. And this is something that I think OS designed is always provides a very interesting case study. There's a lot of really smart ways to do things out there. There's a lot of research in algorithms and artificial intelligence and machine learning. The algorithms we tend to study in computer systems tend to be quite simple. The reason for that is that they have to run a lot. And because they have to run a lot, the performance of that algorithm itself matters a lot. There's no point writing a scheduling algorithm that only makes 5% better decisions or only makes better use of the system by some small percentage if that algorithm takes forever to run. It takes like a second every time you run it. And then it's done a really great job of picking the next thread to run for 100 microseconds. That would be a computer that you would not want to use. So we always want our out. And when we're writing computer systems, especially when we're talking about things that run in tight loops, we're always thinking about this. And this is something that comes up on exam questions in this class, and something that you guys will encounter when you go off into the real world of software development. You will have a really smart, clever way of improving some particular part of a system. And you will discover that nobody likes your idea. And it's not personal. It's just that your algorithm is so slow that it slows down the rest of the system so much that no one cares about the part you sped up. Because you slowed down the overall system too much. This is something we need to think about. And we know the scheduling is the result of conflicting goals. In a lot of cases, interactivity conflicts with my desire to keep the system as loaded as possible. Any questions about why we are doing this in the first place, and what we are trying to accomplish? Any questions? All right, so today we're going to talk about three different types of schedulers. We're going to talk about schedulers that essentially use different types of information to try to make decisions. One type of scheduler that's fun to talk about, but nobody should go out and try to implement, is a scheduler that uses knowledge of the future. If you can write this scheduler, then I would suggest that you use your ability to acquire knowledge about the future for something more interesting than writing an OS scheduler. Like use it to bet on football games or something. Or I don't know, whatever. Develop a reputation as a psychic or something. Don't write OS schedulers with it. But these are fun to talk about because it's a good benchmark for evaluating other schedulers. We frequently, in a variety of different contexts, talk about oracle algorithms, algorithms that know something that a normal algorithm would not be able to know. Because we can talk about how well a normal algorithm with its limited information is able to approximate the performance of an algorithm that knows these types of things that other algorithms cannot. So no scheduling algorithm is actually able to know the future. It's not possible. In a lot of cases, what algorithms do instead of trying to predict the future is they assume that what has just happened is going to continue to happen. On some level, that's not a bad way of predicting the future. If I was going to ask you to predict what I was going to do for the next three minutes, you would probably say, I'm going to sit here and continue to tell you boring stuff about scheduling, and wander around, and pace in front of the back of the room, and move my Nalgene bottle from one side of the room to the other. That's not a bad prediction. I should try doing something else. I should go outside, or leave, or lie down, or something just to befuddle you guys. But look, using the past, the recent past, to predict the future is not a bad thing. Now, if you go an hour back and you said, well, I would just assume he's going to stay in his office all day, that's not exactly quite right. So frequently I use the past, but I use the recent past. And I say, the best I can do to predict the future is just to assume that whatever just got done happening is going to continue to happen. Finally, we talk about ways that schedulers can incorporate exogenous or outside information. What does the user want? What does the system think it wants? And schedulers' priorities, which somebody brought up before, are a way of layering external information on the schedule. The scheduler doesn't know why you chose to prioritize that particular task. It doesn't care. All it does is say, OK, clearly the user, the system administrator, or somebody else knows more about this than I do, and I will incorporate this information and use it to try to follow the policy that somebody else has set. But keep in mind, this is really ways that schedulers are trying to use external information. The priority is an abstraction. The scheduler follows them without really knowing anything about them. It's up to you to set them well to achieve whatever you're trying to accomplish. But let's start out with the simplest schedulers possible. The simplest schedulers possible don't know anything. They don't care. They don't try to guess. They don't try to store any information. They don't predict the past. They don't know anything about the future. It's like the goldfish schedule. Every time it turns around, it's like, wow, never seen that side of the bowl before. So give me an example of the simplest possible scheduler. There's no information at all. OK, so I'm going to argue that a batch scheduler is actually too complicated. Because a batch scheduler knows something. It knows the order in which the jobs were submitted. So I'm saying, I don't even need that. Yeah, random, right? All the random scheduler does is it wakes up and it's like, OK, all you got to tell me is give me a group of threads that are ready to run. Doesn't have to be a queue. Doesn't have to be a list. I don't know anything about them. And I just poke at one, you're it. So how does this work? Welcome to my awesome scheduling animations. Super fun. I just, as you can imagine, and what it does is it runs it until a time quantum expires. And then, so here's how random scheduling works. And sorry, I'll slow it down. It's more to be a little more specific. I pick a thread to run. How long do I let that thread run? Forever? Yeah, up to a fixed amount of time. Will every thread that I pick run for that long? No, what else might happen? It might exit. What else might it do? That's a little drastic. It might yield. That's a little nice. What else might it do? It might ask me for help with something. Say, hey, do a write. It might do something that causes it to have to move out of the ready state. It might do a blocking system call, for example. So my random scheduler runs. I picked T3 in this case. The dark purple thread. I have two purple threads. I like purple clearly. The dark purple thread, in this case, ran until its time quantum expired. Where does it go? I need to de-schedule it. Where am I going to put it? Goes back on the ready queue. It's still ready to run. It hasn't done anything that caused it to block. Doesn't need to wait for anything to happen. Now I pick the next thread. Let's say that in this case, T5 stops before it reaches the end of its quantum. Where does that thread now go? It must be waiting for something, because it stopped running. It's possible it exited. OK, fine. In this case, let's say I performed a system call. It did something that caused it to need to wait. Now I pick another thread that's ready to run. This one runs until its quantum. It goes back in the ready state. And this is what I do. This thread blocked. And I just do this over and over. I don't know why I made so many of these. And over. So this is random schedule. Extremely simple. I choose a scheduling quantum. And the scheduling quantum is something that comes up again. So when I'm doing preemptive schedule and I always have to decide, what's the maximum amount of time I'm going to let a thread run? Because there's some thread that may not stop ever. It may be computing digits of pi or something. Or it may be a malicious thread that's just doing while 1 just to see if I'm broken. And I'm going to let it take over the machine from now until the apocalypse. So I choose a scheduling quantum. This is the maximum time I'm going to let anything run. How do I enforce this? How do I enforce a scheduling quantum? How do I make sure that the kernel gets control? Timer interrupt. So normally my timer interrupt has to fire at some rate that's a modulus of my scheduling interval. My schedule interrupt is 500 milliseconds. I better have a timer that fires at least every 500 milliseconds. Probably it fires faster than that. And every time it fires, I look up and I say, OK, wait. The thread's less than time left. I go back to sleep. I'll let it run. I'll let it keep going. Then after 500 milliseconds I say, OK, you're done. OK, I choose a thread at random from the ready pile. Now, again, notice I didn't call this a list or a cure or whatever because this is random schedule and all it sees is a pile of threads. There is no order here at all. I run that thread until it blocks, until it does something that processes it to not be able to proceed, or until it's scheduling quantum expires. And then I choose the next thread. So what happens when a thread gets done waiting? Let's say it was waiting for a read to complete. That read is done. What happens? Where does it go? It's pretty easy. I just put it back on the ready pile. And of course, how would this happen? This would happen because I would get an interrupt from the disk. The disk would say, hey, by the way, kernel, there's this read that's finished, and I would look it up and I would say, oh, I know what thread was waiting for that. It's over here on this weight queue. I'd wake the thread up, put it back on the ready queue. Very simple. That's actually how it would be. OK, round robin scheduling. So now we'll go to the batch scheduling answer. This is more complicated than random scheduling. It still requires essentially no information. All I need is an order. I need a queue. And then what I do is I just establish an ordered queue. I pop things off the top. I put them back onto the end. Simple. This is essentially the same as random, except rather than choosing it random, I just choose the first queue, first thread in the queue. And when a thread is done waiting, I have a decision to make. I could put it at the front of the queue. Seems fair. It's been waiting, after all. That's been in line. Or I could just put it at the back of the line. Who cares? It'll get its turn again eventually. OK, so these are the simplest algorithms that we could potentially come up. They don't take any input. They don't compute anything. They're extremely simple to implement. And they don't try to do anything smart. And these aren't particularly useful algorithms to implement on a real system. Except they are extremely useful for evaluating your own algorithms. So let's say you come up with fancy algorithm foo that's going to do an awesome job of scheduling. And you run it against random and random wins. This happens. Because again, it may be that your algorithm is doing a fantastic job of choosing the next thread to run, but it's taking forever to do that. So if you cannot perform random, take your ball and go home. Bad idea, start over. So these are good, strong men. Because a smart scheduling algorithm should be able to outperform these. And by outperforming, we're really talking about end-to-end performance. We're not just saying, how well does it choose the next thread to run. We're saying, if I run certain workloads, how quickly do they complete? Because that's a benchmark that includes the overhead of the scheduler itself. We'll talk more about dutch marketing performance analysis in a month and a half or so toward the end of the course. OK, so. And these algorithms, the other thing that you want to notice here is that if I'm a thread that blocks frequently, if I run for a small period of time and then I block, then I run again for a small period of time and I block, maybe that's because I'm taking input from the terminal. I'm redrawing the shell prompt or something like that. These algorithms tend to sort of penalize me, because what ends up happening is I block, I wait, I go back in the queue. But because I've blocked quickly within my quantum, I end up with less CPU time than a thread that just runs and runs and runs. So if I really want the most CPU time with something like this, I want to be the thread that's computing pi. Not the thread that's redrawing the mouse pointer. And you can see an obvious problem with this, because most of us want the thread that's redrawing the mouse pointer to be able to do its work, and we don't necessarily care how quickly we generate digits of pi. So this is a problem with these schedulers, and we'll talk about ways to address it. And so just to point this out, you can also do like a round robin. Round robin is sometimes used as a primitive within other scheduling algorithms, essentially saying, when I have a tie, when I have a bunch of threads that in any other purpose are all equal, then I just use round robin to go through them. Or I could do random too, it doesn't matter. But these are sometimes used inside other scheduling algorithms when I've done my best to make a decision, but my decision has not produced a clear win. So now let's talk about the other extreme. Let's talk about schedulers that know things that schedulers cannot know. So if you were designing, if I could give you sort of one superpower, and it wasn't, I heard a podcast about this once. There are two superpowers that people usually want. Anyone guess what they are? They poll people, they say, what superpower would you like? It's like 80% of people pick one of these two. One is to fly, the other is to be invisible. I think those are kind of funny, they're sort of at odds with each other. So they did this thing where they asked people, which one would you like to fly or be invisible? Anyway, now if you're a scheduling algorithm, you don't really care about either one of these, these are not useful to you, right? But what superpower would you like? What would you like to know about a thread that's difficult, that's impossible to know? Yeah, yeah, how long is it gonna run before it blocks in particular? I might also wanna know how much more work it has to do, that's true, that's another thing. How long is it gonna run until it blocks? Does it just have a little bit of work to do before it engages another system resource, or is it doing a lot of work? If I have a thread that only needs to execute four instructions, and then it's gonna keep the disk busy for another couple of seconds, that's awesome. I should run that thread right away, because it runs, it gets out of the way. And then some other part of the system is busy, and that's good. I have a disk that's busy, a CPU that's idle, now I can run Pi, okay? So how long is it gonna use the CPU? Is it gonna block, right? Is it gonna yield, is it gonna stop, or am I gonna have to yank it off the CPU? And once it's on the weight queue, how long is it going to wait? Now, which of these things might I potentially be able to make some guesses about? I probably can't predict the ones above, right? This is control flow, control flow is unpredictable, it probably wouldn't be worth the effort anyway. Which one of these could I maybe take a stab at estimating? Yeah, how long will it wait, right? So I know it's waiting on the disk, something that's predictable, I might be able to guesstimate, right, I'm just saying, the disk usually takes this long to serve as a request. Of course, if it's waiting on something like the network, or you, then all bets are off, right? It's probably pretty unpredictable, but there's certain cases where weights might be predictable. Okay, so let's give an example of a know it all algorithm, okay? So what this is gonna do is it knows now, let's say I know how long each thread is going to run until it blocks. Again, not information that I would normally know. So I'm illustrating this on the slide, here's how long each thread is gonna run. In what order do I run them to minimize waiting time? Yeah, so if I start with the shortest job, then I can minimize waiting time, and I will show you in a second why, okay? So this algorithm knows shortest job first, right? I run, and notice too, with this example, I'm, let's just pretend that I can run these jobs to completion, okay? I'm running these jobs, and this is how long it takes to finish, right? This thread, and in reality, threads don't tend to finish this quickly, they end up doing something else, but you can think of it as an approximation for how long is it gonna take till that thread puts another resource to work and gives up the CP, okay? So, and again, I will show you in a minute that how this minimizes waiting time. But in general, if I start to think about scheduling algorithms, and I think I've sort of given away these answers to this question, right? The reason I'm interested in threads that are not gonna run for very long is because those threads tend to run in order to activate some other resource. So if I'm interested in keeping lots of other resources busy, having a thread that only runs for a brief period of time, and then sends a network packet is great, right? Because now I've got the network card busy and the CPU's idle. This is why, one of the reasons why I'm interested in this, it can also, it's also very common, I shouldn't say it's very common anymore, maybe people stop doing this, but it's also a pattern for threads that are doing interactive work to run for a brief period of time and then wait for a long period of time. Why is that? Why would a thread that's doing interactive, an interactive task, tend to only run for brief spurts and then wait for long intervals? Yeah? What's that? So it's not gonna reward that thread, but why do interactive threads have that pattern? Yeah. Yeah. They're waiting on you, right? And you're super slow, right? You know, again, like between the time you hit the key and you hit the next key, like to you, you're like, wow, I'm an awesome typer, man. Mavis Beacon taught me well. I am speeding along 80 words a minute and bash is like, when is this person gonna type the next key? I've been waiting and waiting, you know? So yeah, so when you're typing and you're bash prompt, the amount of work it's doing is tiny. It wakes up, oh, he finally pressed a key. Okay, I could have told you you were gonna do that. Try hitting tab, you know, types the key, it goes back to sleep for a long period of time, right? So that's the reason they're waiting on you. You're slow, right? So, you know, at long waiting periods, short, brief bursts of effort are sometimes used. They used to be used by the old Linux scheduler, which we'll talk on Friday, to try to identify interactive tasks. Tasks that are actually painting or interacting with these, right? Okay, so let's go back to the shortest job first example, right? So let's imagine that this, so let's imagine that we are, the job runs and then it waits, okay? Once it's done waiting, it could potentially go back on the CPU. So these were the lengths that we looked at before for how long these were gonna run until they blocked. And now what I'm showing you is how long they are blocked for. Notice that T3, my favorite dark purple thread, does not block at all, right? It runs, it just runs to completion and never used to resource. If I run, so this is the worst possible case, okay? The worst possible case, I run the longest one first, right? Let's imagine that T3 never stopped, I'd de-ank it off the CPU. So now what you can see is, here is the total waiting time. This is the amount of time that all the threads spent waiting to run again, right? T3 stopped here and let's imagine I just ran the same threads in the same order again starting here, so it has to wait all the way to here to get to run again. Actually, technically it would be here, I guess, but anyway, yeah, it's close enough, right? Now, if I run them with the one that blocks first, it turns out that the waiting time is much shorter. And the reason for this is mainly because I'm doing a better job overlapping use of the CPU with use of other system resources, right? So imagine that these are all using some of the resource. If I go back to this example, I mean, this is one of my big wins, right? At this point here, the CPU is busy, nothing else on the system is busy, right? Then T1 runs, same thing. The only thing that's busy is the CPU. It's not until here that I actually activate any other resource that I start to allow a thread to wait for anything else to happen. Here, this happens very quickly, right? Right here, now I've got another resource that's going, and every time the CPU is busy, it's overlapped with some of the resources on the system also being active, okay? So I'm making better use of all the resources on the system. Does this make sense? Any questions about this particular algorithm? Shortest job first, yeah. Okay, so hold on, this is an important question. Can you implement this algorithm? No, I don't know, right? This is my oracle algorithm, right? This algorithm knows things that you don't, right? So the answer to that question is you can't, okay? This is a pretend algorithm that knows this, right? You don't know it, but this algorithm is one we're gonna compare against because it does, right? This is the best I could potentially do, okay? So normally, again, we cannot predict the future, right? Control fold is unpredictable, users are unpredictable, and instead, and this is something that we will come back to over and over again, I'm going to use the past to predict the future. In the case of scheduling, what I'm going to do is I'm gonna use the thread's past behavior to predict what it's gonna do in the future. So in order to do this with the scheduler, let's say that I'm looking, and I'm trying to implement something that's like shortest job first. So what do I track about each thread? What do I want to know? Again, I'm gonna use the past, use the past to predict the future. What about the past do I care about? What about the future do I wanna know? Yeah. So what I wanna know about the future is how long it's gonna run before it blocks. What do I know about the past? Last time, how long did it run before it blocks, right? So this is the, again, I'm gonna assume the thread's gonna keep doing the same thing. Is this always the case? No. But this is what I'm gonna try. So I'm gonna say I'm gonna measure how long the thread runs before it blocks and threads that run a short amount of time before they block, I'm gonna assume that those threads will do that in the future. And again, remember that when they're blocked, they're typically blocked because some other part of the system is busy. So allowing that thread, prioritizing that thread, running it before other threads that are just gonna use the CPU makes it more likely that I can overlap other two system resources, right? Maybe the resource that it's waiting for is you, but that's still okay, right? Because you now know that bashed through the character that you just typed and you can continue typing, right? Or thinking or whatever it is you do in front of your computer, right? Okay, so the canonical example of this for scheduling is something called a multi-level feedback queue. Doesn't sound as awesome as it actually is, I guess, but whatever, MLF queue. So here's how MLF queue works. Of course there'll be a fun diagram in a minute. Like every other scheduler, I choose a scheduling quantum. This still defines maximum amount of time and gonna let any thread run. I established some number of queues. Does it really matter, more than one? So I have a plural, but other than that, it's okay. Now each queue represents a level. And in general what MLF queue is gonna do is it's gonna prioritize threads based on that queue. So if there are threads in the high priority queue, I will run them first, as long as they are ready to run. If there are threads in the middle priority queue, I will run them after the high priority queue and I just go down the list, right? So if there are two threads that are ready to run and one is in a higher queue, that thread will be run first. What do I do if I have multiple threads in one queue that are ready to run? I can use round robin, I can choose one at random, whatever, at that point it doesn't matter. So I choose and run a thread from the highest level now on empty queue. Now what I haven't told you is, how do threads open up in these queues in the first place? How does that work? Okay, and here's what happens and this is more, there's less hysteresis here than there would be in a real scheduler. But in general, when threads block before they reach the end of their quantum, I tend to promote them. Those threads get moved into higher priority queues. What's the inverse of that? So if I'm going to promote a thread that blocks before it hits the end of its scheduling quantum, how do threads move downward? Yeah, if I have to yank it off the CPU so something else can run, then those threads tend to migrate into the lower priority queues. So this is really kind of a form of interactivity detection. We go back to my assumption that threads that are interacting with users run for short periods of time and then wait for long intervals. Those threads tend to block before they hit the end of the scheduling quantum and those threads will be promoted into the highest priority queue. Now keep in mind, the highest priority queue, so why does this work? Why aren't there threads that are just always in the highest priority queue and blocking everything else from running? Doesn't that seem like that's what would happen? I mean, some threads are going to get promoted, those threads are going to sit up there, they're going to run and all these threads in the lower priority queues are going to be like, hey man, there's no love down here. I never get to run. Yeah. Well, but keep in mind, if I stop one high priority thread, there might be another one that's ready. So I just might end up doing round robin with threads from the, so imagine I have a bunch of high priority threads that are just blocked, they just run for a little bit block, but then there's another one that's ready. Why is this unlikely? This can happen, actually. But why is it unlikely to happen? They would be expected to happen. What's that? They would be expected to happen. Yeah, these threads are usually sleeping for a while, right? So imagine I have three threads in the high priority queue, one of them runs for a bit blocks, it is now sleeping for a long time, right? The next one runs for a little bit blocks, it's now sleeping for a long time. The third one does the same thing, now I'm out of threads in the top priority queue, they're all on a sleep queue or they're all in the waiting state, they're all blocked. So now I can do work from the medium priority queue, right? But the schedule will start threads in low priority queues in certain cases. We'll talk about that in a minute and what to do about it. Nothing good, just something simple. So let's go through an example of how this works, okay? So let's say I run, I start, I pick T3, okay? I start with all the threads in the top priority queue, right? I take T3, T3 goes to the waiting queue. So when T3 comes back from waiting, where's it gonna go? It's gonna stay in the top, right? It blocked before it finished, it gets to stay in the top. Top priority queue, okay? Now let's say I run T5, T5 hits the end of its quantum, where do I put it? I put it in the middle, right? It's migrating downwards. It has hit, it had to be preempted, therefore I move it down, okay? T4, T4 blocks, T4 goes on the waiting queue when it comes back, where is it going? Back to the top, T1 runs. Now in this case, can that actually happen? What do I think? I don't know why, is there another bug on the slide? Let's see. Oh, yeah, sorry, there's a bug on the slide. So pretend that T1 actually went to the waiting queue and then came back, right? So now T1's back in the top, it runs again. Now at this point, T1 ran, now it blocked, it's on the waiting queue. So now what do I do? Again, this is what I want it to happen, okay? This is why I allowed these threads to be in the top queue, because they tend to run for a short period of time and then block quicker, right? So now at this point, I can start running things from the lower priority threads. They're a little bit lower priority queues. So imagine T5 runs, hits the ends of its quantum and it has this problem, it keeps doing that. So I move it down, goes down into a low priority queue. Now T3 came back from waiting, it's the next thing that runs, right? Hopefully, again, runs briefly, it goes right back to the waiting queue, okay? Questions about multi-level feedback queues. Now again, normally when I implement this algorithm in order to be a little bit less dramatic about how I move threads around, I usually don't normally move them immediately, right? You know, if a thread has a history of bad behavior, I shouldn't say bad. If a thread has a history of hitting the end of its quantum, it goes into the bottom queue eventually. But usually, hitting the, you know, if I have a thread that's in the top priority queue and it happens once to run to the end of its quantum, it doesn't immediately move down, right? I probably, you know, dock at some points, leave it in the top queue for now and if that happens enough, right? If the thread has gone from taking user input to computing digits of pi, eventually it's gonna descend and end up in the low priority queue, right? So CPU bound threads, a thread that is entirely CPU bound, it's not using any of the resources, that thread goes straight down, right? And that thread ends up at the bottom and it's only run when there's no other work to do. IO bound threads tend to stay in the top queues, okay? So we've, I've already pointed this out, there's a clear starvation problem here, right? So it's possible without doing anything special that particularly if I have enough threads and not enough queues, that threads will end up trapped in some of the lower priority queues and those threads may be starved and they may go extremely long periods of time without ever being able to run. So who can propose a solution? Your first day at your new job as an OS developer for Facebook, Facebook is making a new OS for mobile phones, right? They're like, all right, I'm a lot of queues, keep starving threads, what do I do? Yeah, I just like periodically just take all the threads and just dump them back in the same queue, right? It's like I just sort of start over. This kind of gross, kind of dumb, like a little bit of a hack, but whatever. It tends, you know, like this algorithm just suffers from this problem. This is not necessarily the worst solution, but it's not necessarily the best solution, right? So it's probably not the worst. I just periodically rebalance them, yeah. Yeah, so we'll come back to that on Friday, actually. So Friday's lecture when we talk about the, what is it called, anyway, that some of the Linux work on interactive schedulers, they have a scheduler that feels somewhat like MLF queue, but yeah, what it does is it gives priority to threads in the high priority queue while not starving threads in the low priority queue. It's a neat, it's the rotating staircase deadline scheduler. There it is. I knew I could sum it up. Okay, so coming back to priorities, right? And priorities, you know, don't necessarily feature to these algorithms very much. And that's because priorities, like we mentioned before, are really sort of external information that the system can layer on top of other algorithms to try to get it to make better decisions, right? So, you know, if you look at your own systems, normally what you'll find is, you know, has anyone ever inspected the priorities of things that run on their system? You guys run top. You should do this in your virtual box, right? Fire up top to find all sorts of weird stuff running that you don't understand what it is. And those tasks will have priorities assigned to them. Okay, did you assign those priorities? No, I mean that the system was configured that way, right? So someone at Ubuntu decided that, oh, by the way, the cron daemon, which is in charge of periodically running background tasks, that we can put it kind of a low priority, right? Whereas a shell or anything you launch as a user might run at higher priorities because, again, it's you, you're interactive, you want things to perform with it better, right? The priorities are, and you know, you can imagine using priorities in combination with interactivity for an MLFQ scheduler in order to pick what queue to put threads in, right? So I can combine its interactive behavior with a priority that was set from outside, okay? And yeah, I mean you can imagine, you know, simple, establishing simple priorities for things based on whether or not this is, you know, a task that has a responsive component or whether it's a task that's just something that needs to complete eventually, okay? And yeah, the other problem with priorities, of course, the priorities are always these, like, again, a priority is really a meaning, usually a meaningless thing, right? If you look at Linux, I think Linux priorities are also backwards, if I remember correctly, like smaller ones are better. Who would have thought, I think they go negative? This just gets weird, right? Because it's just a number, and all the computer knows is if the priority is three, it's more important than one has priority of five, right? So priorities really only ever establish these relative relationships between things on the system, and so they're host to a variety of different problems, right? So one problem the priorities create is something that, well, there's a bunch of different problems. I mean, starvation is one problem that priorities can create when priorities are treated too strictly. So if I have strict priorities, and I go back to MLFQ, I'm always running things from the top Q if there's a thread that's available in the top Q. This, doing something like that will produce sleet of starvation varies, right? A somewhat elegant approach to this that some of you guys might want to play around with for your assignment two schedule or something called lottery schedule, right? So can anyone guess how lottery scheduling works? Anyone want to try to describe a lottery scheduler? Pretty similar to how a lottery works, yeah. Yeah, I mean, the mechanism you can think about is lottery tickets, right? But yeah, this is normally done with probability. So imagine I gave everybody in this room a certain number of tickets. I give some of you more than others, right? Because I like you, because I think you look like you have important work to do, because you're reading the paper, I don't know, whatever. So some of you guys get more tickets than others, and then the way I choose, this is like a raffle, right? I mean, if people go to these events that have raffles, you can buy like 600 raffle tickets, right? You've just increased your chance of winning the raffle by some amount, right? When it's time to pick who wins, I pick a number. I figure out who has that number that person gets to run, right? So how does this handle starvation? What's the difference between this and something like MLFQ or something that would use strict priorities? Yeah. Yeah, so there's, for every threat on the system that has any tickets, as long as you have a ticket, there's a non-zero probability that you are going to be chosen to run. It may be very small, but it's not zero, right? And so at the limit, if I hold enough lotteries and pass on enough tickets, every threat on the system will have a chance to run, okay? Now, having this probabilistic component to it may not make everybody happy. It can be a little bit difficult to sort of prove things about schedulers like that, but this is a solution to the starvation problem caused by MLFQs and other things, right? And keep in mind, I mean, I can play other games of scheduling too. Another thing I control other than just access to the CPU is how long threads get to run, right? So it's possible that higher priority threads, I may give longer quantum, right? Because I assume they're doing important work and I want to give them more time to do that on the CPU, okay? So, priorities and tickets are examples of these abstractions that we can use to think about scheduling. I just wanted to point this out. These clearly really have no actual basis in hardware, right? It's just an abstraction that we use to make the process of thinking about how to schedule threads a little bit more in tune. All right, so on tomorrow. We're having lecture tomorrow, apparently. Okay, Friday, right? We will talk, so Friday is a lecture about pseudo-modern litics scheduling. It was modern four years ago and I wrote the lecture, but four years have passed. So we'll talk about a particularly interesting interactive scheduling algorithm. There's some little anecdotes about Linux development that are kind of fun. So I hope this is a fun lecture. This will be the last lecture we will do on scheduling and the last lecture that we will do on CPU abstraction in general, which means next week we get to start memory, which is awesome because I think memory is really cool. So, I will see you guys on Friday.