 Welcome back to CS 162 everybody. We have some brave souls that are here on the night before the exam. So that's great Today we're going to Briefly finish up something we didn't get to last time and then we're going to dive into a new topic, which is scheduling so however, if you remember from last time among other things we did a lot of talking about the monitor paradigm for synchronizing and And a monitor is basically a lock and zero or more condition variables for managing concurrent access to shared data And monitors are a paradigm that we talked about and some languages like java actually provide monitors natively For languages that don't have it natively then you can get a a condition variable class that integrates with a lock class and you can program With the monitor idea still And so a condition variable is kind of this key idea, which is a queue of threads Waiting for something inside a critical section. So the key idea here is you allow sleeping inside the critical section By sort of hiding the fact that you're going to atomically release the lock and and reacquire it and this is in Contrast of semaphores, of course, where you can't wait inside the critical section And we talked about the operations. There's three main ones. They have different names depending on the implementation, but weight signal and broadcasts are the basic ideas there And the rule is always you have to hold a lock when you do any sort of condition variable operations And so this was the general pattern that we talked about from mesa scheduled Convitched condition variables. Okay, and basically the idea is that you Typically grab the lock you check some condition and potentially wait if If the condition is not succeeded because it's mesa scheduled when we wake up from the weight We always make sure to recheck the condition. So that's the while loop and really that's because it's possible that somebody will have gotten in and Made the condition invalid again before we actually got to start running again And then you unlock you do something and then there's always a closing condition where You grab the lock again. You'd maybe do some other things But eventually you're going to signal to make sure that anybody who is waiting is woken up And then you unlock the key idea here is to be thinking That you're always within the critical section you grab the lock here You release the lock there and all the code that you see as a programmer between the lock and unlock are always considered to be Executed with locks on Okay, and of course obviously when you actually go to sleep under the covers It releases the lock and then reacquires it before you start up again All right, and then we spent a fair amount of time last time using this pattern to solve the reader's writer's problem So I wanted to see whether there were any questions on monitors before we kind of move on Any remaining questions at all? Okay Good so The thing I wanted to finish up last time. So I was at the very end of the lecture kind of showing you some You know examples of scheduling of threads within the kernel Here the one thing I did want to do was talk a little bit about the i o portion Of the i o interface because we didn't talk about that earlier in the term And if you were really to look at the layers of i o that we've talked about There's this at the user application level You might execute a read which is a raw interface level that translates directly to a system call Inside the libc library That system call basically marshals the arguments together into registers calls a sys call And then the results of the sys call Are then returned from this read system call as if it were like a function Really is kind of like a function that calls the kernel You're getting familiar with this now with project one, of course And then inside that system call is a type of exception But the same thing happens if you have interrupt processing But that goes into the system call handler where you un-martial or take apart the arguments And then you dispatch a handler based on what you're trying to do So it might be dispatched based on the fact that this was a system call To a read for instance, and then you marshal the results back and you return them from the system call And then one layer deeper when we dispatch on this read we might actually call something called vfs read inside the kernel which takes the description file structure for that file and a buffer and a few other things And it basically Does the file access maybe calls a device driver in the process So I wanted to look a little deeper in these lower layers here Just so you have seen them once or twice is going to be useful as we get further into project two and three You should know there's many different types of i o But as we were talking earlier in the term the unix Way or the posix way basically treats all of these like file i o okay, and so That system called interface read write open close Uh Basically translates into calls across the system call boundary in the kernel. That's this blue thing right now And depending on what we're calling whether it's say a file system actual Storage in block devices or maybe it's device control like a serial link. That's what the tty's are keyboard mouse Or its network sockets The same open close read write system calls are used and the question might be well So how does that work? Okay, and that's the magic of standardized api so the magic of the standardized api Is arranging so that all of these very different things can all be accessed as if they were files all right and The you know internally there's this file description structure Which represents an open device being accessed like a file When we return a file descriptor as an integer As you recall that file descriptor that the process knows about is mapped as a number is mapped inside the kernel to one of these Structures, okay, so this is the internal data structure describing everything about the file We haven't really talked too much about it. We mentioned it once before and you've probably straight into it The equivalent of this in pintos now And uh, it talks about everything having to do with that device like where the information resides Its status and how to access it and it's um You know in the kernel typically what gets passed around is a pointer to this Uh file structure. It's a it's a struck file star and everything is accessed with that file structure and You know, we can't give that file structure to the user. Why is that? Why can't we give that pointer up to the user? Does anybody know why that because it's in kernel memory exactly so those addresses don't mean anything to users And the capital file stars we've talked about are different from this the capital file star Represents buffered user level memory buffering a file. This structure represents the actual Internal implementation of the file and you see something here, which we're not going to talk about today called an inode So when we start talking about how file systems are implemented Uh, the inode's going to come up a lot, but that inode can point at a wide variety of things not just file blocks And the thing of interest here for today that I want to talk about is this file operations structure, uh, which is the f underscore op item in this file structure and that basically describes how this device implements its operations So for disks it points to things like file operations in the file system for pipes it points to Pipe operations. I noticed uh, there were some uh questions on piazza about well, how does a pipe Get implemented in the kernel. Where is it? Well, it's a it's a queue inside the kernel And how do you access it? Well, you access it because the file structure is uh pointed at by, um Excuse me the file structure points at both the queue in the kernel and has a set of operations on how to read write etc for the For the pipe All right, and for sockets it points to socket operations, okay So the cool thing about this is really that by putting this layer of indirection in here, um You basically get the ability to have everything look like a file from the user's level Okay, and all of the complexity is buried in this simple idea the simple interface So for instance, here's an example of what that file structure looks like For those of you out there So here are the set of operations that are standardized things like seeking reading writing bytes. There's uh Asynchronous io reads and writes etc. Okay, how to open a directory or read a directory Um how to open how to flush and so on these standardized operations are the ones that devices that want to look like Files to the user have to provide um All right, and uh, you know, so for instance, I think I um, I don't know if I had this slide or not But for instance for a pipe Uh, there are two different file structure That are hard-coded in the pipe implementation For the file operations one for the read end and one for the right end and things like the read end Uh, don't have write Uh calls and things for the right end don't have read calls. Okay So this vfs read that I showed you the virtual file system read We'll talk a lot more about this in detail later in the term But uh, what it's got for instance is uh, this is where the read system call goes and it says for instance It takes the file star in Okay, and then it says read up to count bytes from the file starting at the position That's uh kept out in the file star into the buffer that's given here and it returns an error or the number of bytes read Um, here's an example of something that says well if we're trying to read Uh, do we have read access or not? Okay Check if the file has read methods, right? If we try to read something that doesn't know how to read then we're going to fault So a good example that would be trying to read from the right end of a pipe for instance And the other thing is that's very important that you've started to do in project one Or probably should have been doing already is uh, check the user's buffer is actually accessible to it Okay, and if the user doesn't really have access to the buffer he wants to put things in Then uh Basically this will fault Okay, and then um, you know, can we actually read from the file? That's another question range And then we check and if there's a read operations we do it Using the the specific read operations otherwise we do it's called a synchronous read which will use the provided asynchronous data or asynchronous operations to read from it and then um when we're done We notify the parent that something was read. So this is sort of when you see file browsers Um that you have open on the screen and you create a new file You'll suddenly see the file will pop up in the file browser while there's notification going on inside the file system Um, we'll update the number of bytes read by the current task. So this is sort of scheduling information We'll talk about schedulers in a bit for cpu cycles, but it's possible that um We're and it's probable that we're scheduling also the number of bytes We're pulling off a file system and we may choose to uh suspend a process That's reading more than we desired from at a given time And then we update the number read calls again some statistics and we return Okay, so the idea here is that everything at the top level has been designed in a way to be uh easily plugged and played from a wide variety of devices underneath All right, any questions on that? So, um underneath the covers even further below than what we showed here. Let me just back up. Whoops If you look here, um, I wanted to show you What did I want to show you? Oh, yeah, so in this, um Many different types of i o We have the high level interface which we've just been talking about and then at the bottom We have the devices in somewhere in the middle We have to have things that know how to interface with all the unique characteristics of the devices And provide enough standardized interface up to interface with the the kernel Okay, and those are as you probably are well aware called device drivers So going back to where we were here So what a device driver is is its driver device specific code in the kernel that interacts directly with the device And it provides a standardized internal interface up Which is not surprisingly going to be very close to those file operations. I showed you earlier And the same kernel i o at that lower level can easily interact with different devices So the device driver level is going to give us the uh ability to interact identically with say a usb Key that you might plug into a usb port versus a disk drive that's spinning they have the same kind of interface in the kernel after you get out of the device driver Okay, and the device driver is also going to provide the special device specific configuration from the iockel system calls We gave you a good example of well if you got a device that yes, it does open close read write But there's some special configuration that doesn't fit into that standardized interface Then you'll typically use the iockel calls for that device to set things like you might set resolution on a display Or uh, or baud rate on a serial link And the device driver typically has two pieces to it. There's a top half and a bottom half The top half of the device driver, which is uh interfacing up into the kernel Is accessed at a call path from system calls um, and it implements Standard cross device calls. These are going to sound very familiar to the to you from the f ops. We were just talking about earlier This is a slightly different layer a little bit lower, but it also supports open close read write iockel. It also Has a strategy routine, which is typically the way that you start I o starting and so for instance if the file system design Decides that there's some number of blocks that need to be pulled out of the file system Then uh once that's been determined then the strategy routine can be Put together to start the actual i o happening So that's the top half and the the thing about the top half is that uh processes that are running or threads Can go all the way through to the top half And if the i o doesn't actually have to happen they can return from that and return to the user But if the i o happens to have to happen to a slow device Then the top half is potentially going to put things to sleep And the bottom half runs pretty exclusively as an interrupt routine Okay, so it gets input or transfers the next block of output based on interrupts that have occurred and it may wake sleeping threads In the top half if the i o is now complete Okay, and so here's a an example of a An i o request coming from the user where the user wants to let's say do a read So the user program at the top here is going to request some i o that might be a read system call Or it might be an f read or whatever in the buffered i o and That's going to cross into the kernel. So that's doing a system call And the first thing the kernel is going to do with a say a disk drive file system situation is it's going to say Well, can I already satisfy the request? Okay, and if the answer is yes, then it'll immediately Copy the results into the user's buffer and return up. Can anybody tell me why The kernel might be able to immediately satisfy a request From the user What what might what might be the reason that we could say yes Cash yes, so for buffer. So it's quite for the simplest way to think about this is if I execute read 13 bytes at a time to a file system The disk is transferring blocks that are 4 to 16 k in size. So my 13 bytes May bring in a whole let's say 4k at a time And that'll be put in a cache And there is a couple of different places of cash. We'll talk about as we get further into this later in the term But that cash is going to hold the whole 14 kilobytes. And so The first time that may return that take a little while to go to the disk and return to the user But the next time for the next 13 bytes. It's already in the cache Okay, so But assuming that's not true Then we're going to we realize we got to do some actual i o to an actual device So this is the point at which we're going to try to send the request to the device driver And potentially put the the process to sleep at that point. Okay, or the thread and Um when we get Uh to the top half of the device driver It sort of takes the request and it figures out how did that translate to the particular blocks that need to be read off the disk Um and when we talk about file systems, you'll get an idea of where the you know How we figure out which blocks and then it's going to put that thing to start the Start the actions with the disk drive and then put things to sleep And that's going to be the place where for instance the strategy routine takes over But at the point that the top the device driver is done and has sent the request to the device It then puts the thread to sleep So you can say that the thread kind of started at the user And worked its way all the way down to the top half of the device driver and it's now sleeping Okay, and meanwhile, um, yeah, that's a really good question. Does the device driver have its own process? Not exactly. Uh, so notice that everything i've talked about here is kind of running in Uh Response to the process is request of the system call and so it's running on that process is kernel stack And eventually we get to this point where we can't go any further And what happens there is that kernel stack or kernel thread gets put to sleep by being put on a weight queue with the uh With the disk and then another thread will be scheduled to run. Okay But the the thing that has run Up to this point has actually been the kernel thread of that user process Now you might another reason you might say whether it has its own process is okay So nothing's running anymore. What happens? Well, we've asked the device To go ahead and start executing. So the device is off doing its own thing And eventually when it's done, um, possibly, uh, transferring the data Into memory through through something called dma. We'll get there again a little later in the term It will generate an interrupt To tell the uh The system it's time to to wake up and that interrupt is going to go to the bottom half of the device driver, which is now going to figure out Who might be sleeping waiting for that block? In which case it's going to determine who needs the block and it's going to wake up that process And from that point the process now the original process, which uh, you know Was kernel thread put to sleep now wakes up and says oh, let's transfer the data to the process's Uh buffer and then return from the read system call Okay So any questions on that? Now there's a lot of interesting details in here, which we haven't talked a lot about yet But for instance the uh process of sending no pun intended the act of sending requests to that device It's possible that the device driver is going to take a set of requests from a set of different processes and reorder them Using something for instance often called an elevator algorithm so that the the requests Do the least amount of head movement? Okay, but that's that's a topic for another day right now. We're just talking about the device driver Okay, so That's what I wanted to say about device drivers I think the important things to get out of this for now Is this idea that it's the kernel thread associated with the requesting thread that uh is gets put to sleep and assuming That uh, we have a one-to-one mapping between the user thread and the kernel thread And there's a post that I did on that last night in piazza as well Then putting the the thread to sleep here in the the i o is okay because all the other threads have their own kernel threads and get to Keep going on the other hand if we had a bunch of user threads running in this Environment let's say if any one of them were to do i o they got put to sleep then all of those user threads associated with that one kernel thread did put to sleep okay, so that's That's the danger of sort of multiple of the many-to-one model where multiple user threads are associated with a single kernel thread good so, uh If you're not familiar with it, um, I will make sure you know tomorrow is a our first exam It's five to seven. Um as been stated. Um, we have some special Dispensation for 170 folks This is video proctored I know there was a lot of people that were worried about having to prop their cell phone and so on so there's been a bit of an update on what we're Asking of you, but we definitely do need the webcam Turned in and you logged into zoom with screen sharing while you're doing this And the more details are up on the pin piazza link. So hopefully everybody's got that figured out now Um, the topics for this exam are pretty much everything up to monday. I know we originally said everything up to Lecture eight, but really lecture nine was mostly a little bit of a continuation of lecture eight. We spent some time Looking at monitors in a little bit more detail, so Scheduling which is the today's topic for the rest of the day is not part of the exam So don't worry about that Homework and project work is definitely fair game So The other thing is the materials you can use during this exam. This is closed book closed friends closed random internet people Know open Channels with folks You're allowed to use your brain Yeah, you can use your personal device drivers, which hopefully will drive your fingers properly when you're typing answers in um You can have a single cheat sheet that you've written both sides and you can eight and a half by 11 hand written Um, and I think that's it any questions Oh our device drivers in scope You're um, they're not going to be in scope on the exam So things things from today that didn't get talked about uh on monday are not in scope But you're allowed you're allowed to use the device drivers in your fingers as well Okay Any other questions? Yes, so there will be a sheet at the end I think there was even a piazza post showing you what it was like But uh, there will be a sheet with uh important System calls and function call signatures that you need to know But it won't hurt to it wouldn't hurt to know the ones that you've been using a lot in your project Scope just in case we forgot to give you one All right And things that like thread creation and so on If you don't remember the exact, um Signature and we didn't give it to you then Do your best to come up with The uh the signatures that you need. I think we've been pretty good about that, but if we forgot something You know You know what the signature should look like pretty much because you've been using it So I think you'll be okay if you've got some arguments slightly reordered That's okay All right But most of the things that you need signatures on we will give you For system level things Okay now If there aren't any more questions about the midterm Let's go on to topics for today I'll pause one last time here How hard is it? Just perfect Uh Perfectly not too hard not too easy just the right amount for the time that you have to do it. I don't know Uh So, um, I would say use your primary monitor All right, um If uh in terms of printing the exam, it's an online thing unless you've talked to us, uh and have special, uh Have some special arrangement. So it's it's going to be an online exam You'll be able to put your answers in and when you save on an answer It holds on to it for you. So there won't be any losing of the answers or anything like that All right Okay, and all those details should be posted already So you should take a look make sure you're ready with any setup in terms of being able to log into Getting your zoom setup and so on you should probably try that out tomorrow or tonight whatever just to make sure you're ready okay Today I want to switch topics a little bit, but it isn't really that big of a switch. We've been talking about Uh, how the mechanisms for going from one thread to another. Okay, and we talked a lot about that over the course of the last, you know nine lectures And the one thing we didn't really talk about is how do you decide which thread to switch to? Okay, and that's uh That turns out to be an extremely interesting topic So, you know, I'm showing you a loop here This loop kind of represents everything there is about a thread Or excuse me everything there is about the operating system Which does the continuous loop and it basically says if there are any ready threads pick the next one run it Otherwise run an idle thread and the idle thread is typically just a kernel thread that does nothing keep some statistics And we just keep doing this over and over again And the question about how frequently do I loop? And when do I cut somebody off and pick somebody else? Or how do I choose between the uh, the 12 things that are ready to run? That's all scheduling and it's interesting scheduling because there are many policies many different reasons for choosing one thing over another And so scheduling is actually a really a deep topic that we Spend a couple of lectures on because it's interesting And Another figure that I showed you a while back Is this one which is kind of showing the cpu? Busy potentially exiting or executing stuff and every now and then something happens so that the current thread That's running in some process has to stop. Okay, and examples of that are for instance If we do some i o we were just talking about that a moment ago where the cpu Enters the device driver to do some i o and the device driver puts it to sleep You get put on the q with that i o and you got to wait for the i o to happen. So that's an example of the cpu Relinquishing the thread that's running and putting that that kernel thread on On a q and of course the thing that has to happen immediately after that as well Let's go at the ready q and pull somebody else to run because we don't want to waste cpu cycles And the other thing that's kind of interesting here is we're busy running along and the timer goes off And at that point we say that the time slice is expired And we put that item That thread that was running back on the ready q because its time is up But you know, we got to pull somebody off the ready q to put it on the cpu Okay, and then of course when we're doing fork or we're doing some other Scheduling or so on excuse me some other synchronization that requires interrupts We can be pulled off the cpu as well And the real question is scheduling is how's the os to decide which of several tasks to take off the q? and you know, it's If if you didn't learn about this you might easily say well, this is dumb you just pick the next one Why is that interesting? Well, the answer is that picking the next one is is Rarely the right thing to do that you can have what's called a fifo scheduler And of course we'll talk about that one first, but that's not the best thing to do Um There may be many other things where you got to pick the one that is uh got highest priority for some reason Or you pick the one that's more likely to make you typing at the keyboard happy because your keystrokes get registered Okay, and so scheduling is this idea of deciding which threads are given access to the resources moment to moment And for certainly these next couple of lectures, we're going to be talking about cpu time is the resource We're talking about but in fact very interesting scheduling happens with respect to discs We could say well, I've got a set of tasks and I want to make sure everybody gets equal bandwidth out of the disk drive Okay, so that's a scheduling priority requirement, okay for policy Um, but today and you know and next time we're definitely going to be talking about the cpu Okay, so, you know, we're all big fans of of cues of various sorts Um, and uh, here's a FIFO cue that looks like uh, they're not social distancing. So this is uh, A picture from a little while ago Um, so what are some assumptions? Well in the 70s, maybe this is a picture from the 70s. So in the 70s Scheduling was kind of a big area of research um computers were new enough that people hadn't really figured things out and um The the usage models were were pretty basic because people had mainframes in big rooms and uh, those were Multi million dollar machines and you had a bunch of people using them And so you had to somehow make sure that those million dollar Super million dollar resources were properly shared among different users Because they were just expensive and you couldn't let a user Uh Take too much time, but you also couldn't let a user who's maybe spent money for computer time be upset Uh, because they're not getting their fair share And so The thing that's interesting is there is a lot of implicit assumptions in these original cpu scheduling algorithms Of things like the following one program per user. Okay, or one process maybe per user Um one thread per program programs are totally independent of each other these kind of ideas are Certainly not the case anymore, but they're a good place to start when we sort of dive into scheduling Um, so these are a bit unrealistic, but they do simplify the problem. Uh, so it can be solved initially So for instance the question might be what's fair Is it is fair about fairness among users or about programs? so if you think about it if you have one user And uh, they have five programs and a different user has one program How do you share the cpu? Do you share that by cutting it in six? and giving one six to each uh each program So now the user just by having multiple programs gets more of the cpu than the user who only has one program Okay, so that's a type of fair which is fairness per process, but it's not fair per user, right? um You know if I run one compilation job and you run five then you get five times as much cpu as I do Is that fair? I don't know um The high level goal of course is still doling out cpu time Because when we do this swapping from user one to user two to user three to user one to user two to three Or earlier we saw these were threads or their processes We need to decide which one's next and we got to do that with some policy in mind And the interesting thing hopefully you'll figure out by the end of the lecture is There pretty much isn't one policy that goes well for every situation And it's it's often the cases where people have tried to come up with a single Scheduler that were to to work across a wide variety of platforms that things have gotten in trouble and not worked Well for any of the platforms So one thing that we might use as a model for this here's an interesting idea Which is burst time or cpu burst? And what you look here on what you see on the left is the idea that we run for a while And then uh, we go to wait for some i o and then we run a little longer And we wait for some i o and we run a little longer and we wait for some i o And in each of those instances um During the waiting of course, we're on some q Okay, because we can't run and so we're off the ready q and pretty much other people get cpu time And so the execution model is really that programs are alternating between bursts of cpu and bursts of i o And what's interesting is if you were to measure that On some system. This is totally unnamed here for a moment and you were to put burst time on the x axis And uh, how often you see a burst time? Of that size you see there's a peak at the lower end But a really long tail Okay, now just to be clear in case you're wondering again. What do I mean by this x axis? I mean that if you look at this thing on the left and you say I run for a Some amount of time before I go to sleep that some amount of time is is on this x axis Okay, and so some of them are really short like for instance if I type What does that do I type a character it generates an interrupt There's an interrupt routine that's run that character Maybe gets forwarded through the windowing system to a to a thread that's waiting for it And then that thread goes to sleep waiting for the next thing and that might be a very short burst because the amount of work That happened for typing a character is short On the other hand Certainly you have some processes that might be running for a long time like computing the next digit of pi They're going to be way out at the end of the long tail And so if you were to look at the set of tasks You'd find that there's a it's weighted toward the small bursts because there's a lot of those little ones And furthermore you might infer that those little ones have something to do with interacting with users Okay, so programs typically use cpu for a period of time and then do io and then keep going And scheduling decisions about which job to give the cpu to Might be based on burst time. Can anybody think what might be a policy? That we should use based on burst time for scheduling cpu What might be a good one and can you think of a reason? Okay, give the long duration first. Okay, that would be one policy. So why So if I gave the long durations first That means that things that are short duration are potentially waiting a very long time Right, so the long duration first Might give me a lot of efficiency because i'm using the processor cache as well, but it's going to really be uh A bad impact on the short burst ones because they're just going to run quickly and finish, right? so Okay, we could go a pseudo randomly sure anybody give a justification for why we might want to optimize for the short ones What might be a good reason to try to optimize for the short bursts? So we have maximized the number of processes that get to go in a fixed time that could be one idea There you go. I saw somebody that said for responsiveness. There you go Because if those low bursts are about interacting with users And they're short I want to handle them quickly because The users get the big benefit of seeing their you know, they type the letter A and then it shows up on the screen quickly And the long ones are hardly going to notice if you uh hold them up for a moment to run a short one Okay, and so really optimizing for short bursts may have something to do with responsiveness and I will say something that will surprise you a little bit, but maybe you've run into this and shared resources, but Back when I first started writing papers And I was using a mainframe to do it There were a bunch of people that logged in And we had such a high load with so many people logged in that when I had a emacs up And I started typing You know, you might type, you know to be or not to be that is the question And then a second later or three seconds later the whole the whole phrase would show up So the the scheduling was so bad that you could type whole sentences and then it would pop up on the screen Okay, and so that's not very responsive And you can imagine as we've gotten more and more in a situation where people are You know have cpus of their own so they have lots of cell phones and other things you're going to want to be Cognizant of responsiveness Okay, so So what are some goals that we might have for scheduling? So for instance, uh, and yeah the sum of the waiting time is going to be the smallest That's going to be a metric that we're going to use in a moment. So that's very good. So, um minimizing response time Okay, we want to minimize the elapsed time to do some operation or job and um The response time is really what the user sees. It's the time to echo a keystroke in the editor It's the time to compile a program and for real-time tasks, which we're not going to get to real-time today We'll do that next time, but um, you have to meet deadlines imposed by the world. So, uh, my my favorite example of this is Most cars these days have hundreds of little miniature cpus in them And I want to make sure that when I slam on the brakes in my car that, uh The system is responsive and applies the brakes Quickly and timely so that I don't smash into whatever I was trying to avoid So there's a real-time deadline in that scenario where it's just it's not just Keeping me happy as a user. It's keeping me alive, right? So that's we can get into real-time Where the deadlines are far more important than the kind we've been talking about up till now We'll talk about that next time um another Scheduling uh thing which tends to be in the big cloud services is maximizing throughput So this is completely different than minimizing response time maximizing throughput is about saying I want to maximize the number of operations or jobs per second And the throughput is related to response time, but it's not identical um, and uh minimizing response time Leads to more context switches because to minimize response time what happening is I'm handling a whole bunch of really short jobs And then a long one for a while and then a whole bunch of short ones and as a result I'm context switching a lot And as you know, all context switching has an overhead associated with it And so I'm actually not getting the maximum throughput when I'm context switching Okay. Now there's two parts to maximizing throughput One is minimizing the overhead which is not context switching much And the other is very efficient use of resources like the cpu disk memory, etc and a key on this which uh It's something to start Putting into your viewpoint here and thinking about is by not context switching A lot not only do I avoid Uh overhead, but I also avoid disturbing important things like cash state Okay, so um in 61c you talked about the power of caches We'll talk about the power of caches when we get to file systems But by not switching but rather running for long periods of time What I do is I get a chance for the cash state to build up and then for me to take advantage of it So high throughput Is often in direct contrast to response time those two are conflicting with each other Another one which is really funny is fairness, you know, what does that mean? And there's so many different versions of fairness You could say well roughly i'm sharing the cpu among users in some fair way But fairness is not about minimizing average response time Because the way I get better average response time is by making the system less fair Anybody tell me why that would be so why is better average response time Achieved it make it by making the system less fair That makes sense to anybody There you go People who make more requests because they have a lot of bursts get priority and they get priority over other users So the mere act of having bursts Gives you more cpu Okay, and I'm going to give you a funny instance if we get to that today of uh somebody in the early days of computer Um Othello for instance playing each other Figured out that by putting print statements Into their Othello code they could get more cpu time than the other guy and get an advantage so So let's start with the first obvious thing here, which is first come first serve or FIFO and Really you could think of this as run until done So in really early systems first come first serve basically meant You submit your programs in a big queue at night and you'd come in in the morning and they would have run one after another And FIFO order now that was the original Notion you ran everything to completion today. We basically run the cpu Until the thread blocks. So what it says is if you have a queue the ready queue You put things on the end of the ready queue when you get pulled off the cpu Or woken up for my oh and then the the scheduler just grabs the next one off the head and keeps going down the queue One at a time in FIFO order. Okay And so to show you what this means here's a gantt chart I'm sure you guys have run into these before but it's basically showing Uh a set of a sequence of events in time And what we're seeing here is an example where uh three processes p1 p2 p3 Came in uh to the queue the ready queue the first one had a burst time of 24 second one had a burst time of three the third one had a burst time of three and so we run for um 24 until the burst is done then we run for three then we run for three And um if we were to view the users of these processes We can ask ourselves. What's the waiting time? Uh for them. Okay, so process one doesn't wait at all Okay, because they start right away process two has to wait 24 Time cycles whatever this is process three has to wait 27 And so we could average the waiting time there by so zero plus 24 plus 27 over three and that's 17 And we can talk about average completion time right p1 ends at 24 p2 ends at 27 p3 ends at time 30 And that gives us an average completion time of 27 So um this average waiting time and average completion time end up being metrics that we could Optimize for if that turned out to be something we wanted to do Now the big problem with first come first serve scheduling there are many problems, but let me just show you Uh the the biggest one here is what's called the convoy effect, which is short processes are stuck behind long ones This is also the You know five or less item problem at uh safe way Where you come and you you only want uh, you know a bottle of milk and some chips And you try to go in the short line and some person in front of you has decided that their card full of 50 items fits into the five item Requirement you've just come to the convoy effect because you are now serialized behind that long Job, okay, and so here's the convoy effect with uh first come first Served scheduling what happens is we see when arrivals are coming here. So the here the arrivals are showing up This is the actual execution at the top We had a blue one executing and then the green one arrived At this point and the green one doesn't get to start until that point and so on and now the the uh, dark green one Gets to run. Okay, and then the red one shows up and the blue one shows up, but now the red one is long All right, um, and so that Blue powder blue one now is stuck for a very long time. So if you look at the queue Here, that's what's going to build up the red one is now running But now the blue one is is queued and another one comes along and another one comes along and another one comes along And all of these tasks are stuck while the red one is running and then when it finishes they all Finish out pretty quickly. So this is a convoy Of jobs that are all stuck waiting for some long job. That's why it's called the convoy effect okay all right and The funny thing about this is if I were to just switch the order of what I showed you earlier Where I showed you p1 p2 p3 if they were to come into p2 p3 p1 Then look what happens here p2 Gets to run quickly p3 gets to run quickly and then p1 runs And so the waiting time for p1 is now six So notice that p1 only has to start six units of time before it starts running p2 has zero and p3 has three The average waiting time now is six plus zero plus three over three, which is three The average completion time is three plus six plus 30 over three, which is 13 And if you compare for what we had before Just because p1 arrived first before rather than last notice that We had An average waiting time of 17 when p1 showed up first now. It's only three We had an average completion time of 27 when p1 showed up now. It's 13 so you can see that First come first server fifo has this problem that uh, you know, it's very uneven as to how you service things So the pros and cons of this are You know short jobs get stuck behind the long ones is a definite con. It's simple. Okay, so that's a pro But um, and it's going to be the simplest we come up with in this set of lectures, but This is really the safe way get the milk effect You know, um, I guess the good thing is you can sort of read the the rags there While you're waiting for that other person to get through and find out about the the space aliens that have landed somewhere in Nebraska, but It's always getting the milk. Yep. The milk milk is is the important thing here in operating systems Okay, we haven't spilled any yet though. So we'll have to see what what happens when we spill the milk, but all right, so let's see if we can do better. Okay, because this this, uh Unevenness with scheduling it seems like a downside at minimum and uh, and then you know, there's got to be something better so the simple thing we can do is what's called round robin scheduling And um, this is going to be our very first stab at fixing first come first serve And um, really the first come first serve. I mean, let's look at this. This is potentially very bad for short jobs Which is going to be very bad for responsiveness to users Right here. I'm showing you the best first come first serve the previous Worst first come first serve was extremely bad and we don't know are we going to be responsive or not? and that just seems It's going to annoy the users right and so um What else can we do? So that's a robin there because you know, I can do cut and paste And put in clip art, but the round robin scheme is going to be all about preemption Okay, so every process is going to get a small amount of cpu time and uh We're going to call that a time quanta in typical operating systems today. It's 10 to 100 milliseconds And we're only going to let jobs run for a time quanta and then we're going to preempt them and move on to the next one Okay, so after the quanta expires Timer is going to go off. This should sound familiar to everybody Process is going to be preempted and it's going to be put on the end of the ready queue And the next one in pipe orders is going to be pulled off the front and this preemption Is going to give us different behavior Now we can do some analysis very quickly about this right so if we have end processes in the system um then uh this is uh We can figure out that every process if it's running for a long time gets one over nth of the cpu time Okay, and yes, this is a As uh said on the chat a quantum leap in scheduling So in chunks of at most q time units We can see basically if there's n processes in time Chunks of q time units that no process is ever going to wait for n minus one times q time units So there is now a minimum Excuse me There's a maximum amount of time we have to wait Which is going to mean that uh, we have some sort of a minimum level of responsiveness that we can get out of the system At least so as to not uh annoy users too much Now the system I talked about earlier where I was typing in whole sentences took a long time to show up Was still like this, but it was a situation where n was so large that this time got too large Okay So what about round robin scheduling so the performance well if q is extraordinarily large that's the quantum time We we reconverge back to first come first serve Right if q is really small we interleave Okay, and that actually I guess if you thought really small you might think of this almost like what the hardware hyperthreading does So q Clearly has to be big enough that the overheads don't kill us So the context switching isn't the only thing we do we actually do some computation But it can't be too big or we don't get the benefits from a responsiveness standpoint So here's an example of a round robin with time quantum equal 20 So I have a set of four processes here. I'm going to show you the gantt chart for this Uh process one's got a burst time of 53 process 280 Uh, excuse me. Let's try that again process one has first time of 53 process two has eight Process three has 68 and process four has 24 And so process one being the first one to arrive runs for 20 And the and the timer goes off Okay, and the question a good question here is do we know The burst time a priority No All right, we're going to talk about that in a moment So burst time is some magical prediction of the future, which we're going to have to address in a moment However, I can tell you after the fact if I've observed what happened I know what the burst time was because I know how long it ran Okay, so in the next few slides Assume that what's happening here is I say well, I knew how long that was going to run And I'm just playing with the scheduling to see what's different. Okay We'll get to where burst time comes from in a bit Okay, because nothing we're doing Is based on burst time yet But so process one is it's got at least 53 Cycles it's got to run and so it's going to run for 20 and then the time out It's going to happen And it'll get be put on the end of the ready queue in the next one Which is process two is going to come up, but it's only going to get to run for eight So why does process two only run for eight instead of 20? Because it's done right it doesn't have anyone else to do so it finishes at eight Process three comes around runs for its 20 process four runs for 20 Process one gets back again. It gets to run for 20 Okay, process two doesn't run because it's gone process three runs for another 20 And here's the rest of it We won't bore you with all the remaining details But if we were to compute the waiting time We would see that we've get 72 for process one 20 for process two 85 for process three and 88 for process four And we can come up with average waiting time and average completion time Which are these two numbers 66 and a quarter and 104 and a half okay So the good thing here is Now this is a good question that was posted what happens if somebody Uses fork to create way too much Way too many processes and thereby take this over. Okay, so right now we're talking about a situation where every process Not necessarily every user, but every process is given an equal amount of time Now the way you start dealing with malicious Users like as being talked about in the chat here is that's when you start noticing that a given user has too many processes Or they're creating them too frequently and you put a restriction on how rapidly they're able to create them Or how many they're able to create Okay, good good observation there if somebody creates a lot of processes They can tie up everybody else because we are really giving equal weight to every process right now So round robin pros and cons one it's better for short jobs. How do I know that? Well, if you look This short job p2 got to run starting at cycle 20 Rather than waiting until cycle 53 So the fact that we're running things round robin means those short jobs come up much quicker than they would in the first come first serve faces, okay So how do we let's just look at a couple examples So I suppose we have two tasks one with a burst link of 10 and one with a burst link of one If we run them first come first serve We get an average response time of 10.5 with a quantum of 10 Because notice the quantum of 10 basically just runs t1 to completion and t2 comes afterwards with a quantum of 5 What happens is We run T1 for 5 and then t2 gets to run. So if you notice the slightly smaller quanta Half of the quanta here basically gives us better response time Okay Now you could say well, this is interesting. Why don't we just set q equal to you know 0.0001 Okay, and why don't we set q equal to smallest number we can come up with To get things more responsive Overhead yes, good answer So switching is expensive. So here is an example where the two threads are the same length Burst time and there if the quanta is 10 versus 1 we get equal responsiveness Okay, and why is that? Well, that's just because they're both equal burst length And so the fact that the quanta are equal treat them similarly. Okay If we have burst length that's Quanta equal 1 and the quanta gets too small Notice what's interesting here is our average response time actually went up And the reason is that this green thing actually ended up taking longer It had to wait more because it ended up having to wait a little bit for the blue one because of the interleaving So just because we have a small slice doesn't necessarily mean That the average completion time necessarily goes down. So you have to be a little careful here Okay Now, how do you implement round robin and the kernel? Well, uh, you start with a 50 q Just like in first come first serve But you have a timer. Okay, and the timer goes off on a regular basis. You set the quantum How do you set the quantum? Well, that's actually a configuration parameter in the kernel But as I mentioned, usually it's set to 10 or 100 milliseconds. Uh, if you don't change anything All right, and a timer interrupt goes off and you use that to take the current Thread off of the cpu and pull the next one off the ready q. So this we've been talking about pretty much since Day one actually we just weren't calling it that and of course You have to be careful to synchronize things and so on so that the cues don't get messed up in the process All right, so this is all about project two scheduling So you're going to get to start thinking about this when project two shows up you get to actually implement some scheduling Okay, that's something to look forward to So how do you choose the time slice if it's too big we end up with response time suffering If it's infinite we get back FIFO if it's too small We have throughput suffering because there's way too much overhead and switching Actual choices of time slice The initial unix which was intended more for a batch mode was about a second Which means that when people are typing Rapidly you just you were not seeing responsiveness. Okay And you could end up if you had three processes you could end up with three seconds per keystroke And so you're trying to balance Short job performance and long job long job throughput And that's where this 10 to 100 milliseconds kind of comes into play If you know anything about hci, you know that that 10 to 100 millisecond range is kind of where the responsiveness comes into Play as well for responsiveness for other things that humans can notice Okay, the typical context switching overhead is like 0.1 milliseconds to a millisecond And so they're really targeting about a 1 overhead No more in context switching Now the question of can you have the the scheduler discriminate priority by program id or user id? Absolutely. We'll get there Okay, because we're doing right now. We've only been treating everything exactly the same and so we'll see You can imagine that might or might not be the right thing to do Okay, so comparisons between first come first serve and round robin. So assume there's zero cost context switching just for the moment Is our round robin always better? Well, here's a simple example where 10 jobs each take 100 seconds of cpu time a round robin scheduler quantum of one second And all jobs start at the same time Okay, so if you look if we're doing fifo what happens? It's the first job runs for 100 Seconds a second one for 100 seconds and so on and when we're done. We end up at the thousandth second Second 1000 if we're doing round robin and we're circling every second What happens is the first job isn't done until Cycle 991 and then the second one finishes and so on and so in this situation The average response time is tremendously worse in the round robin case than it would be in the fifo case So if you're talking about a lot of identical very long jobs round robin is just not the right thing to do All right, and that's because you slice everything up in little pieces and therefore the jobs have to run for Um, you have to interleave for many Many periods before they finally finish and of course when we put context switching overhead back in that becomes really bad Okay, and the cash state has to be shared and so I just can't keep uh Can't overemphasize this as an issue In the fifo case if I get to run for 100 seconds I get all of the cash for that one job and the processor runs You know it hums along like a race car here if i'm switching over and over and over again the cash state doesn't get a lot of time to uh to build up Now of course those of you who are really paying attention realize that one second is pretty long in CPU time, but Certainly fifo gives you much better use of the cash And here's a kind of an interesting thing. So here's uh Here's an example of first come first serve scheduling for Uh process one two three and four who happen to you know process one shows up Then process two then process three then process four if I had sort of uh You know I was an oracle and I knew the future I could reorder them to give me my best first come first serve Behavior, which would be to do the shortest one first then the next one then the next one then the next one Okay. Um, yes, the colors are eye-searing. Isn't that wonderful? Um, so if we look at uh This the best first come first serve Basically gives us an average wait time of 31 and a quarter whereas the worst is 83 and a half And the completion time the best first come first serve everything's done With an average time of 69 and a half whereas the worst case it's average 121 and three quarters So this is the vast difference between first come first serve Best and worst case in the middle is for instance a quanta of eight Gives us a wait time of 57 and a quarter that's kind of between the two right And a completion time of 95 and a half which is kind of between the two So the thing you can get out of this other than uh, aren't these uh vibrant colors vibrant is that um by using round robin We can find a way without knowing anything about the jobs or when they arrive to come up with a fair to middlin Uh response time wait time and completion time. Okay, and that's why round robin is often used as a default simple policy because just by switching you kind of Get rid of the worst behavior of fiefel Okay, and then we could go in the middle here and look at some other options It's kind of interesting that you know, there isn't any obvious best Quanta because as you notice, uh as I get out from eight On either side things go up and so on and so you know the pro another problem with Uh round robin is you know, what's the ideal quanta? Well, you pick one that works pretty well for everything and you stick with it. It's the standard thing All right Notice here that the p2 is the shortest uh job Okay, and notice that the difference between best and worst is horrendously bad for p2 Right, so the best first come first serve is zero wait time and the worst is 145 The best completion time is eight and the worst is 153 so that poor p2 Man, it's affected by scheduling because it's short p3 Is the longest look at that it hardly even notices right the best first come first serve You know, it has to wait on average, you know on average 85 the worst zero Um the completion time is 153 or 68 Yeah, you know, there is some difference there But mostly the long jobs don't notice and the short jobs really do So we're getting out of these kind of scheduling decisions. We're making are really targeting. How can we Give ourselves better responsiveness While not disturbing the long Things that need to run efficiently and by and large the long things don't really notice too much right unless you really Give continuous priority to short things so the long ones never run. That's a problem. But by and large the short ones Basically do better if you if you schedule And those are the ones that the users care about Okay So how do we good? So there's a question here. How do we know What's short and what's long Right now all of the things we've done are oblivious to the length. We don't know anything about the burst time What we're doing is an analyzing what happened after we found out But you could imagine we start remembering things like the last time p2 Woke up. It was really short. Therefore. It might be really short again. Very good okay, and So the hardware is doing the interrupts to handle the keystrokes coming in but ultimately the operating system has to take over to actually Forward the resulting keystrokes to the right application. So yes, the hardware does what it can But eventually the device driver has to take over and then that has to forward on to user friends Okay now Suppose that we want to talk about handling differences in importance between different threads, okay That was brought up earlier and we could start doing something called priority queue okay, and the priority queue Is Something like this where we have different priorities and we run everything from the lowest or from the highest priority first And then go on to the next ones down and as long as we have really high priority jobs Then we run those at the expense of the lower priority ones Now this question that showed up in the chat Can't we hard code keystrokes to be handled every 50 milliseconds? Whatever you have to have a scheduler that can know how to take over or pre-empt From a long-running job in the instance of a keystroke Okay, and priority Is something you could do so you could make um threads handling user Events to be higher priority than ones not. Okay, that could be an option okay, and Unfortunately, it's very hard to know for sure Uh that this thread always ought to be given the highest priority Because there could be situations where things absolutely have to run and your highest highest priority thing just keeps running And everything else doesn't run. Okay, and you start getting into live lock problems So, um, and you don't always know what ought to be the highest priority job unless a user tells you And then you don't always believe them because everybody's going to always say well, my thing is the most important So that's why scheduling is such a tricky thing right? How do you know the right? scheduling policy to make everybody happy So the execution plan in a priority scheduler is always execute highest priority rentable jobs to completion And you could say that each queue in this could be processed round robin with some time quota Okay, and so in this scenario here, perhaps priority three is highest Make sure you always know for sure what your highest priority is before you make a conclusion Um sometimes zero is the highest in some scenarios But here we're going to say three is the highest we could handle the jobs in priority three In robin where we just keep cycling through All the priority threes until there are no jobs left and then we move on to priority two And if a new priority three one comes along will immediately pre-empt and start running priority three again. So that's how um a priority scheduler works the problems that show up are among other things starvation Where lower priority jobs don't ever get to run because the high priority ones And ultimately there's forms of deadlock or priority inversion, which is closer to a live lock Which happens with a low priority task grabs a lock needed by a high priority one So imagine here the job six is run along and it grabs a lock And then job one comes along and job one tries to run And it tries to grab the lock, but it can't because job six has got it Okay, so that is a priority inversion now that simple case I gave you where there are only two threads in the system. It turns out it's not a problem Why is that so why job six has the lock job one tries to run and grab the lock And uh, but it's being held up How do we how does that resolve? Maybe figure that one out. So job three tries to grab the lock. It goes to sleep who gets to run again Job six job six eventually finishes Gives up the lock And immediately job one gets to run because they're high priority. So that one Resolves, okay but This comment about giving priority to the job with the lock is important because it could be If there were many jobs in the system, what really ought to happen is priority three Out of hand its priority to priority zero Job six long enough for six to release the lock and then let job one Run that's called priority donation and you get to try that out In in project two get to figure out how to implement that but the other thing that's interesting about this high priority This priority inversion problem is if you have a third task Okay, so job six has the lock job one's trying to grab it gets put to sleep Job four is an intermediate task. It starts to run and it's running continuously Now we have a problem And the reason we have a problem is job one needs to run But it can't because it's waiting for job six which won't run because job four is running Okay, and this is This is a situation where this won't resolve Okay, now the question is why is this a priority scheduling issue? The answer is because we've set up a scenario where priority two is running Continuously, but priority one is higher priority But it can't run because it's waiting for the low priority six. So we have a priority inversion where Job four is essentially pre preventing you could look at it this way job one from running Because it's preventing job six for releasing the lock Okay, now what's interesting about this Is this kind of priority inversion is exactly what almost toasted the martian rover And i'll tell you a little bit about that Maybe next time But there was a situation where a low priority job grabbed a lock on a bus an immediate priority job was running But the high priority thing needed to get in there and there was a Timer that figured out there was a problem and it would keep rebooting the rover But it would get stuck in this priority inversion situation So how do you fix this? Well, you need some sort of dynamic priorities Got to adjust the base level priority somehow up or down based on heuristics And one thing like I said is this priority donation where job one as it's going to sleep Gives priority to job six because it knows who's weight who's holding the lock All right, so what about fairness? So this strict priority scheduling between cues is unfair. You run the highest and then the next and so on Long-running jobs may never get the cpu There was an urban legend for a long time that in multics, which was one of the original multi processor In multi process machines running at mit that they finally shut the machine down years later And they found a 10 year old job that was still running now. That's just an urban legend. It wasn't true But the idea is there right the idea that things That are running in a priority world might prevent something else from ever running There was a background task and so there's a basically the trade-off here is fairness Which is everybody gets to run is being hurt by average response time Okay And if you're asking about would the priority Inversion Be resolved when the intermediate task finishes? Maybe unless there are other intermediate tasks I mean the fact that you have a high priority task means you want it to run right away And the fact that something lower is preventing it means you've completely Taken over the priority scheme that was the designer came up with that's a problem So how do you implement fairness? Well, you could give each cue some fraction of the cpu So if there's one wrong long running job and 100 short ones You know, what do you do? Well, it's sort of like express lanes at a supermarket, you know, sometimes the express lanes get so long You get better service by going into the other lines. And so maybe there's a way to figure out how to give some cpu sort of to every cue Maybe you increase priority of jobs that don't give service next time I'm going to tell you about several variants of unix schedulers Including the order one scheduler that was linux standard for a long time up until like 2.4 and It basically Had this dynamic scheme where it would figure out and it would continuously adjust priorities up and down Based on things like figuring out. Well, this is must be an interactive task because the bursts are all short So the priorities go up But then this thing runs for a long time the priorities go down And there are all these really complicated heuristics trying to adjust priorities up and down Based on what it thought that was happening. Okay, so that is Something people have tried but it's really hard to get right Okay, but what if we knew the future? Okay, so shortest job first Says you run whatever job has the least amount of computation to do Okay, and this is sometimes called shortest time to completion first. There's also a preemptive version Which basically says Whatever job has the shortest remaining time first let it run A preemptive version of sjf would be you know, if a job arrives and has a shorter time then it gets to run Okay, but what's The problem with this and the reason I'm showing a crystal ball here is you have to have an idea for every job in your queue Which one's the shortest remaining one? Okay, and you can apply this to the whole program or the current cpu burst What have you but the idea is to somehow if we knew the future Get the short jobs out of the system It has a really big effect on short jobs and responsiveness But only a small effect on the long ones and the result is better average response time All right So shortest job first or shortest remaining time first are the best you can do at minimizing average response times You can prove that these are optimal if you knew the future To compare srtf with first come first serve What if all the jobs are the same length? Well the shortest remaining time first just becomes the same And the reason is if you have a bunch of jobs that are all the same length and you start running even one of them It's now shortest From that point on and it just runs to completion Okay, so srtf degenerates into fifo when everything's exactly the same length If the jobs have varying lengths then the short jobs always get to run over the long ones So this is almost like in that That eye peeling colored slide earlier where I showed you the difference between the best first come first serve and the worst This is like srtf could somehow Figure out what the best First come first serve was by always picking the shortest jobs and running them first Okay, so now a question and could we use a neural net policy? Maybe um, but let's let's Talk about the benefits here for a moment Okay, so assuming we can predict the future So, um, here's an example where we have a and b our long cpu bound jobs that run for a week c is an i o bound job where you run for a millisecond of cpu and then you enter the The disk and run for nine milliseconds And then you run for a millisecond to figure out what to grab next in nine milliseconds So this c job is kind of like something you might get if you're copying from one disk to another Okay, and um If c is running by itself notice that you get 90 of the disk Okay, only if c runs by well by itself If I somehow disturb this one millisecond in here and take much longer Then my my disk portions are always going to take nine milliseconds But the time to get there is going to be longer and i'm not going to be keeping the disk busy Okay, a or b always get 100 of the cpu and so they can they can run well For long periods of time Okay, and you know here i'm saying that they run for a week and c runs for short periods of time So with first come first serve what happens is once a or b gets in then c Uh, it doesn't get to run for a week and I get no bandwidth out of the disk What about round robin or shortest remaining time first while here? Let me show you an example Here's an example of round robin with 100 millisecond time slice where c runs for a millisecond And then while its disk is going on for that nine milliseconds a runs for its 100 millisecond time slice Then b runs for 100 milliseconds and then c gets to go for its millisecond and then it gets disk i o And what happens in this round robin with 100 milliseconds? Which might be default on linux is you get a four and a half percent of the disk rather than our target 90 percent of the disk use If I get my round robin to be a millisecond, I've got all of this switching and overhead And now I get my disk utilization back to 90 but boy, this looks very wasteful, right? On the other hand srtf does exactly the right thing. Why is that? Well c runs because it's short And then while the disk is going on c is not even on the q a let's say starts running And a as soon as it starts running it's now shorter than b. So a will always get to run in preference to be a runs until the disk Interrupt comes back and now c is scheduled to run but c is shortest and so c gets to run and in this instance I get back my 90 percent disk utilization Entirely, but I have a hundred percent cpu utilization. So this looks pretty good Right again, assuming I know the future So problems here might be starvation if you noticed earlier This particular oops example I gave here is not running b. So b is starved until a is done and then b gets to run Okay So srtf can lead to starvation if you have lots of smaller jobs and large jobs never get to run So you need to predict the future And uh, well some systems might ask the user. Well, how long does this test take? How long does this test take? I challenge you When was the last time you knew exactly how many milliseconds some code was going to run? it probably not Until after you ran it once, right? So And the other thing is, you know users not only are users clueless, but they're not always Honest, okay. They may be dishonest purely because they're hoping to optimize their use Okay, so the bottom line is you can't really know how long a job will take for sure if we could predict that You know, I have some stock to sell you, right? but We can use srtf as a yardstick For other policies because it's optimal can't do better. So the pros and cons are it's optimal Because it's got the optimal average response time, but the downside is it's very hard to predict the future and it's really unfair Okay, now if you hold on for a moment. I want to give you a little bit more on this So first question and this was great. It was already brought up in the chat Is how do you predict the future? Well, we don't we predict the future using all of those techniques that people use right now to predict the future The great thing about cpu's And typical applications is there's a lot of predictability in them And so if we want to change policy based on past behavior We basically exploit that predictability. So an example might be that srtf with an estimated burst length We could use an estimator function. Okay, like a colman filter All right, or here's the simplest colman filter, which is really exponential averaging Where I have some alpha and I just predict the average. Okay. Oh, no colman filters has just been declared on the chat I will I will say that colman filters do have their place You're welcome to put some sort of machine learning in here There's always a trade-off. However between the cost of doing the prediction and And the benefit because there is overhead to doing the prediction So Anyway, as you can see there are ways for us to predict the future Okay Another thing we could do Is Now let's target some of these different places. So one of the things that is wrong with srtf is it's very unfair So here's another alternative, which I want to introduce called lottery scheduling This is going to be very short But the idea is you give each job some number of lottery tickets and on each time slice You're going to randomly pick a winner and On average the cpu time is proportional to the number of tickets Okay, and so you assign tickets Just like in srtf you could give short jobs more tickets and long jobs less tickets And then probabilistically the short jobs will get more probability to run than the long ones Okay, and to avoid starvation since every job potentially gets at least one ticket We know that when we cycle our way through all the tickets That everybody will have gotten to run a little Okay, so this unlike srtf, which could actually shut somebody out indefinitely lottery scheduling doesn't and lottery scheduling is closely related to other Types of scheduling which are basically Trying to optimize for for average cpu time. Okay So the advantage over strict priority scheduling it behaves gracefully. So as you add more items Then um, and you redistribute the the tickets Graceful gracefully everybody still gets to run okay so here's an example um and We'll we'll get you out of here pretty closely to study some more but For instance, if I have one short job and one long job And I give 10 tickets to short jobs and one ticket to the long job then um The percent of cpu each short job gets ends up being 91 and the long jobs get 9 percent Well, how do I know if they're short or long while I use something like My predictability my average filter earlier. Um, if I have uh, two long jobs, they each get 50 percent. Well, that sounds good If I have two short jobs each get 50 percent Okay, now if there are too many short jobs to give a reasonable response time Then perhaps we're overloading the the overall machine Okay, so there is a point At which scheduling just can't fix the fact that you don't have enough resources Okay, so that's a that's a topic for another day So how do you evaluate a scheduling algorithm and we'll leave you with this thought here? um You can model it. Okay, so these scheduling algorithms are mathematical things you can come up with a queuing theory uh evaluation and apply some Some uh jobs to it mathematically and figure out how that goes although that's Typically a very fragile type of analysis. It's very hard to be generalized You could you know You would come up with something just using average queuing theory rather than sort of transient kimi queuing theory That's a little simpler, but still not exact You can build a simulator which actually puts uh Puts in a trace of how things are actually going and then simulates the results That's often what happens with schedulers sometimes people just go ahead and Toss a new scheduler onto a system and run it and see what happens schedulers unfortunately as we're going to talk about next time can get so complicated that people have no idea What they're doing and why and that oftentimes leads to people complete breaks in code so the o1 scheduler in um In linux was tossed out rather unceremoniously by linus to uh come up with the cfs scheduler And that was because it was getting so complicated. Nobody understood it anymore Okay, so we'll finish this up next time, but we've been talking about scheduling now We talked about rom robin scheduling, which is the simplest default scheduler You give each thread a small amount of cpu and it executes and you cycle between all the ready threads The pros is it's better for short jobs Which gives us a way in to optimizing for responsiveness We talked about shortest job first and shortest remaining time first So the idea there is you run whatever job has the least amount of computation to do and uh It's optimal for average response time the downside is you have to predict the future And we talked about uh various ways of predicting the future Including uh various versions of colman filters like just the moving window average You could have some machine learning example or something more complicated We're going to get to multi-level feedback scheduling next time Which is like a happy combination of a couple of things where you have cues of different Priorities and those cues each have a different scheduler on them And at the top we have short quanta and at the bottom we have fifo And so we're trying to approximate srtf in a way that Optimizes for really short jobs and gives them cpu quickly, but still gives you Good behavior for the long jobs. All right, and we also talked about lottery scheduling Giving each thread a priority dependent number of tokens. All right, so I think with that will bid you adieu. Good luck tomorrow I'm pulling for all you guys. I'm sure it's going to be great And I hope you I hope you have a Wonderful weekend after that so that once you're done With the test you can get yourself a little bit of relaxing time. All right. Goodbye everybody. Good luck