 Alrighty, welcome back to Operating Systems. Everyone's most fun class to come to while there's midterms and other courses, yay. All right, so today, pretty much just more simple stuff, more review stuff we don't actually have to know. Well, first half we'll probably want to know for the midterm, other half is mostly for your enjoyment and it's a good thing to know. So we can explore some dynamic priority scheduling, may also be called feedback scheduling, depending on the literature, just another scheduling algorithm, so we're taking a rewind back. So in this algorithm, we essentially let this algorithm manage the priorities. So it scales the priorities and the only thing you get to set is the initial priorities. So looks kind of like round robin, there's set time slices, we measure CPU usage across the time slices and the idea behind it is we would increase the priority of processes that don't use their time slice. So they're more likely to actually get scheduled next and we will decrease the priority of processes that use their full time slice. So they essentially give up the CPU and hopefully make it a bit more fair to the rest of the processes. Yeah, yeah, so they can still use their full time slice just after they're done it. They get a lower priority so that other processes can use their time slice and then they'd get a lower priority and the one that was waiting would be higher so eventually it would run again. It's essentially, this is just like kind of the same ideas round robin except we're just adding priorities. So we'll see how that works in the example. So like we've been doing in Linux, the lowest number means higher priority. So for this, each process gets assigned some priority when it starts that you would be able to specify PN and the algorithm works as follows. So we pick the lowest priority number that is the process to scheduled. If it yields, we just pick the next highest priority process. If there is a tie, we would just go to a rival order or something like that. If it's lower priority, if a lower priority process becomes ready so lower number, AKA higher priority, then switch to it immediately. So we have preemptions here. Then at the end of our time slice, so there'd be like an accounting time slice and then a time slice for a process just to make things easier. So at the end of a accounting time slice, we would just record how long each process runs for in this interval. So it would be some number C to the end where N is the process number and in this timer interrupts still occur. So the calculation we do is whenever we recalculate priorities, we calculate the new priority P, the N as the last priority divided by two. So it gets smaller and smaller over time because we're dividing by two and then we add however how much time it actually used in the CPU in this time interval. And after we do that accounting, we reset every process, it's CPU usage back to zero. So this would be an example. We have four processes ready to execute in arrival order X, Y, A, B and then we have CPU and IO bound processes. So A and B are something called CPU bound processes, which means they just want to execute on the CPU. They will use as much time as you will give them. While X and Y are IO bound processes, which means they are actually waiting on for some resource or something like that. So they'll only execute for a little bit of time and then they'll have to wait because they're interacting with a file or something like that, something that's much slower. So it doesn't need that much CPU time to make the request to the kernel and then the kernel gets to block it until it's finally done with that request and then it can unblock it. So in this case, we have a timer interrupt every one time unit. The maximum time slice for a process to execute in is 10 time units and the priority interval whenever we recalculate the priority for every process is also 10. So I could make it that I recalculate the priority whenever something gets context switch, that's just a design decision you might choose. But if I have processes that are, if I'm context switching between them really, really fast and they don't use their entire time slice, well, I don't want to really waste time doing accounting every time I do a context switch as well. So I might just delay the time that I pick to recalculate the priorities. So in this schedule, well, if all the processes have a initial priority of zero, then they're all tied. So they just go in arrival order. So I would schedule X first. It is an IO bound process. So it only executes for one time unit and then it's blocked for five. So it won't be available again until time equals six. So in this case, I've executed X and it is now blocked and now I get to pick between Y, A and B. I pick Y just because it is the next, the next soonest in arrival order. So after that I'd schedule Y for one time unit, then it would block for five. So it would be ready again after seven time units have passed. So in this case, I just schedule process A for the rest of the time. So even though the other two processes are ready sometime in here, because they all have a tie for priority, you wouldn't get preempted unless you have a lower priority. So process A is the next runnable process at time two and it would just run until we have to recalculate the priorities. So at time 10, we recalculate the priorities and see if it's still going to run. And here is that showing that X is ready at time six, Y is ready at time seven. And at time 10, we recalculate the priorities. So all their initial priorities were zero and then we just add a constant number with how long it actually executed for in the last time units. So process X executed for one time unit, process Y executed for one time unit, process A executed for eight time units and process B executed for zero time units. So these are all the new priorities. So because priority B now, or sorry, process B now has the lowest priority over every other process, we would preempt process A and we would just schedule process B and it would fill up the whole time slice. So it's done, yep, which one? So the eight in A was not arbitrary. So I'm doing, I'm recalculating the priorities every 10 time units. So I just took a break here whenever I hit 10 to recalculate, if A was still the lowest or tied for the lowest, it would execute for the rest of its time slice. So like two more units. But here I preempt it because now process B is now the lowest priority and I kick it out. So any questions about that? Yep, yep. So here at the top it says, they all have an initial priority of zero. So arbitrary, yeah, but it's there. All right, any other questions? Yep. So what it means that X and Y block for five time units means whatever they're done executing for their one time unit, they're actually waiting on some type of resource or something like that and they can't execute anymore. So the kernel is waiting on some file to happen, something like that and then after five time units we can actually execute them again. So they'd be ready. Yep, yep, yeah, so if A came in first at the beginning of this arrival order, it would just hold it for 10 time units, then we would recalculate, then its new priority would be 10 and it would be the last and then it's a tie for the rest of them. Okay, so let's see how that we can do the same thing again, except set the initial priority for A and B to be six and X and Y can still be zero. So in that case, starts off the same. So although, whoops, should be flipped the other way, whoops. Anyways, yeah, so X and Y are higher priority, it should be a lower number but I screwed up and put it as a higher number for some reason so it should be flipped. But anyways, same thing would still apply. X would go first because it is a higher priority process, then Y would go first and then A would only execute for four time units and then at time six, X is ready because it gets unblocked. So it's done waiting for whatever the kernel was gonna do with it and because it has a higher priority, it would be unblocked and since it's a higher priority then process A, it would be scheduled to run immediately, run for one time unit, be done at time seven, Y is ready, it would execute for one time unit and then it would get blocked and then here A would execute until the A executes until we recalculate all the priorities and then here we would recalculate the priorities as so. So X and Y, yeah, so, oh sorry, it says A and B have a priority of six, okay. I said it wrong, sorry. All right, so process X, it's new priority while it's old priority was zero divided by two. It ran for two time units in the last 10 time units so it's new priority is two. Y ran for two time units as well, it's old priority was zero so it's new priority is two. Process A, it's old priority was six so divide two gives you three, plus it ran for six time units so it's new priority is nine and process B, it's old priority was six, it ran for zero time unit so it's new priority is three. So X and Y still have a higher priority or lower number than A and B but we see that we have now flipped and we would prefer to execute process A over B or sorry, process B over A. So at this point X and Y are still blocked so X and Y would still be blocked for two more time unit or yeah, two more time units so we would schedule process B, B would run for two time units and then at that point X is ready to execute so X would execute again because it is a lower number or higher priority so it executes for one time unit, Y would be ready, execute for one time unit and then B would execute for four time units until X is ready again, then X would be ready, then Y would be ready. So any questions about that? So just hops back and forth. Yep, yep, yeah so you could have it recalculate at the time of every context switch but that would probably be too costly especially how many times we're switching here so generally you just recalculate every at some fixed priority or some fixed interval, yep. So then our CPU would just be idle, right? Yeah, so if A and B finished and we only had X and Y and they were blocked and those were our only process, if they're blocked we can't execute anything our CPU would just be idle. So we would get what? Yeah, it'd be idle for like two thirds of the time. Yep, yeah sorry so in here X and Y have priority of zero and then A and B have priority of six. So we would prefer to execute X and Y because they have a lower number. So yeah, I said it wrong before the initial priority of X and Y are zero now A and B have a priority of six so we would prefer not to run them. But if everything's tied like before whenever we scheduled A it just stays until we recalculate the next priority interval. All right, any more questions about that? Fairly boring. More follow the simple algorithm. All right, so let's talk about large language models then and more fun stuff you could do. So large language model is very large so like 30 billion parameters and the model itself is like 20 gigs and on some C++ project they're like oh well I have this 20 gig model but I only need four or 6.4 gigs of RAM. How is this possible? And it's so efficient. People were discussing it on the GitHub and saying oh how is this ever possible? And I will teach you the trick because now you can understand it and this is why people should take an operating system class because everyone thought this wasn't possible but it's not. So I alluded to this last time that we can control our processes virtual memory. So memory map or M map is used to map files to a processes virtual address space. This sounds much worse than it is. I see some groans already but this is actually a useful thing that makes your life a lot easier. So what it does is it gives you like a pointer to a virtual address and that pointer you can just access the file through that pointer. You don't have to use a buffer. You don't have to use anything. You don't have to worry about going to the right location in the file. You don't have to worry about literally anything. You just access it as if it was memory. No read or write system calls. You interact directly with the file system with just using memory addresses and this is possible through the magic of virtual memory. So this M map system call takes like six arguments. We can just dive into an example instead because this is fairly boring. So let's just go into a fun example. So in here, here's an example. So remember whenever we were opening a file before well if we wanted to read the contents of a file and then if we wanted to read the contents of a file and shove them out over standard out while we had to set up a buffer we had to open the file, got a file descriptor, did a read system call on it and then did some stuff and then did a write system call on it with a buffer making sure that all the sizes were right maybe having to retry doing the read system call over and over again and it was a giant pain. So instead we can make this program. So this program has the open system call so we get a file descriptor from that. So I'm going to do a little meta thing here and I will open this own source file with the flag read only and then I set up the struct stat. So we haven't seen the system call yet. We won't really see it so but it's a good one to know. So this stat struct tells you information about a real file. So you can fill in that struct using an F stat system call so you give it a file descriptor you want information about and then the address of that struct so it can just write the information there same type of deal with the rest of the system e-calls we saw, negative one if there's an error otherwise it sets error no and you should probably check it. So here I just check that it is the success error code and then here is my M map. So M map takes six arguments. The first argument, it looks like a lot of arguments but most of them you don't really have to use. The first argument is an address if we were very picky about what virtual address we wanted. So you could say I want this memory to be mapped at this specific virtual address. 99.9% of the time you will not care so you can just give it a null that means I don't care where you put it. Then you have to say what the length of this is. So here in this stat struct there is a field called stsize or underscore size that tells you how big that file is. So I want this virtual memory map to be as large as the file that I want it to represent. Then there is some protections. So it lets you essentially modify that PTE entry if you want different permissions. So here I have to set protection. I want to be able to read from it and I don't set write. So if I try and write to any address I get back from that it'll give me an error. If I wanted to write to it I can set a write flag if I really want. Then there's a bunch of flags with how this should behave. This is called private. So that means this memory mapping is private to your process. If you fork the other process does not get this same does not get the same mapping or anything like that or it gets an independent copy of the mapping. If this flag is shared then both processes will map to the same physical location and then you'll be sharing memory, which will be fun. And that's another way to communicate between processes that's really, really fast. This argument is the file descriptor. So whatever if you want it backed by a real file what is the file descriptor of that file? And then the last argument is just the offset. So how many bytes from the beginning of the file do I want to start this memory mapping? Yeah, so that's a good question. So if I use the share flag is that alternative of what I'm doing with pipes and you can think about it that way but it's like a buffer that your processes can actually see. It's not managed by the kernel. Kerl just says, hey, both of you you have the same memory mapping for virtual addresses goes to the same physical memory so you can share values. You can do whatever you want. So it doesn't have to be like a first in first out thing like pipes, you can just do whatever you want. So this stat struct is specifically for getting information about actual files. So if you read the documentation like so here it'll tell you exactly what everything it's for yeah, it gives you all probably too much information. Basically it just gives you information about a real file like who owns the file. Like if you do LS or something like that all that information about the file. So like here is the size, here's who owns it here's the permission. It would all be in that stat structure. Yeah, so whatever you fork both processes are gonna be exact copies. So if this was, if this is private and I fork they'll both have that same virtual address but they'll be independent of each other. So private is basically like how fork behaves normally. So all your virtual memory is private to you. Okay, cool, any other questions? Yeah, so this is fun, this is useful. So here we can check if it failed or not and then after we have the memory mapping we don't need the file descriptor anymore so we can just close it. And then here I can just this data pointer that I got back from Mmap, it is a virtual address and I can just read it like memory. So I just read it like memory I don't have to do a read or write system call and how it works under the hood is the kernel knows that that virtual memory region is associated with this file. So as soon as you try to access a memory address it would get a page fault on like the first page and then it would read the first page of the file and then actually read that into physical memory and make sure that that virtual page maps to that physical page. So I just access it as if it was memory. So here I just access it this file as if it was an array. So I just go have a for loop that iterates over every single character and I just do a printf for that character and then afterwards it has an associated so like malloc has a free M unmap is essentially the free version of Mmap. So if I go ahead and execute this my key is not working, then guess what? I just get the contents of the file and no read system call, nothing, no fuss, no must actually pretty awesome. So any questions about that? Yep, so data is a virtual address where as soon as you start accessing it it accesses the file. So this address could be anything but if I'm like, you know, byte zero here then it would be the first byte of the file then the third byte da da da da da and so on and so forth until the end of the file. So pretty cool and actually this hopefully you prefer this over read and write system calls, it's actually a lot easier but most people won't know of this because they haven't taken this course and they don't know they're scared of virtual memory or don't even know what it is. So since you know what virtual memory is you can use this and it's not too bad. So why did I talk about this? So that Mmap system call is lazy so it just sets up the page table and initially it does not read from the file. So it just creates page table entries that are invalid that have some bookkeeping information that says, hey, whenever we try and access this virtual page I should probably read from this file. So the kernel would store some information in the page table entry because if the valid bit is zero well it just generates a page fault anyways and the other 63 bits the kernel can use for whatever information it wants so it can just store a pointer or whatever that says, hey, this page should actually map to this file. So what it would do is it would just keep track on where on disk is this actual file and then whenever you first try and access the page you get the page fault and then the kernel reads from that disk into memory so you can't even see that it's doing it and then it would make that entry valid and then you can just access it, no fuss, no must and the nice part of this is since the kernel is lazy well you will only read in whatever pages of the file you actually use. So if you use the whole file I have to read the whole file still it's not doing anything magic but if I only read like the first byte and the last byte I would probably only read the first page and the last page as long as that file is quite huge. So back to the question so they asked well that file is 20 gigs you had to read the whole thing so how is it not taking 20 gigs of memory? How is it only when I'm using it it only uses 6.8 gigs? So the reason this is is because this whole discussion the whole patch they made to make this super efficient blow people's minds is instead of opening a file and just reading 20 gigs of it they just M mapped it that was it that's the whole patch they just M mapped it they replaced read and write system calls with just accessing memory and that was it and the reason it was so great is because large language models are really sparse you don't use for each inference you don't actually use the entire 20 gigs of the model you might only use a few gigs of the model or something like that so if you just M map it and you just use the memory directly well you don't have to worry about trying to be efficient or anything you just let the kernel handle the page faults whenever they happen and just let it load it into memory whatever parts of the file it actually needs. So in part of this discussion one of the experts was like oh there's talking about like how big the page tables need to be in order for this magic to happen so it would have to set up the page tables. So we can actually answer that question how much space does the kernel need for the page tables so someone posted you only need 40 megabytes of page tables for that 20 gig file. So they argued it was 20 times 10 20 gig file by four times 10 24 times 10 24 so this is two to the power of 30 so this is a gigabyte and then divided by this magic number what is this magic number? Page table size and so we made our page tables fit on a page it's a page size so yeah this is the page size and then times eight so times eight why would we multiply by eight so this is the page table entry size so eight bytes so in this case for 20 gigs of space well we take 20 gigs divided over the page size so that is how many virtual pages I would need for 20 gigabytes and then each of those virtual page numbers needs a page table entry so they just multiply it by eight and then divided by 10 24 times 10 24 which is just a megabyte and then someone clarified that yeah this calculation is just 20 gigs divided by the four kilobyte page size times eight bytes per page table entry divided by one kilobyte even though that should actually be one megabyte so is this correct or not so what do we think about this? So they just changed this and said oh it's one kilobyte but 10 24 times 10 24 is actually one megabyte so they just said it wrong yeah so if you do a map with a 20 gig file you need a page table entry for all of it even if it's not going to be used you still need a page table entry so it might not actually like load the page and read from the file but it still needs a page table entry so is this the worst case or is this the best case? So yeah this is like the best if we only include L zero right? So this is how many entries we need in L zero page tables how many entries do we need in L one page tables probably too much to think about now so in fact this is correct but this is like best case scenario assuming that we only used L zero page table entries and they're all well they would all be beside each other but there's no multi-level page tables so in this case if you want to be statistic and do like actually this is how much it uses well it's right that we need to the 20 page table or 20 times to the 18 page table entries so those are the page table entries we need across L zero page tables so if that's how many page table entries we need well we know that we can only fit two to the nine page table entries in each L zero page table so we can figure out how many L zero page tables we need by just dividing it and then we get 20 times to the nine or 10,240 full L zero page tables so these full L zero page tables would be 40 megabytes except we can count how many of them we need so if we need 10,240 L zero page tables we need that many entries across our L one page tables so because we need that many entries across our L one page tables well each L one page table can point to up to 512 L zero page tables so if we just divide how many L zero entries we need divide by how many we can fit in an L one page table we would get 20 full L one page tables so in fact we wouldn't need just these 10,240 full L zero page tables we would need also 20 L one page tables and we wouldn't count our L two page table because we would always need an L two page table it would already be there for us so actually and this is like whatever that meme is is that nerd guy with the glasses that definitely needs to be 100% correct so in this actually our best case we would need 10,240 full page tables which if we divide by the page size and how many entries we need we actually need 40.07 megabytes worth of page tables not 40 so Panantic, yes, yeah you only have one L two page table yeah so in this case this assumes that we would already have an L two page table because our process is already running so it would just add entries to that, yeah so each level of page table can only point to two of the nine things cause we only have nine index bits so that means it can only point to two of the nine different things which is 512 all right any more questions about that that's like fun, well it's not that fun but you can answer it and it should kind of make sense yep so why did I divide by what sorry oh so the question is why did I divide by two the 20 here and I divide by two the 20 here because I just want it in megabytes so two the 20 is just megabytes or MIB like the real megabytes not the marketing fake megabytes all right so that's actually our best case did we ever need to go that far no but do we know multi-level page tables hell yeah we do all right so that's pretty much it any other questions lab three lab whatever yeah so in this case we wouldn't have the worst case because we're mapping all that virtual memory together so all the addresses would be contiguous so they would be beside each other in the L zero page tables so the worst case would be like if we were like on a boundary of our L one page tables so we might need another one that's pretty much our worst case so our worst case is gonna be maybe they don't exactly fill a page table maybe instead one has one entry and the other has 511 or something like that but it's only gonna affect the first and the last so it's probably not gonna be that much different worst case would only be off by like maybe four page tables no two maybe two all right any other questions lab three lab anything we can take a break you guys are probably going to be screwed for your other midterms so yeah we can just end early and I will be around so just remember for you we're on this together