 Welcome back to 353. So lab 2. Everyone having fun? So, yeah some of you may, at least some of you have been debugging for hours and hours and hours. If you use Signal, go down to the end of the lab document and see those flags. Use those flags and that will save you a few hours by itself. Yeah. Yeah, do not mess up your file descriptors. They are hard to debug. Other useful tip, the directory is a file descriptor. So, don't close it while you're going over the directory. Bad idea. That's the major causes. Yeah, if you use a signal handler, remember at the end of the document, it says, hey, if you use a signal handler, what happens if things randomly start failing due to system calls and like just weird errors? It says flags restart. So remember when we had that lecture where we were trying to ignore signals and we had like read and then we got an error with like interrupted and we had to handle it. Well, there's a flag for the signal to prevent that from happening, like that essay restart thing. It will automatically restart any system call that gets interrupted so you don't have to deal with that. So, that's one of the main causes of errors that I've seen so far. Other than that, it's been people messing up their file descriptors. After that, you're golden. So, yeah, when you're changing your file descriptors to bit hard to debug because printf doesn't work. If you screwed up your file descriptors, printf also doesn't work. So, that's fun. So, one hacky thing you can do that kind of works is just exit with no numbers. So, like exit with 42 or 43 or something like that and then you can debug that way. So, at least know where your child process made it to. But, yeah, other than that, that one's real fun to debug. All right, speaking of fun to debug, let's continue. So, something that's a bit easier, we can go back to scheduling and see, you know, another scheduling scheme or algorithm we can use called dynamic priority scheduling. May also be called feedback scheduling in the textbook. And this is a way to have priorities and let the algorithm manage the priorities for you. So, you just set an initial priority and then it goes ahead and it adjusts them on the fly. In this case, for this algorithm, it uses set time slices, measures CPU usage across the processes that are running. Idea here is that we just increase the priority of processes that don't get any time in that time slice and decrease the priority of processes that actually get to run during that time slice. So, hopefully, you know, through adjusting priorities, we'll be able to be a bit more fair and let every process run. So, for this scheme, we can just pick the lowest number as having the highest priority. So, each process just gets assigned some priority when it started. You know, we can give it names, so P subscript N. And the way the algorithm works is it just picks the lowest priority number to schedule. If that process like yields and gives up at CPU time, then we just pick the next lowest number or next highest priority. And if we have to break ties, we just break it with the rival order. And if a lower priority number becomes ready, well, we immediately switch to it. So, we have preemption and we assume while we have time units, so we can't switch between time units or within a time unit, but we can between. Yep. The formal definition of preemption is you can just take some resources away. So, in this case, preemption means while these processes are using the CPU, means I can stop you from using the CPU and take it away and manage it. So, yeah. Mostly operating systems like controlling resources, preemptible and non-preemptible. So preemptible, things I can take away. Non-preemptible are things I can't take away. So, like, if your process wants to use memory or wants to use some files or something like that, I can't just take that away willy-nilly without causing gigantic issues. So, we can, yep. So, if it yields, so that means it can just give up its CPU time. So, it can just say I'm done executing for now, run something else. Or it could be blocked, like it could be waiting on something. So, basically means it just gives up its time. So, how this works is we record how much time each process executes for in that, well, we can call it a priority interval. So, that's like the interval of time that we will use and we'll recalculate priorities at the end of the interval. And we'll have, like, CN, we can call it C subscript N. And here, timer interrupts still occur. So, timer interrupts would probably happen. Throughout the priority interval, so we can go ahead, recalculate and maybe context switch to a new process. So, at the end of the priority interval, the formula to update the priority of all the processes is, well, the new priority for process N is equal to the last priority for process N divided by two. So, it decays a little bit. So, it gets a bit lower. So, it's more likely that this process will run. And then, plus, however much time it has actually executed for. So, if it gets to execute in that time slice, then, while we add to that, so that would get a lower priority and we would get, hopefully, a different process running. And then, at the end of the, and the C is just keeping track of how long that process executed for. So, whenever we recalculate the priorities, we recalculate the value of C back to zero for all the processes. So, we get to see how long they execute for in the next one. So, that's a bit wordy. So, we can just have a quick little example. So, we can assume, for now, that all processes just have an initial priority of zero. So, we'll say we have four processes ready to execute and arriving in order x, y, a, b. And the reason that they're named slightly different are they behave differently. So, typically, with processes, there's like two main ones we care about. So, a and b are called CPU bound processes. So, they're just doing some calculation. They just want the CPU as much as you will give it to them. And then, x and y might be like IO bound processes. So, they're processes that are blocking, that are waiting for something slower. So, like, waiting for the user to input something, which is really, really, really slow, waiting for the disk to give back some information, waiting for a network, something like that. And in this case, these processes just execute for one time unit. And then, we assume that they block for five time units. So, they're unable to execute while they are blocked. So, we have a timer interrupt occurring every one time unit. And then, each time slice is 10 time units, and their priority interval is 10 as well. So, they don't always have to match, but typically, they will match. So, the priority interval is just every time we recalculate the priorities, and the time slice is, well, how long a process could actually execute for. So, we want to figure out the scheduling. Well, since everything has the same priority, we have to break our tie with arrival order. So, in this case, we would schedule process X. So, because it's one of these IO bound processes here, so it only executes for one time unit, and then, it is no longer available for five time units. Yep. So, time slice is, like, the maximum time that a process can run without getting thrown out. And priority interval is whenever we recalculate the priorities. Typically, they're the same. So, yeah, in this case, X runs for one time unit, then it can't execute again for five time units. So, the next process in arrival order is Y. It's one of these IO bound processes, so it only executes for one time unit, and then, it's blocked for five. So, after this, we would have A run until we recalculate the priorities. So, X and Y would be ready again sometime at, you know, time equals five or six or seven or whatever. But, because they're all tied and they have the same priority, well, we would just prefer to keep executing Y and, or A and not context switch because they're tied. And this, you could adjust if you wanted to, but typically, it'll just keep on executing A. And then, at time ten, we will recalculate the priorities. And, yeah, here at time six, X is ready, Y is ready, we don't switch to them yet. So, when we recalculate the priorities, well, the initial priority of all of the processes is zero, so that explains this column. And then, we just add however long they ran for in that last interval. So, in the last ten time units, X ran for one, Y ran for one, and then A ran for eight. So, our new priorities would be one, one, zero, eight, or sorry, one, one, eight, zero backwards. So, in this case, well, now we have one process that has the lowest number or highest priority over everything else. So, we would just execute that for the next ten time units. And then, we'd recalculate after that and keep on going over and over again. So, we can, yep, yeah, yeah, yeah. So, in this case, X and Y are blocking, they're waiting for something to happen, like the kernel made a request to a hard drive or something like that, and then start executing another process. So, there's only one process running on the CPU at a time. Yeah, well, which is here it's X, then it's Y, and then it's A. So, there's only one process running on the CPU at a time. So, X is unblocked here, so because it blocks for five time units, it means it could execute if we decide to give it the CPU. Yeah, yeah, block just means it needs five time units because it's waiting for like a hard drive or something like that, waiting for something that's slow. So, it can't make any progress until that thing is done. So, it stops at one, so after five time units, we say it's ready because we could execute it again if we wanted to. But the kernel just, in this case, decides not to. All right, so we can do the same thing, but now processes, A and B have a priority of six, and the others have a priority of zero. So, because of that, X and Y, well, their priority is zero, so we'd prefer to run those, and X came before Y, so we would run X first. Run X for its one time unit, it gets blocked for five, so it's not available again until time six. So, next process is Y, runs for one time unit, blocks for five, so it's not available until time seven. So, in this case, well, all the processes with priority zero are blocked, so I have to pick something else to run. So, the next highest priority is a six, which is A and B, and A came before B, so I would pick A to run. Now, when X is actually ready to get unblocked and it can actually execute again at time six, because it has a higher priority than A, it immediately just kicks it out and we contact switch to X. It runs for one time unit, and then it gets blocked again for five. And then at time seven, well, Y is ready again, so it would execute for one time unit, then it gets blocked again for five. Then, well, there's nothing else to run but A again, so A would run for the remaining two time units until we have to recalculate the priorities. And if we recalculate the priorities, well, for process X, the initial is zero, ran for two, so its new priority is two. For Y, initial was zero, ran for two, new priority is two. For A, well, its original priority was six, divide by two, that's three, and then it ran for six. So its new priority is nine, and then for process B, its new priority is just three, cuz it didn't run for any time. So now, because of this, well, at time ten, X is currently still blocked, and Y is currently still blocked. So the next highest priority is process B, so I would choose to execute process B for two more time units until X becomes unblocked. So now at time 12, X is unblocked, it has a higher priority than B. So we immediately execute X for one time unit. Again, it gets blocked for five. Then Y is ready, execute for one, gets blocked for five. And then our next highest available process would be B. So it runs for four, and then X is ready again, goes X, and then it goes Y, and it goes on and on and on like that. So questions about this? It's just one way to just let the algorithm handle the priorities for us. Not terribly exciting, right? But it's a thing you can use. Let's go on to more exciting stuff. We still like virtual memory, right? Ish, no, what the, virtual memory is great. All right, I'll show you why virtual memory is great, because you will appear to be a wizard, cuz LLMs are a thing that people like, right, they're cool, they're hip, they're trendy. So if you have like a 30 billion parameter large language model, well that model itself is like 30 gigs of RAM, like it's 30 gigs large. And if you have taken this course, you can actually optimize things a little bit. So that 30 billion parameter model can actually run using only like seven gigs of RAM, and how? So it's so efficient that people are like, they don't even believe it. So we can see the discussion if it works, plus. So yeah, I got the number on, it's only 5.8. But we can kind of read that, I don't know how this is possible. Like is someone smarter than me that can tell me how the hell this works? Like how is this even possible? And we should be able to explain this because we, from taking the course, should count as someone smarter than this random person on the internet. So let's take a brief detour and see. Wait, question? So question, how does the hard coded priorities we saw in the previous lecture being incorporated with this? Is it just the initial priority, negative priorities get divided by two? So just depends in this, like you can have hard coded priorities that don't change, you can have them dynamically change, just depends on the algorithm. All right, so let's have fun with virtual memory. So we can actually control our processes virtual memory. So there's this cool system call that is actually cool called memory map. The system call is called nmap because of course we have to make everything shorter. So it is used to map files. You can use it to map files to a processes virtual address space. So has anyone like opened a file and then you have to do read system calls to it and it kind of sucks, right? Because you need to like declare a buffer and all that stuff. Well, this actually makes it a lot easier to actually read a file. So let's just see what that API looks like. Actually, let's just go into an example. All right, so if I was to read a file with like normal system calls, I have to like do open, I get a file descriptor back and I just constantly do read system calls until eventually it returns zero. And then I know I'm at the end of the file, I have to create a buffer. I have to do all that fun stuff. It's no good. So instead, we can actually do something called, I mean, memory mapping a file or M mapping it. So here, we'll be a bit meta. So I'm going to open the file mmap.c, which is this file. So it's the current source file. So I'll open it as just read only, so I want to read it. Of course, we should be able to guess what file descriptor number we get because file descriptor zero, standard in, file descriptor one, standard out, file descriptor two, standard error. So next available should be three. So I'll just write a little assert there to just formulate or not formulate. Just make sure that what I think is true is actually true. And then we can see this other fun system call. So there's a struct called a stat. So it just has a bunch of information about a file. And the system call fstat will fill it. So this is another system call. So we do fstat for the file descriptor, give it the address of that struct. And then it goes ahead and populates it for us. And the reason we're doing this is because one of the things it will tell us is the size of the file. So once we know the size of the file, then we can set up this M map. So this takes six arguments, which is a lot, but we can go through them. So the first argument, the address, it's a bit silly. So you can actually politely ask the kernel exactly what virtual address you get as a result. If you want, if you say null, you let it pick it for you. So we will let it pick it. Next is the length, so the length of the mapping. So I'll just map the entire length or size of the file. Next are some permissions. So what you're allowed to do with that memory. So if I have prod read, that means I'm only allowed to read that information. I can't write to it. If I write to it, I would say fault. And I'm just making sure that I actually just read the contents of the file without accidentally writing to it or modifying it. Next is the flags, so map private. What that means is what happens when this process gets forked. So if it is map private, that means it is private to this process. So when you fork, the new process can't access the same virtual addresses. Because they're mapped directly to the file, otherwise you'd be mapping the same things. If you want to, there is a flag here called map shared. And that will make sure that when you fork, doesn't change anything. Both processes see the same thing. And that's a way to actually share memory if you want. Then the next is the file descriptor. So the file descriptor to actually map, and then the offset. So how many bytes into the contents of that file descriptor do you want to start? So we'll start at the beginning. So Mmap kind of looks like malloc a bit where it just returns a pointer. So we just check, hey, is it the special Mmap field? If it is, we don't handle an error or anything. We just have our assert handle it. But if we wanted to, we could check error no and all that stuff. And then after that, I don't need the file descriptor anymore. I mapped it. That's all the file descriptor was good for. So I can close it. Now, instead of doing read system calls or anything like that, well, I have a virtual address now. And what that points to is the contents of the file. So in order to read the file, I just access memory now. So I just have a for loop that goes over every single byte of the file and then just prints the character. So just a normal run of the mill for loop. And that's it. So if I go ahead and I run this, I see boom. I see the contents of the file and I don't have to do any read system calls or anything like that. Kind of cool, huh? Yeah, yeah, yeah, so yesterday I talked about using mmap to allocate memory. So this file descriptor here, you can set it as negative one and then it's not represented by a file. It just gives you some new memory. Yeah, so right now it's setting up those virtual pages and it will map those to the file. So it will make sure that the virtual address we get back, if we try to access that memory, we'll actually be accessing the file. Instead of just random physical memory. So if I don't give it a file descriptor, then that memory won't actually correspond to the contents of a file or anything. It'll just correspond to just random physical memory. And yeah, so that's one way you can share. Like that's one way you can do IPC across processes. So if I change FD to negative one, so it's actually memory and then change that flags from private to shared and then I forked. Both processes have the same virtual addresses that map to the same physical memory, so they can actually share memory. And you can communicate with them through memory, which is a lot faster because, well, you just need to do a map system call to set it up. And then after that, you don't have to do any slow system calls. You're just sharing memory. So that's mostly what we do. And that's fast, but kind of weird, weird and error prone. All right, any more questions about this or any weird things we want to do with it? But this is kind of cool, right? Yeah, yeah, so the offset is with respect to the contents of the file. So zero just means start at the beginning of the file. If I changed it to, I don't know, let's change it to offset then I have to recalculate my size, so I'm not going to change it, but yeah. Yeah, so the question is for the flags. Can it either just be read and read only or write only? So you can combine them. So you can combine them. The usual things with these flags are like their bit flags. So you can go ahead and do something like bitwise or and you can do things like that. So that will let you have that memory mapping for reading or writing or and writing, so it'll set both of them. Yeah, yeah, so the question is reading, writing the only two options and there is at least, so there's execute. So execute is like, you can see how this kind of exactly corresponds to the flags in the page table entry too, right? So like there was, well, this would set the valid bit, but there'd be read, write, execute are the main ones, yeah. No, if I set it to execute, all that means is that if there are instructions there, I can execute instructions there. So it just means that those bits, I'm allowed to essentially point my CPU to it and start executing them. And if you didn't have this, so this would be like some voodoo crap magic thing where I just load some raw instructions in the virtual memory. And then I essentially say go to and say go start executing that. And if I set this flag, it'll happily start executing instructions. But if I don't set that flag, I'll probably get like, I'll either get a sag fault or I'll get illegal instruction or some type of error in my process will just, but yeah, only unless you are doing really low level operating system things, only psychopaths would use that flag. Or if you're like trying to hack into something, yeah. Yeah, so like adding the executable flag here doesn't make any sense because I mean they're ASCII values. So if I tried to execute it anyways, there'd just be random instructions that just happened to correspond to the ASCII numbers and it'll probably crash and burn anyways. So it doesn't really matter. But right, maybe I want to do right because maybe I want to just read over it and then change everything to, I don't know, an A or something like that, who knows. So if I do that, I don't get any errors. And if I remove the flag, I should see that bad things happen. So, sag fault, yay. So, protect, so as I'm now teaching the first year, segmentation faults are actually good because they're easier to debug than just random things occurring and then you have to try and figure it out, which I guess you're kind of learning in lab two. But, yeah, yeah, no, because I modified it after I printed it out, right? So I could have looped over it again and it would just been all A's. No, like the actual file, that's a good question. Hopefully it doesn't, yeah, so it doesn't overwrite the original file. Yeah, just copied it in the memory in your virtual address space and then you're just playing with the memory and it's fine. But I think there's some flags you can do to actually modify it and then write it back out. That's just what we did, right? We had it right and I wrote all A's to it. So right now it's just when that process is running and then it doesn't write it back out to disk or anything like that. Yeah, and I forget how to force it too, but this did not. But I think there's a way to force it too, but I don't know it off the top of my head. All right, so that's kind of cool and kind of weird. So, yeah, there's our example. Anything else weird you want me to do with it? Yeah, so in this case it didn't print out all A's because I changed the byte to an A after I printed it. Yeah, so if I do, right, so if I just looped over it again, then I'm going to see a bunch of A's. It looks like it's screaming. Yeah, the last what, sorry? The last one's screaming. So, unmap is basically so like, M-map is kind of like malloc and this is kind of like free. Yeah, so it's just saying I'm done with it, undo that memory mapping. And then if I try and access again, I sag fault, right? Everyone likes sag faults, I mean, they're not that bad. All right, any other questions about that? Okay, so let's try and answer, let's be smart. So, M-map is actually lazy, especially even if you don't like back it with a file or anything like that. So all that M-map call will do is set up the page tables. Whenever you do that, it doesn't actually read from the file. So what it actually does is it creates an invalid page table entry during that M-map call and it sticks some information there that's like, okay, if someone tries to access this, they actually want to access this file through this file descriptor. So it just stores some information in that page table entry. Again, like where on disk is this entry? And then the first time we try to access that page, our M-MU would call as like a page fault because it wouldn't be valid, right? Well, instead of just passing a sag fault up to your program, the kernel and essentially its interrupt handler can be like, okay, you faulted at this page table entry. I know that's supposed to be the file. So I'll just go ahead and read a page worth of content from that file. And I'll map it and like I'll fix up the entry, read it into a physical page, change the mapping to point to that physical page, and then just let the process resume again. And then now you're actually accessing the contents of the file through that. And this actually ensures that only the parts of the file that get read, like only the parts of the file you use actually get read. So in this case, if that file was like absolutely ginormous and say, I don't know, it was who knows, like 30 gigabytes. And I accessed the first byte on that while that would be on a page. And it would be like, okay, well I need to read a page of content from that file into memory and then set that up. And then if you wanted to just read byte, I don't know, 10 million. That would be on a particular page and say, okay, I'll go ahead and read that page now, and read it into physical memory, fix up that mapping. And then suddenly, all you have to do is just, yeah, all you have to do is you just read two pages. Doesn't matter how big the file is. So if we go back to that question, how is this possible? So it could be a 20 gig file, 30 gig file. How is it possible that it only uses like 5.8 or 6.8 gigs of real memory? So the reason because this is, well, whoever wrote the first version of the process or program just loaded the entire model into memory, whether or not you used it or not. And then some very smart person who has probably taken a course like this or at least played around with the kernel was like, okay, well, when you do model inference, they're typically sparse, you don't use all of it. So instead of reading the contents of a file that has contents I might not touch, what did they do? They M mapped it. So they just did like a one line change where instead of doing read system calls over and over again, they just did one M map system call. That was it and that's it. They just M mapped it. That was the whole wizardry behind that. So questions about that wizardry. So they immediately cut off like was like 24 gigs of memory that the process used by just doing M map. Pretty good trade off for having to sit through and listing to me drone on about virtual memory for a while. So you can actually get some real savings from it. And yeah, so the question, why would M map use less memory here than read? So again, this is a sparse file so we wouldn't actually access all of it. So all we have to do is actually set up the page tables. So M map, again, like I said, all that does is set up the page tables. So I know if I try and access that memory, where I actually have to load things into physical memory. So if we have a 20 gig file, can we come up with how much space we would actually need for the page tables and do a bit of review? Everyone likes virtual memory, right? So I'm M mapping a 20. Let's just say 20 gigabyte file for the sake of argument. So how many pages do I need to map a 20 gig file? Assuming, yeah, no, just how many pages do I need? Yeah, so kind of. So this number, so like 20 times 2 to the 30 divide by what? Divide by 2 to the 12. Yeah, so that's how many pages I would need, right? So that's like 20 times 2 to the 18, which is some weird number. So that is how many pages I need. How much space would be consumed by the page table entries are just required for this? So each page table can hold to the nine, but just in terms of how many page table entries I need just to keep track of all these pages. So like across all my L1s, how many page table entries do I need? This, right? And how much space do they take up? So they'll take up this times eight because it's eight bytes for page table entry. So in total, we have about 40 megabytes. So 40 megabytes for the page tables is the answer someone gave on this post. So is that actually true? I guess no, right? Because these are all the entries that are in the L0 page tables. How many L1s would I need in the best case? Yeah, couple kilobytes worth. Yeah, exactly how many, yeah. Yeah, it's essentially exactly this much, like this many pages, divided by 512, right? Because, well, they can only hold 512 each. So it would be like this many pages divided by 512, which would be like 20, I think it is. Which, yeah, that is a lot. And then, well, we only have one L2 page table that we can assume to be there. So this 40 is approximately right. But if you want to be technically correct, then you have to keep track of the L1 page tables and be a giant nerd, I guess. So the person who said 40 megabytes exactly was wrong because they didn't account for L1s. So slightly higher than that. So if I have 20 full L1 page tables, well, in actuality, I will have like, so this number is, I think this is like my L1, L0 page tables. My full ones, or wait, no, sorry, that one. So I use 40 megabytes for my four L1 page tables. And through that, there are 10,240 full L0 page tables and then 20 full L1 page tables. So my grand total, I would need actually 10,260 page tables. And then the size of each one of those are going to be 4,096. And if I take that to a megabyte and divide that whole thing by what a megabyte is to the 20, you'll see that, actually, it's like 40.08 megabytes. Wow, what a difference, right? Why did we do all that when we just approximated it? I don't know. I'm slightly crazy. But it kind of makes sense, right, that I would have, so if I have like a 39-bit virtual address space, that means I can address up to 512 gigabytes. And that makes sense that if I need to map 20 gigabytes, well, that's essentially like 20 entries in my L2 page table. So each entry can actually map up to a gigabyte. So it kind of makes sense. So each entry in the L2 can, if everything is full, figure out the mappings for up to a gigabyte of memory. And here, if you want, so yeah, a question, if we want to be specific, should we also count the L2 page table? So every process needs to have an L2 page table to exist, so you can assume it already exists. Otherwise, that process wouldn't be running. All right, any other questions or anything like that? All right, so I will be here around and use the extra time if you want to ask me about your problems with Lab 2. So just remember, we're all in this together.