 Okay. Hello. Hope everybody had a nice weekend. Starting to get nice outside. Just want to point out that you guys are in a race with the weather. Just not helping you this year. So I would suggest you get assignment three done now. Then you can play outside in the 70-degree weather that we're going to have next week, it seems. Sorry. I mean, normally Buffalo is good about this, but you guys are just tough here. So today we're going to finish, we'll see how far we get. Might end a little early. We're going to finish talking about virtual memory. So as we did with the CPU, we've come sort of up the stack, talked about mechanisms that hardware provides, talked a little bit about how operating systems use those mechanisms. And now we're at the point where we're going to talk about policies that govern how various decisions are made that affect the overall performance of the system. So that's where we are now. You guys know the mechanisms that are used to implement virtual memory and to also to implement swapping, which we talked about last time. And so now today we're going to talk a little bit about how some of the decision-making logic works, or at least be able to frame the problem as far as what we want, right? What are our goals when we start talking about page replacement and what we should move out to disk or to some sort of secondary storage when we need more space? Okay. So first I just want to plug this one more time. I know I already posted it on Discourse, but please go to see Dan's talk tomorrow. Dan is a super cool guy. Obviously his company has been very successful, so he has, how many people heard about that? How many people have ever heard of 43 North? Okay. About half the class has heard of like a massive business plan competition that occurs in this city. So I don't know if that's a good, bad, and average or not. How many people know who won last year? Okay. How many people know that that's a tech company that won? That's the same number of people. Okay. So this is interesting, right? You know, Dan's someone I've known for a little while. He's a UB alum. He's very personable. He's a great guy. There's a lot of interesting stuff sort of going on downtown in the downtown taxi. Has anyone ever been down to the incubator before or to the, yeah, it's super cool, right? I mean, I wish I could, I wish I could move my group in there. The only problem would be that I'd have to come up here to teach, which is a pain, but it's a super cool space. There's cool companies there. They're doing interesting things. This is one of them, and I'm sure he'll have plenty to say about tomorrow, so I won't ruin it, but it's cool stuff. I mean, they have built, they built an app and they made money with it. So if you have some desire to do that yourself, please come. So after mentioning this three or four times, at some point soon we will probably finally get around to changing the one or two lines of code necessary to make the Test 161 leaderboards public. So does anyone have any questions about this? If you go to your profile page on the Test 161 website, it explains how you can set visibility settings so that you can decide how you want to share your glorious or maybe not so glorious results on the assignments with the world. So if you earn a perfect score on the assignment, there is a leaderboard you can be on. If you would like to reveal your name and set up a link so that people can find you, like an employer who might know about these things and be interested in hiring you or talking to you about a job, you can do that. If you would prefer to have made anonymous, you can also do that. If you'd prefer for the leaderboard to look like you didn't exist, you can also do that. It's the beauty of virtual anonymity. So, but yeah, so does anyone have any questions about this? This is an opt-in system. It was completely voluntary, but we do hope it's a nice way to give you guys some recognition for your work. Yeah, so yeah, it's a great question. So for right now, the only things on the leaderboard are the targets we've set up for the different assignments that track points. However, the whole Test 161 system has always been built in order to also do performance testing. So we may eventually over the next few weeks roll out some assignment three performance targets and expose those as well. So yeah, good question. And for the assignment targets, you're on them if you get a perfect score. We might put together an overall assignment three target that will actually be the leaderboard for assignment three rather than the little ones that are the checkpoints. So, but for the other ones, it'll be performance based. So we'll run parallel VM five times and it's just a race to the finish line. It's a good question. Any other questions? Okay, cool. All right, so who can tell me the difference between a TLB fault and a page fault? Two memory related exceptions. Yeah? Okay, so they can like pay 20, 20 answers, right? So that's one fact about this is a true, which is that a TLB fault precedes a page fault. What else? Who else can tell me something about these two types of faults? Right, so the TLB fault, you can just think of it's the TLB generating the fault. A TLB generates a fault when it can't translate a virtual address to a physical memory address. So that's what a TLB fault is. A signal from the TLB telling the operating system, I do not know how to translate this virtual address, at which point the OS can either load a translation or determine that the process is accessing some sort of memory that it's not supposed to and take the appropriate measures. What is a page fault then? Steve, not you again. Yeah. Yeah, so if the page is not in memory, right? So remember, all the TLB tells the OS is I don't know how to translate this virtual address. In the process of loading the translation, the operating system may discover that this page is no longer resident in memory because I moved it out to the SWAT file to make more space. Or it may also discover that the page is not initialized. This is a heat page. It's an address I've given the process permission to use, but the process has never accessed it before. In that case, I just create something with a bunch of zeros in it and let the translation continue. And every, as was pointed out, every page fault is preceded by a TLB fault. And you can, I'll leave that as an exercise to the reader why that's important. Okay. So we talked about swapping a page. So what is, what does it mean to swap a page out? What am I doing? I like that there's like, there's a hand motion here I like, right? That's like the international symbol for swap out. Well, maybe this is, I don't know, one of them swap in the other swap out. I don't know, you guys can figure it out. What is the, what does that mean? What am I doing? What do I have to do to swap out a page? You're so close. It's way more than I ever could have imagined from an answer. One last thing. How do I make sure I can find it again? Yeah. Oh, I already did that. Yeah. Update the page table entry, right? Three steps. Remove the translation from the TLB, copy the contents of the page of this and update the page table entry. I have to update my own internal data structure so I know where this page is. If I don't do that, the next time it gets faulted, I don't know where it went. Yeah. Yes, it does. Great question. So one of these steps can kind of float. The other two have to be in a very specific order. What can happen if I reverse the order of one and two? Or let's put it this way. Why is it really important that one be done before two? Yeah. Yeah. I mean, you know, remember, the I.O. is not synchronous. The I.O. occurs in a series of steps. So you can think of the I.O. that's copying the page out as sort of starting at one end of the page and walking across the page and streaming all the contents out to disk. And if the page is changing while I'm doing that, it's possible that I've copied the first half of the page and the process makes some change to the part I've copied and then I copied the second half of the page. And now what I have on disk is some terrible hybrid monster like a chimera with a head of a man and the body of a horse or something. Right? That's not what I want. I either want the man or the horse. So right. I was a stupid way of explaining that. All right. Yeah, so this makes sense. I do not want the contents to be changing. Updating the page table entry kind of can happen along the way, right? Okay, so I'm not going to do the diagram again. Great. So the first question was swap in well, let me go back to swap out. Why am I swapping pages out? Why would I do that? Yeah. I need more memory. I might have run out. I might just be doing this preemptively to make sure I have a certain amount of memory that's always available, whatever. Right? I need more memory. I could be out. I could be being careful. I could be trying to plan ahead and make sure I have some spare pages, but I need more memory. So this is how I do it. Once I've moved the page to disk, I have another 4K that I can do things with. Unfortunately, at some point, it's possible that I need to swap that page back in. When does that happen? What forces me to swap in a page? Now, I can swap in pages whenever I want, right? I can always swap in a page, but when do I have to swap in the page? Yeah. Yeah, when the virtual address is used. Remember, the virtual address contract requires that I make it look like memory. So now I'm doing a store to that virtual address, and I better make it look like memory again. And that involves bringing it into memory and mapping the virtual address to a physical address. So at this point, at this point, you can think of it this way. I have a virtual address that the TLB has to be able to translate to a physical address. It's a valid address in the processes address space. That's the assumption. So I had better create a virtual address for it and make sure that that virtual page has the contents that the process is expecting. So in this case, there's more steps. I need to stop the instruction, right? So this instruction generates an exception because it cannot complete until I do these other things. I need to bring the page back in, update the page table entry so I know where it is in memory, and then restart the instruction. And I skipped like three steps. Yeah, I need to get a page, right? That's important. Load the entrance into the TLB and restart the instruction. Any questions about swapping in, swapping out before we go on? This is the mechanism. Yeah. Uh-huh. Yeah. Yeah. So that's that's one of the things. And as you guys start assignment three, that's one of the questions your page table has to be able to answer. Where is this page? I've got a virtual address. There's a couple of cases that you might need to encode. One is it's in physical memory. It's just not in the TLB. That's your bare TLB fault. Another is it's on disk and I have to go get it. Another is it hasn't been initialized. I need to fill it with something. There's other cases as well. Those are the main ones. But yeah, your page table has to be able to answer this question. And so ultimately your page table entry ends up containing this information. Yeah, good question. All right. Yeah, okay, we're not too tired. Okay, so so now let's talk about page eviction. So that's the mechanism. Let's move on and talk about the policy. So the mechanism here is I can safely by applying these two primitives swapping out swapping in, make a portion of the disk look like backup memory. So I can move things out to it. I can move things back in. Now I have to do that manually. I can't I can't directly map virtual addresses out to disk. But I can move things out there when I need some extra space. So the big question I that that's involved here is when I'm trying to create memory. What page should I choose? So what should get moved out to disk? Now, and there's a couple reasons I might need to do this, including I'm actually trying to swap in pages for another process. So these two things can be connected. It's possible that part of that swap in operation, and this is something you guys need to think about when you build your own VM system, part of that swap in operation might have to stall waiting for a swap out to finish. So if I that allocate page step may cause the kernel to do more swap. Okay. So what's the fundamental cost benefit calculation here? If you think about this from sort of an economic perspective. What is the cost? What is Yeah, how would I measure the cost here? Yeah. Yeah, it takes it's it takes time. Why do I care about that time? What what? How does how does that time affect the system? I mean, so what it takes time? I'm always doing though, as it's always doing a bunch of things all at the same time. Why is this a problem? Yeah, the context which ends up being a pretty small percentage of the overhead here, this does create a couple of context which is that's true. But the overhead here is is dominated by well, well, first of all, what dominates the overhead here? Right? What causes this to be costly? The IO. So the faster the disk, the slower, you know, the smaller the penalty. So, you know, you can imagine how flash has changed some of these these calculations flashes much faster, faster substrate. So, you know, now, what during the time that I'm that I'm doing this that I'm moving things, what can't happen? There's a couple of things that I'm that that cannot happen a couple places where I might be stuck. What is that time delaying? Somebody else. What can't proceed until I finish this operation potentially? Yeah, what's that? Well, I mean, so I'll take one word from that answer, right? The page itself is totally useless. So while this is going on, I have a page of memory that's actually worse than, you know, it's sort of it's totally useless. I can't modify this memory. And it's not on disk yet. So it's stuck in this sort of no man's land. So I lose a page of memory for the amount of time that it takes to actually perform this operation. What else? What else does this generate? Yeah. Yeah, some thread is very likely blocked. It tried to access a page. And that access triggered a swap in. Maybe that swap in itself triggered a swap out. So something is waiting for that to complete and cannot complete. Remember, that memory instruction just cannot complete until I finish this operation. What else? There's another place where this generates load. So I have the time there's another component of the cost here. How do you tell when the machine is swapping a lot? Old machines. Yeah, so I'm generating IO. Now, maybe you don't think this matters, but it turns out that IO is slow. And the more IO I'm sending down to the disk subsystem, the slower it all goes. So I'm generating IO that's competing with other IO that's also going on, generate by the file system and other things that are happening. So I'm also slowing down the disk. And that's causing everything that uses the disk potentially to slow it out. You know, the disk only has a certain amount of bandwidth. And I am I'm adding bandwidth to it. Okay. So it's the time the cost is the time and disk bandwidth required to move a page to and from disk. What's the benefit? What do I get out of this? Why would I do this in the first place? Run? Well, break it down for me, though. Like that's that's a very meta level answer. What do I get out of one swapping one page out? Yeah, that's good. That's true. Yeah, but but but I mean, both of these things are a little too high level. I'm swapping one page out. I get something out of it. What's that? No, I haven't got there yet. We haven't we haven't started to talk about what to swap out. I'm just talking about why, right? Yeah, I get a free page of memory. There's a page of memory that was used a second ago. And now it's free. I get exactly one additional page of memory. Here's the thing. So if you look at this, again, you're an engineer. Maybe maybe you're a scientist like me, but whatever. What can you do? Like how can I make this? If I'm trying to optimize this system? What can I do? I mean, when I'm trying to optimize cost and benefit, there are two options. I can reduce the cost and I can increase the benefit. So what do you what do you think the right way to approach this problem? Yeah, well, okay, so if I swap multiple pages at once, the cost is roughly twice, right? And the benefit is roughly twice. Now you're right. If I you could certainly make I probably make some arguments if I do a lot of swapping at once, maybe I can use the disk a little bit more efficiently or something like that. But I would argue that this pretty much is this will pretty much scale linearly, right? The cost if I don't change something else, the cost and the benefit to scale 100 pages is 100 times the cost of benefit to scale one. So what do I do about this? Yeah, so that's great. So one way to make so that's not really reducing the cost. It's just making the cost less damaging. If I can do swapping when the system is idle, then that's good. I can find times that will also help because it will reduce the interactive delay. But there's one big knob here that I want you guys to identify. There's one big place here if I make a good decision, I can win. Yeah, let's let's assume I have a fixed amount of swapping to do. I've got I need 100 pages, you've got to optimize this cost benefit calculation. So I don't really have a choice about this. The system has a certain amount of memory and the user did something that requires more than I have. So what what can I change it? Yeah, so here's here's the knob, right, right here. I the benefit here depends on how long the page that I remove stays unused, right? So remember someone, what's the worst case here, right? I think we'll get to this, right? Someone taught before talked about a case that was probably like the worst possible case for this. So what what's the worst case scenario? I think I have a, okay, we'll talk about thrashing a sec. What's the worst case scenario? Yeah, define immediately. I could do worse than that. Yeah. Yeah, so imagine process A is on trying to swap in a page. It needs a page of memory. The system is full. It starts to swap out a page from process B. What is it going to do? What is process A going to do as soon as it starts to swap that page out? No, no, no, what is process A going to do? So process A has chosen process B's page to evict. What is process A going to have to do while that completes? What's that? No, it's going to block. It has to wait. Because remember what it was probably trying to do is access some sort of memory. And this is how I ended up on this long, tortuous, terrible nightmare where I needed a page and I had to swap out a page. But whatever, okay. So process A now goes to sleep. Now what happens? Process A is sleeping. Process B is scheduled. What does process B do? Access is the memory that process A is in the process of swapping out. Yeah, so that's the saddest, saddest possible case. Because then what I have to do is either find some way to pause the IO, which I don't suggest you try to do. Or the worst case scenario is I have to wait for it to get to disk. And then either I have to go and whack process A and be like, by the way, you don't get that page, in which case it's even slower. Or in certain cases, if that memory is already gone, I have to bring that page all the way back in. So in that case, I have paid the cost. I've paid the full cost of swapping a page. The full cost of swapping a page is both bringing it out and back in. And I have received zero benefit. Because the page was available to me for no time. Like even before I got done doing the IO, I needed it again. So that's the worst case scenario. So what is, okay, so this is what I said. So what's the best case scenario? I'll come back and we'll do some of these slides in a sec. What's the best case scenario? So that's the worst case. Even before I'm done, the page is already on its way back into memory. What is the best case? Yeah. Yep. So there's two ways that that maximizes the benefit. So in that case, it turns out, well, I shouldn't say there's two ways. And if you maximize something, then you're done. So that maximizes the benefit. Because the amount of time the page is unused is infinity, which is a good number when you're trying to maximize things. It also reduced the cost. Why? Yep, don't have to swap it in. Didn't have to do that IO. Didn't have to stall the process. So my cost is roughly half. I only have to do wine IO. The benefit is infinity. So by construction, this is by far the best page, right? And this isn't so weird. Okay, so you might think that this is strange. This is like never going to happen. Can you come up with some examples of pages that might never get used again? Ever? Yeah. Okay, yeah, someone else is going to pick up Microsoft Word with me. I like that. Yeah. Well, remember, so those pages had to come in at some point, right? So it has to be something you used. But again, maybe, you know, you tried to use some new weird feature that, you know, that you couldn't figure out. And so you went back to doing it by hand, your old way that allows you to sort of zone out a little bit while you do something mindless and repetitive. Yeah, what else? Yes, that's a great, that's a great observation. A lot of startup code, a lot of initialization stuff. So, so just a fun fact here, it turns out if you start hacking around Linux, modern operating systems. So you guys are going to start to realize on assignment three that your kernel takes up memory. You know, the kernel code is loaded into memory, and it takes up memory. And it turns out that some of that memory, as the person just pointed out, is only used during initialization. So what Linux actually does is, and I'm sure Windows and Mac do this as well, is it marks those code regions as only used during init, and then it unloads its own code segment after it starts to run. It's pretty wild, right? So once it's booted up, like, I don't need those pages anymore, and they're gone. So there's a portion of Linux code that is only there during boot, and then vanishes. I think that's pretty cool. The way that they do it is really gross. But anyway, Linux is like macro city. There's all these weird macros everywhere. I have no idea what any of them do, but one of them is like early boot or something that marks a segment of code as not needed in other times, right? And then I'm sort of the linker packs things together, anyway, whatever. The details are uninteresting, but the fact that they're doing this is neat. Okay, what else? Well, whatever, those are good. I mean, Malik sometimes can produce this behavior. A lot of Malik allocators aren't necessarily great at freeing pages from the heap. The heap they usually look at as being contiguous. So if I have, let's say I had a bunch of data structures that were all on one page in the middle of the heap that have all been freed, Malik will just kind of let the operating system clean that page up, because all Malik can do is adjust one end of the heap. Malik can't really tell the operating system, oh, by the way, that page I'm done with. So what will happen is, you know, hopefully I can recognize that that page is inactive and move it out to disk, where it will never return, because the data structures on it are dead and are never coming back. Okay, so let's go back, let's talk about this a little bit thrashing, right? So this is sort of like we said before, this colloquial description for what happens when this goes incredibly wrong. And this is the state that you can get a machine into where the, I don't know if there's really an equivalent with CPU scheduling, but the overhead of doing paging and swapping has overtaken the system to the point where it's impossible to get actual real work done. Now there are times on a system, and you guys may have driven a system to this point, where it can become very slow and unresponsive for interactive use, but it's doing a lot of work. So that's okay. Now it may frustrate you when you're trying to SSH in the system and reboot it or something and you're sitting there and they have to take each character and wait five minutes, but if it's sitting there and it's like serving a million web pages a second, that's cool, right? I mean that's kind of what you want. Thrashing is not that. Thrashing means that I'm so busy just trying to move things back and forth to disk that nothing is really happening. You know, it's like things have slowed down to a crawl and the overall throughput of the system is, has dropped to a very unsatisfactory level. Yeah. Well, that's the thing, right? So nowhere here have we assumed that the OS has any mechanism for a process to tell it that a page is actually dead. Now there are pages like this, but for example a page from the code area. That page may never be used again, but it has to be able to be used again, right? I can't throw it out. Now with heap stuff you might be able to do that. You might be able to say this page is just totally dead, but in most cases I just, you know, the heap allocator just keeps that page in case it needs it for other allocations. But yeah, that's a good, that's a good question. You might argue that I could use some more mechanisms here that would allow processes to say to the kernel I am completely and utterly done with this, this virtual page and I will never need to use it again. But we don't have those, we're assuming we don't have those mechanisms and so I'm guessing. And if I guess I have to be prepared to be wrong, maybe even five minutes from now this page needs to be able to come back. Yeah. So that's a great question. The swapping is optional, right? Remember, this is a game we're playing in order to solve some problems related to virtual memory allocation. We're assuming when we swap, swapping well requires making some assumptions. If I have a system where every process has very high utilization of all of its memory then swapping is not going to help me very much because there's no good pages to move out, right? If every page is really hot and getting touched a lot then even if I want to swap it's not clear whether it's going to help, right? There are systems that don't swap. So for example, Android phones don't have a backing store. When they run out of memory something either they start to kill off stuff or the app doesn't load. So there's no requirement that you swap. It's something that you do to sort of try to take advantage of the fact that in a lot of cases there's pages that come in and are used briefly and then are never seen again, right? But yeah, that's a good question. Any other question? All right. All right, so we've identified a page that will be never used again. Now let's talk about how we actually do this. So what's the problem with identifying the page that will never be used again? Yeah. Takes time. Interesting. I wish all it took was time. I am going to sell you an algorithm that will tell you which pages will never be used again. Yeah. Yeah. What does that require being able to predict? The future, right? Like I said, if it just took time, that'd be awesome, right? I would take a very nonlinear algorithm for predicting the future any day, right? Like it can be like, oh, factorial. I'll take it, right? Like, even if I could predict five minutes from now, that would be nice. I guess it has to be able to run in less than the time that it's going to predict, right? Like, I will give you an algorithm that predicts one day from now in one day, right? That's 100% accurate. So yeah, we'd like to be able to predict the future. We're not going to be able to do that. So what do we do instead? When we can't predict the future, we use the past to predict the future. That's our approach. Now, what would we like to know? Now, when we talk about page replacement, compared to scheduling, there's some additional challenges here. And the challenges have to do with the OS's visibility into page activity. So let's go back and talk about, remember the good old TLB, what is the best case scenario as a process is running along? Process is running along. Does the operating system have any involvement in its memory usage? Yes or no? No, right? I don't want the OS to be involved. Because every time the OS gets involved, it's because something went wrong. There was a missing page from the TLB. I had to load a translation. I moved a page out. I wasn't supposed to, whatever. So the best case scenario is that the operating system literally has no visibility into the memory usage patterns of the process. And this makes things very challenging. So with scheduling, I see everything. I see when you block, I schedule you. I can measure how long you ran. I can measure all sorts of other things about your activity. With memory, not so much. We almost, the equivalent of, if I tried to write a scheduling algorithm that depended on the instructions that you ran, but at least in the case of the instructions that you ran, I can actually see those maybe. But the collecting statistics here is non-trivial. But here's some things I might like to know. One thing is, how long will it be before this page is used again? That's the predict the future algorithm. So the optimal schedule, or that we can frame an optimal schedule, we're in the same way that we did before. And this is, again, useful as a comparison point. So the optimal scheduler always knows the future. It knows what page is going to be picked. Let's, sorry, it knows what page, for each page, how long it will be before that page is accessed again. And it can choose the page that's going to be out the longest. And this maximizes that cost-benefit calculation. Clearly, right? All right, I'm getting very ahead of myself today. So now we're going to design this algorithm. And we need to, like when we design any algorithm, come up with a couple of things. One is, what are the inputs to it? What are we going to track? What can we track, remember, because it's possible that there are things about the page activity that would be too expensive to monitor. We're going to figure out how to collect the information and what to do with it, how to store it. So this is another issue here. I've got a lot of pages. This is another big difference between the page replacement challenge and doing scheduling. I've got a gazillion pages out there. If I try to store one byte on every page on the system, or worse, like four bytes or like some sort of ugly data structure, I'm consuming an enormous amount of memory. And it's not clear that that memory is worth it. It might be better to just use that memory to reduce memory pressure on the system and use a simpler page replacement algorithm. So when I was doing a CPU scheduling, I only have to store some amount of data per thread or per task. So on Linux, like the task, I mean, you guys have a thread structure. The task structure on Linux is this huge thing. I'm sure they've done a lot of work to keep it relatively small, but there's still a lot of information in it. And that's okay, because there aren't a billion tasks on one system. There could easily be a billion virtual pages on a system. Probably more. Okay, so as I pointed out, so the collection process can be expensive. If I'm collecting a lot of, if it requires slowing down the process a lot to collect this information, it's very unclear whether or not that trade-off is a good one, because I'm slowing things down in order to speed them up, and it's very likely that I'm not gonna speed them up in a way that's commensurate for whatever slowdown I've caused. The statistics who take up memory, and that could be better used for other things. So, simplest possible page replacement algorithm that I can implement. Random, at least I've imparted an appreciation for randomness to fit. Again, random, always simpler than FIFO. FIFO requires a little bit of state, random, not so much. Yeah, pick a page at random, yeah. Isn't technically random. Yeah, but it's not really random, right? I'm not using atmospheric noise here. I'm just using my pseudo-random number generator. Although I was, how many people have seen the movie Citizen 4? I think I should watch that movie, it's really good. Anyway, it's about Edward Snowden, so there's this one scene where he's using R-Sync, which I was kind of excited about, right? Because I use R-Sync. And he's R-Sinking down this, I don't know, he's transferring something to somebody, he's R-Sinking down this huge file that's called atmosphericnoise.zip. And I'm like, that's interesting. He's probably using it for encryption of some kind, right? But anyway, we don't need atmospheric noise levels of randomness to do a random page replacement algorithm, right? But you could try that and see how much it outperforms the pseudo-random number generator. Probably not by much, okay? So, pros of random, easy. And a good baseline for other algorithms. We'll go back to what we said about scheduling algorithms. If you can't outperform random, take your ball and go home. There is no point in continuing to try. Cons with random? Probably not going to do very good, right? Just gonna make some bad decisions here and there. Okay. So, now when we come to pages, how do we use, we talked about, when we talked about the MLFQ, we talked about ways to use the threads past to make decisions about what we're gonna do within the future. How do we do the same thing here? Someone is saying an acronym, which is not an answer to the question. How do I do this? What am I gonna? Okay, that's interesting. I could look, that's a good point. But that's sort of like looking to having to record something about the future. But what's, there's some basic intuition here about pages that I might wanna use, yeah. Okay, I might need to do that to implement this, but what's my intuition here? Go back and think about locality. Yeah, once a page hasn't been used for a while, it's the probability it's gonna be used again starts to go down. So, on modern systems, there's this notion of what's called a working set. A working set is the group of pages that a process has to have in order to make progress. That set might include a bunch of code, imagine I'm running some big loop, a bunch of things that are on my heap, the stacks of whatever threads are running, whatever. It's certainly usually larger than one page. So, the way to think about a working set is if I give, let's say I give the process zero pages, it's gonna start faulting in things right away. At some point, when I give it a little bit more memory, a little bit more memory, the page fault rate is gonna drop pretty dramatically. At that point, I've identified the working set for that process at that point in time. The working set changes all the time. I go into some other part of the code where I'm in a much bigger loop. I need more code. I'm using a bigger data search or whatever. So, but the intuition there is that once a page goes out of that working set, it, you know, the amount of time since it's been accessed is gonna keep going up and up and up. And pretty soon, I can probably say this page hasn't been used for a while and it probably won't be used again for a while. That's not perfect because sometimes pages, there's a periodicity to which I use things, but it's not bad, right? And that leads us to a very simple algorithm that's called least recently used, right? Check the page. So, when I go through all the pages in the processes address space, or sorry, all pages in physical memory, and I pick the one that hasn't been used for the longest, least recently used page, and that's the page that I evict. Hopefully, this is a page that I won't have to bring back in for a while. And the hint that I'm using here is the fact that it's the oldest. Pros of this solution, compared, you can say anything, compared with random, this might be as good as we can do, maybe, without being able to predict the future. It's not clear that there's a lot of really strong signals here about what to do. The real problem with this turns out to be about implementing it. And this goes back to the problem with both storing and collecting statistics that I handed out a little while ago. So, if I actually really wanted to implement a full version of LRU, what do I need to store for every page on the system? Timestamp, that timestamp is probably like four bytes. That's unfortunate. Here's the real problem. How do I tell when the page was accessed? So, I've got a process, it's running along, it's got 64 TOB entries that are all sitting there being used, and then I context-switch it out. Ron, do you have an idea about how to do this? Pages as a tree, when it comes back to dropping something off, you drop off a leaf, when a page is accessed, you move it closer to the root of the tree. Yeah, I mean, I can implement a heap of pages, right? And that would allow me to order it in this way by timestamp. We need a couple pointers as four or eight bytes, right? But the other problem here is collecting this. So, let's say I can do this. Let's say I've decided that for whatever reason, the overhead of storing this information is acceptable, how do I figure out when a page is used? I can guess. What's the best guess? What's the best way to guess a time when the page is used? Yeah, when the TOB miss took place. How inaccurate can that guess be? When is it the most inaccurate? Yeah. Well, now, give me an example of when it's the most inaccurate. Hypothetical scenario, yeah. Yeah, or imagine it's the first page I fault in, I touch it once. Then I run along for a while, and my working set's not big enough to kick any other entries out of the TOB, so I never replace that entry, but that entry has been sitting there now for microseconds, or milliseconds even, without being touched. And the OS has no idea, right? Because hardware users can store this information. So this is the problem, right? This is doing this. Yeah, okay. Yep, we just said that. And I don't wanna record this information every time, because it's just too much overhead, right? I really wanna fill the TOB as fast as possible, it's not worth it. Yeah, okay. This is all just the stuff that we've been talking about. So now let's talk about a sort of an interesting take on this approach. So this is a way to implement something that's LRU-like without having to do all of the work of LRU. So this is called the clock algorithms. This is one of these dusty algorithms that I feel obligated to talk about. Otherwise, people won't feel like they've got their money's worth. So how do I do clock LRU? I keep all the pages in some ring, right? You can imagine that they're just in the core map, I'm just iterating over the core map. I keep one bit of time information. That's it. That's one of the nice things about LRU, it's extremely efficient. I don't need to store three to two bits of a timestamp, one bit. And I proceed as follows. So when I load the virtual address into the TOB, I set this bit. This indicates that it's been used and remember that's the only time that I really know that. So I haven't solved that problem, but I can solve the storage problem this way. Set the bit when it's loaded into the TOB. Now, when I need a page to evict, I go through the core map and my algorithm is simple. When I find a page with its bit clear, I choose it. I evict that page and I swap it out and clear the memory so it can be used by another page for some reason. You know, I'm running this algorithm when I need memory. Otherwise, I clear the bit. Does that make sense? So if the bit is set, clear it and go to the next page. If the bit is clear, use that page and stop. That's where the algorithm halts. How do I know that this algorithm is always going to terminate? What's the worst case termination sort of situation for this algorithm? Yeah, if every bit is set, then I will make one pass clearing all the bits in order and then as soon as I get back to the first entry, I'm done. So there will be a clear bit and it will be one of the bits that I cleared. So here is my obligatory diagram. The green bits are clear. So in this case, I'm going to use that guy. The next time I run, I'm doing the same thing. This guy's gone. I guess that was the whole thing. I mean, keep in mind, in parallel with this, these bits are also going to be being set. They're going to be set during the page fault path. So when you analyze this algorithm, what does it mean when my clock hand is turning slowly? So go back here. Let's say that if I, if you imagine just looking at the clock as it goes around, is turning slowly good or bad? Who wants to argue good? What does it mean when the algorithm, when the clock hand is turning slowly? Yeah. Yeah, it's pretty quickly finding stuff or it's not being called very often, either one of which is good. So with the clock hand is turning slowly either, I don't have a lot of memory pressure, or when I do need a page, I'm finding one quickly. And so the pattern of accesses is such that I'm able to very quickly find a page due effect. That's the good case. What happens when the hand is turning rapidly? Yeah, that's either lots of memory pressure or it turns out that the particular algorithm is choosing bad pages, because those pages are coming right back in and I'm cycling. So this is generally a bad thing. All right, we are done with page replacement. We are done with memory in general. Any questions at this point? You might have like two seconds left. All right, so they're not gonna do this, or maybe we'll do it next time. On Wednesday, we'll start talking about disks and file systems. So I will see you on Wednesday.