 All right, all righty, welcome back to operating systems. So yeah, midterm back as of this morning at someone godly hour whenever I woke up. Average is like 76, something like that. So hopefully not too bad. For regrade stuff, I posted a list of what TA marked what. So procedure for that, email the TA first. If there's a problem that like if they just messed up or did a mistake, they'll fix it. Anything more serious, they tell me and then I'd look at it. Yay. All right, so any questions about anything with that? So yeah, average is going up slightly, so yeah, 76. Might even be higher than that now, 76.1, no idea. But yeah, generally was pretty decent-ish, ish, ish, okay. All right, so now we get to talk about page replacement. So on our machine, our memory hierarchy looks something like this at the top, which is the fastest and least plentiful memory, it's like the CPU. So we only have a few registers on our CPU. It's as fast as we can get to store bytes, but yeah, there's not much of it. And per byte, it's really, really, really, really expensive. So below that on your CPU, there'll be caches. So like there'll be an L1 cache and L2 cache and L3 cache of varying sizes. The higher the level, the slower it's going to be, but the bigger it's going to be. And yeah, we might have up to three levels. Below that is memory. So just normal random access memory. So that's like virtual memory dealing with pages. Capacity for that, probably like 16 gigs, 32 gigs. The lucky of us, 64 probably end up there. And then pass that is NVMEs, which is likely what most of you have. That's non-violetile memory, so it doesn't get deleted whenever you turn the power off. That's like your really fast SSDs. So capacity for that, like affordable ish, is like a terabyte, maybe two, maybe 512 gigabytes. And it's slower than RAM, but it's a lot cheaper. Below that is more like SSDs, like cheap ones, that are more like thumb drives and things like that. So they're really cheap, but they're not that fast. Then after that, there are hard disk drives, so that's the spinning magnetic things. And then below that is tape drives, which are magically somehow still used. Have any of you ever seen a VHS tape? Stereo, so where you put it in a VCR player, so it looks like this, it's black, it has two little holes in them. If you look through the little window, it has like a tape and it goes, it kinda looks like a movie theater reel. So you can actually use that same principle to store data. It's really, really slow, but you can fit a lot on it. So Google will actually use those for archival storage, things that they're not going to access in a very long time, because they can store like terabytes of data for like cents and they are very, very cheap, but they're very, very slow. So if you go look at a Google data center, I think there's a YouTube video about it. You can see like a giant thing, giant shelf full of actual things that look like VHS tapes. And in order to retrieve data, you can see a little robot arm go and grab a tape drive and then put it in the thing to read it. It's really, really slow, but that's the only way to store like petabytes and petabytes of data without going broke. So likely you will never see that unless you go work at a Google data center, in which case, let me in. All right, so the whole idea of this is we want to use this hierarchy to hide it from the user and pretend we have the speed of the layer above it and the capacity of the layer below it. So if we're using memory, we want it to look as fast as actually using the CPU, but you'll have a much capacity as like a hard drive. So you might notice sometimes when you use your computer that you are able to use more memory across all of your processes than you have actual physical memory on your computer. And that is done through this as well. So the idea behind that as well, one, all your processes probably aren't using all of the memory at the same time. So what it does is it can throw some of those pages off to disk and use that for additional storage. So if you've used, I assume most of you use Windows. So on your C drive, if you unhide what's on your C drive, you'll see something called like a page file or something like that. And that is where memory goes whenever you run out of physical memory and it's creating the illusion that you actually have more memory on your computer than you actually physically have. So you might notice even if you only have like four gigs of RAM, you can launch Chrome or something like that. And it will, that uses 10 gigs of RAM, but it's still able to run somehow. That's because it offloads some of that memory to disk. So that's basically what the manpaging is. So we use memory as a cache for the file system. So could happen in two ways. So one idea is like that M-map example where instead of just reading a file all at once from the disk, I set up my virtual memory so that one page corresponds to a block on the disk. And the idea behind the manpaging is I won't read that entire file into memory until you actually use it. So whenever a process tries to use that page, there's a page fault, then the operating system or the kernel specifically in this case will only load that page in from disk whenever you actually try to use that. So only uses things that only uses memory or only reads in from disk what you actually use. You can also do the reverse of that. So I could, if this memory page doesn't correspond to a file on the disk, well, if I want to create the illusion that I have more physical memory than I actually have present on my machine, I can essentially just map it to somewhere on the disk that will typically on Linux be called the swap space. So I map that memory page to somewhere on disk called the swap space. Sometimes it'll be a file. Sometimes it'll be a partition. But if you run H-top, you'll see right next to the memory of how much is being used under that bar. It's called SWP. And that is the amount of swap that is being used. So what that is is you ran out of physical memory and the kernel had to move some pages from memory and write them to disk because you ran out of space. So any questions about that? That's how we can create the illusion where we run things. Well, we can support more memory than we actually have on our machine. So with this, we could consider a processes working set. What's a working set? So a working set is just the number of pages that a process use in a given amount of time, typically from like when it gets context switched in to when it gets context switched out. Ideally you would like all those pages to be in physical memory. So you don't have to go to disk to get them because otherwise that would be slow. So if you can't fit all the memory your process uses in a certain amount of time in the memory, your process may do something called thrashing and thrashing is bad because it essentially makes your performance go down to zero. So thrashing is just constantly, your program uses a page and it can't fit in memory so it has to boot something out onto disk and then whatever it booted out onto disk you use it right away. So it has to read it back in from disk. So it's constantly going in and out of disk instead of just using physical memory. It's called thrashing because while it was a mechanical drive, suddenly it would just sound like hell because it would just keep on reading back and forth and it would just sound like it's thrashing around. We're not too creative once we come up with names. This is constantly going in and out of the cache and your performance sucks. So is this kind of transparent unless you're really paying attention to your performance? You might not notice thrashing or swapping actually happens. So if you really care about performance so this will go into future courses or maybe you have lots of money. If you're Google or something like that and you really care about performance what some of them do is completely disable swap space and that way if you have run out of physical memory instead of going slow it just immediately runs out of memory your process crashes and it says out of memory and if you're Google, how do you fix the problem? Buy more memory and then buy more memory, problem solved, everything goes fast. You don't have to try and debug weird performance issues let run out of memory, buy more, throw money at it. Google loves doing that, Microsoft loves doing that, Amazon, everyone. You'll probably love doing that at some point. So once we talk about what to throw in and out of memory there's some algorithms we can use today. We'll go over three but there's four we can talk about. So there's optimal so we can replace the page that won't be used for the longest amount of time that's looking into the future. I'm going to maximize the amount of hits I have where I don't have to move a page back out to disk and read a new one in. Another strategy we could use which I can't really evaluate you on is just random. So instead of making any informed decision I just randomly pick a page and evict that and we'll talk about that a bit but don't really need to explain it. The next is first in, first out which is generally one thing we can go to in operating systems if we have to like do anything that looks like scheduling. Yep, that won't be so you look into the future. So in this you could only do that if you're given all the page accesses so you couldn't implement it but it's the most optimal thing to do. So we usually just use it to evaluate. So yeah, first in, first out replace the oldest page first. It's just a queue like we all know and despise possibly. And the last one is least recently used so this looks into the past. So I replaced the page that hasn't been used in the longest time so I should be actually able to implement this. And these page replacement algorithms, guess what? You'll probably have one of these on the final. They're kind of like scheduling problems but they're a bit easier. So for scheduling we did like average wait time, average response time, number of context switches. For this it's a bit easier. We only care about essentially the number of context switches or in this case the number of page faults that we have to read in from disk. So when we evaluate all of them we assume that our physical memory can hold some comically small number of pages so we actually have to move things out of disk. On a real system you'd have like millions of pages for this because we need to argue about it by hand. We'll just have four. And in this example we'll just assume a single process, nothing insane. And we're going to assume that it accesses just these page numbers. And we'll also assume that initially all the pages are on disk so I have to read them into memory. So we'll use this exact sequence for a bunch of the examples during this lecture. Our goal is, well we just want to reduce the number of page faults because going to a disk is slow. So for every example all we're going to do is find the number of page faults, lower better. So any concerns about that? Be straight, hopefully fairly straight forward. So this case, assume our physical memory can hold four pages. So how I present these is this box here is physical memory. And so I can fit four numbers in this box so this represents what's actually in physical memory. And then above the box I write what page I'm accessing mostly so that I can keep things straight. Make sure that this page should definitely be in physical memory if I'm accessing it otherwise I did something really wrong. So if I access page one and I assume nothing is in memory yet I have to read it from disk into memory. And in that case now page one would be in memory and I write it in red text because that was a page fault that I had to load from disk. So any question about that setup? All right, so what happens if I access page two? Yeah, it's just another page fault and I have to read it in from disk, throw it into memory so it would look something like this. So I access page two, well I wouldn't replace page one because I can fit up to four pages in memory so I'll just put page two in it, write it in red. So if I access page three likely the same thing will happen, right? So I have to load it in from the disk, now it's in physical memory. Next I access page four, I have to read it in from disk, now it's in physical memory. So any questions about that? Should be the first four accesses in this if nothing is on the disk will all be page faults. All right, so now what happens if I access page one? Nothing happens, it's already in memory, right? Don't have to read it from the disk so I just rewrite what's in memory, I don't have to move it, there's nothing in red so I don't have a new page fault. Clear? Kind of. Kind of? Okay, so now I access page two. What happens with page two? Nothing happens. Nothing happens, right? It's already in physical memory. All right, what about page five? Yeah, so we're using optimal right now. So optimal is I look into the future, I replace the page that won't be used in the longest period of time. Throw out anyone? Throw out four, we got lots of fours? So if I look into the future, well I use one immediately so I definitely don't wanna throw out one and then I use two next so I definitely don't wanna throw out two then I use three. So I definitely don't wanna throw out one, two or three. That leaves four as the poor victim so I need to throw out four. So to write that, I just replace four with five and write what just came in in red because that would also be a page fault. Good, any questions about that? All right, so now if we did the optimal thing, well if I access page one now, what happens? Hopefully it's already in memory because I made the best decision, I could. So access page one, nothing happens. Access page two, well I was smart so nothing happens again. Access page three, I was smart so nothing happens again. Now I'm at the end of my rope, I access page four now I have to throw something out. What do I wanna throw out? Yeah, one, two or three. So in this case, I just don't wanna throw out five. There's only one wrong answer and it's throwing out five. So I can just arbitrarily pick one, two or three. In this case I don't like one so I just replace one with four. And then now I access page five which is already in memory so not a page fault. So how many page faults are there? Six, so I just count the number of red numbers, right? Makes it simple. If you want in your exam circle it, underline it, do whatever you want. So six page faults. Do I have any questions about that? All right, let's remember that. So six, optimal, so if we have anything that's better than six, well that doesn't make sense because this should be the optimal thing. So same thing again but we will do FIFO. So can we speed run this lecture and I do the first four? Sure, all right, first four, same thing. We need to load them in from memory. So anyone want me to go back over the first four? All right, so what happens when I access page one? Nothing, what happens when I access page two? Nothing, what happens when I access page five? We throw out one. Yeah, throw out one. So if it's first in, first out, the first thing I load it into memory is a one so I boot it out. So I boot out page one and replace it with page five. Now I access page one. Ouch, I made a bad decision. What do I replace? Two. Now I access page two. I made a poor decision, I made a really poor decision. All right, so now I access, I throw out page three. Now I access page three. Now I remove, yeah, four. Now I access page four. Wow, I suck at this, all right. So now I kick out, which page? Five. Now I access page five. Now I kick out page one. How many page faults was that? 10. And you can also do a quick thing for FIFO so the red numbers, they're all diagonal lines here. So if you want to do it really quick, it's like argued by just diagonal lines so they should all look like diagonal lines if it's FIFO because it's first in, first out. So that was pretty bad, right? So if I had, say I have more memory, like I just solved the problem by buying more memory and I can hold five pages, well I should only have five page faults in total, right? I just load in all five and then everything fits. I have no other hits, everyone agree with that? So more memory, less page faults. Everyone would agree with that? So is the opposite true? So if I have less memory, I should probably have more page faults, right? Makes sense, like depends how you access them. Less or equal? Sounds reasonable that if I have less memory that I would have more page faults because I have less memory, it's smaller. Let's make sure, so this is 10 page faults. So same thing, but we can only fit three frames in memory. So let's do this again. So first three steps will be exactly the same, but now we are full. So now we access page four, what do we throw out? One. One. Oh crap, that was a bad decision. So this already looks, yeah, like we're right. So now we access page one, what do we throw out? Two. Two. Now we access page two, what do we throw out? Three. What, oh yeah. Yeah, so now in this case, we have six accesses, we have six page faults, we're not doing too good. Now we access page five, what do we throw out? Four. Now we are seven for seven? Not looking good. Now we access page one. Do we replace anything? No, that's a hit, we are good, so things are looking up. Now we access page two. Do anything? No, we're good. Access page three, what do we throw out? One. So we throw out page one, now we access page four. What do we throw out? Two. Two, and now we access page five, which should be good, right? Already in physical memory. How many page faults is that? Nine. Oh man. Hmm, you guys lied to me. This is better. I have less memory, less page faults. So if your computer starts running slow, just take out a stick of RAM, it'll go faster. That makes sense? Not really. So this is so special, people gave it a name. It's called Belladies Anomaly, and what it says is, well, if you have more page frames, you can have more page faults, which is not what we expect. And this is only a problem for page replacement, only for FIFO algorithms. This does not exist for other algorithms we will talk about, like at least recently used, already other like stack-based algorithms. So if you are super interested in this, if you wish you were in math or something like that, there's a bunch of math people that wrote a paper in 2010 that said this is actually unbounded, filled with a bunch of Greek letters that I can no longer read anymore. So if you wanna try and read it, be my guest, basically their conclusion was that you can just construct any arbitrary sequence or any sequence of page accesses to get any fault ratio you want. Doesn't matter how many pages you have. So it just depends, you can make a sequence for whatever you want. And they proved it through math. I can't explain it more than that. That means you should probably look at more CS things if that is where your interest lies. Don't ask me, I cannot read Greek, I can read C. So for other algorithms, it behaves like we expect. If we have more page frames or more memory, we have less page faults. But specifically for this, it has this weird anomaly where less memory, less page faults, just a weird thing. And in practice, it essentially just comes down to whether or not you just got lucky in this case. Because, well, you can construct arbitrary sequences, but you don't know what a process is actually going to use. So let us do our final example of the day. Wow, we're really speed running this. So this is least recently used. And again, we're just assuming one process and that is it. So in this case, we won't have ties, but if we had ties, if we have multiple processes for whatever reason, we could do something like five photo break ties. Again, we'll just assume just only one process. And I swear I won't just make a page replacement for the exam that I just pull out of thin air that tries to do something with you for that. So this will probably be on the final exam. It will look fairly straightforward. So first four accesses are gonna be the same, right? Don't have to go over those again. All right. So now access page one, what happens? Nothing, it's already there. Page two, what happens? Nothing. All right. So now page five, we have to kick something out. And this time we're using least recently used. So that means we look backwards in time and whatever we have not used in the longest time, we kick out. So between pages one, two, three and four, what do I boot out? Three. All right, so if I look backwards, means I can't boot out two, I can't boot out one, I can't boot out four. So my remaining victim is three, I kick it out. Any problems with that? All right, so for page five, I kick out page three. And now access page one, that all right? All right, that's all right. Access page two, is that all right? That's all right. Now access page three, all right, I need to kick something out. What do I kick out? Four. So I look backwards, I can't kick out two, can't kick out one, can't kick out five. So my last victim is page four. So I kick it out, and now access page four. So I made a bad mistake. So now what do I kick out for four? Five. Yeah, in this case I make another poor mistake. So I look backwards, I can't kick out three, two or one, so I kick out page five. Turns out that was a bad mistake. Now I have to replace something with five. What do I replace with five? Two? Anything. So this least recently use has to look backwards. So unlike optimal, we can't pick. There's a definitive answer here. So I can't kick out four, can't kick out three, can't kick out two, so leaves me with one. So I replace page one with page five. How many page faults we got? Eight. Better. So crappy FIFO was 10, smaller one was nine, this is eight, and optimal was six. So was all right. All right, so could you actually implement this? How would you implement it? Yeah, if you wanted to, for every page, you could keep track of the last time you accessed it, and then if you were to implement this, you could just, if you needed to replace something, you look at all the timestamps and pick the smallest one. Seems like that would be pretty slow, right? I have to scan. That's like O-N, and the more pages I have, the slower it's going to get, and also I have to keep track of a counter, which might not be too bad, but ideally I don't wanna do that. So the bad part is if I try to implement this, while I have to search all the pages, which is really, really slow, so it's hard to implement it. Can we think of another implementation for this that doesn't use a counter? What about some implementation, maybe using a link list of some type? Double E-link lists? Yeah. Yeah, so I could have a double E-link list, and in your case, if I access it, I put it to the back, and then to replace something, I replace it at the front. So yeah, either I could have a double E-link list where I put the most recently used thing at the front or the back, doesn't matter really, so I could put the most recently used one at the front, and that would make the back the least recently used one, and or I could do it in the other order, doesn't matter as long as I'm consistent. So, would that be a good idea? So every time I access a page, so that's even if you read a single byte, you have to do a link list operation. Probably not a great idea, so better than searching every page, but that's not saying much, so that means like if I access a single byte, well, that means I have to bring that page to the front of the list, and computer science-y, that's like a constant time operation, but realistically, if I access one byte, and then I have to do what? How many pointer operations is a link list insertion to the head? Something, probably like six, something like that, a lot, and I access one byte, and then I do six pointer updates, that means suddenly everything is six times slower, because I have to do that whenever you read a single byte from a page. So, that's really slow, I can't really implement it, and even if I could implement it, what would happen if I have multiple cores? So they would have to, they each use memory, but they would all be accessing this link list. So if I have multiple cores all updating a link list at the same time, and they're writing to it, would I have a data race? Yeah, probably quite a few, kind of looks like, surprisingly it looks like lab five a bit, right? So, in order to fix it, well, it would make it even slower because they'd have to put a mutex around that link list, now it's even slower, so the more cores you have, the worse it gets. So, definitely can't implement it. So, in practice, we can't implement least recently used, we can kind of argue about it. So, we do what any good computer engineer does, where if you have a difficult problem, all you do is change the problem. So you just make the problem simpler. So, the thing we can do is we can implement a approximation of least recently used, so we'll take the idea and try and implement that, and we can argue about it that like, hey, well, least recently used is kind of an approximation of the optimal algorithm. So, if we take an approximation of the approximation, it's probably okay. So, it's kind of like scheduling, there's lots of different things you could do that all have different trade-offs. So, this thing takes time to run, so even if you have the best algorithm in the world, if it takes too long to run, you can't actually implement it, and well, if it takes a long time to run and sometimes makes a bad decision, then that might be even worse than just making a okay decision really quick. So, specifically in this course, what we'll see in the next lecture is something called clock algorithm, which tries to approximate least recently used, and the idea is, well, a clock that we'll see, and it's a whole animated clock, and it's a whole thing, it's lots of fun. So, other things you can do to kind of approximate least recently used is least frequently used. So, instead of keeping track of like a time stamp, you can just keep track of whether or not it was used in a certain time period, and then just get rid of things that haven't been used. You can also use 2Q, which is like a hardware queue, an adaptive replacement cache, so it just throws a cache on there. There's all sorts of different trade-offs you can make this same as scheduling, there's no one answer for this because, well, it also depends on just arbitrary processes using memory, and just because a strategy is good for one process might make it bad for another process. So, any questions for today? It's hopefully fairly straightforward. Yeah, pretty much. You asked me to speed run it. So, oh, I can make it. So, are we all on the same page? All right. There may be some. All right, miss that joke in the last, yeah, the whole reason I do this course is for that joke, so now I'm done. We can finish. All right. So, today, we saw a page replacement. So, we saw the optimal algorithm, which is good, unrealistic because we have to look into the future. If you want, throw machine learning at that, maybe that'll do something. That's, if you wanna do OS research, hey, that's something you could do. Yeah, probably a bit too much computation, but, you know, the replacement decision happens fairly infrequently, so maybe. For random, random actually works surprisingly well because the idea behind that is, well, if you make random decisions, you might randomly make a good decision and you might randomly avoid the worst decision. So, typically it actually kind of works better than FIFO because it avoids the worst case. So, FIFO typically always makes a fairly poor decision, so making a random decision is better than always making a poor decision. So, FIFO, easy to implement, but has that weird anomaly and yeah, we probably don't wanna implement that. Then, we got to least recently used. We saw that, that worked pretty well, it gets close to optimal, actually works well for most sane programs, but we can't actually implement it and we'll implement something tomorrow. No, not tomorrow, Thursday. So, remember, pulling for you.