 Welcome back to operating system. So yeah midterm didn't turn out too bad. Did it? so everything is returned to you as of someone godly hour this morning, so Seem to go alright average. It's like 75 plus a little bit by now might up also be up to like 76. Maybe I'm not sure Anyone want to check that? Yeah 75.9 all right so close it might end up going to 76 who knows so Yeah, it wasn't too bad I was really scared there for a moment because I was watching as they were grading and like It was like 30% average and I was like crap. There's not that many questions left But turned out it was like that Scheduling question that most people got right so grades went up and yeah, it's good All right, so any concerns about the midterm yet? 76 all right, we've hit the threshold. So yeah pretty good All right today we get to talk about page replacement so This is your computer's memory hierarchy some of these things might be new generally we're mostly concerned about these top fours or what in ours what's in computers most of the All the days now so the top you have your CPU on it. It has registers. There's not that many of them They're really really fast and well, there's not that many of them. So the capacity is pretty low Below that there's a CPU cache like the L1 the L2 and the L3 cache on your CPU There's more capacity. It's not as fast as registers, but it's faster than anything else Pass up after you run out of cash Well, there's RAM or your memory and that's where our virtual memory system comes into place and Well that Capacity most people probably have like 16 32 gigs of RAM Goes up from there and then after you run out of memory. Well, you have non-volatile memory Probably so that's like you're really fast SSDs that are in most computers nowadays They're like terabytes or maybe 512 gigs something like that They don't cost as much per gigabyte as RAM, but they're a bit slower then pass that you might have like SSDs like bigger drives or like thumb drives that have more capacity, but they're slower Pass that you have those spinning hard drives like the original iPod that you may or may not have ever seen in your life And pass that there is tape drives that you have I guarantee never seen in your life How many of you have seen VHS tape in your life? Yeah, it looks like this thing. It has like two things and then spin around and there's like a physical tape on that thing Yeah, well, that's why they're called tape things. So there's a version of that to store massive amounts of data That's not just a crappy movie from the 90s, I guess Before that whenever VHS tapes were used So they'll store like terabytes and terabytes and it will be dirt cheap, but it will be insanely slow and people like Google will use this for like archival storage of things that You likely will never access but they want to keep it around and they'll just be like a giant room full of those tapes And if you want to access it There's actually a machine that goes and grabs the tapes and like manually loads it to read the data from it It's really really slow, but hey if you want to store like petabytes of data, it's dirt cheap So you'll probably never see that. So yeah, we probably end on most of our computers around here All right any questions about that. So now we need to play some illusion So if you're designing these systems the reason we have this giant hierarchy is because we want to hide it from the user and ideally we want every level of that to have the speed of the layer above that and The capacity of the layer below that so RAM is not actually that fast But if we read a bunch of it and cache it on the CPU It can look a lot faster than you can physically read RAM from the device if we read the Write things in the cache and reuse it over again And then ideally you want to have the capacity of the layer below it so you get the best of both worlds so in that case If we do something like this we can actually use more memory across all of our processes Then we actually physically have in the machine So if you only have eight gigs of RAM and you have you know Two tabs open in Google Chrome and that uses 10 gigs of RAM Well, you might notice that might have happened before before and it doesn't crash It still works, but doesn't really make sense So what it does is Something called demand paging so we can use memory as a cache for the file system and also we could map memory pages to system blocks and Like actually on the drive whenever we're reading files and that way well Demand paging is that M-map idea we had before we could kind of just figure out what pages and memory map to what on the Disk and not actually read the disk until you actually use them and then you know you access a page It's referencing a block on the disk then we read it in from the disk What could also happen is well on the disk you might run out of physical pages and You don't have anywhere to put it and you don't want to be out of memory So what you can do is you can make something called swap space on the hard drive itself And you map that physical page and store it on the hard drive Temporarily and free up that block in physical memory, and then you can use it for something else So this is called swap space if we've used windows I assume most of you use windows or have used windows so in your C drive There's like a something called a page file if you enable hidden files So that page file is that swap space So that file will be used whenever you run out of physical memory and you need to start putting pages Temporarily on the hard drive, so that's called swap space and if you run H top on your virtual machine under the memory thing There's another thing that says swap space Likely that'll just be zero because it's not being used But if you start using really RAM intensive things it will start filling up and it will start going slower Because while it's moving memory to disk and moving it back and forth So a questions about that so swap space basically just throwing Memory pages to the disk if we actually run out of physical memory that way you can well you can The total memory usage across all your processes can actually be more than the amount of physical RAM You actually have it's just going to probably be slower So if you like context switch and all of your memory pages are on disk Well, you're going to have to load them from disk back in the memory Throw some things out and it might be really slow So that process of picking something to throw out and move back to this to move in another page Well, that's what we get to talk about today. That is called page replacement So a processes Working set should fit in physical memory of what that means is so the working set is given an amount of time Probably from when you got context switched in to when you got context switched out The working set is the number of pages your process uses in that amount of time So if your working set is in physical memory Well, then you don't have to swap to disk everything is going to be nice and fast if Your working set is bigger than the amount of physical pages you can fit into memory Then it's probably going to be slow and the worst case it can do something called thrashing And what thrashing is is while you can't fit all your working set into memory at one time So you're constantly going to be moving pages to the disk and then loading pages a different page from the disk back in the memory and then You execute a little more access a new page and then you do it again You move it back to you move a different page to disk load another one back in and things will suddenly get really really slow Because well instead of accessing Physical RAM directly you are doing hard drive accesses, which are going to be a lot slower and This is why sometimes If you really really care about performance because all that is invisible You don't actually really know unless you're paying attention to like your operating system metrics of like how much swap space is being used The your process is just going to go slower and you're not going to really notice it unless someone complains So if you really care about performance and you're something like Google well What you might prefer is just you would rather just run out of memory Get an error in your process and have it be dead and then you'll know like oh well I need to go buy more RAM if you're Google you probably prefer to just buy more RAM than just randomly have a process Like goes slow or you just allocate more RAM to your virtual machine or something like that to resolve the issue And you just get rid of swap entirely So sometimes people do that now if you really care about performance. Yep the thrashing Yeah, like someone probably it probably sounded like hell So if it was a physical hard drive whatever it was thrashing. Yeah, it would just go like Like it would just kind of sound like it's grinding because it's constantly accessing when it probably shouldn't be So yeah Likely that's where they coming up because well as you have noticed in this course. We're not very creative with names alright So we get to talk about page replacement algorithms Should pay attention because this process you will probably do on your final But it's pretty boring so we can just go through it So page replacement typically you assume that you have a finite amount of pages and physical memory and Your process will just assume one process. You're accessing memory so you need to bring things in from disk into main memory and Typically you will have more pages you're using than actually fit So you actually have to make a decision which page to throw out of memory and onto disk in order to load a new one in So we'll go over three algorithms today and talk about four So the optimal thing to do would be if you can look into the future and see all the page accesses a process makes Well, ideally you replace the page that won't be used in the longest So I maximize the amount of hits I get where things are already in physical memory so that's the optimal thing you can do but of course you can't implement that because in General you can't predict what a process will do The next algorithm, which is not really an algorithm is instead of just Making a decision of what page to effect. I just pick a random one and just move that one back to the disk Turns out it's not that bad But we don't really have to discuss anything and how could I evaluate to tell you on exam do the random algorithm? Yeah, that would be fun All right, the other thing we could do is the old trustworthy thing of first in first out So I could just replace the oldest page first replace it with the new page Simple to implement. We can all do FIFO. Hopefully at this point same ideas with scheduling The last one is least recently used So instead of looking into the future I look back into the past and the idea behind that is I replace the page that hasn't been used in the longest time Because likely if you have just used the page you will likely reuse the page because while your process is running You access the variable that's on page likely you will read or write that variable again So you'll act you're more likely to access the same page So that's why we do least recently used So how we evaluate all these things it's kind of similar to our scheduling questions, but a bit easier We assume our physical memory holds in this case We'll do four pages and then we access the pages in this following order So we just give the numbers so we access them in this order And then we assume all these pages are initially on disk We just context switched in this process and we have to load it from disk into memory and This is the order we will use for all the examples in the lecture in order to actually compare them with each other and for this Unlike scheduling where you have to find like average waiting time average response time and the number of context switches for this It's just number of page faults that happen. So let's work to do so Here we'll start off by doing the optimal algorithm which should maximize the number of page hits we have and Minimize the number of page faults we have So in this case how I draw all of these things is each box here represents a memory access and Inside of the box is what is currently in physical memory so initially that block box is supposed to be empty and then above the box I Draw in or I write the number that we're accessing at this step So in this case we access page one because it is the first one It is not in memory yet. So we have to read it from disk into memory So it goes into this box and because we had to Page fault and read it in from disk I put it in red and that way at the end to count the number of page faults Or things we had to read in from this you just count the number of red numbers So in this case this will be fairly boring. So if I access page two Well, it's not in memory. I have to read it into memory in this case I can fit four pages into memory. So I wouldn't just get rid of page one that wouldn't make any sense So I read it in now in memory. I have page one two and in this case. I just read in page two any questions about that Hopefully fairly straightforward. What should happen if I access page three? Yeah page fault loads it into memory We load it Next one's gonna be the same thing the first four steps are always quite boring. So after Step four now physical memory is full in this silly example So page one two three and four are currently in physical memory and my next access in is to page one What happens in this case? Yeah, it's already in physical memory. So that's great. I don't have to read it from disk Don't have to do anything. So in that case, I just rewrite the same boxes Nothing's in red because it was essentially a hit. All right. What about page two? Is that a hit? That is also a hit All right now when we access page five is not currently in memory and memory is full. Yep. I dropped four in this case Yep. Yeah, so that's skipping ahead. So in this case I have to pick one of these to actually evict if I'm doing the optimal thing, which one should I drop? Well for so why should I drop four? Well, the optimal thing is to look into the future and see which page isn't going to be used in the longest So starting at this access I look into the future. Well, it accesses page one next So I shouldn't get rid of page one then accesses page two I also shouldn't get rid of page two then accesses page three. So I shouldn't get rid of page three So the only thing that's left to get rid of is page four. So I get rid of page four So in that case I replace page four with page five So everyone good on that? All right now what happens if I access page one Well, that should be a hit because well, I looked in the future and the next three should all be hits So page one That's a hit page two also a hit because that's still in physical memory Access page three also a hit now. I access page four. What should I get rid of now? Yeah, so in this case as long as I don't get rid of five it doesn't matter So as long as I don't get rid of five It's fine. So in this case all this is assume. I hate one So I'll just get rid of one. So I replace one with four But of course we could have replaced two or three because well, it's over at that point So then we access page five. It's still there because we did the optimal thing So how many page faults do we have for this optimal algorithm? Six so you just count the number of red You can do that on the exam if you want or circle it or do whatever you feel like underline it Doesn't matter. So six page faults. Yep Yeah, so this is just so we have to move things in and out of memory if Your whole memory had four pages and there are four kilobytes That means you have like 16 kilobytes of memory Which I guess with the thing in the 70s But yeah, if I asked you to do this now you have thousands and thousands of millions of pages So only using Google Chrome will get you to this point of having to move things in and out So it's just for these questions realistically, right? We take millions of pages to finally fill up main memory Yeah, yeah, these could also Remember when we end mapped like a file So these could actually represent a file if these memory pages didn't actually represent a file They were just part of the program that was running and we just threw them to disk because we ran out of memory So they could either represent real files or we just ran out of memory All right, so six page faults remember same sequence optimal six page faults Same thing but for FIFO so we can make this lecture go fast. So can I skip the first four steps? All right, so we can speed run this lecture one two three four So we fill it up any questions about that. We have our four page faults. We can't get rid of all right So we access page one that's a hit right everyone agree Access page two Also a hit. All right. This looks the same as before. So now the difference is we access page five So if our algorithm is first in first out, what do I boot out to? Load five into memory One because one is the first thing I loaded. So that was the first thing in first thing out FIFO So I replace one with five and that is a page fault. All right So it turns out I probably made a poor decision. So now I access page one, which I just threw back onto the disk Now what should I get rid of for page one? Two, which will prove also to be a poor decision So now I access page two Which I just threw to the disk. That was a poor decision. What do I replace for page two? three Also a poor decision. All right So now I have this now. I'm accessing page three. Ah What do I throw out now? Page four So I throw page four out and then you can also see this pattern with first in first out You can see there's nice diagonal lines of red numbers. So if it's first in first out, it'll just look like diagonal line So you can really speed run this if you wanted to Because yeah, it's just kind of silly. All right. So now we access page four. What do I get rid of? Page five. All right. Now I access page five. What do I get rid of? what How many page faults did I have? 10 that was a pretty poor effort on my part, right? so Now let's think about this. So This would probably be a lot better if I could fit five pages in memory, right? I'd only have five page faults in total because I can fit everything So if I have more physical memory, I have less page faults Right that makes sense to everyone So the up is the opposite true. So if I have less physical memory, I should have more page faults, right? That would make sense All right, that makes sense. We all agree on that All right, let's make sure So we'll do the same thing, but we'll decrease the number of Frames or pages we can fit in physical memory from four to three So same thing, but we can fit less So we have our first three accesses where we just they have to be page faults We load it into physical memory now we access page four. What do I throw out? page one Wow, that was a bad decision already So I throw a page four and my next access is to page one. What do I throw out? Page two another bad decision. So I'm five for five now So now we access page two It's not in memory. What do I throw out? Three all right, that brings us up to six for six for page fault. So yeah, it's not looking good You guys look like you were right now. We access page five. What do I throw out? Four by waterfall method. It has to be four. All right So throw up page four Now it turns around so now if I access page one, it's in memory. So that's a hit access page two That's a hit Now I access page three. What do I have to throw out for page three? Page one right So now I throw up page one Ooh, that was bad. So now I access page four. What do I have to throw up for page four? Two so finally I access page five and that's a hit How many page faults is that? Nine that was better. That was less page faults than before. So Seems weird, right? So the solution is well You know, if I want to make my program go faster, I just take out some memory and it goes faster. So This has a name Yeah, we have nine page faults. So that has a name. It's called Belladies Anomaly and it says that Contrary to what you believe if you have more page frames. So you have more memory that means you have more page faults and this is a problem specifically for FIFO algorithms doesn't exist for Least recently used or any stack-based algorithms. We will see slightly later and in fact There was a paper in 2010 that some math people wrote in Greek that I can't understand That summarizes the solution if you want you can read the paper Basically, they concluded that Given a FIFO algorithm, I can just construct any arbitrary sequence of page accesses that can Cause any page fault ratio I want. So given a page fault ratio They can come up with a sequence that will cause that ratio Doesn't matter how many pages there are So for other algorithms, they behave how you expect Increasing the number of page frames decreases the number of page faults So you don't get penalized for having more memory, which is what you would expect But if you use FIFO for this specific problem, it has this issue, which is also why Hey, we probably shouldn't use FIFO because this is pretty dumb Yeah Yeah, so this is in specific circumstances. So in general, you don't know how a program behaves So it could not do this in the real world, but they were like, yeah We can just construct arbitrary sequences that can do this But in the real world, you don't control what pages of process accesses So this may or may not happen in the real world, but you could implement it But you might run into this where like you have more memory and you have more page faults And you have to be like Why the hell did that happen? Well, you got unlucky basically and why'd you get unlucky? Well, a bunch of math people can prove to you that sequences can happen with any page fault ratio And again, don't ask me about the details of math or anything I can't do that Greek stuff. It looks like gibberish. I can read C All right So now we can do our last one of the day Wow, we're really speedrunning this So this is least recently used We'll go back up to four physical pages And for all of these, we can use FIFO to break ties But ties will only happen if we have multiple processes running at the same time And then, you know, it's essentially like a data-race problem, all that fun stuff So for the purposes of this course, I'll just assume a single process and we don't have to talk about ties So same situation we had before, we had that weird three-pages thing So memory can hold four pages We had the same accesses Now we're going to use least recently used, which looked backwards into the past So first four accesses are going to be the same no matter what Because we have to load them into memory Access page one, page one, that's a page hit already in memory Access page two, that's a hit already in memory Now we access page five So now I have to choose a page one, two, three, or four to throw out of memory And replace it using least recently used So which one should I throw out? Three So if I look backwards in the past, I used two most recently, so I shouldn't get rid of two Before that, I used one, so I shouldn't get rid of one And then after that, I used four, so I shouldn't get rid of four So the last page standing that I kick out is three So I replace page three with page five, and this is what it looks like So it looks like we made a good decision So if we access page one, that is a hit If we access page two, that is also a hit Now we access page three So now we have to choose something to throw out What am I going to throw out? Four So in this case, I will throw out four Because while I can't throw out two, it was used least recently I can't throw out one, and I can't throw out five, so that leaves four So I have to get rid of four for three Turns out that was a bad mistake, because while I couldn't look into the future Turns out we're accessing page four next So if I access page four next, what do I get rid of? Five So I look at the past, I can't get rid of one, two, or three That leaves five, which is a poor decision Because now I access page five, now what do I get rid of? One So I can't get rid of four, three, or two So I have to get rid of one, so I have to get rid of one, replace it with five Now, how many page faults do we have? Eight Eight So, optimal was six, our first faithful was ten, our smaller one was nine This is eight, slightly better Alright, any questions about that? So that's our algorithms for today Yay So, oh yep, so random depends So this, least recently used typically works really well with like real program usage Because typically if I use a variable X, I'm going to read it soon anyways So I probably actually, typically if you use something you'll use it again So least recently used, for most sane programs works pretty well But if you have a program that just randomly accesses memory just in random places This isn't going to work very well And also if you implement it, well how would I implement something like this? Least recently used? Yeah Yeah, you could have a counter for each What about implementing using link lists or something like that So if it's just used, I put to the front and then the least recently used will be the thing at the back If I just keep on refreshing it, right? So we could implement this but those are both kind of slow to do And well, if the program does random things anyways, I can pick a random number like that So random decisions are actually pretty fast, so probably faster So, if I try to implement least recently used and say I have a counter for everything Well, that seems like it's kind of impractical to actually implement Because if I read a variable, that means I've accessed something, right? So I read like a byte and then suddenly after I read a byte, I have to add a counter to something Which is going to be more than one byte, so suddenly things are eight times slower Probably not a good idea Also if I put a counter on stuff, I'd probably have to search all of the pages in memory Which is also going to be really slow So, turns out if you try to implement least recently used, it's kind of impractical So if I implement it in hardware and say I have a specific counter for each page in memory Well, each time I even change or read a single byte Well, I have to essentially save that counter to whatever I'm using to keep track of that page And that's going to be, that's going to take some time, going to be fairly slow And then if I want to replace things, well, I have to look at all the timestamps And see what is the oldest timestamp and then evict that one That's like O-N, so the more memory I have the slower that's going to be So I probably don't actually want to do that And if I implement it in software using linked lists, it has the same problem So technically, if you're a computer science person, this seems kind of fast Because they're like constant operations So if you do a page reference, so if I access any byte, all I do is move this page to the end of the list So I move it to the front of the list because it was just used I'll just assume the most recently used thing is at the beginning of the list Then eventually it will fade to the back Then for replacement, that's also constant time Because, well, I just picked the thing at the back of the list And that's the oldest page, so I evict that one But if you actually try to implement that, that means even if I access one byte Well, we've all used linked lists before So to do that constant time operation, I probably need like up to six pointer changes Of reading and writing pointer values So it seems kind of bad that I read one byte and then I have to read and write six pointers That's probably a bad idea And also imagine that you have multiple cores running So if you have one linked list that's keeping track of all the pages Well, would you have a data race for that linked list if you had multiple cores? Yeah, you would have data races for that So you would have to have a mutex around it And then it's even going to be even slower than those few pointer updates So if you actually try to implement least recently used It would be way too slow to actually implement and no one that does it Yeah, so you could use an array or something like that But that still speeds it up It's still going to pale in comparison to reading a single byte Because this is something I have to do every single time I read even a byte So if it's something that you do that's more than a byte, it's probably going to be too slow So what we can do is we do the typical thing we do if we encounter a hard problem We just change the problem and make it easier So least recently used is actually kind of an approximation of the optimal algorithm We can't implement the optimal algorithm So we use least recently used to kind of approximate that So just take it one step further So we can approximate least recently used So we can actually implement an approximation of the approximation So there's lots of different ways to approximate least recently used So we can actually implement it So specifically in this course we'll be looking at something called the clock algorithm Which will be next lecture But there's also like least frequently used So it relaxes it a bit instead of the most recently used It just keeps track of the number of times you reference it Or if you've referenced it at all There's one with two cues that try and speed it up a bit in hardware There's adaptive replacement caches We just throw some caches on it There's all sorts of different strategies It's kind of like scheduling where there's all different trade-offs you can make If you make something really complicated Well it takes longer to run And sometimes it might make a bad decision anyway So you may as well have just made a poor decision faster So any questions about anything we saw? So we can speed run Alright So what we saw today optimal So don't use, make sure we maximize the page hits we have Just replace whatever is used furthest in the future And we could also just do random page replacement It actually works surprisingly well in practice Because unlike FIFO which just kind of sucks It avoids the worst case because well it just makes a random decision Instead of always making a poor decision like FIFO It can randomly sometimes make a good decision Kind of avoids the worst case if you simulate it It actually turns out to not be absolutely terrible FIFO, easy to implement but has that weird anomaly Where hey I have more physical memory And I have more page faults, that's not good And then we round it off with least recently used Gets closer to optimal but really expensive to implement And we will figure out an implementation in the next lecture So with that remember 4 and 4 year