 Good morning everybody. Friday. It's a dim response for Friday, man. Friday usually gets like more love than that. Friday? I'm disappointed on Friday's behalf. I feel good about Friday. Alright, so today we're going to continue on with virtual memory, paging, swapping, we're talking about some things. So I've been really enjoying the last couple classes. We've had really nice back and forth, which has been really good. I mean, does anyone feel bored? Like we're going too slow or something? I'm actually asking this for you. Okay, good. Because I write these slides and I have these grand ambitions that I'm going to cover like X amount of material, and then we get through like 0.5X, which is fun, right? I'd rather do that and have you guys understand it. What I think might happen, and I apologize for this in advance, is that I think that I was really hoping that I get through virtual memory before spring break. I don't think we're going to quite get there, right? Because there's some cool stuff that I do want to cover that I just don't think I'm going to be able to jam into the next lecture, and I want to do review on Wednesday. So I think we'll have a little bit of virtual memory left when we get back from spring break. Hopefully you guys won't have forgotten everything about this, just most of it. So we will do a little bit when we get back, but that's just sort of how it works. So anyway, okay, so today we're going to go on. We're going to talk a little bit more about swapping. We'll talk a little bit about exactly how swapping works. We'll talk about how to distinguish between page faults and TLB faults. I'm going to mention briefly how to manage TLBs because I think they're important to understand. And then finally we're going to talk about what's called demand paging, right? Or a procrastinatory technique that the operating system uses in order to avoid doing more work that it has to. Not a kind of technique that I would recommend on any of your assignments in this class or in general in life, but on computer systems that tends to work out pretty well. So how many people here need a partner for the class? Raise their hand. Okay, keep your hand up. Everybody look around the room. Everyone has their hands up, look around the room. There's three of you guys, and you guys happen to be sitting all right by each other, right? So I think that's really convenient. If you don't have a partner, at some point I'm just going to put you in a group with somebody else, right? And depending on how lazy I'm feeling, I may do that completely randomly or I may try to include some other piece of data, right? I don't know. This may be just like a random schedule in an algorithm. I might try to do some sort of priority based whatever, right? But it really just depends on how much extra time I have. People that don't have a partner really need to get partnered up. Maybe there's people that are not in this room that didn't come to class that don't have a partner. Those people also need to have a partner. So today is my imposed deadline for getting you guys into groups. So the three of you guys, if you guys want to stay after class and sort of find it out among the three of you, right? One group will emerge, hopefully. And one of you will be sad and lonely. So that's just kind of unavoidable. But so the forums are... How many people have used the forum on the website? Does it work? Kind of? Okay, okay, okay. Anyway, I hope it works. I didn't write the code clearly, but... So the forum is another place to look for partners. And again, at some point today or over the weekend, if you don't have a partner there's just going to be some moment in time where some sort of script is going to run and suddenly you're going to have a partner, right? So if you prefer to find a partner that way, it's kind of fun, right? Like a lottery. That's cool. Otherwise, if you want more of a non-arranged marriage then use the forum or hang out after class or whatever and find somebody to work. Questions about assignment two, partnering, et cetera? All right. So on Wednesday we talked about how do we store page state, right? So we had this great idea of virtual memory as an abstraction, of address spaces as an abstraction, giving the process an idea that has all this room. You know, virtual addresses gave us all this freedom and flexibility to do a lot of interesting things. But at some point as we kind of worked our way into this, we came down to this issue of now we have these 4K pages or 8K pages that are flying around the system and there could potentially be a lot of them. And so, you know, there were two issues that we kind of unearthed here. One was how do we store the information that we need in order to manage these, right? Specifically, and in particular, to load the TLB and to translate the virtual addresses when they need to. And the second problem was how do we find that, right? So how do we store it and then how do we locate it rapidly, right? When it needs to be located, okay? So anyone have questions? Storing and retrieving page state. Wednesday. Seems like so long ago. Wednesday. Any questions on this stuff? Okay, so let's start. My favorite people, way in the back of the room. So why do we need to store a page state? What's the point? We need to keep what up to date? As the system is running, you know, I need to keep information about pages for what purpose? What's the real thing that I'm doing with this information, primarily? Mapping and what piece of hardware helps me do that mapping? The MMU. Okay, we're getting closer? I heard TLB right here, right? So really what I'm doing is I'm just keeping the TLB up to date, right? The TLB or the MMU is going to ask me. I don't understand how to translate this virtual address and I need to be able to tell it, this is the physical address that this virtual address points to, right? Or this is the memory-like thing that this virtual address points to, okay? Morning. So the first thing I need to do is I need to store information about each virtual page. And then what's the second thing I need to do, Jason? Couldn't tell me. I just gave you the answer. Two seconds. You want to make a guess? I need to store it and then I need to be able to what? Access it. Access it. Locate it. Retrieve it. And do that fast, okay? Because, again, as soon as I'm in the kernel, looking up a virtual address translation, something has already gone wrong, right? What is the thing that went wrong over here? If I'm in the kernel trying to translate a virtual address, what bad thing has already happened? The TLB didn't know how to translate it already, right? I want the TLB to do all my translation. It's super, super fast, right? If the TLB and the MMU can translate pages, I'm in heaven, right? When I get into the operating system, I'm already slow. And if it takes a long time to locate the information about the page, I'm just getting slower and slower, right? Okay. What information? So what does the operating system need to know about each virtual page, okay? So I've got a virtual address, it calls into some virtual page. What information do I need to store about that page, doctor? One thing. One thing. Yeah, I need to know where it is, right? The location. And the location is we started to talk about the last class, right? It could be in memory, right? There might be memory holding the contents of this virtual page. It could be on disk, right? And we'll talk today more about swapping, which is the process of moving things out of memory to disk in order to create space in memory for new things. And then there's one other final option here, which is that the process may have never used this page before, right? And we'll talk a little bit about this today as well, meaning that the page is about to come into being, right? So when the process first uses the page, I'm going to give it a brand new page, right? So the page may be uninitialized, that's another option, okay? What other information might I want to know about the page I need to store? Permissions, right? What's the process allowed to do today on this page? Is allowed to execute instructions from this virtual page? Is allowed to read or write to this virtual page, right? What else? Statistics, I'm hearing it whispered from, like, this group, right? Maybe it's all three of you together. Statistics, right? Has this page been read or written from recently and how many times? And this will become more and more important today and then also on Monday, right? On Monday, we're going to start to talk about policy, right? Policy about how and when to move pages back and forth to disk in order to create the illusion that there's more memory and in order to do this well, we need some information about the pattern of page accesses, right? Or I shouldn't say we need it, but it makes things much, much nicer, right? If we know something about which pages are being used heavily, we can do a better job of moving things back and forth to disk. Rob, you made a noise. I thought you were going to... Okay, laughing. Oh, wow. I hope it wasn't me. No. That's an unconvincing no. All right. And then what was our requirement for how we store this information, right? What's the thing we're trying to do here? Compactness. Compactness. There we go. He's following up his giggling with an answer, right? Because the number of page table entries, right? We called this thing that we used to store the page data, page table entries. How many of these am I going to have, roughly? I need one for every what? What's that? Four cad. Four cad? No, I didn't know what you were saying. But I heard various of it. Virtual page. So any virtual page allocated by any process on the system, I have to store this information. And remember, our address space abstraction is designed to encourage processes to use lots of memory. You've got two gigabytes or three gigabytes or four gigabytes of address space to use it. Yeah. It really depends, right? So a lot of the statistics collection, right? So let me ask this question, right? So assuming that good things are happening and the operating system is not involved in the translation of a particular page, how do I collect statistics? How would I collect statistics about a page? What would I have to rely on to help me? What's that? Well, what would that mean? How do I try to collect statistics by using the kernel? What does that mean? I have to do when the page is accessed. I have to trap it in the kernel. And that's the exact thing I was trying not to do, right? Because it's slow. So what do I need to rely on here to help me out? Usually when I'm doing hardware, right? Because in the best case, when the translation is being used in the TLB, or today we'll talk about cases where the memory management unit will load the address into the TLB without trapping it into the operating system. In those cases, hardware is the only thing that sees what happens, right? The operating system is trying... Well, I'm trying to stay out of the way as much as possible, right? So collecting really, really fine-grained statistics about page use is difficult, right? And it's difficult because normally I don't want to do that on every page accessed, right? So there's a balance here, and we'll come back to this on Monday, right? Between making page translation and page accesses as possible and being able to save some information about how often the page has been accessed, right? If I was really trying to save every... If I was really trying to count and record every time a page was read and written to, I would argue that in most cases, that's not feasible, right? That requires too much work, even by hardware, on what has to be a very, very, very efficient operation of translation, right? All right. It's a great question, though, and we will talk about this more on Monday, all right? So what do we call... Shoot, I gave it away again. What do we call the single entry storing information about a single virtual page? Page table entry, the name of the slide. Oh, man. I outsmart myself all the time. All right. So I did it again. Man, okay. So what do I use a page table? It's clearly called a page table, right? What do I use a page table to do? I'm going to use it to map a what to a what. What specifically is it? I'm using pages so it's a virtual what? Page number, right? So the virtual page number for a process, and God, I keep forgetting to do this, I map a process and a virtual page number, and then what do I want to find? A page table entry, right? Some information about the page. Now, it's not necessarily... The page table will not necessarily map a virtual address to a physical address because the page may not be in memory, right? But the point is the operating system, the TLB tells the operating system, I don't know what to do about this virtual address that was used by this process, and the operating system needs to be able to very, very quickly retrieve information about that page, right? Could be in disk, could be in memory, could not be initialized yet with it, right? Okay, we call these page tables, right? And I just gave away the answer to this, too. Processes have separate page tables, why? Because virtual addresses are private, right? Virtual addresses don't make any sense outside of the context of a process, right? All right. So, we talked about three different designs of page tables, right? I had flat page tables, I proposed a linked list page table design, and then I had this somewhat more mature tree-based multi-level page table, right? So, flat page tables, what was the axis cost here? Oh, one, right? One lookup. The virtual page number is used directly as an index into the flat page table. What about the size? Large. Big. Too big. You know, one big fat array, right? Very, very terrible. And an array that is sparsely used, right? Again, what we've seen over and over again is that big contiguous data structures are a bad fit for our address-based abstraction, right? Address spaces are huge and sparsely used, right? So, big contiguous data structures generally don't work well, all right? What about the linked list page table? What's the speed lookup speed? Assume I've n virtual pages. O of n, right? I have to potentially walk the entire list, right? Maybe I sort it, I'd be a little better than this whatever, right? But the point is generally O of n. What about size? O of n, right? And the nice thing about it is that every, you know, it's as small as it can possibly be, right? There's no entries that are unused. Every entry is in there because there's no, okay? Multi-level page table. Speed of access. How many accesses does it take, potential? OC, right? Like some constant number of accesses, right? You know, depending on the depth. You know, if I have a two-level multi-level page table, then I have two accesses. If I have a three-level, I have three three indexes I need to look at, right? All right, size varies plus smaller than a flat array, right? And this is a design that operating systems have chosen, many operating systems have chosen simply because it provides a nice balance between the size of the data structures that I need to hold this information and the speed of access, right? Remember, speed is important here. But if I use too much memory to store all this information, there's not enough left for the process or any left for the process. All right, after our little review, any other questions? I'm retrieving page state. Questions? Any questions about this? All makes perfect sense. Everybody's going to get all of these questions right next week on the midterm. It's awesome. Okay. All right, so we talked, and we've sort of been hinting at this, right? And now the time has come to finally sort of fully pull back the covers and admit that virtual addresses don't always map to physical memory addresses, right? Sometimes what I want to do is I want to utilize I want to exploit this nice property of the virtual addresses that they require translation to play games behind the processes back, right? And the normal game that I want to play is if I need more memory I want to move things out of memory, right? I need to preserve the contents, right? But I want to move things out of memory and the typical place that I move things out of memory is I move them onto the disk, right? But there's no real limitation here, right? I mean, you could write a system that swapped to the network assuming that you could do it reliably. You could write a system that swapped onto a tape backup drive, right? That would be really slow and terrible, right? But the point is you just there's some, you know, slower, larger backing store, as we sometimes call it that is used to preserve the contents of a page that wasn't memory during a period of time when the page was not used and until the page is accessed again. We'll talk we're going to go through today in detail both cases, right? When do we move a page out? What do we have to do? And when do we move it back in, okay? And again, when I do swapping well, your system that the idea of any when you place a cache in front of something the idea is you want the thing to look as big as the big thing but look as fast as the small thing exceptions, right? And when this is done well that is how your system looks, right? Especially when it's done well, an operating system can hide the slowness of the disk, right? And we're going to talk a little bit today and on Monday about ways that the operating system tries to hide the slowness of the disk, right? Particularly by moving some of that slowness off of critical interactive paths, right? So if your operating system does something slow while you're not waiting you don't notice. But if your operating system does something slow while you're waiting you notice, right? And so one of the tricks here is to move some of the slow pieces off of interactive paths, right? So don't do them while somebody's waiting, right? And again, what can happen here though is if, I don't know, there's not really a good analogy here but this is kind of like being underwater in your house or something, right? I mean on some level things go wrong and the operating system plans badly then what can happen is that the memory can look as slow as the disk and feel as small as the memory that you have, right? So this is... these are the two bounds here, right? There's no way it can look any slower than the disk and there's no way it can potentially look any smaller than the memory unless I start artificially limiting memory for some reason. So these are the extremes, right? And we want to be here, right? We don't want to get down here but this can happen. Questions about swapping about this idea in general. I think this kind of makes some degree of sense by this point, I'm hoping. Alright, so let's say and again bear with me for a minute that I've decided to move a page to disk, right? Why would I do this? First of all why would I be moving a page out to disk? Usually to make room for something else, right? I've run out of memory and a process needs a page it's attempted to access a page and so I need to move something out, right? Okay? What do I need to do? What are the steps here? So I've got a page in memory and I want to write that page out to disk, right? So what do I need to do in order to ensure that this happens safely? What's the first thing I need to do? Anybody? What's that? Okay, so yeah, let's say I have that page. I have a page, I've chosen my page I've got a page in memory and I want to swap it out to disk, right? Gotcha. So your idea is perfect but how do I accomplish this? How do I make sure that a process can't keep using the page while I'm swapping it out? Invalidate it where? Specifically. For the process but where? In the TLB, right? If the translation is in the TLB then the process can keep using it without me even noticing, right? And in particular on multi-processor systems one core might be trying to swap the page out and the process it's using the page might be running on the other core, right? And so this can happen, right? But the first thing I have to do is I have to get that entry out of the TLB, right? If it's in there. Because if it's in the TLB it's possible that the process can keep using it without me having anything to do with it, right? And what might happen is that I might start copying some of the data and the process might change the page and I might not notice that and when I finish copying the data the data that's on the page and the data in memory don't match anymore, right? I have a diagram here. We go through that. I'll point out when that can happen, okay? So get the entry out of the TLB, yeah. What do you mean? Right, right. But remember when I... So on the processor that's doing this, right? I'm assuming... So on a single core system no one else can be using it because I'm running, right? And I'm not using it. On a multi-core system what do I have to do? What was the thing we talked about last time that had kind of a cool name? What's that? TLB shootdown. TLB shootdown, right? So I need to... I have Nerf guns now. One of my colleagues brought something. I thought about bringing one to class but I thought that would be weird. But maybe next week. Maybe for midterm preparation. Anyway, so it's basically like I need to... There's a TLB entry on another process and I need to kind of like shoot it down, right? I mean I need to invalidate it. And you're right. Someone else could be using it, right? And on some level that would probably be a bad thing. Why would that be kind of a terrible thing if somebody was using it right at that moment? Well, but let's say I can stop it safely, right? Let's say that I shoot down the entry in the other TLB and then a second later somebody tries to translate the page. What's the problem here? I'm in the process of moving the page to disk, right? I've chosen this page to move to disk. And now someone else is going to try to access it right away. Well, what does that mean? I've got to put it back, you know? So if the page is being used right as I'm trying to swap it out, this usually means that I picked a terrible page to swap out, right? I mean, if literally somebody is using it right in that moment, you know, that's the worst possible case, right? Because usually what I'm going to have to do is maybe I can abandon my swap out and leave it and choose a new page. But in the worst case, it gets... I get all the way through here and then somebody access it and then it's like, okay, you know, now I've got to kick out some other page, right? And what have I done, right? I've slowed down two processes, right? I'm just waiting for me to do this and the other process that now has to stall waiting for the page to come back in. I've done two disk IOs and memory is the same as it was before, right? So this is like a total fail, okay? Not good at all, right? But anyway, but I need to make sure that I do this just in case, right? Because I can't... I need to make sure the other process can't translate this address while I'm talking, okay? So now I've removed the address from all the TOBs. What's the next thing I need to do? This is a pretty simple process. I think there's only three steps. What's that? Well, of the table, I need the same sub-state, but it's not the table. What's that? Well, I... No, no, no, it's not quite right. What's the next thing I need to do? Copy the page to disk, right? I've got data in that page, right? In order to play this game, I've got to have that data be preserved, right? So I've got to copy the data to disk, right? So I've got a 4k spot on disk that I've chosen to put this data, and I just start a disk I owe to copy all the data from memory into that part of the disk, okay? And now, what's my third step? Change the page table entry, right? The page tables are my source of information about what is happening, right? Now, this is a logical description. When you guys implement this for assignment three, you might need to do portions of this kind of all at once, right? Because again, if there's a TLB fault for this page address right here, I need to know that the page is in the process of being written to disk. So sometimes I need to store more information in the page table than just where the page is. But, Isaac, you had a doubt or a question? Yeah, no, I have a question. Some systems are either required to part of the disk that you're going through. Do you just write in raw data? I don't know. I'm not sure there is a big difference, right? I mean, you know, maybe it's like a multiple... Blocks or sizes of 4k? I don't know, right? But 4k is normally... So that's actually... Well, we'll come back to... I'm not going to go there. We'll come back to this when we talk about file systems a little bit, right? There's a reason that I don't want to write this to a normal file, right? As Isaac said, normally what an operating system does is it uses a part of the disk and essentially... So remember we talked about the file abstraction and how that file abstraction maps down to disk blocks? For swapping, I normally just use the disk blocks directly, right? Because I don't have variable-sized files. Every file, quote-unquote, in my swap file system is the same size. They're all 4k. And so I don't need these extra features. I don't need open-close. I don't need to read and write arbitrary number of bytes. I'm always writing 4k, 4k, 4k, right? And so I can use this. And for assignment 3, you guys will do this. You'll create a raw disk device on OS 161 and you'll use that as your swap device, right? I hope not. And I... Yeah, I don't know much about Windows, but I can look into that. I would be surprised, right? You just don't want all that file system machinery here. It just slows you down, right? Yeah, problem. Yeah? Oh, yeah. No, no, no. I mean, how many of you have ever installed Linux? When you install Linux, it creates a swap partition. And if you do mount or whatever, you can normally see what that swap partition is, right? So, yeah, when you install Linux, it allocates a portion of the disk for swath, right? And that is accessed, like I said, in a special way. There's no file system on top. It's used directly as disk. All right, so this is it. But let's walk... Oh, yeah, thanks. Everything's moving around. It just takes a minute. My complicated diagram. Okay, so here's our world, right? I've got an address space. I've got physical memory. I have something in the TOB, right? And let's say that the process is using some code page, right? The TOB has loaded a mapping for that. And there's some physical page in memory that's been allocated for that code page, right? In addition, the kernel has a page table entry, which has other pieces of state in it. So I didn't want to make this, like, too ferociously complicated, right? But one of the things it definitely has is it has a location of where that page is, right? Where is that virtual page? And it points into memory, because that's where the virtual page is, right? So, you know, lightning review, what do I need to do to swap this page out? So the first time it was this side of the room, I'm going to go to this side this time. So the first thing I need to do, I need to remove the TOB entry, right? I've got to get this thing out of there, right? Because if I don't, the process could potentially continue to use it, right? Maybe add another core, all right? What's the next thing I need to do? Got to copy the data, right? So I've got blah, blah, blah in my memory, right? Blah, blah, blah, right? Isn't that great? Yeah. Man, you guys are cheap labs sometimes. But I'm glad I did this this way. I thought about it last night. It was late. I was like, do I really want to add three more slides? I go blah, blah, blah. I'm glad I did. But actually, the other reason I'm glad I did it is because, so this illustrates the problem that we face when it comes to getting that entry out of the TOB, right? Because if I didn't do that, what could happen, right? So remember, disk is slow. So this process right here takes a long time, right? In operating system speed, in instruction speed, right? In memory speed. This is slow, okay? If I had left the entry in the TOB and the process continued to use this page, right? I started writing blah, blah. And I'm going to finish writing blah, blah, blah. This is going to be great on the video, right? So the process might overwrite the first blah. It might write like foo blah, blah, right? But I've already written that byte. And so I have blah, and this isn't on the slide. I have blah, blah, blah on disk, and in memory I have foo blah, blah, right? And why is this a problem? What's the next thing I'm going to do with this piece of memory? I'm going to replace it. And normally when I exchange memory between processes, I zero it first. So I'm going to clean it off, right? This is actually a great question. It's not covered in here. Why when I exchange pages between two different processes do I generally fill the page with zeros? Why would I do that? Why not just hand? It seems like it's going to slow me down. Why not just hand the page to a new process? Talk to you. Exactly, right? I'm leaking information between processes, right? If I take this page, there might be another malicious process over here that's like what is that cool process doing? What is it going on about? And if I give it this page that says blah, blah, blah on it, well it's going to know, right? So in general, it's a security issue, right? I don't give, and it also helps with some other things, but when I give either a process, a new page, even if it came from that own process, I always zero it out, right? I give you a nice clean page of memory to start. I mean, this is a move, right? Like on some level, moving something to disk, if it's a memory, requires doing the copy, and then deleting it, right? So this is like a move broken down into its individual pieces. So yeah, that's basically it, okay? All right, so I write this, and now what's the third thing I have to do? Over here, this side of the room. Update the page table entry, right? Because next time I try to translate this virtual address, I'm going to find this PTE, and the PTE better tell me something about where that page is because I'm going to have to go get the contents, right? Put them back in memory before I can restart the instruction and give the process the illusion that the thing that's been in memory the whole time, right? All right, good. So let's think about the parts of this that are slow and the parts that are fast, right? Three steps. First step, removing something for the TOB, slow or fast? Fast, very fast. Boom, done, hardware. TOB shoot down might be a little bit slower, right? Because I have to interrupt all the other processors, but in general, this is fast. Copying the contents of the page to disk. Slow, really, really slow, right? Update the page table entry to indicate that the page is on disk? Fast, right? Now let me ask you something else. Why do I care? Why do I care about swapping out being slow, right? Or, let's put it this way. Remember I said before, I care about doing things that are slow when somebody's waiting, right? So if I'm swapping out a page, who's waiting for the page to be swapped out? Another process that's what? Wanting some more memory, right? So the reason that I'm swapping out, the reason I would be swapping out on, trying to think of it, like the reason I would be forced to swap out memory, synchronously, is that another process has tried to allocate some memory. Maybe it called fork, maybe it called espray to get more heap, maybe it just started using some code pages that I hadn't brought in from memory yet. We'll talk about that later in the class, right? So in general, this is a, if I'm doing this on a synchronous path, meaning somebody is waiting, it's because they're waiting for the memory that I'm about to free up by swapping out, right? So I care about making this fast. So, okay, the second step here is slow, right? Copy the contents of the page to disk. That's really slow, okay? Does anyone want to speculate about how I can make this faster? What could I do? So I could have, what does that mean? I like the idea. You guys are getting out to some good OS principles, how do I apply an idea from a cache here? Let me see if this, yep. Anybody else want to make a guess? Ah, ah, it changed since when? Right, so there's two things here, right? So like I said, this can be on a critical path, this can be on a synchronous path, this can be slowing somebody else down so I want to make it fast, right? What I can do is essentially what you suggest. But there's even a better wrinkle to it, right? So normally when I start allocating pages, the system will actually have a dedicated place in on the swap disk reserved for that page. This is kind of important because it means that every time I'm swapping out, I'm swapping out to the same spot, right? Now what I do is that when the system is idle, when there's nothing else interesting happening, like maybe when you walked away for coffee or when you've let your laptop go idle because you're paying attention to this fantastic lecture, the operating system will be trickling data from memory onto the swap disk, right? So it'll take pages from memory, it doesn't swap them out, but it copies the contents. And what does this mean? It means that I have, hopefully, when it comes time to swap out a page or when it comes time to allocate memory, I have a group of pages that I call clean. And what clean pages are are pages where the contents on the swap disk match the contents that are in memory, right? And what does this mean when I go to swap the page out? What can I avoid doing? What's that? I can just throw it out, right? So if I have a page that doesn't map its swap disk content, it's called dirty, right? And now here, instead of always copying the contents of the page to disk, I can only copy them if the page is dirty, right? If the page is clean, it means I already have the contents of it cached, you might think of it, right? In this case, I'm using a slow thing to cache, right? I'm using the disk to cache the contents. And the idea is, but essentially, there's an aspect of this that's kind of, you know, this is anti-procrastination, right? This is preparation, right? The system knows that at some point you might launch a new process, right? So your thing's been sitting idle for five minutes, you're going to come back and you're still going to start a Photoshop, right? So rather than sitting around on its butt doing nothing, right, while you're gone, the operating system is like, you know, I'm going to prepare, you know? I'm going to prepare for the worst, you know, I'm going to get ready so that if suddenly there's this massive request for new pages, I could just start tossing stuff, right? And I know it's in the swap, okay? Does this make sense? This is kind of a neat idea. Now, what is your name? Nitin. Which is really interesting, right? So he said, if the contents of the page haven't changed since it was brought into memory. So what type of pages have contents that in general almost never change? Code. So this is another, and I didn't have information up on the slide, but it's a great point, right? So code pages that are used by a process generally don't change, right? And they're usually marked as read only because people stopped writing self-modifying code in the 80s, right? Because it's really bizarre and hard to do. So in general, the contents of code pages don't change, which means the code pages on some level are great because I can always just throw them out. Now where do I find that code page, right? So I could just, the first time I load the code page, copy it to swap, but where does the code page come from? It came from a file on disk. In fact, it came from a file on disk in the ELF format, right? That's how I loaded it during exec. And so, I mean, there gets to be some subtlety here because executables can change, right? But in general, I could attempt an even further optimization. I might say I don't even have to allocate swap for code pages, right? Because when I need to reload a code page, I just reload it from the ELF file that it came from in the first place, right? So this can be kind of a clever optimization. I don't know actually how this works out in real systems because the point is if I change, you know, let's say I install a new version of bash and I have ten copies running, right? I actually don't know what happens in that case, right? Because I'm changing the code and so what could happen is that, you know, if I'm relying on the executables being on disk and now I've changed it, I might get, you know, a weird, bizarre hash of different parts from two different executables. So I'd have to look into this and see how this worked. But it's possible here, especially for code pages, I can do some additional things because they're never changed, right? So let's talk about swapping stuff into memory, all right? Stuff coming in, right? So we have to swap a page in when the virtual address is used, right? And at some level this is a part of what's something called on-demand paging which we'll either get to later today or Monday, right? But on some level the thing to consider here is that unless I'm doing something clever, whenever I swap in a page, it means that the process is waiting for that page to be there, right? The process has tried to translate the address and again, I've told it, oh, that's a memory address. Yeah, trust me, that's a memory address, you know? And then it tries to use it and I'm like, oh, crap, it's on disk, right? I've got to run, go get it, put it in there, you know? So this is in general kind of a very important path to be fast, right? Because there's a process waiting, right? So what do we need to do? So I'll help you out. First thing, the process executing instruction that tries to use that address. The first thing I'd better do is stop that instruction, right? Because that address is not now. And so this goes back maybe a month, right? So I know it's ancient history. So remember there were three ways that we entered the kernel, right? Does anyone remember what those three ways are? It's a good midterm review. System calls or software interrupts? Hardware interrupts and what? Exceptions. And when we talked about exceptions, we kept coming up with our favorite, which was divided by zero, right? Which is an exception that might call me to kill the process, but trying to translate an address that's not in the TLB also creates an exception, right? And this is how the operating system gets control, right? So if the address is in the TLB, I have a TLB miss exception and the operating system will start to run, right? And essentially what has happened already, what the processor did for me is it stopped that instruction, right? Rather than executing it because it can't, it raised an exception, right? What's the next thing I need to do? Let's say this page is on disk, right? I'm swapping in. It's coming in from disk. What's the next thing I need to do? I better find the page, right? Well, okay, so there's two ways to do this, right? And some of these don't have to be done in this order, right? I need somewhere to put the contents, right? So I'm going to bring in a page from swap, and so I need to find a page in memory that I can use. If the system is really, really heavily loaded and most of the memory is in use, what might this require? Swapping out some other page, right? So this can get even slow, right? So a swap in can actually trigger a swap out, right? Okay, but let's say I have a page lying around that maybe I've already written the contents of disk, so the swap in is very, swap out is very fast. Maybe I've trimmed some memory over time. A lot of times processing, a lot of times systems will actually try to maintain a certain amount of free memory at all times, right? So rather than just waiting until they need memory, they'll be constantly sort of, it's like having your hair cut, you know, it's like constantly trimming off little bits to try to keep it ever from getting too long, right? Rather than waiting until it gets long and they're cutting the foot off, you know, they're constantly trimming, I don't think you need that page, I'm going to put that on a page, I don't think you need that page. So they're constantly trying to have some sort of reserve around, because frequently when a new process starts, there's this barrage of new pages that have to be brought in, right? So if page allocation were like this slow gradual process, you might not need to do this, but the thing that causes big booms in page allocation is like you start Microsoft Word, right? And suddenly like 10,000 new pages need to be located, right? Or whatever. It actually might be that much. Okay, so and then as Michael said, now I've got to find that page, right? So now I need my page table entry to be correct. So I've got to find that page on disk. That page is somewhere on disk it better be, right? If it's a page that the process has used before, that's got some data on, I wrote that data to disk, so I need to find the data. What's the next thing I need to do? Copy it back, right? So I've got to copy it into memory, okay? And then what's the, I think there's seven here, what's the fifth thing? Update the page table entry, right? Because now the page is in memory. What's the sixth thing? What's the last piece of system that needs to find out about this? The TLB, right? Got to tell the TLB here's the new translation. And then finally what do I do? What did I do? What's the first thing I did? Now I've got to restart the instruction. Sorry, I know we're getting to the sort of basement of the slide here, right? So now I've got to restart the instruction that was trying to address that virtual address, right? Because now it's memory again, right? Again here's the illusion, right? The illusion is that the addresses are always memory, right? But because the illusion wasn't true when the address started to execute, sorry, the instruction started to execute, I had to take some time to recover that illusion and now the address is in memory and it will act like memory, right? Okay, so let's go through our example. So I have this page on disk, right? The page table entry is correct. The TLB doesn't have an entry for it, great, okay? So what's the first thing that happens in our example that triggered this swap in? Well, I executed an instruction, right? The process said, oh, I want to load 0x10 and that's the virtual page number and I didn't bother with an offset, right? So I want to load some address on page 10, right? And that creates an exception, right? Which stops that instruction from executing. What's the next thing I need to do? Over here. I need to allocate a page, right? Well, uh-oh. This doesn't match the slide. Okay, no, it does, right? Okay, allocate a page, although for some reason that's 3, right? Okay, so I need to allocate a page, right? And I think this was supposed to refer to find the page on disk, right? So in some level, I have to do both these things, right? I need to figure out where the data is and find some more to put it, right? Okay, after I've done this, I know where the data is, I know where to put it, what do I do now? Copy it in. And this again is a slow process, right? Now it's updated, now what do I do? I've got to move the PTE, right? Making sure that the next time, maybe if the century falls out of the TLB, I'm going to need to reload it, so the page table entry needs to point to where it is in physical memory, right? Alright, the sixth thing is what? Update the TLB, right? And you'll see here that if you remember the last example it pointed to physical page number 50, and I've changed it, right? Because that's what happened, right? This page is not is rarely, if ever, going to come back into the same physical page location that it was before. What matters is that the virtual page number is the same, right? Because the process doesn't know anything about physical memory, okay? And finally, what's the last thing I do? Let that instruction go, baby, let it go. Let it roam free, okay? So, so again, so how this whole process works is extremely dependent on how we choose the pages to write out to disk, right? As you mentioned before, if I choose a page to write out that's in the process of being actively used, I'm in trouble, right? This is bad, you know? And ideally, I choose a page to swap out that will never be used again, right? The process will go to his grave never having touched that page again, right? That is the ideal scenario, right? And if I can't get that, then I want the process to go for as long as possible without touching the page, right? Because there's a cost, there's a fixed cost to moving stuff back and forth to disk. I have to do several IOs, which takes time, and I have to, it's really, the cost is in two components. One is the time it takes, right? The other is the disk IO that's necessary, right? And the longer I can use the memory that I freed up by swapping the page out to disk before I have to bring that page back in, the more I amortize the cost of doing those operations, right? So that's my goal, okay? So let me just introduce you to the idea of on-demand paging before we finish today, and then we'll come back to this on Monday, okay? So sometimes, rarely, procrastination is a useful thing, right? And the best things to procrastinate about are the ones that if you wait long enough, you may never have to do, right? So never do something immediately if you might never have to do it at all, right? Sometimes if you wait long enough, the person who's asked you to do it will give up and ask somebody else, right? That analogy doesn't quite get here, but I am not going to give up and stop asking you to do your assignments, right? So don't try it in this class, right? But in general in life, this can't work, right? So let's think about how this works. Okay, you know, I'm Microsoft Word or I'm OpenOffice. I don't want to pick on Word. OpenOffice is really slow too, okay? So I'm OpenOffice or I'm whatever, and I've got this huge chunk of code and I'm running Exact and the process, the ELF file is saying here is this like big chunk of code, put it into my address space, right? So what could the kernel do? Let's say the kernel was like this, right on the ball, you know, go get her sort of kernel. Well, what would it do? It would take this huge chunk of code, all of it, and it would find somewhere in memory to put it, right? It would be like, okay, you've got, you know, a gigabyte worth of code. That's cool. You must be a really cool program, you know? I'm going to like, I'm going to sit here blasting away at the disk for like two minutes looking for memory for you so that you can load this huge blob of code. Right? Now, let's say that I have a kernel that's a little more, you know, again, I'm on top of things, but I'm not going to necessarily get to everything right away, what might it say, right? It might say, you know what? I'm going to make a note of that, right? I'm going to write down what you asked me to do. I'm going to write down this virtual address region that you asked me to use, and I'm going to make note of where that code is just in case I get it to get to, but I'm not going to load it right now, right? And why wouldn't I load, let's say I use, take about a word processor, okay? Any word processor, because they're all terror, right? How many, like, they all have these huge code bases, right? Because they're really complex. How much of that code do you think you actually execute in a session using that application? I don't know. Make a guess. I think it tends too high. I would go with one, right? It depends on what you're doing, right? Like, if you're bringing up OpenOffice to like a little flyer with your name on it to print it off for some reason, you know, I don't know, .1% of the code, these things have all these funky features, right? That nobody ever uses. They've all got 15 ways to do the same thing, right? And usually it's hard enough to figure out one, okay? So in general there's no reason to load all this code, right? Okay, same thing with the heap, right? The heap I think is a little bit of a different story, right? The processor might say, I need four megabytes of heap. I know I do. I'm going to use all of it, right? And the kernel might say write. Okay, again, I'm going to grant your request. I'll make a note of it. I'll write it down here in my page table so you're allowed to use that memory. But I'm not going to back that with real memory until you really show me that you're serious about this code, right? You're serious about these data structures. You've said you're going to take these data structures out every night, but I'm not going to let you know, okay, I'll pencil it into my calendar. But when you show up at my door, waiting for the memory, then I'll make sure it's there, right? And so the idea here is that until I either execute an instruction from a code page or do a load and store to a data page, basically until I use a piece of memory, a page in my virtual address space, the operating system is going to try to avoid backing that page with any real physical memory, right? And the reasons for this are the ones that we just described, right? A lot of code is never executed, right? Never. In fact, in companies like Microsoft and other big software projects, they run what are called dead code elimination tests, right? Because it's possible that in something as complex as Firefox, there are code paths that it is impossible to execute, right? And the compiler hasn't been able to figure this out because it's all really complicated, right? But when they run testing, right? At least like Microsoft, you know, when they run test suites one of the things they look for is which code paths are being executed. And if there's a code path that is never executed, right? It means one of two things. Either the code path is completely unreachable, in which case, I might as well get rid of it because it's just taking up space. Or worse, that code path is so weird, right? That it's never tested by my test suite, right? And what does this mean? Who might try to get the system to go down this code path? An attacker, right? A lot of times, dead code paths or very, very, very cold code paths that never get hit in test suites have all sorts of bugs in them, right? Because those bugs are never uncovered. And so if I'm a clever attacker and I find out pieces of Firefox that never get used, if I can figure out a way to get the code down that through some sort of very weird thing, right? Very weird set of conditions. I may be able to find a buffer exploit or something else that I need. But in general, a lot of code is never used from the operating system. It doesn't care why the code is there, but it's going to say, until you execute it, I'm not going to put it. Okay? So on Monday, we will talk about demand paging, and we will talk about page replacement, all right? Have a great weekend. I'll see you on Monday.