 So, today we're going to, so last time we sort of finished up looking, and we were sort of in the middle of looking at different ways to translate virtual addresses to physical memory addresses. So, today we'll talk about the modern solution, and then we'll talk about various challenges in actually implementing that solution within the operating system itself. So, okay, so a couple announcements today. Last week for assignment two. So Friday is the deadline on, and good luck. I mean I can tell that people are working on it, which is great, but you know please do as much as you can. One reminder is that the assignments in this class are cumulative, so you don't, you know, we're not going to give you any new stuff after you're done with assignment two. So when you're done with assignment two, you may have to keep working on assignment two, even after you thought you were done, because there are a lot of assignment three, in fact I would say all of assignment three, depends on the assignment two things that you guys are doing. So, you can either fight those bugs now, and get credit for them, or you can fight them later. We probably will distribute a solution set, but we're going to charge a pretty hefty fee for it. So, it will probably dissuade many of you from accepting our offer. So my suggestion is, let's crank out this week, get assignment two as far as you can. Alright, so Wednesday's lecture will be the end of the material that will be covered by the midterm after break. So, it's a reminder on Monday after break we'll do review. On Wednesday we'll have the midterm exam in this room, and the midterm will be up through Wednesday, because we will not have class on Friday. So that was never on the calendar. I realized that this week, so it's not going to happen. We'll be holding sort of final, last-minute, desperate office hours for assignment two at that time, and we'll do that in a little bit of class. So we'll have a lecture on Wednesday, and then on Friday we will do our sort of assignment two turn-in party. It's a party for me. I think it's fun. I'll be there, the TAs will be there, and hopefully you guys will be there. Actually hopefully you won't be there, because hopefully you'll be done, and so you won't care. Final note about this week is that we will not have recitation tomorrow or Friday for two reasons. One is that I think we've covered as much as we can cover with assignment two and recitation, and the second reason is that Jinghao is ill. So he can barely speak, and I'm sure he can barely speak on Friday morning at 8 a.m., when it's probably hard to speak most weeks. Okay, so over the weekend we had a couple new groups that reached the summit. So Rakesh and Ramya are those guys here, there they are, all right, great. And Viral and Pranav here, all right, congratulations, good work. So again, these guys, they probably are going to be in hiding for the rest of the week at an undisclosed location with no access to piazza or people that need help with assignment two. Hopefully the people who are done with the assignment will still come to office hours on Fridays to help out other students, this is a great way of developing good karma for the class that you'll need for assignment three. So yeah, I think hopefully now you guys are realizing that assignment two is hard, assignment three is also hard. This is the first big deadline for the class, after this we'll have a big chunk of time to work on assignment three, but everything up through assignment two is sort of like just volume one of Kill Bill, right. When you get done, you start through this long, bloody epic and then you realize that there's like three more hours afterwards and a bunch of other enemies still to defeat. So all right, so returning to any questions about logistics for this week? So just to sum up, no lecture Friday, no recitations this week. Office hours is scheduled and probably some extra months on Friday afternoon. Okay, any questions about what we covered last week in particular about ways of efficiently translating virtual to physical addresses? So what was the goal of this process? Remember, the kernel was going to establish a mapping between the virtual addresses that processes C and the physical hardware addresses that actually point to real memory. When we're translating between the two of these, what was the overwhelming and overriding objective? Anyone remember? What is the number one most important property that this translation process has to have? Yeah. Nope. Nope. This is a one word answer. Yeah. Fast. Really fast. Memory is supposed to be fast, right, and if we get in the way too often what's going to happen is the kernel is going to make memory seem slow, and that will be really sad because to be honest with you, there aren't very many other fast things on the machine. If you make memory look slow, what are you going to do the disk, right, it's going to look terrible. So in the way that we wanted this to happen was that as many translations as possible should be able to proceed without needing the kernel to do anything. So once I've told the MMU about how to translate a particular address, I'm hoping that I won't have to get involved again, and the reason is of course that entering a kernel is too slow. And the goal here was the kernel is going to set the policy for how addresses are translated and hardware is going to provide the mechanism that makes those translations fast. So we've talked a little bit about the policy so far in terms of how address spaces are constructed, but really at this point talking more about mechanisms in terms of how hardware actually does this for us. It's all also that all addresses are translated implicitly. So there is no point in which user programs ever have access to direct hardware addresses. The process is always that the user program uses an address that it thinks is a real address. It's all that knows about reality, and all of those addresses are being transparently translated by the kernel with hardware help. So the simplest virtual memory mapping scheme that we came up with is called base and bounds. Who remembers? Who can describe how it works? Base and bounds. What's to give it a shot? Nobody? Base and bounds. All right, so I'll give you a hint. I assign each process a base and bounds, right? Clearly that's got us a long way down the line to solving this. Once I do that, how do I translate addresses? Yeah. Right? What does the bounds represent? Right, so I assign it a base physical address. That's where its address space is going to start. That's virtual address zero. And then I assign it a bounds which determines how big that address space can be. And then in order to check whether or not the virtual address is okay, I just have to check whether or not it's smaller than the bounds. If it's inside the address space, I'm good to go. And translation is as simple as taking the virtual address and adding it to the bounds. It's coming back to people now. This was pretty straightforward. What was the main problem with this? This is very simple. There's only two parameters I have to assign to each process. You can imagine I can store all this state pretty efficiently even if I have a bunch of processes. What was the problem? I just know this worked, yeah. Yeah, so now the problem is remember the goal with the address space abstraction was to give, part of it was to give processes this huge chunk of virtual addresses that acted like they were all there. Now the size of the address space that I can use is now limited by physical memory. So that's one big problem. And it's also possible that if there's large portions of the address space that are unused, those portions are, I still have to allocate physical memory for them and those portions are waste. So this was not a good fit for the address space abstraction. So we tried something else. What was that thing? I mean, it's called segmentation. That's on the slide. I'm not accepting that as an answer. But what was it? Yeah, so I assign each process with multiple bases and boundsies. Each covering a segment within the address space we call this segmentation. Give me an example of a segment. Stack, heap, code. These are logical regions of address space where we expect there to be contiguous allocations. That's important. I wouldn't allocate a single segment for both the heap and the stack. Why not? Seems like it's more efficient. I've just removed one segment. I used to have two segments, one for the heap and the stack. Now I'm going to do one. Yeah, there's this huge, vast wasteland of unallocated memory in between the heap and the stack, hopefully. And so if I use one segment to cover both, then I'm wasting a lot of memory. So segments are covering areas of the address space where I expect there to be contiguously allocated memory. I assign each one a base and bounds. And in this case, I need a starting virtual address. So in base and bounds alone, the implicit assumption was the starting virtual address was zero. Here I need three parameters for each because I need to specify where the segment starts in the virtual address space as well. Not just in the physical memory. Checking is easy. And this is also the procedure I can use to find out which segment a particular virtual address is part of. And translation is also quite simple. The problems were segmentation were that I still have this entire chunk of memory that has to be contiguous in physical memory. I can't just bring in the portion of the code that the application is using. I have to bring in all of its code. And it turns out, so how many people think they've ever even used, I don't know, let's just say 10% of the features of Microsoft Word? How many people think they've used over 10%? I'm surprised. Yeah, me neither, right? There's like billions of menus with all these weird options that all look scary and you're just trying to avoid clicking on because if you do, it takes you into this weird dialogue that you can't escape without destroying your document in some weird way, like converting the whole thing to wingdings and you can never get it back. So the thing about it, for all of those options that you're not using, there's code in the Microsoft Word executable that's sitting there waiting for the day that you suddenly decide to click on whatever sort of weird menu you're trying to stay away from. That code's there. It's been there the whole time. So I would prefer not to have to load all that code you're not using and hopefully most sane people aren't using into the address space every time. If I had pure segmentation, I'd have to do that. I'd have to take the whole code, put it somewhere in memory, and any time Word used any of the code, I'd have to bring in the whole code segment. So I'd like to avoid this. And I have this problem with external fragmentation because the segments may want to change size. The segments themselves are a bunch of different sizes. So now we've gotten to the point where we left off last time, which is we came back to this idea that what ideally I'd want is the way to translate any virtual address to any physical address. I can't do this alone, but we introduced the idea of a piece of hardware that would be able to help me. That hardware was sometimes called a translation look-aside buffer or a cam. It's essentially a form of content addressable memory. And this was a common thing that operating systems do to make something fast, is rather than doing everything quickly, I do a small number of things really fast. So that makes the entire process look faster, assuming I can figure out which things to prioritize. So we'll have to come back to this in a minute. But the idea here is if I choose a careful subset of all the things I want to do and do those things really fast, then I can make the entire process look a lot faster. This is called a cache. So the example with the TLB, I can simultaneously search a bunch of different entries in the TLB, not an unlimited number. This is limited in size. And I can use this to map virtual addresses to physical addresses. You'll see here I'm only mapping a snippet of the virtual address to physical address. So the problem that we had identified was that cams are limited in size. And so let's say I have a content addressable memory TLB that has 32 entries. And I've decided, for whatever in any reason, to map each virtual byte to a physical byte of memory. So how many bytes of memory can this 32 entry TLB now cache? 32. And that's probably not enough. Remember, I don't want the operating system to get involved. I want it out of the way. If I had a byte to byte TLB, the operating system would be constantly in the loop having to fill up the TLB with new entries as the program kept executing. So at this point, segments are too big. Mapping individual bytes is too small. So what shall we do? One soup is too hot. The other soup is too cold. There must be a soup. There must be soup, not a soup, some amount of soup that is just right. And of course there is. We call that paging. So the idea with pages, and it's quite nice, is that I'm going to pick a fixed allocation size, which is called a page. And I want that to be small enough to avoid internal fragmentation. Remember, the problem with those big segments is that there was a lot of unused memory inside of them that I didn't want. So the smaller I make the page, the less likely it is that I have to bring in a page of memory to get one byte, despite the fact that I don't want any other byte on that page. At the same time, the pages can't get too small, because then the cam isn't able, the TLB is not able to translate enough of the address space to be efficient. So I want it to be large enough to allow the TLB to effectively cache a large portion of my address space. The size of pages also determines the size of the memory-related data structures in the kernel. And this is another incentive for keeping the page size on the large side. If I have to store some amount of information per page and I have a certain amount of physical memory and I have a certain amount of virtual memory allocated by every process on the system, then clearly the size of the data structures that I need to store all that stuff are going to scale based on the number of pages. So if pages are too small, those data structures get really large. Those data structures consume memory that I do not want to waste storing information that processes don't actually care about, that you don't actually care about. There's another really important property of programs that's going to help me here. And that property is called execution locality. So it turns out happily that program accesses to memory are not randomly distributed all throughout the address space. So if I took a running program and I could take a snapshot of it over a window of time and I looked at where it was accessing memory, again, those accesses are not just randomly distributed over the entire address space and they're not even randomly distributed over the entire address space that the program has permission to use. They turn out to be highly clustered. So who can give me one reason for that? There's many reasons for this. There's a really obvious one. Yeah. Yeah, the standard processor pipeline, right? Load, decode, fetch, decode, execute, increment the program counter and do the same thing. So unless I'm jumping to another area of memory as a result of some conditional, essentially my code is just walking down the code segment. Now are there jumps and branches, of course. But ideally, a lot of my code ends up in these linear sequences of instructions that produce high locality. So a bunch of them are going to fall within a single page. And it turns out, actually, and I think this is really cool, that at least one of the things that Microsoft used to do in the past to try to improve the performance of their applications was actually study those patterns and then reorganize the binary file in order to improve spatial locality. So they were actually able to even get more spatial locality out of their program by, for example, noticing that it took this branch a lot, so why don't I move the code from that branch right after that branch so it didn't have to jump to some other page. Pretty cool. But anyway, this helps us. Give me another example of spatial locality. Yeah, any sort of data structure I'm using. That data structure is probably not like gigabytes in size unless I'm some sort of database. But the little data structures that you're using as you go along, where are the local variables that your program subroutines are accessing? They're all right next to each other on the stack. So I've got nice locality in the stack. I've got nice locality in the code segment. I've got potentially high locality when I'm looking at data structures. This is awesome. This is all very, very good for the system because it means that although I move around within my address space in any given small time window, my accesses are extremely clustered. And so I can load a small number of translations that cover a small amount of memory and get a lot of use out of it. Another way of thinking about this is every time the operating system loads a translation into the TLB, it wants that translation to be used as many times as possible. Ideally, I would load that translation once. And that translation would be valid from now until forever. And I would never have to do anything else. But the worst thing would be if the accesses were random because I'd be constantly loading new translations and it would be very unlikely that those translations would be valid for more than one load. The operating system does work to tell the MME what to do when a translation request is received. I want that work to pay off. And we'll come back to this when we talk about page replacement and swapping because it's a very similar concept. All right. Any questions about execution locality? This is pretty important to understand. It's a pretty powerful concept we guys think about as programmers. It's also just cool. All right. So page size, let's talk about page size. Now we said we wanted it to be not too big, not too small. So for a long time, a lot of systems used a 4K page size. Because of how particularly x86 processors work, there are requirements that the hardware and the software, in some cases, agree on what the page size is. I don't want to get into all the gory details about that. But just trust me that it's true. And so for a long time, there was this magic number of 4K that was the page size, 4 kilobytes. Now sometimes you see much larger pages being used. There are new architectures. Again, now that you guys have decided to load your machines with gigabytes and gigabytes of memory, 4K is a page size that started to look a little bit small. And so now I have a lot of architectures that support larger pages, potentially even jumbo pages that are multiple megabytes in size. So this is a parameter that's going to continue to need re-tuning. Once you guys have terabytes worth of memory in your computers, which I hope I'm alive for that day because it sounds awesome, but you probably will not be using 4K pages. They're just too small. They're clearly the amount of memory on the system also matters. But even, let's say I have 4K pages and I have 128 entry TLB, now I can potentially cast translations for up to one. Oh, this is totally not true. I'm so embarrassed. This bug has been on the slide since my first year teaching this class. Anyway, so clearly 4 times 128 is not one megabyte that's 512K, but you see my point. So with 4K pages and 128 entry TLB, I can cash 512 kilobytes or half a megabyte worth of virtual addresses. Now, this doesn't seem like a lot. My address space is potentially 2GB or 4GB of virtual addresses. But again, go back to thinking about locality. That one megabyte or 512 kilobytes goes a long way. So it can cover potentially a lot of code, a good chunk of my heap and a little bit of my stack, and allow me to make a lot of forward progress without having to do too much reloading of the TLB. One way to think of pages is as fixed size segments. So rather than having a bounds that's different, the bounds is the same for each. And the bounds is set by the page size. So how do we perform page translation? Well, it's a lot like doing segments, except for the fact that we can play some games with bit masking, which is always fun, because you guys are computer scientists and clearly love to play bit masking games, in order to improve the process slightly. So what we can do is we can separate a virtual address now into two parts. And depending on how the page size is chosen, with the 4k page, I can literally take three of the digits off of one side of the hex address and just rip them away and call them the offset. And whatever is left over, I call the virtual page number. If you go with 8k pages, then it's messier. So maybe that's why people like 4k pages for a long time. But 4k pages take the rightmost three digits. Those are your offset. Everything else is a virtual page number. Each virtual page number maps to a physical page number. I'm translating every 4k segment of virtual address in any process to potentially some 4k region of physical memory. So paging also implies that I'm splitting up my physical memory into 4k chunks. And if it wasn't obvious, all addresses on the same virtual page map to the same physical page. So once I've told the TLB how to translate one virtual page, then it can translate all of the 4k addresses inside of that. Again, you can imagine a program is running along. It's going through its code in linear order. Fetch, execute, fetch, decode, execute. You commit the EPC, sorry, the program counter. And it hits the top of a new page. There's an exception trigger. The OS gets involved. It loads the next page translation. And then I can continue executing a bunch of instructions before I trigger another exception that the OS needs to handle. So the way, so here's how this works in the framework we've been using before. I split the 32-bit address into my virtual page number on the offset. I check if a virtual page to physical page translation exists for this page. And this is the part where things get gooey. And we start having to talk about data structures in about 10 minutes. So this is the part that starts to become more difficult. Because potentially I have lots of virtual pages. And so I want to be able to do this quickly. But once I know that a virtual translation exists for this virtual page number and the physical page that it translates to, the resulting translation is very simple. I take the physical page number. I take the offset. I slam them together. And I'm done. So oh, OK. So just a brief interlude here. So I don't know how many people have seen this movie. In theory, the professor here in this scene is my advisor. I don't know. More or less attractive than my advisor, I'm not sure. But anyway, so this is a scene from the social network. They're in theory talking about what we're about to talk about next. It's about 30 seconds. So I just want to point one thing out here. So when I saw this movie, I saw the slide in the background. You could see the slide. And I thought, there is no way that Matt ever used to slide that complicated in lecture. I mean, look at that. It would probably take you hours and hours to draw. And then after the movie came out, I went back through a slide deck. And I'm pretty sure I found pretty much that exact slide. So there actually was, in fact, a slide this complicated. Let me just watch the rest of the clip. So now let's talk about the TLB. We will get back to this example in a minute. So let's say that we're using 4K pages. How do we actually translate a virtual page to a physical page? So let's say this is my virtual address. Who can tell me, what is the virtual page number corresponding to this virtual address? Again, no fancy division is required. Just the ability to separate objects visually using your eyes. What is the virtual page number? 800. Take the rightmost three digits and just pull them to the side. That's your offset. Everything else is the virtual page number. So my virtual page number is 800. What physical page does this translate to? So now notice something here. The TLB is now caching virtual page number to physical page number translation. Because remember, the offset doesn't matter. The offset's the same. So I looked this up in the TLB. I see that there's a valid translation located for virtual page number 800. What's the resulting physical address? Again, no fancy math required. 306, 346. Replace the virtual page number with the physical page number, and you are done. Very easy. Let's do another one. Virtual page number. 10. What's the physical translation for this virtual page number? 50. What's the resulting physical address? I've got to get through my silly animations. 50336. Does this make sense? This is probably, I think, again, assuming you can perform this visual manipulation. This is probably the easiest form of address translation we've talked about, something because there's no actual math required here. It's just pattern matching. Again, if you did pages that weren't 4k and fell on a weirder boundary, you'd have more problems. But I won't do that. So here's a question about the TLB. So clearly, the TLB is an important part of this process. Where do the entries in the TLB come from? It's a little bit of review. How do they get there? What's that? Yeah, the kernel loads them. So go back to our example before, when I was telling the MMU what to do. Now what the MMU is going to ask me is it's going to say, some of you guys have seen this as part of your assignment to debugging. Some of you guys thought it was a virtual memory problem, where the system said, oh, by the way, I have a fault on this address that I don't know what to do with. And it turned out to be a fatal fault. And usually that's because you've done something dumb. But what the system is saying to the kernel is, I don't know how to translate this address. And the kernel at that point could say, here's how, or has to deal with the situation. So again, so the operating system is asked for help via what's called a TLB exception. Usually that exception will indicate something else about what happened, like whether or not the address was being read or written to. And at this point, the operating system is now in the loop. And also keep in mind, at this point, the clock is ticking. The faster I can handle what happens here, the better for everybody involved. So what are some of the things that we like about paging? Does anyone see some nice benefits to paging compared with the approaches that we've discussed so far? Yeah? Should be less small. Yeah, so actually it's nice. There's less of both kinds of fragmentation, right? So because I'm choosing something that's smaller than a segment, I have less internal fragmentation, because especially due to spatial locality, because it's an execution locality, because it's very likely if I execute one instruction on the page, for example, I'm going to execute other instructions on the page. It's very likely that if I touch one part of a data structure that's located on that page, I'm going to touch other parts of the data structure that are located on that page. But there's also less external fragmentation. In fact, you can predict with precise and complete accuracy how much external fragmentation takes place because of paging. Anyone want to guess how much it is? None. Zero, right? Because I have a fixed allocation size. Remember before I had this worry that I'd have two things, it's like I need 32 bytes of memory, and I have two things that are 30 bytes apart. Never happens here. I need 4k of memory, and I've got 4k chunks. And so either I've got one or I don't. And I never have to worry about external fragmentation. So this is extremely nice. Now, keep in mind, I can take a lot of the things I liked about segments, and I can just map them right down onto pages as well. And a lot of operating systems do something that you would kind of describe as a mixture of both. They use segments to describe properties of large, contiguous parts of the address space. For example, the code can only be read and executed from. And then below those segment data structures, they have pages to store the virtual to physical mappings for the pages that are within that segment. So I can really set up things so that I get a lot of the benefits of both approaches. Okay. Now, the problem of course is that potentially, compared with the other approaches, that hardware is gonna be able to translate a lot less memory for me. With base and bounds, I could translate the entire address space with just two pieces of data. Then once I got to segments, I still only had to have a number of pieces of information that scale with the number of segments within the process. But here what I've decided to do is trade off much, much smaller amount of memory that the hardware can translate for me in exchange for much better fragmentation and much better ability to reuse memory inside of allocations. The other problem here that we're gonna start talking about today is that now the operating system is forced to store information about each page. And this is a bit of a challenge. How many page, let's say I have a data structure that might have information about a page in it. How many of those data structures does the kernel have to have? Anyone tell me? Yeah. What's that? Oh, if only that were the case. So there's a claim here that the number is the amount of physical memory divided by 4k. Who thinks that number is too high? Who thinks that number is too low? You wanna revise your answer? Yeah. We're getting closer, so now I've introduced this idea that I might have a per process data structure, but keep in mind I'm talking about the underlying structures that store information about every virtual page. So hardware memory size divided by 4k too small, why? It's one of the things I'm gonna start doing is allowing processes to allocate more virtual memory than there is physical memory on a machine. And so in order to support that, I need to store a page data structure for any virtual page that's been allocated by any process that's currently running. And on most systems, the total amount of virtual address space that's been allocated by all the processes on the system is potentially much larger than the amount of physical memory. So this is even worse, right? They can potentially be unbounded. And as I pointed out before, this memory is pure and utter waste. It doesn't accomplish anything that a process cares about. So the goal is to keep this as small as possible while still being able to translate things quickly. So, okay, here we go. Here's the, oh, where'd it go? Oh, is it here? Okay. So in order, so the goal here is to keep the TLB up to date. When the TLB asks for help, what I want to be able to do is quickly find out information about the virtual address that it's questioning about. It says, I have this problem. I don't know what to do with this address. Let me have some help. Yeah. Yep. What's that? Yeah. Right, so the goal was, remember, with the earlier approaches, I could potentially have hardware map the entire address space without requiring any additional information. So for example, if I'm using segments, and let's say the hardware has a limit where it can only know about 16 segments for each process, that's still enough. So I could have the entire process address space all translated in hardware, right? The problem is that there's these huge swaths of unused space, right, or potentially poorly used space inside those allocations. With pages, I'm, the percentage of the address space that the hardware is gonna be able to translate is gonna be small. But the trade-off is that I'm not wasting memory inside each segment. I can move part, for example, all those features you don't use in Word. I can just get rid of those. I can see after a while, oh, by the way, that code's never been executed. I'm going to get rid of it and use that memory for something else. Does that make sense? It's a good, great question. So, the way to think about this and the thing that you guys will do for assignment three is it's your job to help the TLB. At various points, the TLB will tell you, and this actually happens fairly frequently, I don't know what to do. Process tried to execute an instruction that requires translating in a virtual address and I don't know how to do that. And so your job is to tell it as fast as possible because at that point, the kernel is involved and so things are already slow. So we refer the data structure that we're gonna talk about today and probably next time is what's called a page table entry and you'll understand why it's called a page table entry in a minute. So each page table entry stores information about a single virtual page and that information can include a bunch of different things and usually we try to, so this goes back to the social network example. So this was sort of the type of question that Mark Zuckerberg was answering in that because we usually try to do some fancy tricks to jam as much of this information into as small of a space as possible and we're particularly happy if we can, for example, jam it into a 32-bit or maybe 64-bit or 48-bit or something else that the compiler knows how to bin pack properly. If I put it into 33-bits, you know, you're not gonna get much help but if I can get into 32-bits I'm golden. So things that I might need to know about each virtual page, where is it? So we will talk next time and in the future about other places the pages can go other than in memory but it's certainly the case that some of the time when they're being used as memory a virtual page will need to be in actual physical memory and so the location will be a physical page number. I can assign permissions to the page very similarly to the way that I would do for a file. I may want to know if the page is in memory so if the location should be interpreted as a physical page number or as something else I might wanna know and then I might wanna keep some status about how often the page is accessed and we'll come back to this when we talk about swapping because at some point I might need to decide who, what page gets kicked off of the island and when I do that having some statistics about how often pages get used could be helpful. So now, so I've got these great data structures, they're all floating around but now here's the problem. The TLBS, the kernel for help and again now the clock is ticking so now the kernel really needs to have its act together and be able to quickly look up information for this virtual page because again I'm already on the slow path here. This is already my worst nightmare, the thing that I didn't want to come true is happening. The TLB asked me for help, at this point I already know that I'm causing the process to slow down and I wanna just slow it down as little as possible. So there are two sort of conflict, somewhat conflicting and you'll see directly conflict in a minute requirements for storing this information. The defining one is speed. This is again, this is a hot path, this code gets executed a lot and it needs to return a value as quickly as possible and there are all sorts of games that operating systems and modern hardware architectures have actually started to play to get this to happen as fast as possible because this is by far the most common exception and a serious threat to performance if you don't do a good job here. Now this is the type of code path that if Microsoft engineers tomorrow found a single instruction on their paid fault hand lane path that they could get rid of, they would go out, have a beer and not come back to work for a month. I mean this path gets executed so often that even tiny little savings will make a huge performance difference. Compactness, so here's the other problem and these are sort of diametrically opposed. The data structures that I used to do this can't themselves consume too much physical memory because if I do that then I'm just creating more memory pressure on the rest of the system because the kernel, which isn't even a useful program just sort of sits there getting in my way as I'm trying to do useful work is now not only making me ask it for help to do basic stuff like read things from the disk but it's also consuming most of the memory on the system. This is not something that applications want or you want. So the data structure that we used to do this is called a page table. So a page table maps a virtual page number to a page table entry. And page tables, so how many page tables will the system have? One, what do I have to tell you for you to tell me how many page tables there are in a given system? Per process, right, remember. This is where this process dependence of virtual to physical translation comes into place. So we're gonna talk a lot about at this point translating a virtual address to a physical address but keep in mind the data structure that we're using is per process. So that's how two processes can have the same virtual address that points to different pieces of physical memory. In this case, I could have two virtual page numbers in different processes that point to completely different physical pages. In fact, most of the time they will point to completely different physical pages. I just said that, sweet. Okay, so here's potentially the fastest and most basic approach. Which is that I use one huge array to store every translation for the process. And the way I do this is pretty simple. There's my flat page table, there's my page table entries and I use the virtual page number as an index into that table and that allows me to look up things extremely quickly. And every valid page table entry is gonna have its own slot in the table. The rest of them could be null or something to indicate that there's no valid translation loaded there. What's good about this? There must be something good about it, yeah. Very fast, oh one, right? One look up, very, very fast. What's the reason that we don't do this? Yeah. It's huge, right? Huge, it's four megabytes of physical memory per process. Your MIPS system has potentially up to 16 megabytes, which means that if you ran two processes, the kernel would now be consuming half of the physical memory for a paging data structure, right? No, not what we want. And the other problem of course is that all of this stuff in here, I mean some of these are gonna map to some of these other ones, whatever, but most of this data structure is empty. So it's sitting there hogging memory, hogging memory in a contiguous way which can be a problem for the kernel in certain times and most of it's unused. So this we don't want. So let's go to the opposite extreme. Now let's think about how we would use a linked list to map virtual pages to physical pages. So if I do a linked list page table, I organize the page table entries into a list and when I'm translating an address, I look through that list until I've located it and then I'm done because that page table entry contains the data that I need. So what's nice about this? What's that? It's very compact. This is taking essentially the least amount of memory that I could possibly consume in order to translate all the addresses. What's the problem? Slow, right? I have to O-N lookups to, and yeah, you could do fancy things. You could store pointers to the middle of the table, whatever, right? I mean feel free to throw cord at this and see what happens. But, so one earliest, and this is, I always, I really wanna say this this semester. I haven't said this before. The page table data structure we're gonna talk about next, which is commonly used, is not something that you're expected to implement for assignment three. In fact, this page table data structure will work just fine for assignment three. In fact, it'll work well and you will never notice the performance problem. Trust me, are you talking about a system simulator running inside a virtual machine? Who cares, right? It's not gonna be that slow. So, but let's talk about a different way to do it that tries to get the best of both worlds. So now we have something that's too big but fast and something else that's the right size but too slow. So what we're gonna do is essentially build a multi-level tree. It's gonna be a two-level tree and these are called multi-level page tables. This is not a particularly uncommon data structure if you guys have seen trees before, you've seen this tree, a particularly simple model. What I do and there's some nice features of 32-bit addresses that come into play here. I think 40 or 48 or 64-bit addresses probably do some nice things like this as well but I'm not sure. So for example, what I can do, the offset is for a 4K page, it turns out is the bottom 12 bits of a 32-bit address that leaves 20 bits remaining. So what I do with two-level page tables is I take those 20 bits and I divide them into two pieces, the top 10 bits and the middle 10 bits. So I mean the bottom 10 bits of the virtual page number. So the virtual page number is 20 bits and I divide that in half. The top 10 bits I use an index into my first table, the next 10 bits I use as an index into my second table. And then the real thing where this all feels like it was somehow ordained by a higher power is the fact that a array with two to the 10th four byte entries in it turns out to be exactly 4K. So somehow this all just comes full circle here. So the levels of my array are now themselves single pages of memory, which is again very nice. So here's my example. For each process I need a single first-level page table. Every process is gonna need to have this to translate anything. So there's 4K overhead just out the door. Now for any valid address, I need to look it up in a second-level table. So I take the top 10 bits of this address, I use it as an index into my first-level table, I take the next 10 bits, I use it as an index into my second-level table like that. And that points me at a page table entry. So the nice thing about this is it's still constant time. It's one more lookup than my single flat array. And so it turns out here that the structure of address spaces comes into play yet again. So because address spaces tend to be fairly sparse, it turns out that I don't need that many second-level tables. Again, if I had a process that decided for whatever reason to allocate memory with equal probability across its entire address space, this abstraction would not work out very well. But that's not what happens. Processes put all of their code in one tight little region. And then they have a heap that starts at a particular place and grows in a particular direction. And then each stack sort of starts and grows, sort of oscillates between the top of the stack and the bottom as I make function calls. So I don't randomly distribute my pages throughout my address space. And because of that, you can imagine for an example process like this that I might need one second-level page table to cover all of the code. And then maybe another one here to cover all of the heap. And maybe one per stack depending on how far they are apart. So now I've gotten away with only five 4K data structures and I can still do this in constant time. If all this was allocated at random throughout the address space, it would be bad. I would need a lot of second-level tables. And pretty soon, essentially, what I'd have is I'd have that constant, I'd have that one big array, which is what I don't want. OK. So let me see where I am on time. I think this is a good place to stop. OK. On Wednesday, we will talk about what we do to create more memory than processes think is on the system. And I'll see you then.