 Hope everybody had a good break everybody have a good break. How many people had a good break? How many people worked on? Assignments for this class during break, huh, and it was still good. Yeah all right, so today we are gonna keep going with memory management and I hope so how about this you'd be hackathon They have a 421 521 team wins the hackathon Then I will release one of the exam questions early Fair What? There aren't that many One of the multiple choice questions, right exactly one about the M&M. That's what I'm gonna. No No, no, we did this way We had this last year we use some carrots at the end of the year last year was for something different But but yeah, so if a team wins I'll release one of the long answer or medium answer questions early And everybody can can know what the question is and have an answer ready, etc. So how about that for a For incentives you guys should sign up. It's fun. It's a it's a fun chance to you know Get together with a bunch of people and you know program something up and you know If you have a project that's been kind of kicking around the back of your head It's a chance to sit down and just you know devote some good time to it So I'll probably be there and I may have something to work on All right, so on Wednesday We will hand back the midterms midterms are graded at some point figure out how to release their grades maybe on the website before Wednesday, but if not then Wednesday in the class The assignment to Auto grader is working so you guys can submit submit your patches and have them be tested There were some notes released on piazzas about that Because gurus out of town And I'm hosting a faculty candidate this week there's a few tweaks to the office hours for the next two weeks so please check the calendar before you come in and Any questions about this sort of stuff? so basically we guys have you know roughly a month and a half to Finish the rest of the assignments, and there's actually you know, there's like quite a bit to do so You know so I hope you guys rested and and rejuvenated and ready to kick some butt on December 2nd December 3. Yeah, Jim. I do know those things Well, let's see here I Mean I can you know how about this why don't we talk about that on Wednesday when we hand back the exams? So on Wednesday, I will also get out the statistics. I think they have the average was pretty good. I think it was I Won't say I I don't remember exactly. I don't want to frighten people So we'll hand back the exams on Wednesday and summary statistics will be will be released about the exams of the questions Any other questions? other questions all right, so today we're going to do a little bit of kind of a somewhat longish review of memory management because it's been a little while and then we'll talk a little bit about Haging and hardware support for Efficient address slowness. So does any I mean I find it unlikely that people have questions because how many people remember anything about memory management? So, okay, so as I have any questions popped up over break before we go and do a little bit of review Okay, so We had these four Goals of memory multiplexing. So we're talking about Dividing the memory on the system. There were four things that we were trying to accomplish. Does anyone remember what they are? Well, that's up on the slide. So can anyone describe one of these to me? Yeah, great Yeah, so we want to we need to allow processes to use the memory on the system That's a pretty basic requirement of multiplexing, right? There is a resource. We're in charge of distributing it So we need to allow process to use it. What about enforcing those allocations Frank? Yeah, so we need to to be able to say if I give a process a certain amount of memory that it can use that memory But it cannot use use other pieces of memory What about reclaiming memory? Yeah Yeah, so we're going to talk about over the next week about ways that the operating system might be able to identify Memory and not necessarily memory that's not being used by a process This is memory that I've allocated to a process but maybe the process isn't using it anymore or isn't using it very much and In certain cases if I'm on a heavily loaded system I might need to I might want to be able to take that memory away and it's sort of Transparent sneaky way and give it to another process to use another process is going to take better advantage of that resource, right? And what about revocation? Sean Yeah, so I should be able to take away. I should be able to take away allocations from process, right? So This is pretty much what we talked about, okay? And In order to account, so we talked about some of the problems of direct physical Add, you know direct physical memory Allocation and we came up with this nice abstraction of an address space, right? So an address space is our memory management abstraction It's what we're going to try to implement we started to talk about some of the challenges in implementing it But what was the goal of an address space? What were some of the things that we liked about the address space of abstraction? You just answered a question Yeah Yeah, so we want to well It's I so first of all we get each process an identical view of memory, right? and a view of memory regardless of the machine that they're running on regardless of the Time that they're running regardless of how many copies of that process are running regardless of what other processes are running on the system, right? So this is a nice Thing that makes it easy to write process memory layout and for processes to be able to to use memory at runtime your memory Your address space looks the same Regardless of whether you're running on one machine or another machine, right? And then one of the things about the address space abstraction was we were able to you make memory look uniform What else were we able to do? Yeah, but yeah So one thing we were able to do is first of all make it look like there was a lot of it, right? We can give a process of we had these address spaces We were providing a processes that on your OS 161 system are two gigabytes big. I mean which is huge, right? I mean that's a that's way more memory than the system has, right? We make it look like it's contiguous, right? So it's all laid out next to each other which is very nice despite the fact that that actual address space of abstraction is going to end up Fragmented all over all over real memory, right? We'll talk about how that works That it's uniform, right? That it you know kind of all behaves in roughly the same way And then it's private it's about any pointed out. This is a private abstraction So there are ways to share memory between processes, but they require explicit coordination and approval by the processes involved, right? So we had Our goal here is we're trying to implement this abstraction So we had this nice abstraction of these great features. We want to make it work, right? And clearly what we discovered is that there's no way to implement that without breaking breaking the direct connection between physical Between memory addresses and physical memory, right? So we have to put something in between What was this what was this level of indirection that we that we came up with? Question is I can't give I can't give processes physical addresses anymore I can't allow them to use physical addresses. So what am I going to allow in these virtual addresses, right? So I'm going to introduce the idea of virtual addresses that require translation, right? So virtual address doesn't necessarily point to a specific physical address It's up to the kernel to translate it we talked about some of the nice things that this allows us to do, right? So if I give out references rather than, you know, direct access to an object, I can revoke the references That's one of the things I wanted to do in order to Multiplex memory, right? I can share references and there's some clever things that the operating systems will do to try to identify Cases where sharing can be done safely, right? Remember, we don't want, you know We need to preserve the privacy of the address-based abstraction So we might be able to share pages, but we need to make sure that the doing that doesn't violate any of the guarantees That we're trying to establish, right? I can move things around and that's actually going to consume us for the next couple weeks It's talking about when memory stops, when virtual addresses stop pointing to memory What do they point to and how do I use that ability to move things around behind this reference to allocate physical memory more efficiently And finally I can also alter the object behind them without, without changing the reference, right? What's the interface to memory? It's an interesting question people usually don't think about. Wembley Load and store, right? This is not an interface that's provided by the operating system, right? This is a hardware interface This is an interface that's provided by the CPU, right? These are instructions that the CPU will execute on your behalf, right? But what we do with virtual addresses is we change this interface, we change the semantics, we make it much richer, right? And that operates us and participates us, right? All right, so we talked about virtual addresses So when we talk about physical and virtual addresses, just to review, a physical address points to memory, right? A physical address refers to a, you know, a specific byte of physical memory on the machine, right? That holds value and is, you know, cleared it, you know, is able to be loaded and stored it and loaded and stored it Load and stored, I don't even know how to say that, I'm just going to stop trying We're able to load and store to it and it, you know, powers off at runtime And a virtual address points to something that obeys the memory abstraction, but isn't necessarily actually memory, right? And again, we talked about how using virtual addresses allows us to layer on a bunch of additional semantics to physical addresses, to memory addresses, which we liked, right? All right, so who can tell me how base and bounds mapping work? So now we have this abstraction, right? We have an address-based abstraction We've discovered that the way to implement it is by introducing this level of Indirection into our memory addresses themselves, right? So we've come up with this idea of a virtual address. Now the problem is how do we translate this quickly, right? What happens if every translation, you know, let's say we had an architecture that required every memory address to be explicitly translated by the kernel? How, what would that architecture be like? Really slow, yeah, like unusably slow, right? So that would not be good But so what we want is we remember we have this nice We've talked about in this class one of the design principles that operating systems try to adhere to is the separation between mechanism and policy So we want the mechanism here is the actual process of translation The policy is how the translation should happen and what we want is we want a way for the hardware to allow the operating system to set the policy That's used to translate addresses without having to translate every single address by itself, right? So the hardware is going to provide the mechanism operating system is going to provide the policy, right? So base and bounds address translation, this was the first and one of the simplest approaches address sensation we talked about how Does it work? Heaven need a base address, right? And what else do I need? There's two things I need to tell the hardware memory management unit, right? The base address is one of them, right? What's the other one? Yeah, so I need a base and a bound, right? I need a base address, right? And I need a bound, okay? So I assign each process a base for this address and a bound that this determines essentially the amount of memory that this Process is allowed to use, right? We're talking about a single base and bounds per process. How do I translate that address? AJ, okay, how do I do that? Well, remember here, so there this is similar to segmentation, right? But how in base and bounds addressing how do I check if an address is valid? It's a little bit It's even simpler than you think German check the virtual address against the base Does anyone else have a different theory lovely? Yeah, remember that the idea here is that the the base virtual address for every process is assumed to be zero, right? The base is if the base here is a physical address, right? So the bound all I do is I check if the virtual address is okay by just checking if it's less than the bound How do I translate the virtual address, right? So now I have a virtual address. That's okay. How do I complete the translation? Doesn't I add it to the base physical address? Okay So what's nice about this? It's simple, right What's bad about it? What does this require about my address spaces? Let's say remember I wanted to give each process this really nice huge view of memory So if I give each process a 2 gigabyte address space, how big does the physical Address space that I have to give it how big does that have to be? Yeah, so now now I'm now I you know I'm really not able to to provide one of the things I wanted to provide about my address base abstraction, right? Basin Bounds means that you know the bounds is the same for virtual the virtual addresses and for the physical addresses So the size in order to implement an address space abstraction The size of the address space determines the amount of contiguous physical memory. I have to give every process, right? So either I have to make the addresses spaces very small Which I don't want to do or I have to have a huge amount of memory on the system, right? And remember this has happened in like 1960, okay, so you didn't have a huge amount of memory on this Okay, so this is not this is not good, right? And this and this also leads to a lot of internal fragmentation because the address space abstraction encourages Processes to spread out right to use a lot of the you know Let the heap start here and let the stack start a gigabyte Above the heap in virtual address space right in physical now in the physical address space that whole gigabyte is wasted Right because most of it's never going to be used, okay? So so we extended this idea right so we said okay This isn't a good fit for the address space abstraction because it doesn't address the sparsity of address spaces right most Virtual address spaces in my address space abstraction are not ever going to be allocated right if you look at virtual address spaces There are huge You know look at my address space There's huge Swassif it that are never used right and then little bits of data little bits of code little bits of stack right So how can we extend this idea to better? Better match that abstraction Yeah, so I came with this idea of segmentation right I like basin bounds the comparison and the translation are very easy And very easy to do in hardware right but Instead of one basin bounds, which is bad. I just say I can have an arbitrary number of basin bounds per process Then we call each one of them a second Right and we now what we can do is we can essentially take this address space Which is extreme again remember it's extremely sparse most of it is unused and we can cover it with a number of segments Right and now the total amount of physical address space Physical memory that I give to the process is essentially more of a reflection of how much of its virtual address space It's actually used right don't have to cover the large areas that aren't allocated right I can just take four or five or six segments and use them to cover the parts that are in use right? And each one can can grow and shrink independently each one can be protected differently So now each segment remember I I took the basin bounds idea and I had to add one additional thing right So now I need to know the basin bounds We always assume started at virtual address zero with segments. I don't I can't make that assumption So the segment I need to know where it starts in the virtual address space Where it should be mapped to in the physical address space and the size right? So I have a starting virtual address a base physical address and about okay Then my translation works as follows I'd see if they're if that virtual address falls inside of some Segment that's been defined for the process if it does then I can translate it by taking the offset within that segment and Applying it to the physical address right so I figure out What's the offset inside the segment by subtracting out the segment start in virtual address space and as I apply that to the physical address Right so in case people don't remember how this works Well, we can run through our example again, right? So what happens? When I the process tries to translate address 10,000 Yeah Okay, an exception right why? How Right because the MMU knows nothing right like this is the sum total of the MMUs in You know knowledge about the world right? So it says I don't know how to translate this address right so it asked the kernel right now Let's say this and so what does the kernel need to tell the MMU now in order to allow it to translate this segment? Tim Well, there were three things associated with the segment right the three pieces of data that the MMU is going to need to Translate this address Alyssa Right, so I need I need to know where I need to base address right so I need to know where is this in physical memory? What else do I need to know? Yeah Right so the bound which Alyssa got and the start address in the virtual address space right so I tell it The kernel tells the MMU yes in fact there is a segment right so the MMU at this point doesn't know whether This is a valid address or not what the kernel is going to tell it is yes MMU. This is a valid address There's a segment for this process that starts at virtual address 10,000 It's located in memory at physical address 43,000 and at a size 1,000 so Here's something else. I want to point out that we didn't really talk about last time How many addresses so when we started this example how many addresses did the MMU know how to translate? zero right How many does it know how to translate now a thousand right though every address within this segment can now be translated without Causing an exception right so this is something you want to think about when you think about how this works because we've caused this exception Right, we didn't want to cause exceptions right, but in certain cases there's no other way around it Right the MMU has no idea what's what's happening doesn't know how to do this But now the nice thing is the kernel has just told the MMU with three You know with just three pieces of information how to translate a thousand different virtual addresses right so and you know assuming that this process is going to use other addresses inside that segment then All those addresses can now be translated without without requiring that the MMU asked the kernel for help right So again now I know how to translate this I can find this in physical memory and I can complete the you know the loader store How many people remember how this works now? Okay, let's let's do another one right so what happens here goes to the kernel ask for help the kernel tells it Here's another segment. So how many addresses does the MMU know how to translate now? 1500 right for two exceptions. It's pretty good right And remember that The other thing that's going to happen here is that a lot of times So so may ask you a question when a when a process is running right if you looked at the stream of addresses that a Process is is accessing Do you think those that stream is just completely? Randomly distributed all over the address space What would you say? What do you think? That's in your intuition about how processes work Jeremy Well, I'm not even talking about allocation. I'm just saying let's say you were sitting here in the MMU and do what you were just watching for a particular process right and You know you were just looking at the addresses it was using and you and you just kind of map them into its Virtual address space into its you know into the address space that it's using Do you think those like we would you see sprinkles of addresses all over the place? Yeah, I'm only talking about addresses translated by this process. Yeah, maybe the same ones Okay, so that's a start of an answer. Yeah, why yeah, okay. We're getting warmer here Well, okay, so the process will use the same virtual but I'm saying like more on a temporal basis, right? So first of all what what provides at least one stream of addresses that the MMU needs to translate, right? So loads and stores from memory Provide some indication of where the data is that the program is using but what provides another stream of of virtual addresses that the MMU needs to translate Looked up Yeah, what about fetch right fetches and executes So when you start thinking about the code that the process is executing right first of all it all kind of lives in the same part of it Right, but but what's the usual? Cycle when up when a process is running right? How does how does the how is the program counter normally manipulated at runtime? This is like Computer organization review, right? What's the typical cycle for the one that the processor is going through right? Fetch decode and execute and then in the normal case unless I'm doing a jump. What do I do what? No, no I fetch decode execute and then where's the next instruction I'm going to execute? Manish Yeah, it's the program counter plus four right Unless I do a jump right? I'm just you know linearly moving through the you know the code right now There are jumps in your code right? That's how you do program flow but the idea here is that Processes exhibit a fair amount of locality right address locality So the idea here is that let's say I'm running some loop right and that loop is located in this piece of code Let's say there's code here, right? So now not only have I told the CPU the MMU how to you know translate a thousand addresses I might sit there, you know executing like 20 of those addresses over and over and over and over right? I'm in a loop that you know that the loop count is like a thousand or something I'm gonna sit there spitting in this tiny tiny little bit of code and the nice thing is the MMU never needs to ask the Colonel again how to translate those out right so one of the things that helps us here is locality Now wish I had some statistics about locality to show you guys, but but assuming their processes exhibit good locality These types of address translations could be quite efficient right because you know if it was jumping all over the place Right, I'd have to be constantly telling the MMU about new regions of addresses and we're gonna talk about paging today Which makes this even a little bit worse, right? All right, so yeah, we decided to kill the process because it tried to add just that even though dead beast is a perfectly Okay address okay, so any questions about this before we talk about Before we talk a little bit more about segmentation and introduce a new idea Any questions about segmentation in particular? Yeah How okay, it's a great question. How do processes decide what is the physical address? Who thinks they can answer that question? No, Jeremy. I'm gonna know you yeah So the process never knows a physical address. So what does this mean about the process's decision about what physical addresses to use? Yeah, great it doesn't make a decision about it Right doesn't know right if it doesn't know it can't make a decision process doesn't get to say Hey, I would like that piece of physical memory over there. It never knows it doesn't know anything about physical right all the addresses processes ever use our Virtual addresses right processes never see physical memory Okay, it's a good question any other questions. Oh How the okay good okay fair. How does the kernel decide how to allocate physical memory? We haven't discussed that yet, but we but we will yeah, right Yeah, so so first of all when you start so this is a this is a good review, right? So where do a lot of addresses come from right? Where's what's one time during a program's execution where it tells the operating system about a lot of different parts of the Virtual address space that it's going to use. What does this happen? I hear muttering but no answers Paul No Pick on somebody new yeah on earth Okay, when does that when does that happen? We'd be more specific That's correct Nathan, when does that happen when I call exec? Exact is Tells the kernel here's this blueprint of how I want my address space to be laid out right and here's a file that contains a Fair amount of the contents that I want you to put into my address space for right That file has this l format that you guys are gonna learn a little bit about for assignment to not a huge amount But a little bit right so there's this blueprint and the Program tells the operating system. Here's how I want my address space laid out Okay, so at that point the program is essentially telling the operating system I need a segment of this size. I need a segment of this size. I need a segment of this size, right? So the operating system knows what physical memory is allocated and has to somehow figure out Where to find segments of that size, right? So this is essentially an allocation problem, right? It's quite similar to the types of allocation problems that are solved by things like malloc Right, we're actually gonna so I'm gonna come back to your question Because we're talking about today an approach that makes this allocation problem much easier Right that avoid some of the fragmentation problems but in general the operating system needs to keep track of the physical memory that's been allocated and when it receives a Request for memory it needs to find an area of physical memory that it can use to satisfy that allocation, right? This is a good question Any other questions and you guys will write this for assignment 3 for assignment 3 You guys will be in charge of keeping track of what memory is allocated on this, right? And you will have to figure out how to do this, right? Your system right now Doesn't keep track of it at all, right? That's why it runs out of memory and dies, right? It keeps no track, right? You guys will be in charge of making sure that that you can you can track and reuse allocations, right? But there's a nice idea that we'll talk about today that makes this a little bit easier, okay? Right, so we came up with this idea of segmentation and segmentation seems pretty nice, right? It seems like I have this nice abstraction. I can use it It's it's a nice balance between base and bounds the simplicity of base and bounds But it it meets the features of the address space that I want, right? It gives me a way to allocate only the parts of the address space that are actually used, right? So that's kind of nice, right? The the if you think about the pros of segmentation, right? So I can Segments are pretty still fairly simple. I mean base and bounds are simpler, right? But base and bounds also don't do what do we want, right? So with segments The translation requires one addition So I have to find that you think about it You have to think about what the hardware has to do, right? The hardware has to be able to keep track of some number of segments, right? It's gonna be a fixed number because this is hardware, right? So hardware needs to be able to to keep track of a certain number of segments When it translates each address it needs to find the segment that that address is in and then the translation takes one One addition, right? So essentially, you know The hardware does needs to do a search somehow through all the segments that it knows about and then it needs to Once it finds the right segment assuming the segment exists, and it doesn't need to interrupt the kernel It needs to perform one addition, right? And Again, the nice, you know segments are a nice concept, right? So for example when you guys, you know If you guys have right written C programs before you've seen this error about a segmentation violation, right? a lot of modern operating systems despite the fact that they Utilize a technique called paging which we're going to talk about today to actually translate memory Still organize the address space into segments, right? Segments are kind of a conceptual idea that allows me to say this segment is code This segment is data because usually a segment has permissions that are the same, right? So a data segment may be read write a code segment may be read only, right? and This was also a nice fit for our address basis at least it led to less internal fragmentation, right? So it was based on balance there was this huge amount of memory in the address space that was wasted in the segments I got to wait with this, right? But there are still some problems with segments okay One of them is that if you think about it the segment the whole segment still has to be resident in memory so the operating systems decision is now You know should you know first of all as segments get big I have to find large amounts of contiguous memory, right? So let's say that a process has to heap and I found a nice space for it in between a couple of other Allocations and then it says I want to extend my heap, right? I want some more space because you know, I'm running this really cool algorithm I've just allocated a bunch of big data structures using malloc, right? Well now the kernel has to find it a whole another piece of contiguous physical memory, right? And that can become difficult, right? There I might have this you know in external fragmentation problem, right? Where I have pieces of memory, right that add up to an amount that's large enough to satisfy the allocation But I don't have a contiguous piece now. What could I do if I was the kernel? Let's say I had this problem, right? I had a segment that somebody tried to grow and I couldn't grow it because it was penned in by other segments I need their side of it. What could I do? Somebody asked about this last time What what do I have the power to do, right? Remember I've introduced these virtual addresses. That's fantastic, right? So now I what can I do to all of it to any allocation on the system? Yeah, yeah, so I could say hey, you know I'm gonna like run some sort of compacting algorithm on my you know on my memory, right? So I'm gonna take all the allocations out there. I'm gonna start moving them around, right? But what do I have to do in order to move them? I have to change the virtual to physical mappings But what else do I have to do? This is important. Well, yeah, I mean I need to make sure that they don't try to change it But but again what it I wish it was that simple, right? I wish all you had to do was change the virtual to physical addresses, but what else do I have to do? Yeah No, that's not that's going to be very easy to yeah I have to copy the data, right? I can't just like let's say I had a segment of data over here that somebody had had written some really important data into I can't just move it to you know change the mappings and say hey use that memory over there I have to choose I have to make the contents the same right that would be really Unfortunate for processes if when I moved their allocations around the contents changed right that that would break the guarantee I wanted about memory right virtual addresses are supposed to act like memory so when I do a load After I did a store. It's supposed to be the same thing So if I start moving allocations around I have to do all this copying in physical memory, right? Which is you know not necessarily terrible. I might have to do it sometimes, but it starts to sound kind of terrible I guess I said it wasn't terrible, but it's kind of terrible right You know it's potentially high overhead to start moving segments around and now right and I still again. I still have this problem where I have this external fragmentation I wish I had a graphic for this right, but I could have cases where you know I've got a bunch of segments allocated and I have space in between them But I can't I can't satisfy a new segment allocation Without moving all these segments around which is a potentially pretty high overhead thing to do. Yeah, Jim Yeah, but you can imagine like if I had a badly fragmented I mean this is very similar to how disfragmentation works right if I had a bad a fragmented memory space I could stop all the processes and just carefully rewrite all the segments to the beginning of memory and and give myself like one big contiguous piece that I could use that right But that would also be slow, right? So let's let's take a step back right so Really the reason why we started this is we wanted a way to map virtual addresses, right? And we started talking about Ways to do this that would kind of be a reasonable approach given hardware capabilities, right? So we know that hardware is going to have to do most of this itself, right? I still want to be able to tell hardware how to do it And I don't want to have to tell hardware too often, right? But ideally what I would love is to just be able to flexibly allocate any bite in my virtual address space to any physical point, right? So we know that the operating system can't do this, right? And what we've been talking about are ways that the hardware or can assist us, okay? and you know on some level we've been talking about the MMU up until this point as This kind of cash, right? So the MMU has been caching these address translations, right? The MMU asked the kernel the first time it doesn't know something about how to translate a virtual address And then in the future assuming that it understands how to translate that virtual address It doesn't have to ask the kernel again, right? And this is a common trick that we use in operating systems, right? If there's a big thing that's slow, right? Or a certain type of process that's slow in the general case Well, we try to make it as faster in a specific set of cases, you know And one one way of thinking about operating systems and computer systems in general is computer systems are a series of caches, right? So, you know the disk is a cache for memory, which is a cache for the L3 cat Wait, sorry. I'm going backwards Registers are a cache for the L1 cache, which is a cache for the L2 cache Which may be a cache for the L3 cache, which is a cache for memory, which is a cache for your disk, right? Like that's how it works, right? And the speed, you know that I can access those different caches goes You know get slower and slower and slower as they get bigger, right? Registers are the fastest thing on your system to access. You only have a few of them The disk is the slowest thing to access. You have a huge amount of disk, right? So Specifically the piece of hardware I'm going to use here is something called a translation look-aside buffer How many people have heard of have know about TLB's already? Okay, handful, right? So What TLB's are is a form of what's called content addressable memory And TLB's allow Hardware to look up address translations fairly quickly and we're going to use TLB's to implement it to implement a new idea Which is which is called paging, right? so Hold on a sec. I think I might have gotten ahead of myself here Okay, so Okay Let me go forward and see if Okay, so let's talk about pages and then we'll talk about this is a little bit backwards, right? We'll talk about pages and we'll talk about how we can use the TLB to table, right? so So our problem with segments is that segments Segments allow us to map fairly efficiently, right, but they can get too big Right and what so here's the other problem, right? If I have a let's say I have a data the Program has a portion of its address space that it's using in as a heap, right or for data or for its code or something That's going to all be one second So the operating system is such as to find a piece of physical memory for that segment, right? But what's also true about that segment in memory is is every part of that segment going to be equally used Let's say I have a segment that has all my code in As every piece of code the operating that the program could possibly use What percentage of the code that a program could possibly execute is it probably executing in and any given point in time a large amount Small amount Yeah, I mean there's like features of the programs you guys use that you've probably never activated, right? You've never used and never know how it will never use right, you know features of you know Microsoft Word that like our six are buried in six different sub menus, right? And that that part of the code space you're never going to use, right? So now I've got this big blob of code that's sitting in physical memory and Only a small amount of it is is an active use So what kind of fragmentation is this? Okay It's internal right because I have this big allocation. It's sitting there in physical memory I can move it right I might be able to remove it from physical memory and put it somewhere else But what's true about the segment abstraction? I have to move the whole thing, right? So if I have a big piece of code and Let's say I you know I say well I want to reclaim that memory So I'm going to copy that that piece of memory somewhere else, but as soon as the Program tries to use an address inside the segment I have to get the whole segment back into memory right regardless of whether or not it's using big pieces of it So I have this internal fragmentation, right? So segments are kind of too big right, but if I started mapping individual bytes, what would what would happen? Let's say I let's say I decided that my segments are going to be one byte in size, right? I can do that. You know the bound is one for every segment, right? There's a base virtual address and there's a physical address So so I basically what I'm doing is I'm telling the hardware how to map addresses one at a time What would happen remember we talked about before I interrupt the kernel the kernel tells me something and then I can run for a while, right? What would this mean nothing? Yeah, so remember the hardware can only store a fixed number of these translations, right? So let's say the hardware can store 32 translations and each segment is a thousand bytes is on average a thousand bytes So now I can map 32,000 bytes of the virtual address base without requiring the kernels help now I make the segments one byte, right? So how many bytes can I map now? 32 right if I'm lucky the process is sitting in a teeny weeny little loop, right? Act that axis is like 16 memory addresses, but that's not very common, right? So is the segment, you know is the size of the amount, you know essentially if you think about it what we're doing is we're telling the MMU how to map regions of addresses, right? As those regions get small the number of times the MMU has to ask for help goes up, right? Is those regions get really big the internal fragmentation that occurs inside those regions gets gets back, right? Does this make sense? This is kind of like the big trade-off that's involved in memory management, right? As the regions get small I have to tell the hardware all the time what to do, right? As the regions get big I have huge chunks of physical memory that have to be resident, right? Because any address inside of them could be being used because the hardware knows how to translate it But large portions of it are not going to be used, right? So what we do is we come up with something we try to choose a fixed Granulary, right? What we're going to do is we're going to say we're going to try to choose a middle ground You know a middle ground that's what we want is a unit of Virtual memory that's big enough That the hardware can translate large amounts of virtual address space efficiently But small enough that there's not too much internal fragmentation. It's going to take place, right? The other thing is by choosing a fixed size, right? We're also going to choose a fixed size, right? We're going to say, you know, essentially every segment has a fixed size, right? So we're not allowing segment size to vary anymore, right? What's nice about having fixed size segments? What does that make easy? There was this problem before that we had about What that we're going to be able to solve now. Yeah It makes kernel allocation really easy, right? Because remember before I had this problem where if my segment size changed I Might have to move it and move other things around, right? Now my segment size never changes So what I go what I'm doing is I'm allocating fixed units of memory, right? I break up the memory into fixed units and I can if I have a fixed unit of allocation I can never have External fragmentation. Yeah It does mean that right so now what I'm going to do is I'm going to say the bound is fixed, right? So the hardware doesn't have to know the bound, right? That also makes translation a little bit little bit easier, right? So now let's go back and talk about the TLB. I'm sorry. I got this backwards. I don't know why I ever thought this made sense this way So so right so the nice thing here is is we can use this Content addressable memory in a clever way in order to do this virtual to physical translation, right? so so what we're going to do now is we're going to identify identify Areas of virtual and physical memory By there what we call them. We call them page numbers, right? So I take physical memory. I divide it up into a fixed size, right? It doesn't matter what the fixed size is for this example, right? Could be anything it could be a bite It could be a thousand bites, but it's fixed, okay? Now the nice thing is what I care about is what page of virtual memory is this particular address it, right? And it turns out on modern systems. Usually I use a page size of either 4k or 8k The nice thing about using a power of 2 it is allows me to lop off some bits to get the page, right? But let's say let's say I'm doing this right so I have a virtual address that comes in right and it's in page 800 What content addressable memory allows me to do right? This is what TLBs are very efficient at Well content addressable memory does it essentially searches all of the entries in this buffer in parallel so there's a fixed cost to kind of And again, you can just you can read about TLBs and hardware if you're curious about how this is actually implemented, right? But the idea with the TLB is that there's a fixed cost for looking up this address in the TLB And in some sense it is searching all the addresses in parallel hardware, right? So what's going to happen is it's going to match to this address, right? And then what I'm going to do is I'm going to say virtual page 800 for this process maps to physical page 306 This is how I use this particular feature of hardware to help me translate page numbers, right? I want to slow down a little bit because I feel like I'd be wielded maybe some people so So again because we have this problem with cans, right? We're going to We're going to choose a fixed segment size that's big enough to allow the super so for example I could have done this on bytes, right? Let's pretend that we've decided that we're going to translate byte to byte, right? A byte of virtual address to a byte of physical address, right? So again, here's my virtual to physical mappings. How many bytes of memory does this cam know how to translate? four Right and in modern systems, I think cams are usually limited to maybe like a thousand addresses, right the the the size of this particular piece of hardware scales Exponentially with the number of addresses that it's able to translate, right? So you can't just make one of I mean it would be great if you can make huge Content-addressable memories, right? They'd be really useful for a lot of things, right? But it turns out that to make them fast the the the complexity of the cam scales Exponentially with the number of addresses, so we can't make them arbitrarily large So if let's say my again, let's say my segment my fixed segment size is one byte So there's four bytes that this that this TLB knows how to translate So let's say I wanted to use this TLB to translate 16 bytes, how would I do that? Yeah Yeah, so if this if each one of these addresses identifies four bytes of memory, right? Now my cam can translate 16 bytes of memory, right? Let's say I wanted it to translate 32 kilobytes of memory How big would each segment have to be? This is Fairly simple now 8k right if each one of these virtual segments is 8k now I can translate 8k worth of worth of memory, right? So Okay, so we talked about pages, right? The the nice thing that pages that page sizes also do is Fixing a page size also limits the size of kernel data structures that are associated with managing memory, right? Because if you imagine I so if I have my virtual address space Let's say my virtual address space is 4 gigabytes and I break it into 4k pages How many what's the maximum number of pages that could be present in a given processes address space? like a million Rough and the other thing that help so the thing that we come back to that helps us here, too Right is that execution locality and process locality really makes this work, right? because the the Hardware is never going to be able to translate large portions of the process address, right? It just becomes too inefficient to do it quickly and hard, right? There's a trade-off in hardware between the amount of address space I can translate and the speed at which I can perform each individual translation So what I've done is I've came up with the compromise but again the nice thing here is that most processes You know they do jump around within their address space But you know while they're executing they're basically loading and storing from the same thing and then they might jump And I might have to reload a new translation and then they're going to execute there for a while And then they might jump somewhere else, but there's a fair amount of memory locality, right? So as I said before 4k is a very very common page size, right? Nobody really knows why this is true, right? This page size was picked probably in the 1970s and But you know it seems pretty robust like there are some systems now that are using 8k pages or sometimes even larger pages like 64k pages I think some systems have support for much bigger pages I mean modern systems have a lot more memory than they used to right so it's kind of interesting that we're still using roughly the same page size Right, but but there hasn't been a huge amount of changes So again if you imagine that I have 4k pages and I have 128 entry to B now I can catch translations for a megabyte of virtual memory, right? That's pretty and that's pretty good, okay? And again, we think of pages as fixed-size segments, right? I think I have an example here Okay, so now how do we do page? Translation so let's walk through this and then and then we'll be done for the day. I think so I'm going to break up my address space into Let me just get through this and then I'll let you guys go and we'll come back to this on Wednesday So I'm breaking up my address space into fixed-size pages. I'm breaking up physical memory into fixed-size pages, okay? Every virtual page in my address space that's valid maps to a physical page Or it might be somewhere else But the point is that if I'm using it it has to be in memory somewhere and there's a page to page mapping Right remember essentially I'm thinking of everything now in units of 4k right 4k of memory So If I'm going to translate an address inside a page It turns out that the page number of a 32-bit address is now just the top 20 bits right again These are fixed-size segments the bottom 12 bits identifies the offset into my fixed-size segment or my page Right the top 20 bits identifies the virtual page number, right? So what I do and we're going to talk about address We're going to talk about different data structures to do this right is the kernel if the MMU doesn't know whether or not this virtual pages Ballot it's going to ask the kernel and the kernel has to figure out if there's a physical page translation for this virtual page Right, so we're going to talk about data structures for doing this right and Now what I do is I look at the physical page I take the physical address I take the physical page number and I add on that offset right so this is very much like Segmentation right this is like segmentation with a fixed segment size which is 4k, right? It just turns out that because the segment size is 4k I can do this fancy, you know bit masking to do what I want to do right This is essentially like taking this and subtracting off the offset in the react right, but I can do it using using bitmasks, right? Okay, so on Wednesday, we'll start with this example and we'll get through paging and we'll talk about some kernel data structures that are used to make paging more efficient you