 to today's lecture on virtual memory. So, we have discussed how the performance of memory system can be improved by organizing memory in a hierarchical manner and we have focused our attention on cache memory and also we have focused our attention on main memory. So, we have discussed the hierarchy between cache memory and main memory. Now, we shall go for another level of hierarchy that is between main memory and the disk memory. So, this is the outline of the lecture. First I shall discuss about why do you really require virtual memory. Then the important issues related to virtual memory we shall discuss and as we shall see it will involve a particular approach known as demand paging and we shall discuss about virtual to physical address mapping. It will require some address translation mapping of virtual address to physical address. Then there will be page faults, page replacement and page table entries. Possibly I shall be able to cover all these topics in today's lecture. The remaining topics translation look aside, buffer, page table organization, replacement algorithm, memory management, these topics I shall cover in the next lecture. So, let us first try to address the question why virtual memory? We know that our observation is that a large number of programs or inactive programs are stored on the disk. That means, a disk can be considered as a kind of repository where you store all different types of programs, but some of them very few of them are active at a particular instant of time. That is our observation and a program must decide in the main memory file they are running. This is another important restriction that is that whenever you are trying to execute a program it must be main memory resident. That means we have large programs, large number of programs stored in the hard disk, very few of them are active which needs to be present in the main memory. This is our first observation. Second is let us consider the multiprogramming scenario which is very common in present day environment. That means most of the computers are running in this multiprogramming mode and whenever we are running in multiprogramming mode then we shall be having large number of programs by different users. Now the question is which programs will share memory with others is not known at compile time. So, at compile time whenever we are compiling a program of a particular user, we do not know which part of the memory will be shared and how it will be shared with other programs. That means that the it is not known at run time only it is known at run time. Now at compile time this is not known because it changes dynamically as programs are running. So, programs are running I mean there will be context switching a program will be taken out of I mean that processor will not execute will be suspended then again it will be program will be executed will be resumed because we are using in a time sharing I mean time division multiplexed manner. So, question is how do we ensure sharing as well as protection. So, this is a these are the two conflicting things we want to protect one users program from another users program. At the same time we also want that there should be a kind of sharing if necessary if required. So, this is the these two conflicting things are to be satisfied with the help of this virtual memory. Second observation is formally the size of a program is larger than the main memory size it was up to the user to take care of it. So, if you go back to the history of the development of computers we will find that earlier it was the responsibility of the programmer or user to load the program in the memory and to fit it in the main memory. So, in such a case there is a restriction that means if the size of the program is larger than the main memory then what will happen? Can it not be executed? So, what was done the programs were divided into pieces and identified the pieces that were mutually exclusive. That means the programs which were mutually exclusive I mean the programs which are the only some of the programs will be loaded in the main memory not all. So, these overlays were loaded and unloaded under user program control during execution that means it was the user the user was responsible to swap in and swap out of their programs from the hard disk to the main memory as and when required during the execution. So, this calls and particularly when calls between procedures in different modules should be lead to I mean whenever you have got calls between procedures in different modules actually those programs that which you divide into mutually exclusive manner those were called different modules. So, if these there is calls between procedures in different modules that would lead to overlaying of one module with another and obviously this imposes a substantial burden on the programmer. So, this was a scenario in 80s and you can imagine what was the burden on the programmer fortunately now that burden is overcome. So, and virtual memory has been is now available in all computer systems and it satisfies two important requirements number one is to fulfill the dream of unlimited memory size. That means to the programmer user it gives illusion as if the size of the main memory that it has got or size of the memory that it has got is unlimited. In other words there is no restriction on the size of the program. So, it allows you to execute programs of any size bigger than the main memory and to allow efficient and safe sharing of memory by many programs and essentially it is up to support multi programming this is essential. And this can be achieved as I told by swapping in and swapping out of programs but that should be transparent to the user. So, user will not do it the virtual memory system will do it automatically. So, we can say that I mean how really the virtual memory is implemented. First thing is does is it uses physical DRAM as cache of the disk, cache memory of disk. We have seen that the cache memory and main memory hierarchy you have got main memory and a part of that was stored in the cache memory and which is obviously faster and smaller than the main memory. So, that is how the cache memory main memory hierarchy was implemented. Now, we are going for another level where this in place of main memory we shall be having disk memory and main memory will be considered as the cache of the disk memory. So, that is the basic idea and address space of a process can exceed physical memory size. So, that means that restriction that was that a particular program need not be bigger than the main memory that restriction is no longer arises and some of address spaces of multiple processor can exceed physical memory. So, we are taken care of two things. Number one each user each user's program size can be bigger than the main memory size. Moreover, the sum total of multiple user's program size will definitely be bigger than the size of the main memory and it will simplify memory management multiple processes resident in the main memory. So, that means simultaneously multiple processes will be kept in the main memory and each process with its own address space. That means each of these process coming from different users will have its own address space and that we shall call virtual address space as we shall see and only active code and data is actually in memory. That means we are not loading all the all parts of different user's program in the main memory only active code and data are actually in the memory as we do in case of cache memory. So, allocate more memory to process as needed that means since it is dynamic as we need more and more main memory whenever we are executing program that can be dynamically increased more memory can be assigned and it will also allow you provide you protection one process cannot interfere with the other. That means it will be through the virtual memory user will not do it directly because they operate in different address spaces and user process cannot access privilege information. So, this is another important aspect. So, different sections of address spaces have different permissions we will see that later on we shall discuss about it and permissions for different parts can be different like read only access only like that. So, we can in nutshell we can say that this virtual memory was developed to alleviate the burden of the programmers and a technique that uses main memory as a cache of secondary storage and it will indeed share the basic concepts of cache memory, but using different terminology because of historical reasons the different terminologies were used in the context of virtual memory and those are different from the terminologies that are used in cache memory as we shall see. However, it will involve two important steps or issues mapping translation of virtual address to physical address and management control sharing and protection and protection in multiprogramming environment. So, these are the two basic issues to be dealt with and as we shall see mapping techniques can be broadly divided into two types, paging in fact a special type of paging known as demand paging and segmentation which we shall discuss in my next lecture. Now let us have a feel about the different sizes as we use the main memory as cache of disk. So, full address space is quite large if we use 32 bit address then we have got 5, 4 billion bytes that means 2 to the power 32, 4 billion bytes are to be stored and whenever we go for 64 bit address as it is true for 64 bit processors then the size of the full address is quite large 16 quintillion bytes. So, you can imagine the size of the address I mean with the help of 32 or 62 bit address that can be generated. And another observation which I mentioned earlier that disk storage is much cheaper than the dynamic RAM storage. So, and particularly 150 times cheaper than dynamic RAM for example 8 GB of dynamic RAM will cost may cost 10,000 dollar and 8 GB hard disk may cost only 64 dollar. So, these are the typical prices with time these prices are going down however that ratio remaining more or less same. So, to access large amounts of data in a cost effective manner the bulk of data must be stored in memory. So, this gives a motivation that we can have very large disk, but relatively much smaller dynamic RAM and most of our data and program should reside in the disk because we can store it in the disk in a much more cost effective manner. So, this has led to two levels of memory hierarchy one is between cache and main memory another is between main memory and hard disk. And as you can see the two levels of hierarchy is depicted here you have got CPU which primarily accesses from cache memory. And the transfer that can take place is may be 8 byte transfer between CPU and cache. And on the other hand between cache and main memory as we have seen multiple words get transferred. So, may be 32 4 words so 32 bytes are getting transferred. On the other hand if we look at the hierarchy that is that we are discussing today virtual memory the transfer will be 4 kilobyte that means later on we shall see we shall be using a term called page a page will be transferred not a block that means page is considered to be a block in the virtual memory hierarchy. And these are the typical values the size of register as we know 32 byte it can be I mean size of a register then 32 bit it has to be bit then cache memory size 32 kilobyte to 4 megabit main memory can be 128 megabit this can be 32 gigabit so speed varies 32 nanosecond whenever the processor is accessing from register whenever it is accessing from cache memory it will be may take one or two cycles 6 nanosecond from main memory 60 nanosecond. But from disk it takes enormous time 8 nanosecond so it is very large. So, we may say that the miss penalty for this virtual memory is too large and cost as I have already told is becoming smaller and smaller as we go from cache memory to disk. And line size is also changing as we can see 8 byte 32 byte and 4 kilobyte that means block size and phase size. So, as we are moving from one hierarchy to another hierarchy we are going for larger slower and cheaper memory. In the early years when virtual memory was not used was not prevalent in those days the process the CPU was directly generating the physical address. So, example is most of the clay machines early PCs and nearly all embedded system does it. So, virtual memory is not present were not very concept were not present in the early systems. So, address is generated by the CPU point directly to the bytes in physical memory. However, in the present system as we move to virtual memory system and nowadays all the workstations servers modems PCs modern PCs all use virtual memory system where you have to go through an intermediate step that intermediate step is known as address translation. So, CPU will generate virtual address and before that address is the address is presented to the main memory we have to go through a translation process that is known as address translation and that hardware converts virtual addresses to physical addresses via an operating system manage lookup table and that table is known as page table. So, a page table is shown which really maps virtual address to physical address and that physical address will be presented to the main memory. Of course, there we can have another hierarchy as you have seen cache memory, but we shall refer it to main memory only and the address in the page table not only it will refer to main memory it may refer to the disk whenever it has not been transferred to the main memory. So, in the page table there is provision for keeping information about the main memory address as well as the disk address where it is kept in the disk and these are those typical comparison parameters I have already mentioned block size, first level cache 16 to 28 bytes virtual memory it is pretty large 4k to 64k hit time as I said 1 to 2 cycles and virtual memory may take 40 to 100 clock cycles and miss penalty as you can see is very large in virtual memory 100,000 times larger than the cache memory and access time is also quite large. However, only good thing is that miss rate for virtual memory is correspondingly much smaller and data memory size as I have already told increases in case of virtual memory. These are for the purpose of comparison and as I mentioned there will be differences in terminology we shall be using page in place of block and we shall refer to cache miss by page fault in the context of virtual memory and other differences are placement on cache memory misses by hardware that is done in case of cache memory as you have seen where there is in case of virtual memory it will be handled by software operating system. So, the size of processor address determines the size of virtual memory where as cache size is independent of address size. So, we have seen that CPU will be generating an address so CPU will generate an address and that address size will determine the size of the virtual memory because that address will be I mean you cannot really change that size because CPU will keep on generating the address and bigger than that you cannot have virtual memory but in case of cache memory we have seen it is independent of the address size because the address size in case of physical memory can be smaller can be different of course in the earlier systems there was no distinction the virtual that the address that was generated by the processor was directly applied to the main memory but in the present day context it will be different. So, virtual memory can have fixed of variable size blocks that we shall see that page size can be user defined can be variable. We have seen in case of cache memory the size of a block is decided by hardware that means hardware is responsible for deciding the size of a block and accordingly the cache memory is implemented but in case of virtual memory that size of a block that is your page it can be variable user or programmer can changes and it can vary from say 4 kilobyte to may be 16 kilobyte and nowadays people are still using it still bigger page sizes it can be 32 kilobyte or even 64 kilobyte. So, I have already discussed all these issues. Now let us consider what are the important design issues based on what we have discussed so far number one is page size what should be the page size the page size should be large enough to try to amortize the high access time. We have seen since it is in the hard disk that size of page can be variable but what should be the size it should be large why it should be large the reason for that is it should amortize the size of the high access time and as I mentioned the size can vary from 4 kilobyte to 16 kilobyte and organization that reduces page fault rate is important and that means the miss rate which we know which we call at page fault rate that should be small and you have to design in such a way that page fault rate is reduced how can it be reduced it will be dependent on the design of the virtual memory and one important parameter as we know we have seen that associativity is a important parameter that can affect the miss rate and as we know there are three possibilities and as we know there are three possibilities direct mapping or it can be set associative mapping and fully associative mapping in the context of virtual memory as we shall see we shall always use fully associative mapping so that the miss rate is reduced that means fully associative placement in pages of the memory what does it really mean that means that the page that you transfer from the hard disk to the main memory can be placed anywhere so that is your fully associative placement in pages so that is how the virtual memory will be organized then page faults did not be handled in hardware we have seen in case in the context of cache memory the whenever there is a cache miss it is handled by hardware but it cannot be handled by hardware the reason for that is if we use software it can use clever algorithm for page replacement to reduce the page fault rate that means whenever we are using hardware it has to be simple we cannot use complicated algorithm sophisticated algorithm for page replacement so page replacement has to be done in such a way because we are using fully associative mapping there it will involve replacement of pages and page replacement has to be done so that a particular page which is replaced is not required in near future that can be achieved by with the help of sophisticated algorithm and that will help to reduce page fault rate. Another important design issue is write through approach cannot be used we have seen in the context of your in the context of cache memory we normally use write through policy why write through policy can policy can be used in the context of cache memory the reason for that is the difference in speed is very small so instead of one or two cycles for reading from cache memory it may take may be say 60 to 100 cycles whenever it is read from main memory so the difference in speed is not very large may be 10 to 100 times but in case of in case of your virtual memory we have seen it is 100,000 times slower so that requires that you have to use write back approach. Moreover a particular page is quite big it can be 4 kilobyte to 16 kilobyte let us assume so there is a possibility that you will be performing many writes in a single page before it is replaced so write back policy will allow you that that means you bring in a page keep on writing into it many times there is no need to write it in the hard disk you keep on writing in the main memory as long as it is present it is not replaced so that is why write back policy is the suitable approach for virtual memory so these are the basic design issues and we have got answer and what type of things to be done in virtual memory that we have discussed. Now, let us focus on the mapping aspects we shall be we have to do virtual address mapping to physical address how exactly it is being done so a processor generates a virtual address it has got two components virtual page number and page offset so this is what is generated by the CPU CPU will generate that this is the CPU CPU will generate virtual page number VPN plus page offset and then the number pages addressable with the virtual address is virtually unlimited because that number of bits present is quite large whereas the number of pages addressable by the physical address is limited because you have got a very limited number of such pages in the physical memory so mapping from virtual to physical address is necessary and you will do a kind of translation so this virtual page number will go through a translation suppose we are using N bit address out of which P bit corresponds to page offset and remain P 2 N minus 1 corresponds to virtual page number so this is the virtual address that is generated by the CPU and the page offset directly goes to the physical page number so physical page number is divided again into two parts one is your page offset that directly comes from that virtual page number and here we perform a translation so it need not be same as your N it can be smaller than that so let us give a number M minus 1 so you have got M bit physical address I mean M bit physical address size and having 2 to the power M bits of the power main memory you can say and this is your physical page number so virtual page number is generated by the CPU there is a translation that translation is generating the physical page number and this is concatenated with the page offset to generate the physical address so this is the physical address this has to be done with the help of a table memory resident page table and as you can see virtual page number is generated by the CPU which is applied to a page table that page table does the translation from the virtual page number to physical page number and here as you can see we have added a bit known as I mean valid bit why this valid bit is necessary the valid bit is necessary because only those pages which are required by a program will be transferred to the physical memory or main memory so when the valid bit is one they will that page table will point to the physical address so here you will get the physical address on the other hand those addresses corresponding to I mean when the valid bit is 0 they will they will be pointing to a particular location in the hard disk so you can see the main memory resident page table can have I mean it will point either to the physical page or the disk address so it will have two different possibilities that is present in the page table and this is how actually it is being done you can see this is the page table base register so as you are storing the page table in the main memory and there is a page table base register which points to the base address of the page table so with respect to that virtual page number provides you a kind of index so this virtual page number will act as an index to that page table and with the help of that you will get the physical page number and you can see in addition to the physical page number it has got two more fields valid bit so if the valid bit is 0 then the page is not in the memory that means it has not been transferred to the main memory it is still present in the hard disk we are assuming that a program is always present in the hard disk and from this table we shall be getting that physical page number and with that the page offset will be concatenated and this physical address will be presented to the main memory and of course cache memory will be there as part of the main memory system so reading a word from memory involves translation of the virtual address to physical address comprising frame number and offset using the table called page table actually this is called the page frame number this that physical page number which is called the page frame number how the page table operation takes place is illustrated here first it goes to a translation of operation it is separate you have got separate page table per process as we know you can have different processes that is present in that is I mean simultaneously getting executed by the CPU of course in a time division multiplex manner so for each of the process there will be separate page tables and virtual page number forms index into the page table points to a page table entry as we have seen then it will do it will compute the physical address page table entry provides information about page and of course it valid bit is one then the page is in the memory and that will use physical page number to construct address as we have seen with the help of the previous diagram and if valid if the valid bit is 0 then the page is not on disk so page fault will occur and then you must load the page from the disk into the main memory before continuing so this is how the page table operation is will continue and this is illustrated with the help of this diagram so here you can see the virtual address is generated by the processor comprising n bits and this is that page table pointer that base address it is providing so then this virtual page number with the help of that it is getting indexed and you are getting an entry in the page table where you have got a page frame number and that page frame number is the physical address page number where the physical page number is stored and that physical address and that page offset is concatenated together and that is presented to the main memory so this is the this is how the address translation in the phasing system works as you can see it involves the I mean program is generating the virtual address then the phasing mechanism will generate the physical address and that will ultimately go to the main memory this is how it will work so you can see here that virtual address is 20 bit let us assume and the offset is 12 bit so this 20 bit virtual address is telling which page and of course that which page of this page table and you can have 1 m entry in the page table because you are using 20 bits so that page table is quite large as you can see and that 4 byte in each entry so 4 megabyte is the size of the page table itself so this tells you that whenever we are using virtual addressing that you have to store the page table of each user so you can imagine that you will require 4 megabyte page table for each user not for one process one process for all the processes that has been created will be having such a page table and then this you will be having the physical memory page that will be in your main memory and this page offset will give you which byte of this page and of course address of the memory page will be obtained from the page table that where that page frame number is stored that will give you the address of the memory page and this essentially shows you a particular page and that particular page will be having the data that is requested for that is your which byte that is available in the main memory. Now, we shall discuss about one important aspect that is your page fault we have seen that a particular data may be present in the main memory or may not be present whenever it is not present in the main memory this will lead to a page fault that means as we have seen what if an object is on disk rather than on memory. So, if it is not in the main memory that will lead to page fault and page table entry and entry indicates virtual address not in memory we have already seen that in the page table we shall be having an entry valid bit that valid bit will tell that page table that particular page is not present in the main memory. So, operating system exception handler invokes is invoked to move data from the disk to memory. So, here it will be taken care of by the operating system and current process is suspended and other programs other processes can resume and operating system has full control over placement. So, you can see we are using a combination of hardware and software operating system and some software hardware and this is the situation when before fault you can see these two are mapping to hard disk the address is corresponds to hard disk. Now, the CPU was pointing to this particular address this was the virtual address and this was since it was not in the main memory that means that it was 0 and virtual address was I mean that the disk address was being stored here. Now, after the control goes to the operating system and it handles the page fault then after the fault the scenario changes that means you are in the page table you are having valid bit and this is the page frame number this can point to main memory and whenever it is 0 it will point to hard disk and whenever it is serviced this will become 1 because this corresponding page has been transferred from the main memory to the hard disk to the main memory and it will be converted to 1 and corresponding that page frame number corresponding to the physical memory will be present in the page table. So, page table will be updated and this is what is being shown here that means that page table has been updated and 0 has been converted to 1 and that particular address is no longer pointing to the hard disk, but it is pointing to the main memory. So, this is the details how exactly it happens operating system takes control when the valid bit of the physical pad disk is 0 and how does it finds the requested page on the disk that is another question when a process is created the operating system creates space for all the pages of a process in the disk. So, in the disk all are present and this disk space is called swap space and it also creates a data structure to record where each virtual page is stored in the disk. The data structure that we are referring to is essentially the page table and this data structure may be a part of the page table or it may be separate, but whatever what we have shown right now is essentially the page table and operating system also creates a data structure that tracks which processes and which virtual addresses use the physical page. So, as we have seen in this this is your data structure that takes care of it and if all the pages in main memory are in use then operating system must choose a page to replace the to replace that page based on the past history. So, this here it will be necessary to change the that means if this particular page which is being used this page if it is this page has to be replaced and a new page has to be taken from the hard disk to the main memory. So, the replaced pages are written to the swap space of the disk. So, what we are doing we are transferring between main memory and cache memory main memory and hard disk and this is how the swapping is taking place. So, this particular slide elaborates how exactly it takes place. So, number one step is initiate block read processor signals controller read block of length p starting at disk address x and stores starting at memory address y and read takes place whenever there is a page fault then you have to read it from the hard disk and as you know we normally use direct memory access DMA. So, a DMA technique is used that DMA approach transfers from the hard disk to the main memory. So, in step two the data from the hard disk is getting transferred to the main memory with the help of the IO controller. So, this first step was that was initiated, but since it was page fault we found that it is no longer present in the main memory. So, from the hard disk it was read and it has been transferred to the hard disk and then the third step is IO controller signals completion that means after the DMA is over DMA direct memory access data transfer is over then it interrupts the processor indicating that the IO controller interrupts the processor indicating that. So, here is the interrupt signal going to the processor indicating that the page fault has been serviced by the operating system and then operating system resumes the suspended process. So, in the third step since the data is now available in the main memory then it resumes its operations. So, this is how exactly the page fault servicing takes place with the help of the operating system. Coming to the page replacement find a free frame to store a desired page if there is a page frame use it. So, that means it has to find out whether there is a page frame already available in the main memory or there is no page frame. So, if it is not there then use page replacement algorithm to select a victim frame. So, write the victim page to the secondary storage and change the page and page table accordingly. So, you have to modify both the page and the page table entry. So, the read the desired page into the free frame and change the page and page table and you will restart the user process. This is how the page replacement will take place with the help of the operating system. Now, another important aspect is write miss. So, as I have already mentioned we cannot use the write through approach because size of the page is quite large and because of the you know about the locality of references both the temporal and spatial because of temporal and spatial locality. It is very likely that subsequent accesses by the processor will take place from nearby memory locations. So, the page that has been transferred will be accessed may be in near future that is the reason why we use write back policy. However, you will require an additional bit known as dirty bit. So, page table requires an additional bit known as dirty bit. So, in addition to the virtual bit that you have already mentioned you will require a dirty bit in the page table. So, dirty bit means whenever you have modified you will make it 1 that means you have written into it you will make it 1 if it has only read it then it will remain 0. That means initially whenever you transfer it then the dirty bit is 0 and whenever you perform writing into the main memory then it is converted to 1. So, as I already mentioned many writes can take place before a page is replaced. So, once a dirty bit is set you can perform many writes without modifying anything. Coming to the protection as I mentioned you have got many users many processes present I mean the programs of many users are present or many processes are concurrently getting executed by the processor in a time division multiplex manner. So, many processes will be present in the main memory. How one users program will be protected from other users program that is achieved with the help of protection bits. So, access write field indicates allowable access that means read only read write execute only. So, typically support multiple protection modes that means that kernel versus user will see that these modifications can only be done by the operating system not by the user. That means here in addition to this virtual bits you will be having those access write bits which can be say read only or it can be read write if you allow read write it can be read write or it can be execute only this type of flag bits can be stored in this access write field. So, you can have several bits present there. So, you can say this is your access write. So, you can see the page table will have several other bits. So, protection violation fault if user does not have necessary permission that means whenever you are trying to read a data from the main memory not only you have to check whether it is present in the main memory or not you have to also check whether the corresponding that means the corresponding permission is available. Suppose you are trying to write, but the access that is provided is only access only. That means you cannot perform write. So, in spite of the fact that a particular page is present in the main memory you cannot perform write because it is violating the access write. So, it is leading to additional complication. So, based on what we have discussed so far we can see that the page table will have the following entries number one is address pointed to location in page table or if page is swapped can be disk address instead of control bits. Then valid or present bit if said page being pointed to is resident in the memory we have already mentioned about it valid bit modify or modified or dirty bit set if at least one word in the page has been modified. So, page may have a large number of bytes, but even if a single byte has been modified you have to modify then dirty bit will be set to one. Then reference bit set if page has been referenced with either read or write. So, used to support software replacement later on we shall discuss I mean it is essentially implementing kind of LRU list recently used. So, with the help of this list recently used a simplified version of LRU is being implemented then you will be having protection bits used to restrict access as I have already mentioned read access system only access that type of protection bits can be added in the protection field. So, with this we have come to the end of today's lecture we have discussed about various aspects of virtual memory particularly why virtual memory. Then the different issues related to virtual memory and particularly we have elaborated about the translation process which involves translation of the virtual address to physical address. Then we have discussed to also two other important issues like whenever you have to perform replacement then whenever you go for replacement what type of techniques can be used later on we shall go into more details and in case of write operation what are the additional complication that can arise that we have discussed. Thank you.