 to the 10th lecture in the course design and engineering of computer systems. In this lecture, we are going to study the basics of what are virtual machines and containers. Let us get started. So the story that we have seen so far, the mental model we had is that there are user programs which run on top of a system software like the operating system which manages all the underlying hardware like the CPU, main memory, IO devices and so on. But sometimes what we want to do is we want to virtualize the system even more that is we want to have the underlying hardware, but we want to run multiple copies of the entire system stack on top of it. That is we want to run different operating systems each with their own applications and all of these operating systems are sharing the same underlying hardware, right. So such instances of systems that we run are called virtual machines, right. So this is a common requirement, we want to run multiple virtual machines each with its own operating system on the underlying hardware or the underlying physical machine. So why do you want to do this? You want to do this to utilize the hardware better if your CPU is lying idle due to one system you might want to use multiple systems on the same underlying hardware to utilize it better at the same time have better isolation from each other, right. If you just run multiple threads to use the hardware better then you are not isolated from each other. So you want good sharing of hardware at the same time good isolation more than what you just get with processes or threads. So one way to do it is run multiple virtual machines each with their own underlying guest operating system. Another way to do it is use what are called containers which are lightweight virtual machines. And there is a renewed interest in virtual machines and containers today due to what is called cloud computing because these things form the building block of cloud computing. So let us see what is cloud computing. So a cloud is nothing but a large data center or a warehouse that has multiple pieces of hardware like servers and storage all of these connected by a high speed network, right. So a cloud is nothing but a large set up of a lot of hardware like CPUs. And this is usually set up by public cloud providers like Amazon, Microsoft, Google all of them have their cloud offerings, right. This is a public cloud that is set up by one of these big players and this is made available to other users for a payment on demand. So you as a user can use this hardware after making a payment, right. So this is called cloud computing and there are many ways in which a user can interface with the cloud. So these cloud providers can just give you access to the hardware like they can give you like a virtual machine that is running on one of the cloud servers and on that hardware on that infrastructure you can install your own applications, right. So this model is called infrastructure as a service where the cloud provider is only giving you access to the hardware and you are running your software on it. A different model is the cloud provider will set up both the hardware as well as a software platform. For example if you want to run a web application then the cloud provider might give you a software framework using which you can build web applications and using the APIs you can build your application and run it on the cloud, right. So the cloud provider is now giving you hardware plus a software platform. But still you build your application using the APIs. This is called platform as a service. So the thing is going even further the cloud provider is giving you the hardware as well as the complete software not just APIs using which you build the software but the complete software is also provided by the cloud provider that is called software as a service. For example if you use Google documents on the cloud, right the software is running on the cloud is set up and managed by the cloud provider and you are using it. So you can just use the hardware, you can use the hardware plus customize the software using the APIs or you can log in and use the full software plus hardware bundle on the cloud. These are different ways in which users can use a public cloud and these multiple users also called tenants who share the cloud server that sharing is done via virtualization. For example, if different users are running their cloud applications you want to separate them isolate them as much as possible and so each user is given a VM or a container on the cloud. So there are the traditional way of building applications is you know your application all the different components you run them directly on the hardware servers that model is called the bare metal model, right. You run your applications on bare metal directly on the hardware. A different way that is being done today is to run applications on the cloud, right. So a cloud can either be a private cloud that is you know you have set up some hardware a lot of servers you know different servers each with a lot of CPU cores and memory and all of that you have set it up that is a private cloud within your organization or it can be a public cloud that somebody else has set up the hardware for you and you are accessing that, right. So applications can run either on a private cloud or on a public cloud. And on this cloud hardware you can have multiple components of your system you know your front end, back end, web server, database all of these components can run on this underlying cloud hardware, okay your applications running on the cloud hardware. So what is the advantage you might ask I mean instead of why do not I just run bare metal? Why run it on the cloud? That is because usually this cloud hardware is managed by a cloud management or orchestration software. So there is another layer of software running here which is the cloud orchestration software which actually does take care of a lot of things for you that makes it easier for you to run your application. For example if you are running your application inside a VM and this VM crashes the life cycle of the VM you know creating it, deleting it, restarting it after a crash all of that is managed by this orchestration software. Then if you want to you know move your component to another machine, right finding a physical machine that is free and moving your VM to that physical machine that these cloud software will do it for you, you know placing the VMs on the underlying physical machines. So if you are running an application bare metal and if your hardware resources are exhausted then you have to find which machine is free and move your software there, right. But these cloud management softwares they take care of all of these things for you. Then you know another example is migrating across machine. Suppose your hardware this hardware server has to go down for maintenance or a repair then you have to move whatever application is running on it to another location. And if you are running it on the cloud the cloud management software will take care of moving these components to other machines, right. Similarly instantiating extra replicas if your web server is overloaded having another web server to share the load, right that is called auto scaling, you know scale up your system when under load that is also done by cloud management software. So there are several examples of such software like OpenStack and Kubernetes which if you were running the application bare metal a lot of things that you would have to do yourself these softwares do it for you. They make it easy for you to run a large software application over multiple servers, right. So this is the primary reason why people are preferring to run applications as cloud applications rather than as bare metal applications. So to summarize here are some of the pros and cons of using applications on the cloud which is of course there are multiplexing gains, you know multiple VMs can share the hardware resources better with better isolation. Then you have this orchestration, you know moving software across different servers all of this is automatically taken care of by cloud orchestration software. Then if you are in fact using a public cloud and not a private cloud you do not even have to worry about the hardware, right. The hardware is also taken care of by the cloud providers it is all completely hassle free maintenance for you. The other advantage is pay as you go model, right. If you only need few servers you only pay for few servers in a public cloud if you need more servers you pay for more servers, right. If your usage needs are less you can reduce your cost which you cannot do bare metal once you have bought all the servers what you do they are with you and you are incurring the cost of their maintenance. Another advantage is quick provisioning now suddenly the demand for your application has increased and suddenly you want more servers instead of you buying the servers yourself you can just pay for more and get more servers on demand on the cloud. So all of these are different reasons why people are moving their applications to the cloud and of course there are certain disadvantages also which is performance for cloud applications is usually more tricky because you are sharing the server and you have to access a server that is far away there are some delays. So performance tuning is a little bit harder and also the cost may also come out to be higher if you are heavily using the cloud you may end up paying a lot and it might be cheaper to just have your own setup locally. So these are some of the pros and cons of cloud computing but despite some of these disadvantages cloud computing is fairly popular today to run a lot of applications and a lot of computer systems you use in your daily life are actually hosted on either a private cloud or a public cloud. So now let us try to understand briefly the building block for cloud computing which is virtualization. So how are virtual machines implemented right? So virtual machines are managed by a software called a hypervisor or a virtual machine monitor that is you have the virtual machine monitor on top of it you will host multiple virtual machines each with their own underlying operating system and applications. Now this hypervisor can either directly run on top of the hardware and you know do the functionality of an OS also or it can run along with an existing operating system called the host operating system. So type 1 hypervisor runs directly on the hardware type 2 or a hosted hypervisor instead of re-implementing the OS functionality it will reuse an existing operating system and run along with it right. So whatever whether it is type 1 type 2 the underlying concept is of course the same of a hypervisor which is a hypervisor will multiplex multiple VMs on the hardware just like how an OS multiplexes processes if you realize it is the same thing right we also said an OS virtualizes the CPU for different processes similarly the VMM virtualizes the CPU and memory and everything for different VMs by doing a machine switch it will run a VM for some time save it state switch to another VM save it state switch to another VM and so on right just like how an OS multiplexes the CPU and memory across processes a VMM also does the same thing across VMs but an important difference is that a guest OS expects full control of the hardware whereas a user program does not therefore it is a little bit more tricky to you know multiplex hardware across multiple guest OSs each of which expects to have full control of the hardware at the same time you cannot give full control to any one guest OS why because you want to isolate these VMs from each other you do not want one user in a cloud to have full access to the hardware and deprive another user of the hardware right or do something bad we do not want that. So we want to restrict access to a piece of software that expects unrestricted access that is the challenge here with designing VMMs. So the standard technique to build VMMs is what is called trap and emulate that is you will just create another layer in between another privilege level in between will be used for the guest operating system. So normally what you do you have ring 0 you have your OS running and then like in x86 CPUs have multiple privilege levels or rings right and ring 0 you have OS and in ring 3 you have user applications with the virtualization what you can do is you can run the VMM or the host OS in ring 0 and the guest OS in an intermediate ring like say ring 1 or something and then the user applications can run in ring 3. So you are able to protect the guest OS from the user applications at the same time the guest OS is not very powerful any privileged operation it does will trap to the VMM right. So the privileged operations trap to the VMM and the VMM will emulate the action ok trap to VMM and emulate the action on behalf of the guest OS. If the guest OS wants to do any privileged operation it will trap to the VMM VMM will do the operation on behalf of the guest. So for this to work of course the assumption is that any instruction that accesses hardware whenever the guest OS tries to access the hardware it will always trap to the VMM right it will be a privileged operation that will trap to the VMM. This will ensure that the VMM can restrict the access of the guest. So some examples of how trap and emulate VMMs work anytime the guest VMM tries to do any privileged operation for example it tries to set up the interrupt descriptor table we have seen the interrupt descriptor table tells the CPU which OS code to invoke for a certain event like a interrupt or a system call. So whenever the guest VMM tries to set this up this is a privileged operation it will trap to the VMM and the VMM will set up its own IDT. Similarly when the guest OS tries to do any IO operation it will trap to the VMM and the VMM will do the IO operation on behalf of the guest. So any hardware access that the guest tries to do it will trap and the VMM will do it on behalf of the guest. Similarly whenever there is a system call it will once again trap to the VMM VMM will invoke the guest OS code. The interrupt arrives it will trap to the VMM VMM will find which guest code to run to handle this interrupt right which guest VMM to deliver this interrupt to and which interrupt handler to run the VMM will take care of it. So between the user and the guest OS all communication happens via VMM you will first come to VMM then VMM which is the most privileged software will take you to the guest OS. Similarly any hardware interrupt will come to the VMM VMM will take you to the guest OS right in this way everything is trapping to the VMM and the VMM is emulating it. This is the trap and emulate idea but of course this is not straight forward to implement in modern CPUs there are multiple reasons which we need not discuss in a lot of detail in this course but some of the high level reasons are that some guest operating systems they do not run well at a low privilege level a guest OS is built to run at the highest privilege for example some CPU registers you can read and find out that oh I am not in ring 0 I am in ring 1 and the guest OS may not run correctly right. There are some instructions which actually do not trap to the VMM instructions which access the hardware state but still need not always trap to the VMM they will run both in privileged and unprivileged modes right which is not what we want whenever the guest OS is trying to access hardware we want it to trap to the VMM so that the VMM can restrict hardware access but some x86 instructions what will they do they will access hardware they will run in privileged mode and unprivileged mode and they will behave differently when running in a privileged mode and an unprivileged mode because of which what will happen your operating system will run incorrectly in ring 1 it will not trap to VMM to access hardware the way you want it to it will run incorrectly in ring 1 right. So the summary is that guest operating systems are not designed to run at a lower privilege level right they are always built to run at ring 0 and if you run them at a lower privilege level either you know they might detect it and you know not do their work properly or there might be some incorrect behavior on part of the guest OS. So therefore for these reasons whether it is operating systems or x86 architecture they are not built with this virtualization in mind and therefore they will not behave correctly all the time. Therefore virtualizing x86 building a trap and emulate VMM with x86 is actually quite challenging and modern systems have of course overcome this challenge in multiple ways. So one technique now given all of these challenges how do you build a VMM with x86 there are many techniques and one of the techniques is what is called para virtualization that is you say okay fine let me just go ahead and change the guest operating system and rewrite it so that it does not invoke these privileged operations and it does not have all of these issues right. So that is called para virtualization where you go and change the guest OS so that it runs in an unprivileged mode. So such guest operating systems will not invoke privileged operations they know that they are unprivileged and they will make hypercalls like system calls a guest OS will make hypercalls to the VMM to get some work done right. So this is one technique this is used by the Zen hypervisor but of course the problem with this is that you have to rewrite the guest OS code. Another way if you do not want to change the guest OS code is what is called full virtualization which is you take your guest OS binary and you translate it if there are some problematic instructions that access the hardware but do not trap to the VMM or instructions that are checking the CPU privilege level all of these things you want to hide from the OS that you are running at a lower privilege level. So all those problematic instructions will be translated whenever the instruction is running you will translate it to something else and then you will run it ok. This is of course higher overhead right whenever every CPU instruction somebody has to translate it as and when it is running. Therefore this is higher overhead than para virtualization but you can work with unmodified operating systems you do not have to rewrite your operating system code that is the advantage of full virtualization. So VMware was the first company that pioneered this approach and the earlier VMware hypervisors use this full virtualization idea. And the third technique that is most commonly used today is what is called hardware assisted virtualization that is the CPU makers like Intel x86 CPUs have realized that there are certain problems with the CPU instructions which is making it harder to virtualize and they have changed their CPUs to make virtualization easier right. So what modern CPUs have done is they have added a separate mode for virtualization it is called the VMX mode in x86. So what happens with the VMX mode now in a regular mode of the CPU you have 4 privilege levels say ring 3 all the way to ring 0 your VMM host OS is running in ring 0 and your user applications are running in ring 3 then what happens is this VMX mode creates another set of 4 rings like this ok you again have in VMX mode you have ring 0 to ring 3 then your guest OS can run in this special VMX mode ring 0 and your guest user applications can run in VMX mode ring 3. So what is this doing this ring 0 this VMX mode ring 0 is not as powerful as your regular ring 0 so the guest OS wants to run in ring 0 you give it a separate ring 0 which is not as powerful as the regular ring 0. So now the guest OS is happy that is it is running in ring 0 all of its instructions privilege instructions will run fine but the VMM still retains control VMM can set triggers which it can say that when certain instructions happen when certain events happen that I do not trust the guest OS to do on its own for such events you can exit back into the VMM right. So this ring 0 is not as powerful as the regular ring 0 the VMM can still set triggers that cause a VM exit back into the regular ring 0 and the VMM can handle these events right. In some sense this is the best of both worlds right you have an unmodified guest OS which is running in ring 0 the OS is happy everything is going fine you do not have to rewrite the operating system to you know run at a lower privilege level you do not have to translate the operating system instructions which are problematic you do not have to do any of that you have your regular OS that is running in ring 0 but at the same time this VMX ring 0 is not as powerful as the regular ring 0. So the VMM the virtual machine monitor retains control it can prevent the guest OS from having unrestricted access to the hardware right. So hardware assisted virtualization of course understanding this you have to take a full course on virtualization we cannot cover more detail than this in this course but it helps you to know that there is a way to run virtual machines today with as low overhead as possible but at the same time providing all the isolation guarantees that we need using hardware support and of course there are also some optimizations you may have to do to improve performance for example if you are seeing a lot of VM exits every time you are going switching back and forth right this VM exit is an expensive operation like a context switch or switching across processes. So you might want to minimize these VM exits to improve performance. So the next concept that we look at is containers which is a way to do lightweight virtualization. So with VMs what do you have? You have multiple guest operating systems all of them residing in memory and all of them sharing the underlying hardware via say another you know host OS or a VMM and so on right. So you have multiple pieces of operating system code and overall your system memory consumption becomes very high and switching across this VMs is also very expensive a VM exit it is like a context switch but it is much more expensive because you have to store more context you have to restore more context and so on right. So running multiple VMs in general is there is some overhead associated with it. So containers gets rid of that overhead. So containers is a way to do lightweight virtualization. Containers have lesser overhead that VMs but of course also there is a cost you have lesser isolation but also lesser overhead. So what are containers? Containers is basically you have the same underlying operating system but you have different views of this underlying operating system that is there is the same underlying OS but the process tree the list of processes or the list of files, the file system, the libraries all of these things you can separate out in different containers. So a container C1 and a container C2 will you they can have a different view of what processes are there in the system. The processes in C1 will not see the processes in C2. You can have different libraries, you can have different files, the applications in C1 will not see the files of C2 and so on right. You can provide some basic isolation while sharing the underlying OS image and you can also ensure resource usage limits that is you can say this container can only consume this much CPU. This container can only consume this much memory that will let you also do some basic isolation across containers but at the same time of course since the underlying operating system is shared you do not have full isolation like you have with VMs but you have lesser isolation and lesser overhead also compared to virtual machines. So these containers are implemented using two mechanisms in the Linux kernel. So the Linux kernel today provides what are called namespaces that is you can isolate certain resources for different processes. You can say you know these set of processes will only see this process tree or this file system and another set of processes will see you know P1, P2 will see only certain resources of the system, another processes P3, P4 will see only a different slice of the system. In this way you can isolate processes into namespaces in Linux today and you can also set limits on how much of hardware resources each of these processes will use. You can say this group of processes can only use two CPU cores or you know one GB of memory you can set such resource limits. So you can isolate their view and you can limit their resource usage. Using these two mechanisms you can build a container framework. So there exist already implementations of containers like LXC and Docker which use these mechanisms right which use the techniques available in the Linux kernel to do this isolation and limit resource usage. So there are different container frameworks have different purposes they are built for different ways of using them. For example LXC is a general container framework right it gives you a VM like shell interface whereas a Docker container is for running a single application. So inside a Docker container a single application and all of its files, dependencies, libraries everything can be packaged right all that the application needs all the system software that the application needs can be packaged into one Docker container and this can run on the underlying operating system right. So different containers are used for different purposes and of course you also have container orchestration framework just like for VMs there are software like Docker swarm or Kubernetes which let you manage you know you have multiple containers on multiple machines and these frameworks will let you manage all of these containers like you know start containers on machines move containers across machines. So Kubernetes has a concept of what are called pods that is you place these pods on the underlying hardware and you can move these pods around and manage the life cycle of these pods scale these pods when there is overload right. So these container orchestration frameworks make it easy for you to run a large application with many components on a underlying hardware. So that is all I have for today's lecture. In today's lecture we have seen what are VMs, what is a hypervisor and what are containers and how these techniques of virtualization VMs and containers are used for cloud computing. In a cloud you have a large pool of hardware resources which are shared using VMs and containers and cloud computing makes it easy for you to run your applications because the cloud orchestration software actually manages the multiple different components of your application on the underlying hardware. So as an exercise please try to set up a VM or a container on your machine there are many resources available online and many free open source frameworks available for VMs and containers you can install them on your machine and understand what is this isolation that VMs and containers provide and what are some of the overheads of them. If you can just play around with them that will help you understand the concepts in today's lecture better. Thank you very much.