 That all right. So last one on virtual machines. So we'll go through this and then We can answer your questions about the lab or final or talk about that and do that stuff so virtual machine it's Won't be on the file But it's a good thing to know because you will encounter it in your life and knowing all this stuff We're not actually going to probably learn that much new It's going to actually be kind of review because everything we've learned actually applies to virtual machines The only difference is a virtual machine abstracts the entire machine. So like how a process think is Thinks it has access to all of the memory with virtual memory Well, a virtual machine thinks it has access to the entire machine when it may not So you can run multiple operating systems all on the same machine They each believe they have access to the full machine But none are aware of each other and they're all actually sharing it So there's some terminology with virtual machines the first term you'll hear is the host So the host is the thing that has direct control over the hardware So if you're actually implementing virtual machines the thing There's something that manages all the virtual machines and that is called a hypervisor or a virtual machine manager You'll probably see it more oftenly referred to as a hypervisor So that controls virtual machines controls, you know creating them managing them what resources it gets so what hardware It's actually able to access so some virtual machines might access different subsets of hardware Or you might make up your own hardware to give the virtual machine and the virtual machines is not going to know any better So when you actually implement hypervisors, there are two types and why you might care about them is one Well, it's kind of interesting and two one is a lot slower than the other one So type one is a bare metal hypervisor So it runs directly on the hardware and this requires special hardware support as we'll see and The other is a type two hypervisor. So that's called a hosted hypervisor which simulates a hypervisor But it just runs as a normal application. So you are running a kernel in user mode So there are some tricks you need to do if you need to actually accomplish that task typically The type two hypervisors are going to be slower because it doesn't require any special hardware It's just running as a process and we're going to do some tricks So it can actually run as a normal process even though we know the kernel actually runs in kernel mode and then the other term is a guest a guest is just the Operating system you are running with that hypervisor and Each guest sees its own copy of the host thinking it has access to the whole machine when it may not So this is our kind of lay of the land so far We've been used to the picture on the left so figure a there where we have a bunch of processes It talks to the kernel through a programming interface Which we know for Linux at least our system calls and then the kernel goes ahead interacts directly with the hardware Now when we have virtual machines, we still have that same system call interface Nothing changes the processes and the kernel are all the same but now the kernels are running under a virtual machine and then they're all managed by this virtual machine manager or Hypervisor which then has direct control over the hardware. So we have another layer of indirection here So as a note you might hear the term emulation so virtual machines are typically not emulation, but they can do it So emulation is typically when you translate one ISA or instruction set architecture to another so your phone runs on ARM and Your laptop as long as you don't have a Mac is x86. So if you want to run, you know Say you want to run an iOS application and actually run it on your computer while you're gonna have to emulate it You're gonna have to change ARM instructions to x86 But typically when we care about virtual machines Whether or not we emulate it doesn't really matter to us and we'll just assume that we're not emulating at this point So if you do have to emulate it's gonna slow down because for every instruction you have to actually translate it Which is gonna slow it down So this is okay for some uses So if you're interested for a little front project you can emulate a Original Nintendo system quite easily and you don't you can emulate it because your hardware is like Two billion times faster than the original Nintendo, which is like an 8-bit CPU running at like half a megahertz So with some things you can go ahead and emulate it with other things You might have to do some more advanced techniques that we won't get into in this course so If your virtual machine does have to emulate an instruction set to the performance is going to be worse So we'll just talk about things that are actually the same underlying instructions for the same CPU So some fun things about virtual machines as they enable pause and play exactly like changing processes You can change virtual machines So a hypervisor can go ahead pause an operating system switch to another one to execute Go back and forth between them at will even move them between different hosts if you want It's not a problem for virtual machines so your hypervisor is going to need to context between virtual machines and it Works the exact same way as context switching between processes. So the only difference here is our Processes are actually going to represent the entire CPU including all like special kernel mode registers and everything But aside from saving some more data. It's fundamentally the same thing as switching between processes. So It's all stuff. We already know about They also provide some protection through isolation so guests like processes by default are all Isolated from each other and the host that's actually running the operating system So if you're the hypervisor, you can go ahead you can set limits on like how much CPU time each Each guest can take how much memory they can take maybe how much network bandwidth they can have and That way if one of the machines gets hacked or something like that Well, it only has access to its own virtualized hardware So it's probably fine You can even disable it if you want without affecting everything else like for example If you download anyone know what red star or Linux is No one's heard of that. So that is the official operating system of North Korea So you can if you want to try that out Typically you wouldn't want to install it on your actual hardware because you have no idea what it will do But if you're interested in it Hey, you can install it in a virtual machine and probably a good idea not to give it network access And then you can just kind of see what it does and it's fine because it doesn't have access to anything So that's some fun thing you can do. Yep Because it's from North Korea Red Star Linux it's called so it's Hopefully you're not if you're using it That's probably bad. So yeah, so if a virtual machine, you know becomes infected with a virus or gets compromised with someone hacking into it Or it's pre compromised by North Korea. Well, you can just go ahead and remove it when you're done with it So other things it helps with are if you've ever used cloud services You've all used virtual machines because that's all they run so in data centers with lots and lots and lots of machines well often they don't make of use of all the resources and Being able to actually move virtual machines around is actually great for helping balancing the load so like with processes They're kind of instead of being tied to a CPU they're tied to a natural machine So we could have the case where hey We assigned a bunch of virtual machines to a server and now everyone's using all those virtual machines So all the utilize like they're pegging a hundred percent of the CPU and now the physical machine Doesn't have enough capacity to serve them all so what you can do is you can transparently just move a virtual machine to a host that doesn't have any load while the user's still running it and they won't notice and Suddenly everything will be a much faster from them because they just got moved So cloud services will do that all the time and they only care about running virtual machines for you so And also because they're virtual Well, it also helps you with your architecture if you're building like a big web server or something like that Well, you could put your database on virtual machine You could put your web server on virtual machine and then when you're small with like only a few users Well, you can put run all the same virtual machines on the same physical machine since there's not that many people using them And then as you get more users Well, you can just buy another machine instead of running both virtual machines on the same computer Each computer can run its own virtual machine where that's the only thing running So you don't have to change anything. You just Reconfigure what virtual machine runs where and then you're done. Yeah We'll see that at the end. So the question is what's the difference between a virtual machine and a container? So we'll get to that at the very end So for this the key of stretching is a virtual CPU that represents the entire state of the CPU Including all the specific kernel mode registers and other stuff you wouldn't be able to see in user mode So for processes, it's like part of the process control block It kind of acted as a virtual CPU but only had access to user mode registers There's special kernel mode only stuff that we haven't really gone into all you need to really know is that there are special Kernel only mode registers and things like that that you can only do in kernel mode For example, you know changing the page table or something like that so the vCPU structure just holds the entire state of the CPU and When the guest isn't running it gets context switched in like normal for processes so it would just save all the additional registers and When the guest isn't running it just saves that and when the guest gets context switch back in again It just reloads it all again, and it's all good So This may seem kind of weird especially given what I said about type 2 hypervisors and the fact that the kernel Is running and it's completely unmodified So you are running a user mode You just have like a normal user process and you're running the Linux kernel that is unmodified So it's going to execute instructions that are only available in kernel mode. So how does that work? so Remember on x86, you know everything in user mode is Running in ring 3 which were the names of the modes and then kernel mode was ring 0 so You are essentially running your guest in mode 3 only and it's going to make Use of instructions that happen in mode 0 because it thinks it's the kernel so If you're doing a type 2 hypervisor, you have to do some tricks if you're doing a type 1 hypervisor This is supported for you So if you have hardware support for a hypervisor on x86 You can tell how old this is because if they design this when virtual machines were actually a thing They would probably have called it ring 0 But now since they ran out of numbers hypervisors ring negative 1 So that's where that number comes from because when they first designed the instruction set. Well hypervisors did not exist So for that's for hardware support There's additional mode and then for type 2 hypervisors everything runs in user mode So if you're implementing a type 2 hypervisor One strategy you can do is something called trap and emulate So if you just naively just run kernel code as a user process Well, you're going to get a signal Which is going to generate a trap in the kernel whenever you try and execute that instruction That is only supposed to be run in kernel mode So you'll get a signal that says hey you have an illegal instruction So if you are actually implementing the hot hypervisor, you'd have to know some details about the hardware So you can handle that illegal instruction call and you can simulate what actually goes on what's actually supposed to happen with that instruction and Update the virtual CPU so you'd have your own structure for a virtual CPU where you have to actually know about the hardware and keep track of stuff so then you just simulate that instruction and What this is going to do is one cause for or slowdowns because Every time I'm executing instruction that I'm not allowed to execute It's going to generate, you know, it's gonna have to be handled by the actual host kernel It's going to generate an illegal instruction to your process and then you're going to have to do something to simulate that single instruction So there's a lot of steps whenever you encounter a kernel mode only instruction So this is what kind of looks like visually so at the top of the big line is the guests And then at the bottom is the hypervisor or whatever is running it So in this case for a type one the whole things a user process So if your guest kernel makes a privilege instruction, which is an instruction running in kernel mode Well, it's going to generate the trap and then instead of just failing then you're supposed to handle that and Simulate it and then you update the virtual CPU and then you just keep executing the next instruction keep going on so That's nice if you have a very well-designed instruction set, but this doesn't always work especially for x86 So if you have this clear boundary, all you have to do is trap and emulate and you can actually implement a type 2 hypervisor but x86 didn't know about virtual machines and The importance of this nice clear boundary when it was designed so it has this nice property if you can call it that of Instructions behave the same instruction behaving exact or completely differently based off what CPU mode the CPU is currently in So one example is the pop F instruction So what it's supposed to do is load a flags register from the stack and This behaves differently whether it's in kernel or user mode So if you are in kernel mode and you execute this instruction it updates kernel specific flags And if you're in user mode and execute this instruction it updates user mode specific flags So it does two completely different things depending on what CPU mode you are in and this is a problem for our type 2 hypervisor Because everything's running in user mode So if your kernel actually expects the kernel mode version of the instruction and you just blindly execute it well, then you're going to execute the wrong thing and Probably there's going to be some very hard to debug thing going on so You can't generate a trap in this because it's still a perfectly legal instruction It just behaves completely differently. So these are called special instructions and need another approach So they need something called binary translation so What happens is if the guest virtual CPU is in user mode We can just we don't have this problem because The guest expects to be in user mode and we are currently in user mode So we can just execute the instructions You only have to do something if the guest thinks it is actually in kernel mode So if the guest is actually in kernel mode you have to inspect every instruction before you execute it and see if it's a special instruction and Then if it's a special instruction, then you do the same emulation or simulation of it And then skip that instruction and go to the next one so This is going to be a slowdown and another way or sorry and another thing you can tell when that You know when the guest expects to be in kernel mode because there's going to be instruction that switches from user mode to kernel mode and that would you know generate a normal trap instruction or a You know a normal trap which would be an illegal Instruction and as part of the simulation you would update that hey my virtual CPU should be in kernel mode So since it's in kernel mode now, I also have to Start analyzing and doing this binary translation just in case I run into a special instruction So your performance is going to be worse because now when you are in kernel mode If you are running this type of architecture You have to start inspecting every single instruction, which is going to be much much slower And this is what it looks like visually. It's kind of the same thing But instead of just relying on a trap to get generated when you try to execute instruction this will look at the instruction see if it's one of these special instructions and then Simulated by updating the virtual CPU and then just returning and Continuing execution as normal So this is where the type one hypervisors come in. So there's more hardware to the rescue so they were introduced in 2005 Intel introduced virtualization calling it VTX to an AMD did in 2006. It was called AMD ve and if you Start playing with virtual machines. You might have to go into your BIOS and enable some settings to actually you know, enable this hardware support and Annoyingly it's called different things. So Intel codenamed at Vanderpool was in development and published it as virtual machine Extensions VMX So if you have an Intel motherboard and you try and enable this It might be under one of three names and it might change depending on what motherboard Manufacturer you're going to so on some it might be called Vanderpool Some it might be called VTX on some it might be called VMX But they're all the same thing. It's just that hypervisor support Then AMD same thing they called it Pacifica and then published as secure secure virtual machine So again, there's three different names. It might be called if you have an AMD motherboard But all it does is add the concept of negative one or hypervisor mode So what happens if you enable this whatever you boot your operating system of choice Well, it's going to claim the hypervisor and also act as a hypervisor So if you boot into Windows, it will know that hey this mode's enabled I can grab it and actually be a hypervisor and that way if anyone runs a virtual machine I can provide hardware support for it. So it'll be nice and fast and Same with Linux links can do the exact same thing So that way it can set isolation for the guests and what hardware it wants to virtualize So Now that we have virtual machines to and multiple virtual machines. We have to schedule them Luckily for us the scheduling is exactly the same as we saw before so We can do some things. So maybe you care more about with virtual machines like that processor affinity Maybe you want to keep something on a single core just so you have closer to realistic performance So if you only have like one CPU on a on the physical machine You could still have a host with multiple CPUs. You just have to contact switch between them and Yeah, so now With where the hypervisor we have to map virtual CPUs to actual physical CPUs and then schedule them like processes But this isn't anything new. We can round robin them We can do completely fair scheduling or we can try and just like if we have way more physical cores than there are vCPUs We can just assign them directly to them and not move them around So yeah, like I said, if we have more than one more physical cores and all virtual CPUs We just map them one-to-one and don't even bother context switching if you have to share things get a bit more complicated, but if you have like equal numbers of vCPUs and actual physical CPUs you can still map them one-to-one Because the hypervisor threads won't actually run that often if everything's just assigned to one core The hypervisor just does some housekeeping might be called if you're accessing some hardware, but other than that it stays pretty quiet so This has some additional problems where Well now the guest operating system might now run too Unpredictably for soft real-time tasks So if you're running something on actual hardware Where you're relying on your like your round robin to be every like 10 microseconds and be it very very consistent Well, if you are running it on a virtual machine something that previously met all of your deadlines might no longer meet all your deadlines because between your round robin ticks you could be context switched out and then later get context switched back in so the virtual machine might think only think 10 milliseconds has actually passed but It will be longer than 10 milliseconds, and then you'd probably miss your deadlines So you might actually see some effect of virtualization if you really really care about soft real-time systems because there's gonna be some additional overhead and now Also virtualized memory management gets a lot more complex So all the kernels are going to still have all their page tables But now the kernels usually just maps a virtual address to a physical address But now since it's a virtual machine that physical address is actually a virtual address So the hypervisor has to map what the virtual machine thinks is a physical address to a real physical address So it thinks it's managing the entire address space, but it is not so we have to virtualize that too and This problem gets even worse if we are over committed on memory So that means all the virtual machines take up more memory than we physically have and in which case your hypervisor is going to have to implement page replacement and Do all the paging for you which gets much more complicated if you dive into the details So this would make use of something called nested page tables So again, the guest thinks it has control of physical memory and does all of its normal page table management But now the hypervisor needs to actually Do that so if you have hardware support it will maintain a nested page table that retranslates for the guests So you only have Leo an additional hop to translate all the addresses and because you have hardware support You can actually in your TLB you could go ahead and cash the whole Virtual address from the virtual machine to a real physical address. So on cash hits It's still fast. So you get all your performance back, but you need kind of hardware support for this And then if you're over committed on memory, it gets really complicated. So you might have double paging so your Kernel unmodified would still might page to disk and do page replacement and then your hypervisor might also do page replacement as well and You might want control over this because typically the guests knows how memory is accessed better than the hypervisor So there's some trade-offs here where you actually want your page replacement done and it gets much more complicated That will kind of ignore for the rest of this But there's some other concepts we learned that Would apply to this so for example? Sharing read-only pages while we do that in processes all the time But if you want to do that in virtual machines Well, you could could still do that So if you have you know ten virtual machines running and they're all using the run or they're all using the same kernel Well, ideally I just want one copy of that in physical memory But unlike that, there's no like fork for virtual machines or something like that So similar to that so we're going to use the same idea But now to identify pages that can be shared the hypervisor is going to have to scan through all the pages And if it had to scan through them bite for bite and see if any are the same between virtual machines That would be very very slow So what they do instead is check all the hashes of the contents of the page And then if two hashes are the same then it's possible. They're the same or possible It is not so only if the two hashes are the same which is relatively fast Then it will do bite for bite checks to see if the pages are actually the same And if the pages are actually the same then it will go ahead and share them and then it will enable the same Optimization as we saw before called copy on right So that way if either virtual machine actually modifies the page Then it would make a copy that had a copy of the page and actually do the modification there So another thing it can do is virtualize IO devices So if you have a network card one thing you can do is multiplex one device to multiple virtual machines So you have one network card, but all your virtual machines could use it if you so choose Other options you have is you could map one physical device to a virtual device in one VM So that that virtual machine has exclusive access to the device But the hypervisor is still going to be in the middle of it and actually still translate any usages of that device. So There's a new ish Hardware solution for that to remove the hypervisor and it's called IOMMU. It's only newer systems will have this and That is a way for the hypervisor to directly map the physical hardware to the guest and that guest Actually gets access to the physical hardware and has to manage it itself so the VM now for sure has exclusive access to the device and has Exactly the same access to the device as if you didn't have a hypervisor at all So this allows something like GPUs to run at native speeds and virtual machines Because if you running something on GPU like some machine learning thing or a game Well, you want all the performance you paid for so you don't want anything in the middle of it slowing it down So if you have this you can run, you know, you're you're AI training or whatever at full speed and still get all the benefits of using a virtual machine So when you actually want to boot a virtual machine, the last thing is a disk So virtual machines just boot from a virtualized disk So all you have to do is create a disk image that has all the contents of the physical disk So it would contain like the partitions Each and then each partition would have a file system like fat that we saw before an iNode based one So usually it's just one big file But some formats allow you to split it up or do some compression to optimize things a bit But this file the kernel recognizes or sorry the guest kernel recognizes that file It's just a normal disk and thinks it has full control over it And then if you want to use a virtual machine like a normal machine all I have to give you is a disk image So that's all you need for a virtual machine makes it easy to move around as well So if we still choose which might have been better Especially for lab 5 is we could have just created a disk image for the UG machines And then given it to you and told you to run it as a virtual machine And then all the hardware would have been the same Except you're guaranteed that no other user is on it fighting for resources and he actually get slightly more consistent results So that's all you would need and then if you download some of these virtual machines You might see something called an OVA file But that basically is just a disk image and then some configuration settings for the guest like saying How much memory should the guest have how many CPUs should it have etc? But all you really need for a virtual machine aside from some configuration is a disk image and you can use that virtual machine And any changes you make while running that virtual machine while they just get saved directly to that file So you could even move that virtual machine to a new computer later and everything is going to be exactly the same So that's what I'm using for you know all the lectures all the examples I've shown for like code examples while those have all been in a virtual machine Especially because you guys have like made me like tried to kill a knit and all their weird things that I wouldn't want to do on my real system So it's just a virtual machine and whatever happens to it doesn't really matter And because it won't bring down my actual hardware because it's a virtual machine I can just reboot it without anything bad going on So this is where we get to the containers and what Docker is so one idea people had when people started using virtual machines It's like wow, this is great. Everything's so nice and consistent Updates don't break anything. Everything's completely isolated. So people started like what we had before with dynamic libraries Well, if you're maintaining a big system a and someone is Hasn't followed this course and writes a very bad dynamic library that completely breaks some applications Well, some applications might update and use the newer version Some might be old and use the old version So if you update the version of the library Half your programs work half them don't and it's a complete mess You either need to have multiple copies and fix it up or do something about it. It's just a complete mess so what you could do is actually create a virtual machine for your application that has the libraries on it that it Needs and you're for sure guaranteed. They're the correct version. So everything just works So your application is just in a virtual machine Everything's nice and consistent one update doesn't affect the other one and everything is good so you freeze your dependencies you deployed in production and No matter what it's going to be the same experience as long as the machine can run virtual machines It can run your application So I could give it to you running if you're running Windows if you're running Mac growing Linux Doesn't matter. I give it to you. It works simple as that so the drawback of that is if I have an application and I package it as a virtual machine and I give it to you well The virtual machine is going to contain a kernel So all of your virtual machines for all of your applications are going to contain a kernel and your kernel is going to be like at least 700 megabytes or something like that and your application might be like, you know, a handful of kilobytes in which case I am transferring, you know, almost a gig so you can run my application When most of it is kernel that isn't actually super relevant to the application So I'd rather just not transfer you a whole kernel. So that's the idea of containers So containers, so something like Docker Aim to behave exactly like that where you can just package up an application real nicely with all of its dependencies I don't have to worry about anything But it wants to be faster and actually use the same kernel So if we go back and we remember a why we actually want to do these things So in addition with freezing all the dependencies and everything We also get some nice benefits of the hypervisor where you can set limits on the CPU time Network utilization memory things like that So the Linux kernel supports these hypervisor like limits just on normal processes So the Linux kernel actually supports this with something called control groups or C groups And they allow you to set limits for processes and isolate them to a namespace Even isolate their access to the file system so they can't even see the rest of the file system Limit their network access, how much memory they can use, how much CPU they can use and all things like that so Docker uses that and so Docker uses control groups to run a package application Without, you know, the need for the application to actually give you the whole kernel every single time so it reuses the same kernel gets all those benefits and You don't have to transfer the kernel because it's just going to use the already running Linux kernel so that and Docker is specifically built on top of the Linux kernel So if you're using Linux and you're using Docker, you're just going to use your kernel But if you use it on Mac or Windows well Docker is going to run Linux kernel in a virtual machine for you and then run all your Applications using that virtual machine So there will only be one virtual machine for the Linux kernel if you are not a lucky benefactor of actually running the Linux kernel natively and Then go ahead and run everything on top of that one Linux kernel So you don't have to distribute a kernel with your application So the idea of this is they're going to be much lighter weight. They don't need their full kernel They don't need to have nested page tables or anything like that They can just go ahead and use you know, if you're running the Linux kernel You can just go ahead and run it as a normal process, which is nice and nice and fast. So they just use a normal kernel So that's it so virtual machines virtualize a physical machine So they allow multiple operating systems to share the same hardware They provide isolation and a hypervisor is the thing that controls them So type one hyper or type two hypervisors are slower because they run in user mode and they have to emulate or simulate any Crinol only instructions either using trap and emulate if the ISA is designed correctly or Actually do some binary translation and look to see if it needs to handle it differently Then type one hypervisors actually have hardware support. They've IO something like IOMMU is going to speed up Access to devices by just giving that virtual machine access to the physical device And then hypervisors might over commit on resources and need to physically move a VM Which is able to do with virtual machines You can move them across servers and that's why any single cloud provider. They only use virtual machines And then containers try to have the benefits of virtual machines without the overhead by basically not providing a kernel So that is it. So just remember boom for you. We're on this