 Time here for more systems and we're gonna talk about virtual machine memory management and XCPNG and how you set the dynamic memory and how you can change it dynamically on a running VM. Now, I bring this up because there's a few caveats that I wanna make sure people understand about dynamic memory management and what happens when you over allocate as in, you've assigned a group of VMs that cumulatively have more memory needs than the host can provide, but there's ways you can set a dynamic sizing on there so that they can, well, keep running and keep working on that particular host. Now, where someone can get into trouble is if they live migrate this, this came up during a consulting call because when you do a live migration and there's dynamic memory options set, well, it will reduce the host to its lower dynamic number. That's what that dynamic number is for. Before it moves it over to another host and then re as it's called balloons, the memory backup to the dynamic maximum that's been set. So I wanna walk through that process and yes, this does work with virtual machines that are built in Linux and in Windows, provided they have the Zen agent running and loaded on them. That is a prerequisite for this to work properly, just so you know. But before we dive into details of this video, let's first, are you an individual or a company looking for support on a network engineering, storage or virtualization project? Is your company or internal IT team looking for someone to proactively monitor your system security or offer strategic guidance to keep your IT systems operating smoothly? Not only would we love to help consulting your project, we also offer fully managed or co-managed IT service plans for businesses in need of IT administration or IT teams in need of additional support. With our expert and solve team, we can also assist you with all of your structure, cabling and Wi-Fi planning projects. If any of this piques your interest, fill out our hire us form at laurancesystems.com so we can start crafting a solution that works for you. If you're not interested in hiring us but you're looking for other ways you wanna support this channel, there's affiliate links down below to get your deals and discounts on products and services we talk about on this channel. And now back to our content. Now, the first thing I wanna talk about is just how we set the dynamic and static and what those mean when it comes to setting memory limits. So we're gonna go into the VM, we're gonna go over to advanced, we're gonna scroll down over here. The template that you use to build your virtual machine will set the static minimum right here. So the static minimum is two gig. The difference between static max and static min and we'll actually go ahead and try and change this here. We'll change this to 32 and we're gonna get an error message that setting on VM Tom Wendell's memory, VM state is running and should be halted. You cannot change the static limits while the virtual machine is running. These are the limits that you have set but what you can change is the dynamic ones because if I don't wanna start a VM, restart a VM or stop and start the service and change memory settings, maybe I wanna bring it down to its minimum and then bring it up to its maximum of, well, 16 gigs. But if I tried to say 24 or even 32 gigs, it won't let me change it to a maximum above what those static limits are set. It's really that simple. You set these so the system understands the potential for this VM when it's not running and then the dynamic and lower and upper limits are set so you can change it while it is running. Now the things and considerations to take. When you migrate the VMs and we're doing that and showing that in a Linux VM to show how it's reduced, when I move this live migrated, it will actually reduce down to its minimum and that's one of the reasons that set. The reason it does that is to, at least the way I understand it, when you migrate a VM, it makes it faster by saying, well, we aren't gonna need all of this memory so we're gonna reduce it to its minimum, migrate it over to the other host and then expand it or balloon it back up to its maximum that you have set in these dynamic ranges right here. But this can obviously cause a problem. If I have it set to two and the application is running, still require more, it may have some trouble during the migration. So you may wanna set that to at least the minimum that this would need when you're doing the live migration. If you're not doing live migration, it'll still dynamically go up and down but actually let's just show you how you can dynamically in Windows, change the memory settings and what happens. So currently it has the maximum right here of 16 gigs that we can set. We can see that it's eight gigs now so we look at here in the stats and you can see I've actually played with this a couple times already, changing it just before I started the video. So let's go ahead and change to 16. So we'll just take this running Windows VM and hit 16. It turns yellow for a second since it's changing dynamic memory. Now it's changed. Let's go over to the console on this real quick and it's a little big because of the screen size I have here but we'll go ahead and open up the task manager, scroll up and you can see, hey look it has 16 gigs of memory showing in here but we're gonna do this in Linux and play around with the settings and show you what happens when you over provision things and how it balances all of that in some of the things that change with it. So we're gonna go ahead and stop the Windows one and let's go over to our Linux ones. Now first thing I wanna do is go to this first VM that we have, go to advanced and just see what it's set at. So we have it set to four gigs max. The template allows it going all the way down to 512, pretty lightweight if it needs to be but we have a dynamic set four by four and I wanna leave it at that because I wanna show you what happens when we start this virtual machine and when we move it. So it's gonna start, it's right now running on the Ryzen 2 host. It'll boot up relatively quick here. Okay, now it's booted up and it's using four gigs of memory. If we log into the console, we can see that it's using four gigs. There's the memory right here at the top and let's migrate this and we're gonna do a live migration while it's running. So we're gonna say migrate, I wanna move it over to Ryzen 1. This is gonna happen incredibly fast. So actually click on task here, two, three, one, now go three. Okay, well like four seconds, we moved it over and you can see now it's on Ryzen 1 and it's still running. We don't have to restart the machine. Now because we left the lower and upper the same, it did nothing to the memory. It says, hey, the minimum this goes to is four, the maximum it goes to is four, therefore we are going to move the four gigs of memory from this host to the other host without making any adjustments when we do it. Let's see what happens though when we go to advanced here and we're gonna set this down to one. You gotta remember this is dynamic and we just can't go below this 512 number here. So we'll set this one at one gig, four gig and we'll do the same thing again. We're gonna migrate it back over to the other host that it's next to. So go here, hit okay. And this is gonna take a little bit longer and it's taking only slightly longer because it's going to downsize it a little and bring it back up. And we're gonna do a bigger move so you can actually see that in action and see the memory change. Right there, you can see it blipped back and forth and so we can do that one more time back on the other way. If we do it this way, you're actually gonna watch it do it. So we'll keep an eye right here. It's reducing down the memory and then ballooning the memory back up. Now the console flickers for a second when it does the change because it moved to the other host. Hey, and there's our memory and now it's going to reinflate it back up so we're at 827. Then it's gonna climb back up as it reallocates on the next server. This is why those dynamics are important because if I had applications that were running, it's going to actually cause it to slightly hang doing that because it's going to get stuck trying to evacuate that memory while things are running. So let's go ahead and split the screen. We'll use Tmux. We're just gonna put htop here and then we're going to get it down here at the bottom and we're gonna run this command here stress-ng-vm-1-vm-bytes four gigs to allocate four gigs of memory. So we're gonna watch the memory usage at the top go, well, all the way to four gigs a little over and it's gonna start using swap because we're over allocating everything here a little bit beyond so we can leave any room for the operating system run but that's kind of the goal. This is now a heavily loaded up machine in terms of memory and we're going to try to do a migration while these dynamics are still set down really low. So we still got the dynamics set down to one gig and four gig but obviously if we're using up all that memory what's gonna happen? And I'm not exactly sure what state the VM will be in but let's go ahead and do a migration to the other host. Hit okay. One, I don't think it's gonna happen as fast. It's gonna try to do some reduction but it's not gonna have anywhere to reduce too so it'll probably increase the swap would be my guess what's gonna happen here but obviously it's already taken substantially longer because it's thinking and trying to sort that out while it does the migration. So it's still on the rise of one it's still trying to figure out how to get this memory allocation it's running some processes on there and I'm wondering if it's actually going to crash the VM all together when it does I think we may have locked it up here folks and this is why it's important to have those dynamics now this would run and it still might continue to run cause it's just stuck thinking right now oh there we go it's still going but this is when you don't have that dynamic low set to a reasonable number well let's just push it all to swap and is pushing it back to migrate yet maybe we're doing this all in real time cause I wanna show you how these problems occur when people are setting these up and why it's important to think about the workloads you have on there because when you dynamically are allocating them this is the type of scenario you'll run into easy to set up a really basic VM with no workload and migrate them in a matter of like five seconds much harder to do it when you have a workload and well I think we've actually just got this maybe it's stuck it still seems to be going here in the task so it's slowly getting here I can also just stop the process running the stress NG process and kill it and it would probably do in it so do suspend freeze process failed yeah freezing task failed after 20 seconds one task refusing to freeze I like the phrasing it uses here so the system's under high load trying to do some memory swapping around here to get this over to the other side so I think we'll stop this process real quick here before it crashes and we have to force a reboot so we'll there we go I stopped stress NG that'll probably allow this process to carry on and continue but this is why it's important that when you're setting these minimums you're considering the workload that's on there and that your minimum doesn't go below what the workload will at least need to get things done now a lot of times different tools may use the extra memory for caching and that memory for caching may be able to be reduced and you just have to test this on an application by application basis or whether or not you can do that I mean ideally if you live in the ideal world you have enough memory for all of your applications but you know the reason these tools exist is because the real world doesn't always have the ability to do that so you have a more challenging environment where you wanna get these done and I think this is gonna spend a little bit longer so we'll go ahead and actually kill this server see what status we leave it into or if it'll actually finish or it's still estimating it'll move let's go ahead and see if we'll give it a few more minutes here we'll just fast forward all right all we ended up with was an operation failed it failed to migrate the VM because of those dynamic memory problems being hit but it's on the rise in one server I had to restart that server to stop it after it failed and restart it it's ready to go again now with nothing running on it of course we're gonna go ahead and migrate hit this and now it's gonna pop itself over to the Zen 2 server back into our couple seconds to get this task done there we go migration completed it's over on rise in two now the next step I wanna talk about is over allocating the available memory and how it dynamically is going to resize these on a given host so if we look at a rise in two system here we currently have 62 gigs of free memory so there's the different things that are running on here and now let's over extend this system so let's assign some VMs with more than 62 free but they have these dynamic options that will allow them to go lower and we'll go ahead and include our window system in this as well so take the window system and it's shut down so we can adjust all these different limits and we're gonna assign windows 64 gigs on both sides we're gonna say 64 over here now the minimum windows is down to two and I don't think windows should go below eight gigs on here so here's this system and we're gonna specifically start it on that particular host of rise in two so we'll go over here we're over allocated already because there's well we're right at the limit because our 66 gigs free and we got this one signed 64 we'll go ahead and let windows start up on here and it's thinking about it right now and the reason it's pausing and thinking is cause goes wait a minute there's just not quite enough memory so it's going to make some adjustments on here and see if anything else can give up some memory get this host started all right windows has started on there and we click on this we've pretty much now filled it all up well there's 12 megabytes free so just a little bit of free space on there now let's go and look at this system here this one's got four gigs it's also running on here let's go ahead and just stop it though go over here and we'll set this one at 32 but let's only give it 16 for now we'll set a maximum of or a minimum of eight and a maximum of 16 here so we'll go ahead and also we're gonna start on and we'll put this one on rise in two now remember rise in two that particular system is already pretty much over allocated so we go over here look at rise in two and it's got to make some decisions now who's going to lose some memory well these other ones don't have those same minimum so this one does so this one right here is going to say we're gonna shrink how much windows has here so before that VM starts this is why it still hasn't started it's reducing and evacuating memory out of this one because it can adjust that dynamic and that will leave us in a free space to then start that next VM so now this VM is starting and we'll see how much memory is once it boots up we'll see how much memory is allocated to this particular system which it looks like it was able to allocate the 16 to it once it starts it'll talk to the processes running inside there and then decide exactly how much memory it needs so let's go ahead and start one more and we'll do the same thing on this one we'll say this one's gonna get this max of 16 minimum it's gonna leave it four but we'll put it at 16 maybe this one will give 32 let's go all out here you can have up to 32 but dynamically we'll leave it at these ones here and once again go up to the top and we're gonna start it on that particular VM if you just hit start it'll start on a host that has enough resources to handle that but we're gonna force its hand and say we don't care that this one doesn't have enough resources you're gonna have to figure that out and that's what the system's doing now so now it's gonna start resizing these other ones to try and figure out where it can find free memory so it's taking it away from the Tom windows again and where's that other one on here there's my other one that's already reduced this one it says you only need 13 right now it's gonna reduce this one a little bit more and then find room to fire up this virtual machine and once this one here oh there we go it's booting up and so it's understanding what processes are in there so now we have all these VMs which are technically provisioned at a rate higher than the available memory on that particular host but it just keeps reducing them so we have the min and maximum set now what will happen again and we'll wait till all these VMs are up and running which I think they should be now this one's got just a few more seconds to go and there's that ballooning message that you'll see because it's then ballooning waiting for initial ballooning down having finished that's that process that's going on because it's adjusting this dynamically to allow for that space to be had it's pretty simple once you start understanding what these numbers are doing with the dynamic and static and what's going on in the background but there's a lot of complicated things going on in the back end to make sure these hosts are working and this is where as I said you can get yourself into trouble especially if you don't understand the workloads and the minimum requirements of your workload in order to make this happen because right now with these having pretty much no workload on them with lots of free memory it's easy to balloon them but that ballooning process will run into some limits if there's just tools like StressNG running that say nope I refuse to give up my memory I need this much memory it's locked in here and you'll start having VM migration problems but we'll show you what happens now when we migrate it because now that we've got this completely filled up again with these different VMs how do we free up some space dynamically well that's pretty easy so we'll take and log into one of these like this one here run H top so we can see the memory we've got 12 gigs we've got a maximum set right now that it's 16 gigs and how do we get that extra 16 well now we got to take this and migrate it over so let's go ahead and migrate this over to the other one that has plenty of free memory in it and that will happen relatively fast but you got to remember it's actually gonna shrink down first because we have that minimum set so as it kicks off that process it's going to reduce the amount of memory so here we go watching it go down and that's gonna be based on that dynamic minimum we set of eight so it's gonna go all the way down to eight and once that completes it'll migrate over to the other VM while they're host and then expand it back out probably to its maximum because if I had to guess and even though it's eight right now let's go ahead and look at this we look at this one here oh there's plenty of RAM we've got 72 gigs free so it shouldn't have any problem at all expanding that one out and it's already done that so we go back over to the council it's expanding back up ballooning it out again and you're back at the full memory now hopefully this demonstration gives you a better understanding of how the dynamic memory management works how it will allocate and why those dynamic minimums are so important that they do not go below whatever the workload minimum threshold is for that particular virtual machine and the things you have running on it because well I think that's where you can really find yourself in some of those problems of causing the VMs to crash be in an unknown state or just tell you the operation's gonna fail because it doesn't have the ability to do that now if there's enough interest I might do this for processors as well because yes you can overallocate processors it's much more forgiving when you do that when you have well 24 cores available and maybe you assign five or six VMs all those cores but as long as you're not all using it once there's some advantages to doing that leave your thoughts and comments down below or head to my forums for more on that discussion and thank you for making it all the way to the end of this video if you've enjoyed the content please give us a thumbs up if you would like to see more content from this channel hit the subscribe button and the bell icon if you'd like to hire a short project head over to laurancesystems.com and click the hires button right at the top to help this channel out in other ways there's a join button here for youtube and a patreon page where your support is greatly appreciated for deals discounts and offers check out our affiliate links in the description of all of our videos including a link to our shirt store where we have a wide variety of shirts that we sell and designs come out well randomly so check back frequently and finally our forums forums.laurancesystems.com is where you can have a more in-depth discussion about this video and other tech topics covered on this channel thanks again for watching and look forward to hearing from you