 Tom here from Learn Systems and we're going to dive into how storage works in XCPNG. Now this is being recorded here in October of 2022. And I'll be talking about, well, in the future when there will be some changes coming to the way the storage system works. So this is the current implementation, but I will also leave some articles linked down below to talking about the future and the future things they're going to be doing with XCPNG. And maybe you're watching it in the future when some of these things I talk about, well, have changed a little bit. So kind of keep up with their blog and check it. For now, this is how storage works. It'll work very similar in the future. It'll just be better and faster. We're going to keep today's video narrowed in scope to having your resource pool with shared storage and local storage and what that means and how you migrate between them. I won't be diving into some of the really esoteric details of things like how you set up SEP and Gluster. Those are available, but those are a little out of scope of the topic, but we'll be talking about the different storage types supported overall. And of course, referencing all their documentation and that's already linked down below before we get into details of this video. Let's first, are you an individual or company looking for support on a network engineering storage or virtualization project? Is your company or internal IT team looking for someone to proactively monitor your system security or offer strategic guidance to keep your IT systems operating smoothly? Not only would we love to help consulting your project, we also offer fully managed or co-managed IT service plans for businesses in need of IT administration or IT teams in need of additional support. With our expert install team, we can also assist you with all of your structure cabling and Wi-Fi planning projects. If any of this piques your interest, fill out our Hire Us form at laurancesystems.com so we can start crafting a solution that works for you. If you're not interested in hiring us, but you're looking for other ways you want to support this channel, there's affiliate links down below to get you deals and discounts on products and services we talk about on this channel. And now back to our content. Now, this is the article I was mentioning. I really recommend taking the time to read this. It'll give you a good understanding of where they are with the current storage implementation and where they're going in the future. This was written in June of 2022, but it's a great read to kind of understand some of the history of, you know, the 2008 version of storage versus the 2022 and beyond storage of how things are changing and how things are going forward. It's a good write up. A lot of details in here. Their blogs are actually very insightful just to understand some of the complexities of virtualization and specifically this one about how drives are handled. Now to jump right over to the storage types, this is really important to think about when you're designing storages, where's it going to live? How many hosts are in a resource pool? A resource pool is a grouping together of different Zen hosts, and that allows you to migrate the different virtual machines between the different hosts. And this is common in your enterprise world where you have two or more, sometimes as many as 64 hosts in a resource pool. And we can simply migrate the virtual machines between them. But how they migrate will change based on whether it is a local storage or a shared storage medium. Now just to keep it simple, we're only going to talk about NFS and iSCSI now. Those are the ones that are natively built in when you set this up. You have to load extra drivers, which of course this is all in the documentation if you want to do something more such as setting up the ZFS for local storage, XFS, GlusterFS, ZFS, ZFS. That's a little bit hard to say. That's all in here on different ways you enable those if you want to use those. So out of scope of this video, but it is in the manual, but the menus are essentially going to be the same for setting this up. Now let's talk about how things look in the pool. So we have two hosts in my pool, Ryzen one, Ryzen two, really simple naming. Both of these have local storage. Now XC PNG is set up to boot off of these devices, which are just a simple mirror of MVMEs for the hardware and specs and everything else. That video is linked down below for the servers. But because of that video, I figured, hey, I'd explain how the storage works. So things that are running on local storage besides XC PNG itself, the extra leftover from the partition goes to a local storage that you can set up either as EXT or LVM. I'm partial to setting them up as EXT volumes. It keeps them as easy files to manage. If you're more familiar with LVM or don't have a problem with LVM, you can do it that way and manage it. But with EXT, they're thin provisioned and it works really well. That being said, when you have a VM only on the local storage, it can only start on that particular host. Now you can migrate, you can live migrate and we can demo that between these. You can migrate storage between these, but if you started the server on Ryzen one, you have to do a full migration to this Ryzen two host in order to get it set up. And that's going to require the VM and it can be done live, as I said, but to copy the entirety of the storage over, which may take several minutes or longer, depending on the speed and the interconnect you have between these hosts. Now, when you have a NAS or SAN storage, when the virtual machine lives on the NAS or SAN, it is really easy to move it between the host, because the only thing that has to be migrated is the memory between there. This is one of the advantages of the shared storage and also a prerequisite for HA in order for the HA to figure out which host it needs to start up on because when you have three or more, you can put these in a HA mode. You have it talking to the storage. If one of those host dies that happens to be running VMs that you have enabled the HA mode on, then you would say start that VM on one of the remaining hosts. It would not be able to do that if the host dies and the local storage dies with it. So it does have to live on the NAS or SAN storage. Now you can connect multiple NAS and SAN storage devices. It's as easy and walk through this as when we connect these, you're able to connect them and then keep connecting more of them as needed. There's not really, well, there's a limit of how many you can connect overall, but you can connect quite a few before you're going to hit any of those really large upper limits for just how many different storage devices you can have. And they can be added and removed dynamically. So we'll talk about how to add these shared storage devices. Now for the way my NAS and SAN system is set up, I currently only have one. So we'll be adding a second one as part of the demo, but they have both. And this is because I'm running a TrueNAS system. The TrueNAS system presents as iSCSI and has presented as NFS. And you can mix and match between them and you can migrate between them on the shared server. So it's a single connection to a single device in terms of the hardware, but you can migrate between the two different storages that are presented between there. So let's talk about how that works and talk about the functional parts of that. Now we'll start here with storage to show what storage is currently attached to the system. We have this called Trinity. Trinity is just the name I gave my TrueNAS. So we have the pool of ISOs that we have on there. We have the production NFS share. And we have the Trinity SCSI that I set up. And I guess it is production. Technically, I just didn't add the word production. You can rename these dynamically. Like that. ION, well, let's spell check. There we go. Production and naming them doesn't make any difference because everything on the back end is handled by these UUIDs. This allows you to dynamically change tag. We want to add another tag to there for filtering any of the storage devices and XCPNG without affecting anything that's currently running on it. For example, I have gray log running on this one right here. So no changes happen when you change name. That's just all for your presentation on the back end. Nothing has to start or stop to make that change happen. This currently lives and we'll look at the disc on that Trinity production NFS and everything's kind of clickable here through the storage. So we can pivot back over here to the storage. We can look at these stats. We can look at all the discs that are on there for all the other things running. But we're going to focus on our Ubuntu 2020 2004 storage demo. And let's go ahead and start that up. Now, if I just hit start, it's going to pick the host with the least amount of resources because this is on the shared storage, the NFS storage, it will look at the two hosts that are available. Or if you have more than two and go, all right, this one looks like the one best to start that on also under advanced. You can actually tell it to start under a particular host if you want. But as I said, it's shared storage so it can go on to either one. So we'll go ahead and let this start up. All right, this VM is up and running. And because it's on the shared storage, we'll talk about how it can be migrated. So I just ran bash top down here at the bottom so you can see it's alive and I'm not restarting it. And we're to say, let's move this to the other Zen server. So we'll choose it rise in one. Currently, you see up at the top here, it's on rise in two. And we hit OK. And it's going to take only a few seconds to move over because we don't have to deal with the storage migration. And now it's already moved over. As I said, literally a few seconds, it's now on the other system. You get the refresh to console when it switches. Nothing you have to do to the VM, it's still running. It's now on the other system. Now, the next thing we can do is move to another shared storage. So this is the NFS. And from here, we can migrate it this way or we can just say migrate this particular VDI. So we'll choose this and we're going to go over here to the Trinity scuzzy. So destination SR hit OK. And now it's going to take probably just a couple of minutes and it'll migrate this completely live. I don't have this restart or shut down the VM. I can now migrate these to a new storage. Now, this is really advantageous when you have production systems and you want to add another sand or another nasty to the pool without having to shut anything down. This is really handy for setting up when I have a migration of a customer has a really old servers, they have some old storage devices. And well, we would like to get them off of those, but we don't really want to shut anything down. Well, you can add these in addition to the systems and then migrate them live over also a new shared storage. But let's talk about even live in production here, adding another storage server. And what does that look like? So we're going to go over here to new storage and then select the host. Now, this is where sometimes people get a little bit confused. It looks like I'm selecting either this one or that one because I am, but it doesn't matter which one I attach it to in the name of a shared storage. So if we type in Synology, because this is going to be a Synology Storage Test. Give it a description. What kind of connection is it going to be? I scuzzy, I scuzzy shared. Now, if I were to choose something like these, they would be local and it would matter which one I choose. It would only search locally on that device. But because I'm choosing a shared storage, such as the ice because he it'll actually attach to both of these. So if we put in the IP address, query it, go ahead and have it select. It's going to think it's going to query it real quick here. It doesn't take too long. It's going to depend on your setup. It recognizes there's a hundred gigs analogy option here. All right, we're going to hit create. Let it create that new connection. So we've just added a new storage device, shared one to our resource pull with our servers in it. And I think this one's probably a way to those finishes creating. All right, now that's created, shows it available. There's no disk on it. Let's go back over here. This VM, which is still running. But as we look, the disk are now migrated over to the Trinity scuzzy production. Let's migrate it again. We can move it over to that new Synology right here. So there's our Synology Storage Test, hit OK. And now it's migrating over there. This is the advantage that they have for the way the storage is very dynamic within here of I can just migrate it to any location. Well, unless there's a problem actually in sufficient space. That's a that's a pretty fair error. And this is where you may run into some problems because this particular VM doesn't have enough space because there's a snapshot probably attached to it so that I deleted earlier. So let me provision that with a little bit more space real quick and continue the demo. And while I'm here, all you have to do is increase this total capacity is now 300 gig hit save, switch back over, and we instantly have the availability now of 299 gig. So now I have enough to continue that test. I'm going to leave this in the video because I said, Hey, why not? This is a interesting learning lesson when you don't have enough space. What happens? Does it all come crashing down? No, you just get an error message, you go and increase the storage and we can kick off the migration process again. And by the way, we didn't damage our running VM. So we didn't mess up anything that was in operation while it's doing this. So this is going to take a couple of minutes to move this over. We'll fast forward through this. So you don't have to wait. Now, one of the things I want to point out while it's migrating is it looks duplicated because in some ways it is. One downside of using ice because he is that it is not then provision. So it has to have the entirety of each time there's a copy of the VM in there. This can lead to a lot of storage. And when it's doing the copy, it has to have a base copy. It has to then have anything related to figure out the live running data and then keep copying that live running data as it migrates the base data. This can be, well, a little bit better explained by diving deeper into some of their write ups they have. But overall, this is the process by which the migration takes and why, even though it's a 60 gig VM, it requires more than the 60 gigs to do the migration. So it needed 120 while it's doing the migration, but then it'll actually reduce down once this migration is complete in about 58 seconds. Okay. The migration is complete. It's already coalesced one of them. There's one left to coalesce. So if you look at general, we're still using it says two X disk, but once these coalesce down, there won't be any other ones used. So we'll just give this a few minutes. But in the meantime, let's jump back over here and look at the server, which of course we've now moved between three different storage locations and it's still working perfectly fine. This is, like I said, the flexibility you get with the Zen storage. It treats all the different storage repositories, all the shared ones, even of different types and allows you to uniformly move these between any of the different storage, even from thin provision to thick provision and back again, without having to shut down the VM. This makes, as I said, migrations and moving things around a lot easier. Now, one more thing we can do, and let's see if this completed, let's go back over to the disk. Did it finally coalesce? Not yet. It'll coalesce over time. Coalescence is an entirely different topic that I highly recommend understanding and taking the time to read. They have a good article that I'll leave linked on how coalescence works when you're dealing with multiples of storage. And especially when you start having a lot of snapshots, which could lead to a coalescing error. It's some nice technical details go on the back end. But one last thing I want to show how storage works is what happens when you move things to a local storage. So we're going to go over here to the desk. We'll click migrate VDI and it's running on Ryzen one, which means we can migrate it to Ryzen one local storage. And let's see what happens. So now it's going to migrate. And we're pulling it off of there and moving into local. So got about two minutes left. It's going to go to the local storage on that particular system. Now, once we do this, I am going to lose the ability to migrate quickly between the two hosts. And the reason for that is once the storage becomes somewhere local, it can't start because you can't start the local storage from one host, but start the VM on another host. Someone may point out there's a tool out there that allows for some of this where you build shares across hosts. This is completely not an officially supported methodology. But yes, I'm aware of its existence. It's called HA lizard. If you want to look it up where you can build hosts that actually have replicated storage between each other on there. It's an interesting thing. It's not what I would recommend. And it's not something I would use in the enterprise, but for those of you in the home lab would like to use it. Hey, have at it. It's certainly fun playing with all these things and learning how they work. But you can see it moves quite fast to the MVMEs because, well, plenty of right speed on those. So it just read and moved in the time it took me to talk to over to the local storage on here. So it's finishing a couple of little details and some cleanup. And now we got to see it jump in real time to tell me it's now on the other storage. And of course, all this was done without the VM even being shut down. Pretty simple. How all of that works. You're able just to migrate these dynamically between the different locations. This is also good and can be built into automation where you can move and migrate multiple machines at once and do the migration. This can be coordinated for doing things like rolling updates with their enterprise versions. So if you're updating a series of the hosts when there's package updates and they're going to have to restarted, you can actually migrate all of these and build automation around that. So there's some real advanced functions, but it all starts with this base functionality of being able to dynamically move the storage to different locations. Now, of course, we're going to go ahead and if we stop this VM, we're going to stop it right here. And when we start it, there's no way to start on. It should give me a funny error if I do this because the storage currently is, and we'll look at the disk, so it's on the rise in one storage. We're going to go ahead and start on rise in two and see what kind of error message we get. And this is a little bit vague here, but VM requires SR and it's saying it's requiring that storage repository that's not available to it. That's essentially, this is an XCPNG year. Yes. It is Zen server XCPNG year. So we're going to go ahead and go back. And if we just hit start or start on, if you want to choose it, the rise in one. So even though that's there to start on the other one, it won't actually let you do it. So you, when we hit start, it's going to automatically start on the place where it's local storage is accessed. It's pretty simple to figure out where it lives because well, just hit start. It'll start where it's supposed to by default. Just don't try to get fancy with it, even though those options aren't grayed out on there. Now, as far as adding these storages or removing them, and we can actually go back over here, look at storage and because I didn't tag it with anything, it's not filtered. Now this has nothing on it. If you want to get rid of a storage like this, Synology storage test, we've migrated all the disc off of it. First, we want to disconnect from all hosts. Doing this will clear any connections those hosts have to this and break those connections. Once those connections are broken, then we can do the forget hosts and they can be forgotten, or we can go back and connect to all hosts. And this is an important feature being able to go back and forth and do these. And this is where sometimes people just go to the forget and sometimes it will hang because there's things going on there. So you do want to make sure you do the disconnect from all hosts. Or if you need to connect to all hosts, when there's a problem with it, especially when a storage was offline, when one of the hosts was started, it may be connected to only one of the hosts to pull, but not all of the hosts to pull. So these buttons, you can do them while there's VMs living on there. The last thing I want to talk about is maintenance mode. This is a really neat feature. And for that mode, we just click enable maintenance mode. Maintenance mode allows you to take these offline and put them online. And it actually, you know, puts this little tag right here maintenance mode. What it's doing, though, is looking for any VMs that are on here and it'll warn you and can shut them all down. That way none of them are running. So if you have to do an update to my Synology or an update to a Shurnass, I can just click on maintenance mode. It powers down all of those virtual machines. Then I can process all of my updates and when I'm done. You take it back out of maintenance mode. Let's actually go over here to my storage and show you what it looks like on a lab one, because I don't want to click this on production. If I click something accidentally, here's my lab. And there's a lot of disks in our lab, but not all of them are running. Some are in running states, some are in stop states. But if we go here to maintenance mode and we hit enable maintenance mode, it lets me know that these two virtual machines are going to be shut down and in maintenance mode. And when you take it out of maintenance mode, it will start those ones back up. Also, while in maintenance mode, it won't allow any other changes to be made to the storage repository until it's taken out of maintenance mode. Pretty handy little feature when you have to do updates. It makes storage a lot easier for managing an enterprise. So last piece we're going to do, go back over to this Synology, because this is how I clean things up when I'm done. We're going to go to that disconnect all hosts, like I said, let it disconnect. And then we're just going to tell it to forget this storage repository. And then I don't have to think about it again until I do my next demo, because well, this is just a demo system. We keep set up for testing purposes. Are you sure you want to get rid of it? Yep. And it's gone and that's it. So hopefully this gives you a better understanding of how storage works in XP and G. Leave your thoughts and comments down below or head over to my forums for more in-depth discussion or spend some time in their forums. I really recommend going through and spend some time reading. You can learn a lot about how storage works just in general, but they have a lot of really cool write-ups in their blog. And of course, just all the different back and forth in their forums is very enlightening to get deeper understandings of how all this works, not just with Zen server, but it gives you some ideas just how complicated and how interesting storage is when you have everything virtualized. All right. And thanks. And thank you for making it all the way to the end of this video. If you've enjoyed the content, please give us a thumbs up. If you would like to see more content from this channel, hit the subscribe button and the bell icon. If you'd like to hire a short project, head over to LauntSystems.com. And click the hires button right at the top. To help this channel out in other ways, there's a join button here for YouTube and a Patreon page where your support is greatly appreciated. For deals, discounts, and offers, check out our affiliate links in the description of all of our videos, including a link to our shirt store where we have a wide variety of shirts that we sell and designs come out well randomly. So check back frequently. And finally, our forums. Forums.LauntSystems.com is where you can have a more in-depth discussion about this video and other tech topics covered on this channel. Thanks again for watching and look forward to hearing from you.