 In this video, we're going to show you how to set up XCPNG with TrueNAS and NFS. And some of you may be wondering, what about iSCSI? Well, iSCSI is thick provisioned with XCPNG versus NFS being thin provision, so that is going to be the preferred way I set these up. But if you want to use iSCSI because you're thinking it will be substantially better in performance, I've got some benchmark videos I'll link to down below to show that, yes, there can be some advantages of iSCSI, but it's not night and day. It is not going to double your performance. There are certain factors that may have advantages with iSCSI in small rights, but overall NFS performs quite well on your TrueNAS system provided you either have a log drive to synchronize the rights or turn off sync. And we'll cover those settings in this video. So let's get started. Are you an individual or forward-thinking company looking for expert assistance with network engineering, storage or virtualization projects? Perhaps you're an internal IT team seeking help to proactively manage, monitor or secure your systems. We offer comprehensive consulting services tailored to meet your specific project needs. Whether you require fully managed or co-managed IT services, our experienced team is ready to step in and help. We specialize in supporting businesses that need IT administration or IT team seeking an extra layer of support to enhance their operations. To learn more about any of our services, head over to our website and fill out the higher us form at lorenzsystems.com. Let us start crafting the perfect IT solution for you. If you want to show some extra love for our channel, check out our swag store and affiliate links down below that will lead you to discounts and deals for products and services we've discussed on this channel. With the ad read out of the way, let's get you back to the content that you really came here for. I'm doing this with TrueNAS scale 2404 beta 1. The full version is due out in April. So I figured may as well do this before it comes out here in 2024. And it doesn't look much different than it does in version 23. Pretty much all the options are the same. We're going to be here to data sets and we want to add a new data set. And we want to call this NFS YouTube demo. Now we can look at the advanced options. These are auto expanded in the previous versions of 23 of TrueNAS scale. And you can fill in more. We can put a comment in here and we'll talk about this in a moment. The sync settings, we're going to leave this all at default and we're going to hit save. So now we've created it. And the next step is simply scrolling down over here, going to the permissions, checking the two middle boxes, leaving it at root, leaving in the group at root. And we want to apply the permissions recursively, hit continue and save. Now from here, we can go over here to our shares. And we can add that as a share. So we've got NFS YouTube demo. We can choose the description if we want and hit save. Now I'm going to go over to xcpng. We're going to go new storage. We're going to select the host. We're going to give it a name. We're going to choose NFS, put the IP address to the server, choose the NFS version. If you want it to be using 4.1, that's fine. Then we're going to choose our share and we're going to head hit create. And that's it. We've created the NFS share. So let's go ahead and throw a virtual machine on there. And now it's copying over this virtual machine to that new NFS mount. Now, while this did get everything working, there's two more things I want to talk about that are really important. Now you can just stop here and be happy that it's working. But let's talk about those synchronous rights. This is something that causes a lot of confusion. And let me explain it very simply. If you want the best level of data integrity, you leave it at the default with the synchronous rights turned on. What that means is whenever NFS is talked to via the hypervisor and it says, hey, can you commit this data? The system, the TrueNAS system will say, I've committed the data. Now you can go on and assured that this data has been committed. But if you turn that on, you get a substantial slowdown. You can enhance this and not have it slowed down as much by using a log device on there. But you're still going to still see a performance hit when you have synchronous rights turned on, when you turn them off. You get a warning in TrueNAS that tells you there's going to be potentially some data loss if there's a catastrophic failure. What happens is when the ZFS gets a commit and it immediately is told the data is committed, even if it's not, in that moment from the data being told to the hypervisor that it was committed, but TrueNAS not having yet committed it because it's in the process of writing it and you suffer a catastrophic failure, such as immediate power loss of the system, that right may not get committed to the drive. So now you have a discrepancy between what is on the TrueNAS and what the hypervisor thinks is committed to the TrueNAS system. This difference usually doesn't hurt much in terms of virtual machines. Obviously, if there was a transaction that happened, that transaction won't exist. And usually just a simple checking of the disks next time that VM boots up solves the problem, other than it's a missing transaction that just won't be there. The integrity of the ZFS is still maintained. This does not affect ZFS integrity. ZFS is a copy and write file system. I have a whole video link down below called ZFS the cow. So this is not really a problem for ZFS. It will have the last good commit that it had, even though that is not exactly what the hypervisor thinks should be there. But hypervisors are fault tolerant and I've pulled the power many, many times. And generally speaking, this has not caused a major issue. But if you're running this in a production environment, and especially if this is a business environment, those transactions are incredibly important details of the business. It is probably best that you leave those on and you enhance the performance of your TrueNAS by having a really fast log device to kind of compensate for the slower speeds you get. We'll do a benchmark here just in a moment to show you the direct difference. Second, we're going to talk about how to secure this because you may have noticed there's no usernames or passwords with NFS. This is going to be locked down based on IP address. This is why I always recommend NFS, especially when you're setting up multiple hypervisors and your TrueNAS server, that you have a dedicated storage network. You don't want other traffic mixed in there. You don't want other devices potentially causing problems or any security issues with it. So having a dedicated storage network, I'll show in my production system in a moment here, is the easy way to do this. You dedicate and bind the NFS to that storage network. Then you filter by IP address, and now you're not worried about anyone getting on there. And if someone has physical access to be able to get onto that network and spoof those IPs, you hardly have a bigger security problem than what we're trying to address here. So let's get started and show you those other little details. Now, the first test I want to show you is with our NFS YouTube demo, and the sync right here set to standard. So that's the default the synchronization is on. So we are not at risk of losing the data. This top here is running on the TrueNAS. It's SSH-DIN. It runs every one second. This is the command. This is zpool iostat-vy demo space one space one. This just shows us the four RAID-Z drives that we have set up, the read write operations and the bandwidth being used. And we're going to run this command right here, just a DD command. And we want to write out a file to show you how fast it can write out the file. And we'll do this. And we can see how fast it writes. You see the writes coming across here, spreading across the drives. And these SSDs are allowing with the synchronization turned on about 350 megs per second. So, okay, a little bit slower once it completed at like 321. Let's go ahead and go up here. By the way, we're not rebooting the VM. We're just changing this on the fly. We're going to change this to disabled. TrueNAS does give us a warning that in the event of a crash, as I mentioned, we could potentially lose data. So we'll go ahead and hit okay. And we're going to hit save. Go back down here. As I noted, we didn't have to reboot or restart anything. Got the VMs still running. We're on the same speed test. And you'll see a pretty substantial speed bump we have here. About 495 megs a second. So definitely a lot faster. But of course, that came at the expense of if there was a catastrophic failure, we could lose those couple seconds of data. Now let's go over to the production system here. And I want to talk about the networking on it. First, and then I'll show you how this is secured. So we have multiple networks. The first one is this 192.168.20 network. This is the dedicated storage network, the 192.168.3.225 network. That is just the standard network where we do communications. And we also have this VLAN 69. So we have three different networks. But this one right here in particular does not allow any routing to it. Matter of fact, when you go into this one, that's all it has is an IP address. It is labeled descriptively storage, but it does not have any gateway on it. So if we go over here, then to our shares, and we look at our NFS shares, specifically, we're going to look at this VDI production one. And we scroll down to the bottom here. You see 192.168.20.93, 20.94, and 20.95. We have this NFS share completely restricted to only accept from these IP addresses. So if we try any other system that is not in the authorized list, it will not allow it to mount that NFS share. And if we go take a look at these hosts, you'll see that these hosts are 94 and 95. So if we take a closer look at the networking set up in them, you'll see I have quite a few networks in here. There is the management network to three network. But then if we scroll down, there is our ETH1 tied to the storage network. And it is in the .20 range. And I have a static IP assigned to this. I have another video where I do cover networking more in detail for XCPNG. You'll find link down below. Now, if we look at the other hosts, it's the same thing. Go to networking. And you'll see this same one, one storage. And this one's 95. The third IP address is over here. We go to our VMs and we look at our production. We have two network interfaces for our XO community production. And this is now set up in the storage network right here. And this has the .20.93. That is our third IP address. This network interface is directly attached to it. And this one's still on our standard 192.163 communication network. Now setting it up like this with a dedicated storage network simplifies things quite a bit. That way the storage is actually on its own interface. It's not even a VLAN. I have it dedicated. That way I get the full bandwidth and anything that's happening on the admin side or the side where the VMs themselves are interfacing won't interfere or slow down the storage. It's also a good practice if you can to have dedicated switches for your storage. That way you understand the update and patch cycle for your switches and you schedule it to where you're not worried about the hypervisor potentially losing connection to its storage device or maybe you stack the switches in a way so you have multiple cards tied to multiple switches and this is how you'll have a larger production environment set up. So you can take out one of the switches and have failover between them. And that way you're not having a bad day when one of those goes down and all the VMs are running and they suddenly lose attachment to their storage. Now XCPNG does recover well from this. There are videos I'll have upcoming on that to show some of the different failure modes but it is something you can recover from. It just becomes a bad day for, well any administrator has to deal with it whenever storage gets disconnected from the hypervisor. That's not an XCPNG problem. That's a general system in problem. Nonetheless, love hearing from you. Leave your thoughts and comments down below. Let me know what you think of this video or other videos and head over to my forums. Forums.LaurenSystems.com to have a more in-depth discussion about this and other topics. Like and subscribe to see more content from the channel. And head over to LaurenSystems.com to connect with me on whatever socials you can find me on. Thanks.