 I've done a video before talking about ZFS and the wonderful snapshot feature, which is one of the real good reasons to use ZFS, besides its robustness, the ability to snapshot the system is really important because it allows you to keep spots in time, so to speak, of what the status of all the files were, and be able to refer back to that when people oops things, delete things, or in the worst case scenarios, it's one of the ways that ZFS can protect you from a crypto locker scenario, so if something terrible happens to all the files and you're taking hourly snapshots, the most data at risk is one hour of data, and it's just a great system. Now extending on that, I'll leave a link to my video about snapshots and how they work below, is replication tasks. This is ZFS replication, and it comes down to how do you back up your free NAS server in terms of how do I replicate all the data? First things first, you're going to need some snapshots. One thing you need is another ZFS server, another free NAS server, in this case, that supports as much data as you have and are going to be sending over and those changes. So you have to have an equal or larger server in terms of support. Now you can't get away with a smaller server if you're not filling up your existing server, but you kind of get the idea here that if you have a lot of changes going on and you want to keep these snapshots, it doesn't just replicate the data, it replicates the snapshots, and that's what we're going to cover here. They have this big write-up on here, and I'll leave a link to this as the instructions from them, and we're going to walk through them. But I'm going to use the semi-automatic method because it's easier, and it's how most people will be doing it. But it does support full manual method, and it supports individual users. So instead of doing a backup between two devices, specifically like the whole device like we're going to do here, or at least which volumes we want to do and which data sets we want to do, you can get really fine-grained and create a custom user on another FreeNAS server and drill down and have that set up so it will only sync with this particular user, and you have to set permissions and things like that. They have instructions on how to do this here, but this will get you started on it, and once you have the basis of how it works, it's pretty simple, and it works really, really well. ZFS Replication is a great way to keep different pools in sync with each other in terms of backup. Now, a quick note here about it is the replication is based on the snapshots, hence me mentioning the snapshots. The other thing is the target system will automatically be created in read-only mode to protect the data. So if you just try to, hey, I'm having a problem, I want to get to the data, you can read it, but by default it's read only. The way they say is you create a clone in order to do that, and there's ways you can, of course, change it back to read-write if you need to. If you want to use that system again as a base, or if you're cloning a system, you could do it that way. It is an option, but just so you know, it's not an automatic read-write. It's not a way you can sync two remote file servers together for redundancy. That is different than this. This isn't storage redundancy. This is storage backup into a read-only system. So you have a physical another server that may or may not be in the same physical location and able to send the data because it uses a secure transport method, SSH, to get the data over, which means if someone's listening on the line, or you're transporting this over the internet, it is a secure protocol being used. And I still recommend, if you were doing this, you do it over a VPN. But still, it does support encryption, and that's an important aspect of this. So let's get started. Now, I'm going to show the FreeNAS 11.2 RC1 as the source for all this. The menus are probably what you're going to end up getting used to. I know there's a bunch of FreeNAS 11.1s out there. In reality, this doesn't change much between versions of FreeNAS. I've used this both to the destination for my testing here and as a server sending it. So it works both ways. You can cross over versions of FreeNAS without a problem. There's not any difference between them. There is a difference if you go all the way back to version 9. Version 9 is missing the automatic, or as you refer to it, semi-automatic setup mode that is not available. But it will work with that. You just have to do it manually to get it all to work. It's just a manual setup of setting up the SSH keys. Like I said, we're going to use the automatic. It's quick and easy. So here we have take one with nothing on it. This server is our backup destination server. So this will be the server that collects all that data, which really we don't have to do much on here. We can go here to pre-doc snapshot tasks. There's none. Replication tasks, nothing on here, because this is going to be destination. As a matter of fact, the only thing we need to do here is get the temporary auth token. Now the auth token, when we get to that part, it only lasts for a few minutes. So anyone commenting when I show it on a screen, like, you showed your auth token. One, these are both demo servers that will get reformatted. Two, it only lasts for a few minutes. And you have to be inside my network to attach to it. So just throw it out there for those of you wondering. So right here we have some the beta tank, and we have some really important data. And we want to make sure we back up this data. So let me go ahead and just refresh the pool again. So I copied a little bit more data. And OK, we got about 1.67 gigs of data that I copied over to this. Now, we're starting by baking up the data set. But I'm actually going to do another demo where you can back up the Z-vol as well. We're going to cover how all the differences are. And by the way, this is something someone had asked. And I realize I didn't do video on this. And an important aspect is, can you back up to this server? Many servers. Yes, you can. And we'll cover that part too. So the first thing we need to do is get some snapshots going. Because the replication tasks are based on the snapshots. So if you jump right to replication, not going to work. So we have no snapshots on the system. And we've got to start creating them. So the nice thing is, if you're on this version, the new version, they put everything in one place. If you're on the old version, it's not. Replication tasks are under storage. And snapshots under storage, they're not under tasks. They have moved the menu. So if you're looking for how to do this, the menus are pretty much the same. The layout's a little bit different. There isn't a different location. And it's covered in there right up over to do it. So if you're using a new version of FreeNAS, they're here. If you're using the older 11.1 series, it's going to be like this. So we're going to go here, tasks. We've got a periodic snapshot tasks. Now, if we want to mirror the entirety of this, we're going to start with just backing up some really important data. We're only choosing this one. But if we chose the beta tank, basically the root of it, and then we say recursive, that means get everything below it. That's what the recursive means in here for doing a snapshot. So we only care about right now the really important data. We're not worried about the Zvol one. And I'm going to uncheck the recursive box. But if I had it checked, if I had a series of data sets I created later under it, not folders, but data sets under there, this means recursively snapshot those as well as opposed to just snapshotting this piece of it. Then we set the lifetime of the snapshots. We'll keep it for one day. Begin and end times. These are for when you want the snapshots to start and stop. So if I want them to start each day at 1 AM but stop, so they're not running during the busy parts of the day, you can do that because it does put a load on a server. We're just letting it run all day for the purposes of this. And we're going to run every five minutes, seven days a week. We're going to have a lot of snapshots generally doing this. Actually, yeah, five minutes sounds good. I think I can get this accomplished within five minutes. Because if you start creating too many replication tasks and it has to catch up, well, we're going to put the system under heavy load. All right, so now we have this all set up and a periodic snapshot tasks are going. All right, so we're going to start with a replication task here. So now that we have our snapshots running, we go to replication, and we hit the pull down. Now all the different snapshots we have will show up in this pull down. We have one snapshot task set up, so that's the one that shows up in here. Now this is important. Remote ZFS pull data set. So we have to give this a name. Storage tank one. And we have to create this. So we will create a data set, and we're going to call it from beta server. Data from beta. Add data set. That's it. Didn't have to really do anything else here. Now this is the next step when we're doing this. So we have this here from beta server. And then we have to say remote ZFS pull data set. It's going to be tank one slash from beta server. And case sensitivity, everything is important. It has to match exactly or it won't work. And you can't just put from beta server. It won't know where to put it. So we want it to be tank one from beta server because that's the data set that we have. Then we're going to go ahead and recursively replicate child data sets. Delete stale, snapshots, and remote system. Delete snapshots from the remote system, which are also no longer present on the source computer. That's important. We're going to leave these at default. Now if you have speed limitations, this is where you can set zero is automatically everything. This is so you can set the speed if there's a limiting data factor. So you go, hey, I only want to take up this much bandwidth when it does replication. So that's just kind of planning in your network. Begin and end time. You may not want these if you're replicating off-site. You're like, I only have bandwidth during these times to do this. So you can choose a time by default to just payloads from this all day. Or as the snapshots go, is actually when this kicks off and it starts synchronizing the data. Setup mode. We're going to go for semi-automatic. Remote hostname, 1921683.142. That's just the IP address of it. That's simple enough for the hostname. Now another option is if you have these connected over like a 10 gig direct link or any other, you would just use the IP address that it'll figure it out of the other connection. So whatever the fastest connections, if you are running a higher end freelance with multiples, you may want these if they're in a server directly attached, you may want to replicate them over whatever the fastest connection possible. This is a simple flat network with them using standard gigabit connectors. So we're just going to go directly to the IP address, the same one that we're doing for here for this demo. Now remote auth token. That's the simple part. We go over here. Besides creating a place for the data to land, we got a replication test, temporary auth token. We just go here to copy. And this is where you'd set up a dedicated user but we're not going to do that. And then just hit save. Status waiting. And what it's waiting for is the next snapshot and then it'll start sending over the data. And I'm willing to bet we have a snapshot. Now, this is kind of cool. This is how the replication shows up. You get this little wiggling icon here and it tells you, hey, we're replicating. And it says sending beta tanks, some really important data over here as the auto snapshot. So during this few minutes that we were doing this, we have some snapshots. Let's go over to storage, snapshots. As you can tell, I paused in between because there's now there's two that's been 10 minutes. These videos in case you didn't know where it edited. Sometimes there's some stopping and starting. And go ahead and close this. That auto can expire after a few minutes, like I said earlier. And look, it's duplicating all the data. It's almost duplicated all of IC 948 megs. Let's go over here to this storage pool itself. Looks like there's about 1.6 gigs here. And we can click here and check the status again. We're about 80% copied. I'll wait a couple more minutes. All right, all the data is synced. Now let's take a deeper look at this. So right here we have from the beta server, here's all the important data. It's the same as it is over here. So 1.67, this just goes out another decimal. This one just calls it 1.7 gigs. And if we look over here, there are no replication tasks. They're all happening on the other server. This is just a receiving end of it. There's no snapshot tasks, but all the snapshots exist. So as the snapshots are being created on the other one, the differences between those snapshots copy over to here. Now in the five minutes that occurred between these snapshots, there were no changes to the files. So the snapshot becomes a reference in time to the previous, but didn't take any data. And like I said, I've covered this in my snapshot video. So now you can kind of get the idea. Snapshots are only the differential. And this is also what makes CFS replication so fast is it's only the differential in data. So if there's no differential in data, there's no bytes to really send across. It's just enough data to point to the fact that the snapshot occurred. And then that is synced via the CFS replication at the same time as a snapshot. Or if you do have it set delayed, as you've seen, you can delay when it does it. But it looks and checks at those times to say, hey, is the snapshot been done? And yes, we do it. Now this is really cool, because now we can go here. I can probably mount the volume. Let's take a look at what the data looks like in there. I can look at it. I know it's kind of small, but you can see. I just have a bunch of YouTube stuff copied in here. It's really all there. It's just simple as that. All the data is copied. All of it's in sync. I can do this. I can probably even easily create a share and do in there. But once again, it's read only. But at least I have access to the data. But I also have access to roll back the snapshot on the receiving end, destroy the snapshots, clone the snapshots again. And when you clone them, this is an easy way to get read-write access. But mostly when we're going back, you want to be able to reference this. Reference this and create a share to it so you can get that data back from your backup server. Now let's go back here. Now that's great for data like this. Now what else can we sync over this? So I actually created, and I've got some more demos coming for iSCSI on this, I created a tank called in here, I'm sorry, a ZVAL called SCSI. Now a ZVAL versus a data set. ZVAL is a block storage method. Data set is file storage. Block storage is the way you'd present something like iSCSI. And ZFS handles it differently because it's handling the block and letting whatever you're presenting it to handle the actual formatting of the data on there. So it's not in a standard format of standard files like you would see over here in a normal data set. Can we sync that as well? Absolutely, this is the beauty of ZFS because all of this is happening at the base level of ZFS. So it can become completely agnostic to what formats and data or things are being used because it's a ZVAL, it just knows I need to handle these blocks on here and sync them. Now, once again, it says 91 gigs because we force provisioned this at 91 gigs. Let's see how it snapshots. Pool data set, we're going ahead and say beta tank SCSI doesn't need to be recursive one day of retention and five minute intervals again. All right, so now we have these going. Definitely gonna go over here to replication tasks. Gonna add another one. We have to go back here and give it a place to land. Go here. All right, we have a data set created for this and we're gonna go over here. Remote is going to be tank one slash the SCSI stuff. Same thing here, if there's any recursion, we know there's not any enabled manuals. So I'm gonna automatic again here. Go here, replication tasks, temporary auth token. Get us going. Put in the IP address. Put in remote auth token. Save. Now we have two replication tasks. One's gonna replicate this ZVAL for us and one's gonna go here. Now, one of the things I've pointed out and that can be done is you can go in and just say do everything. So if you did a snapshot that did everything, then you could also be doing a replication of the entirety of the system. And here we go. Sending okay. It's thinking. All right, and it is replicated and working. So even though we force provision it because it's all blank data and because it uses compression, there's nothing actually in this beta tank or let's look at the pool here, there's no actual data in it. So if we look inside of here, we see it's a ZVAL with this, but if we dig a little deeper, actually it shows right here, we're only using 3% because it's just provisioned. It actually doesn't have data in it. So it replicated really, really fast. So yes, you can replicate ZVALs. You can represent that. And as you can see from here is where the SCSI stuff comes from and the beta server. Now, this goes for as well if you wanted to do this from other servers. So last piece I'll show you is how you have it coming from more than one server and how does that work? So here is 192.163.153. Our destination server is 3.142. This is the backup server that's receiving two separate backups coming from the beta free NAS. Let's show you one from here. So pretty much the same process. We go over here, we're gonna create a snapshot task. Oh, and just so people know, if we go over here, there are no snapshots on this. So this is a brand new loaded it for this demo server. Created some data in the volume here and now we're gonna replicate it. So we're gonna go over here to the PRX Snapshot task or over here. Like I said, the menus on the new system I think are clearer than the menus on some of the older versions of FreeNAS here. Add a snapshot. And we actually do this one a little bit different. We'll just do everything. So instead of just using this, we're gonna do recursively, just do all the things on here. Keep it for a day. Same thing every five minutes. Run an all day nonstop, hit okay. All right, now we have the magic task going here. Then we're gonna go over here to replication task. Add a replication task. Remote ZFS dataset name, same concept. We gotta create a place for it. Another server. If we have three of them and is it not FreeNAS, is it FreeNAS? FreeNAS servers, there we go. That's what we'll just call it. Or it's the third one. All right, so now we have the another server right here. Then we go back over to this server. We're gonna head and paste in tank one slash another server. We're doing the entire server. So we go ahead and here and here. All day no limits. Semi-automatic. This version only works with equals of nine, 10, or newer when they added this feature. Get a new auth token. Remote host name, remote auth token. Hit okay. Not ran, sense, boop, status, waiting. And we'll refresh this in a second because it takes a minute and it will start sending the data. Sending. And you can see it's refreshing. Now this doesn't have the cool animation that we get with this one. We have it up here moving around. It just says sending here and we can just keep refreshing this. 14%, 16%. Doesn't auto refresh for those of you wondering, but it's doing what it's supposed to do. Now, because we chose this periodic snapshot task, recursively doing everything, if we add another dataset to this one, it will do it. So there is some advantage if you just want to have it set up to say just replicate everything. That way you can build one task and it just duplicates because you care about 100% of everything on there and you want two copies of it. And then if you add a dataset later, recursively we'll take care of that for you and that's what we did in this example. So now you know how to set up in either the new version of FreeNAS or the old version of FreeNAS. It's really simple. The replication is really straightforward and it will keep everything in sync. So that's definitely great. Is backing up is important, making sure you have enough data is important. And in case you didn't catch one last thing, I will note in case someone asked, even though we are replicating a Z-Val over to this one, please note you still start with a dataset and it can be a Z-Val underneath. So we called this one the SCSI stuff and this we hate as a dataset but underneath it, it has a Z-Val. So it is replicating it. You don't ever create them. That's why if you notice when you're creating a task it always asks you to create a dataset to store it because you can store block storage under datasets. So you don't ever start on the server that's under receiving and with creating a Z-Val. So, but all these works. And by the way, if you've ever wondered about this you can then set your own periodic snapshots. So you have your backups being sent that are also being sent with snapshots. You can then snapshot the data again if you wanted to for whatever reasons or scenarios that you wanna come up with. You can go crazy and recursively do this. Make sure you have enough storage to do it. That's the one limitation you want into because if there's a big amount of changes the snapshots are big. Now this works like I said very fast. It doesn't take long to sync all the data across. We're doing this replication task just across non high speed networks just standard gigabit networking and they transfer relatively fast. Now another option when you're thinking about this if you wanna do this offsite if you need to replicate faster you may wanna have the two ZFF servers on the same local network. Once they're in sync with each other then remove that one to the remote site and that will keep the incremental backups going and keep it all going. So like I said some options here it's definitely a pretty sweet system now. Like I said this is only backing up the data sets on here. It is not backing up the actual free NAS. So a couple side notes. You still need to save config files from here. This is all the configuration for things like well replication tasks and all this but not the data. So that's still here. And if you encrypted the drives you do need to download the encryption keys out in case you ever lose the drive. That's boot and air and I've covered this before. So if you have questions, comments leave them below and thanks. Thanks for watching. If you liked this video go ahead and click the thumbs up. Leave us some feedback below to let us know any details what you like and didn't like as well because we love hearing the feedback or if you just wanna say thanks, leave a comment. If you wanted to be notified of new videos as they come out go ahead and subscribe and the bell icon that lets YouTube know that you're interested in notifications. Hopefully they send them as we've learned with YouTube. Anyways, if you wanna contract us for consulting services you go ahead and hit LawrenceSystems.com and you can reach out to us for all the projects that we can do and help you. We work with a lot of small businesses, IT companies, even some large companies and you can farm different work out to us or just hire us as a consultant to help design your network. Also, if you wanna help the channel in other ways we have a Patreon. We have affiliate links. You'll find them in the description. You'll also find recommendations to other affiliate links and things you can sign up for on LawrenceSystems.com. Once again, thanks for watching and I'll see you in the next video.