 Hey guys, welcome back to Daniel Stack World. So for today's video, well, I should really say, for this week's video, this month's video, who knows, I'm going to be demonstrating a running a backup process on the cloud. So basically, I've been just finishing my backups and written a few articles and videos about this whole process. But one big repository that people care about, I think, is Gmail and G Suite. So in terms of getting your data out from Google so that it's not just held, I mean, there's a few reasons for this. Firstly, anything in Google or G Suite is in the cloud already. So you might be wondering, what's the point? Do I need to back up a cloud? And you can read various opinions about this. The orthodox approach is that you should back up cloud data. Ideally, to two sources, one of those would be a local source, and one of those, that could be a network attached storage system in NAS. It could be a local server. Whatever could just be on your computer. And that you should also back it up to another cloud. So yes, this is very, very unlikely that, for example, Google is going to go out of business or whatever or lose your data overnight, although theoretically could happen. But you might have something like an accidental deletion. And this is something I have seen before, in which people delete accidentally either their entire Google Drive or a critical folder and somehow also manage to delete that from the trash bin. And no, Google does not, by default, have this built-in elaborate backup functionality with different restore points you can roll back to. It doesn't have that, and it's your responsibility to protect your data. So that's another point about cloud backups is that the onus is actually on you, the user. And if you read the fine print of a lot of these cloud services, you might be surprised with these. That's actually the case of the cloud. Cloud providers actually do not take responsibility for backing up your data, even though they do. But that is your job. So Google Takeout is the starting point for this process for getting your data out of the Google ecosystem, I would say. So basically, there's a lot of different apps located there. It's not a backup tool. That should be pointed out. What I'm doing here is creating a last resource backup in the cloud. And that sounds like a contradiction. But it's not itself a backup tool. There are backup tools. If you look for G Suite to S3 backup, for example, you can find a lot of cloud to cloud providers. For example, I'm just trying to find familiar names here. MSP360, there's many more than that, that actually queue backup, that actually provide the service. That's different. That's backing up G Suite, and it's backing up certain aspects for the whole G Suite account. So it can be typically multi-users and typically not focused on the kind of stuff Google Takeout does, which is everything a user generates. So if you're one person trying to extract your data, this is actually, I think, the better source. A G Suite to AWS S3, for example, might just be taking from a foreign administrator to take out everybody's mailbox, everybody's Google Drive, everybody's contacts, and put those into backups. They're not necessarily going to be interested in getting out stuff like your Google Play movies and TV preferences. So this gets everything, everything of Google Service. Now, you can either choose to backup everything. That's a default option. Or you can just go for specific services, like, for example, Chrome. And even within each service, you can select just exactly what you want. So it's actually very, very granular. What I'm going to do for the purpose of simplicity is just to go for everything. I'm not actually going to generate this Takeout because to get this video prepared, I have already gone ahead and done that on this. And this is just a Gmail account. It's just a demonstration Gmail I've set up for making a few videos like this. So you don't need to have G Suite to access this Takeout function. And I do recommend that you do it at least occasionally. If occasionally it's only like me, I only do this once a year. It's just to really make sure that my Google, all my YouTube videos, all my Google Drive, I do more regularly, but my Google Contacts, my bookmarks, that it's not solely dependent upon Google. That if Google were taken out of the picture, for whatever reason, I screwed up, I deleted, or they just lost data, my data would still be somewhere else. And from another location, another cloud, I could go about restoring it on a different platform or Google itself. So basically, now what you can do and you might be wondering why I just wouldn't do a Takeout to Google Drive because they do give you this option and then run it cloud to cloud from our clone. You could do that, but for the purpose of this video, I had to do this process last night, the regular email option because I had done too many Takeouts and this method of just firing up an EC2 instance and doing cloud to cloud has got a lot more applicability than just doing Google Takeout. So I'm gonna show it that way for this video for that reason. Now you can, I typically will generate a TGZ file. If you're using Linux, this should be pointed out like I am using it, you can store up to 50 gigabytes in one TGZ archive. So there's no reason if you're using Linux not to take advantage of that and use zip. If you're using Windows, then zip is easier because you don't need to hassle about opening up a TGZ file. So then you click on the export button. So I've already gone through this process. You will get two emails. One is to say that your Takeout's being requested and that generates immediately, that's how you know it's running by the way. If you do not get that email and I just was on to TGZ support last night for about 30 minutes, there weren't very, I'll try to be politically correct. We did not succeed in resolving the issue. And I kinda knew that was the case that it was and they weren't generating because I didn't get this. So you get this straight away and then you get this. So basically I'm not gonna download this archive on my local here. I'm going to do that in the EC2 instance and do this cloud to cloud. So let us move on to the next step. AWS have put together some useful documentation about installing a GUI onto an EC2 instance. So this is what I read through last night in order to make sure I had everything on my local machine to get up and running. So you can find an AMI, an Amazon machine image that actually has a desktop installed. You don't need to go through the process. This step they have listed out here of installing Mate or you know, some other desktop and they have everything you can go for Unity or you can go for LXDE, which this machine is LXDE. But there is an Amazon Linux 2 machine ready to roll in EC2, which already has the desktop. So you don't need to do that. What you do need to do is make sure you have TigerVNC, a TigerVNC client installed on your local machine. So if I just bring across this screen, I just downloaded this. I'm just gonna move this into my programs directory, type VNC. I think I installed the wrong one. So it's TigerVNC, oops. TigerVNC and just extract this, blow this up here. And now I have TigerVNC and this is it. The executable is here. Just give that a second to open and I'm ready now to connect once I can establish a connection to the EC2 instance. So I'd recommend just getting that ready before you actually fire up EC2. You know, EC2 does have an hourly running cost. It's tiny, so you don't need to skimp about keeping it running for the shortest possible time period. I'm just recommending this as a workflow so that everything is ready after EC2 is running. Okay, so I've logged into AWS here into the management console. As you can see, I'm in the North Virginia region here. Now it's important if you're trying to make sure that you don't have, you know, you don't leave EC2 instances that you only need once every six months running. You might check and then see you're still getting built. So just make sure that you just realize that you can toggle the region, the AWS region you're in here, in here. So I only do my EC2 instances in North Virginia, which is US East One, that region. So I'm just at the first EC2 screen here, didn't do anything besides, you know, go down to the services dropdown and go into EC2. And I simply search for desktop so that I would not have to install one. In AWS Marketplace and Community AMIs, you'll find many, many, many, many more options, including ones with different desktop environments, DEs for Linux, made us find it's a fairly lightweight desktop environment, so it doesn't take too many resources. So that's fine. So I just went for this. And then you choose, now this is important because you should look at how big your takeout is. So the one I have done should be very, very small. It's tiny, it's 2.3 megabytes. So, you know, I could do this for my local in a second, but the one I did last night was over 50 gigabytes. And at my upload speed of two megabytes per second, megabits per second, the time estimate was like three days. And I did this in about 30 minutes on EC2. So just ignore the fact that it's tiny, this will run almost instantaneously on the connection. So basically, you know, you're looking at your storage here, so you want to find something that is going to fit. So you can either go for EBS, Elastic Bucket Storage, or you can go for an SSD size. So for example, I was running a downloading a 50 gigabyte takeout. So I chose to start off with a 75 gigabyte SSD. And the connection speed is given here as well. So if it's a particularly big file, you can see network performance, the rate of data transfer, up to 5 gigabits, up to 20, 10. This is so, you know, if it's particularly big, you'll want to choose an instance type, which is going to give you some kind of a reasonable upload speed. So the next thing you're going to do after selecting the type is you're going to need to create a connection key. So I'm just going to say, I'm going to call this key demo account. It's a new key pair, and I'm going to download this and put this on my desktop. So it's given me a key file ending in PEM, demo account dot PEM. And I'm going to just put this onto my desktop. And I'm just seeing, I think, ah, it's gone into downloads. I'm just going to move that to my desktop. And that'll allow me to connect. So we're now initiating the launch of this instance over here. And I just moved the connection key onto my desktop. And you can see that it's currently initializing. So I'm going to go ahead and get my terminal ready in order to connect to it. While this has been going on, I've logged into my Backblaze account over here. And you can see the bucket here is what I've done. I've created a bucket called demo bucket DR. So I'm going to just take down a few details using any kind of text editor and notepads of some sorts if I can find one that's going to use Kate. So just take down a few details here. And obviously, if you're doing this on an ongoing basis, I would be saving these into something like GlassPass. So just for B2, just you want to put down your bucket name. And you're going to create credentials, an app key for RClone running in the cloud. So you want your app key and your secret key. And I'm going to create those now and just copy those over onto my notepad here. OK, so I've gone ahead and created a key for RClone in Backblaze B2. Now, obviously, if you're using AWS S3, you're going to be running within the AWS ecosystem. So you'll have a little bit less clicking around to do. But it's obviously a comparable process. So I'm now going to go ahead and connect. Just going to move this over here. And I'm going to connect into my EC2 instance. So we just want to make sure that we're on the desktop firstly, because that's where I saved the key file. And you can see it just lets you copy and paste. Now I know that this is going to throw up a problem. I think the username in this example. So ssh minus i, and this is where you tell it where. The key file is mine is demo. So I could have just actually removed the double brackets. And it's not root. It's ECT user. You can actually show you can do this. And it'll tell you anyway what the actual username is. So one moment. So I just quickly realized that I needed to just give. So I'm just going to change the permissions on the key file to 600 onto demo. So just follow the documentation. I forgot to do that step. And then you're going to connect. I'm just going to connect firstly. And this will tell you that it's EC2. So just change the username to what it is. EC2 user at the end point. OK, now we have a connection. So now what I'm going to do is go ahead and start the VNC connection as well. So I'm keeping two connections open, the direct SSH tunnel, and also at the same time establishing the VNC connection. I'm doing that just so that I can quickly run commands over the terminal. And I'm actually going to go ahead and run one now just so that our RClone's ready. So it's sudo yum install RClone. I'm installing RClone onto my EC2 instance. And you can see that it's picked it up from the repositories, 8 megabytes. And just for a bunch of users, it's sudo yum on Amazon Linux just to remind. So that's installed. So when I actually get in now, the package is going to be ready there for us. So again, following the AWS documentation here about installing a GUI in EC2, I'm just copying this command here. It's going to start the VNC and start a local process and port 5901. So obviously we need to modify this command a tiny bit. The key file, we're on the desktop, so we don't need to. It can be just, we don't need to give the relative path. It is simply demoaccount.pem. That is the correct username, ectuser. And the instance ip is here we go. This is the hostname here. So I'm just going to copy that into the terminal ectuser and just backspace through this and just copy that. And it takes about a second. And now we can confirm that we have that connection made here. And now you'll need to start up the, and remember it's already installed, the server here. VM server, VNC server one. So this will start VNC server after we install the package. So VNC server one, I already have it running. So now we need to just create a password for it. So I'm just typing in and confirming a password. OK, so now you can also the, I'm just again following the documentation. You can have this so that it runs if you're reusing this instance that will run on boot. I just use this instance for my yearly job and then got rid of it last night. And you have to check for storage costs as well. So even if you stop an EC2 instance, you need to make sure that the attached storage, there's ways to put that into an archive. But if you're only using it once a year, it probably just makes sense to start a new EC2 instance and terminate it after you're finished running. So we've gone through this, and now we need to just connect locally. So I've just dragged across my VNC viewer. So just remember, we just started VNC server on the EC2 instance. And now we need to connect. So sorry, I'm missing an L there. Local host one. Now this is such, and we're not specifying the port because we've already started the connection with this command over port 5901 on the local host. So we just need to enter the passwords. And the moment of truth, it has worked. So this is our EC2 instance. And we have our clone running. So I'm just going to make this full screen here. This is the final step of the process. So basically, what we could do now is if you just type our clone in here, we can see that the package is installed and we're getting just the man page over there. So what I did last night is as follows. And this, remember, was for a 50 gigabyte takeout as opposed to 2 megabyte one. Just navigate to Google, takeouts. I like to also test, just see how quick this connection is as well if I want to get an idea for how long pushing it up through our cloud is going to take. So you can actually just run. And again, don't forget, this is actually all in the EC2 instance here in a virtual desktop. It's kind of cool that you can just run it as if you were just doing it on your local computer and you can see it's picked up the EC2 instances IP and Amazon as data center. Now notice that the download speed is nothing particularly breathtaking, but wait till you see the upload speed on even a fairly not spectacular connection over here. So it's 400 in the region of 400 megabits per second coming down to small bits. So I'm just going to get rid of that because one thing, unless you install a instance type with a lot of RAM, it can struggle to do even basic things. So I just kind of use it as user browser as lightly as possible with this few tabs. So Google Takeouts, and I'm just going to sign in quickly to the demo account. So I've just gone through the verification and obviously because if you're doing this, you're signing in through a totally different IP address. It's going to trigger Google security mechanisms. I had to just quickly give my recovery email address and I got through. Last night because I'm on two-factor authentication, I needed to do the automated call thing several times over, in fact, a bit of a pain. So our very, very small backup takeout is here, but there's no reason this couldn't be a big takeout. And again, we're getting some kind of messy stuff. So I'll just go through this quickly and get the file onto the desktop. OK, so we passed through that. And our takeout file is downloaded there. So we just need to check. This is actually Chromium. We just need to check where that went. 2, based on the defaults download, probably downloads. Yeah, downloads. So let's just go into our terminal here. And let's just jump into the downloads folder. And there is our takeout file. OK, so now we need to configure our remote directory in ArcLone. I'm just going to head and delete us and close down the browser. Just remember what I said about the resources and some of these EC2 instances or not. If you're going for a light machine, it's not really designed to be using a virtual desktop like this. So it's ArcLone minus minus. Tac-tac config. Some people call it Tac-tac. Flag these arguments. So just a small error in my syntax. I just quickly checked the man page. I need to update my post. Actually, there's no, it's just ArcLone config. So it asks you for your remote. So N is for a new remote. I'm going to call this just B2 because there's only going to be one. Now you get this big list of different storage types here. And you just need to see which number yours is. Of course, you can check this online as well. Back plays B2 is five. So put in the number of the storage type of your remote. And this is where you put in. So I'm just going to bring in the data that I saved here. So we can see what the key ID is. I'm just going to copy this into the terminal. I'm going to try copy this into the terminal again. We should have two-way pasting here. There we go. And copy in the secret key, the application key, the second. This is it. So again, if you're using S3, it's going to be a similar process. So that just didn't catch. And it's going to do this a little bit more manually. And again, there we go. Paste and just verify that's correct. And you can just go through the various options. I don't need to go into the Advanced Config. And that's fine. So I've now created that repository in Artlone. And you can also quit, just Q is to quit. So now we are ready to run the final bit of this. Now, just to remind what is the novelty of what I'm doing at the moment, it's not on my local desktop. I'm running on an EC2 instance. And therefore, this is a cloud-to-cloud backup. It could be direct cloud-to-cloud backup if you were simply using. You could use Artlone on your local and just move between the two. But if you need, for whatever reason, to have a GUI to pull down a take-out that can be in your Google Drive or something else, then this is an approach that you can use. So it is Artlone sync. I'm just going to actually add the verbosity operator there. And just start typing. And then hit the tab to fill that out for take-out. And then we are putting in our remote. So it was, we call it B2. Our bucket name is demo bucket dr. And I'm just going to leave that at the root of the directory. So there you go. It's already run. And you can see we moved up 2.3 megabytes in 2.3 seconds. So that was almost instantaneous. So I'm in the backblaze web UI in my second screen. So I'm just going to bring that over. And as you can see, in demo bucket dr, the take-out file 2.4 megabytes is there. And I just did that quickly to verify that, indeed, the uploaded process successfully. So that's how you do it. Firing up EC2, you could use a VPS as well. Actually, if you're using EC2, it'll work out a lot cheaper. It's just a lot easier. I was thinking about doing a VPS. And then I thought, if I already have AWS, why not just fire up an EC2 instance and set up RClone in a second and run stuff across? So this has broad application in terms of if you're doing cloud-to-cloud backups, they can be done via RClone on your local thing. But if you have the ability to virtualize a desktop and interface with browser-based user interfaces, it just opens up more possibilities and then run RClone on the EC2 itself and just push up to the cloud, you will be able to benefit from an uplink and upload speed of typically above 450 megabits per second. So an awful, awful lot faster than doing it locally. So my 50 gigabytes upload took about 30 minutes yesterday at Google Takeout on RClone, on RClone running on EC2 versus it would have taken, I think the estimate was almost three days if I had done that, pushed it up just for my local machine. So thank you for checking out the video. If you do want to get in touch for whatever reason, this is my website, danielrosil.co.il. I'd love to hear your feedback or anyone else in the whole backup realm that wants to discuss this. I'd be always, always, always willing to chat. Thanks for watching, have a great day.