 Well, thank you all for coming, I am of late to work as senior software engineer in Red Hat and I'm going to talk to you today about disaster recovery in Ogre. So let's start. Okay, so any of you who went to the overworked workshop yesterday? Well, I have to leave halfway, so yeah. Okay, well, poke it in short. OREIT International is an open source project started by Red Hat. It is a virtualization management platform. And it's made on a Linux virtualization, you know, like KVM, QM on Big Bird. It actually manages virtual machines like VMs and templates. Also storage domain virtual, create virtual disks on them and virtualize networks. So this is the architecture of OREIT. I'm going to focus mainly on the OREIT engine part combined with shared storage, shared storage domains. This is going to be the disaster recovery session that I'm going to talk about. Okay, so OREIT supports a few types of storage domains, mainly file storage domain and block storage domains. On the file storage domain, OREIT supports storage domains like NFS, disaster, repository compliance, NFS, and the local storage domain. On the block storage domain, OREIT supports fiber churn storage domain and I discuss in the storage domain. So while I'm building this presentation, I was thinking what's the first thing that crossed my mind when talking disaster recovery. So the first thing that crossed my mind was Murphy's loss. And maybe the most sentence I really liked is, if there is a worst type or something to go wrong, it will happen then. You can really anticipate a disaster. Actually, the difference between other features that you develop and disaster recovery is that you really wish that you would never use this feature ever. You don't really think about disaster when you're writing your program. It seems that every single worker, you don't have any bugs on it. So this is how disaster recovery is really important because when a disaster occurs, you really want your debt to be restored as quickly as possible. So this is kind of a unique feature that every application needs. Okay, so these are four general reasons why we need disaster recovery. Most of them are general, machines and hardware might fail and humans also make mistakes and nature is unpredictable. But I think I want to focus more on customer want access 24, 7, 365 days a week. So this is a bit misleading because customers want it, but actually they more expect us to be accessed 24, 7, 365 days a week. You're connecting to your Gmail server and Gmail tells you, well, we are sorry, we had a disaster, so all your mail got deleted. We are sorry. At the minute that will happen, you won't use Gmail anymore. So in the era today, this is really crucial. You need to have all the data and customer expects you that you will have this data anytime and any place. So how we are using disaster recovery until over 3.5. So in over 3.5 among other features we had a special storage domain called the export storage domain. The export storage domain is a file storage domain and it is mainly a special storage domain that contains all the VMs and templates and all the disks in it. It is based on an OVF and I'm going to explain a bit about what is OVF. OVF is an open virtualization format and it actually describes the VM configuration. When you are talking about management like OVIRT, this is only the configuration of the VMs. The VM actually the process itself, the curing process running on the house. OVF simply tells you if the VM is running or how much memory it got or how much CPU it got. So OVF actually is an external representation of all this data which consists in your database that OVF actually reflects. So once the VM is being exported to the export storage domain to be used as a backup, what actually is being done is that we copy all the VM disks into this export domain and we also take all this data from the database, convert it to an OVF file and put it on the export storage domain. So once we have it backed up in the export storage domain and the disaster occurs, we can just use the export storage domain imported to another data center and import it back into the database. So among other things that you can do with the export storage domain, like I said, you can use the migrate entity between different set-ups or data centers. It can be moved between set-ups, so this is the main difference between the original and the... and this is the main difference between the data storage domain and the export storage domain and it also can be used to backup entities. Okay, so can anyone tell me maybe what could be a problem? There are several problems using the export domain as a backup. Well, I won't get hard on you, but I will just... There are three main problems using the export domain as a backup storage domain. The first problem is main scalability. Okay, using one storage domain to backup all your data centers. So it can be used very good for small data centers, but once you are adding more VMs and more disks, you need to also add more storage to your export domain. So when we are talking about a data center called thousands of VMs and thousands of disks, you will have to maintain some kind of petabytes or a lot of storage to maintain for this export domain. Well, another main problem is that copying data just takes too long. You try to copy terabytes, it will take you days and even weeks. If you want to use backup and you want to use it very quickly for recovery, it's not a practical solution. The third problem is you have a single point failure. Once your export domain gets ruined, then all your data centers decrease. You can lose all your data and you won't have a backup. So what was introduced in over 3.5? So, storage domain until 3.5 contain several properties. One of the properties is the metadata, which is actually the type of the storage domain and the file, for example. The storage pool, which is attached to what kind of storage domain is it and its name. And it also contains all the disks. So what we added in over 3.5 is that we added a new special disk called the OVF store disk. The OVF store disk actually encapsulates all the OVF that I talked earlier in the export domain into one tar file and just put it on the OVF store disk on the storage. So the storage itself becomes more of a self-contained of all its data, because you already have the disks, you also have the knowledge about the VMs and the templates, so you can actually use it very flexibly with other data centers and other setups. The other feature which was introduced is the import data storage domain. So until over 3.5 you couldn't really use the data storage domain in other setups or in other data centers. Now you can import the data storage domain. So once we have the OVF store disk and we also can import the data storage domain, it's actually become much more easy to use it for backup and recovery, because you don't need to copy it anymore. All the disks are already in the storage domain. So with the OVF store disk you simply register those VMs into the new setup. So I'm going to show you an example of how it is being used in OVF. So let's say you have a disaster and now you build up a new setup and you want to import the storage domain and you want to use those VMs which are in it to register them and recover them. So first of all you import the storage domain. This is an example of importing, for example, an NFS storage domain. Simply put the export path. Once it's being imported we attach it to a data center and then you can see here all the VMs which this storage domain contains. It's actually the OVF. We take the OVF from the storage domain, the OVF store disk. We generate this data and we show it in this setup here. So the user can pick which VMs he wants to register and recover. After they recover we choose the cluster and the quota for each entity. Cluster and quota are logical entities in OVF. It actually reflects the feature those VMs supports. So for example if a VM has a shared disk with a feature which was introduced in 3.1 we can't register it into a cluster which supports only 3.0. So after that being done the VMs just register into the setup. So the main differences between the export domain and the import storage domain is that we don't use any copy. All the disks are already in the storage domain itself. So what we really use is only the OVF store disk, only the XML. So this operation takes much less time and it's much more practical to use for recovery today. So as we saw, we don't use the copy operation so it takes much less time. There is not single point of failure because there are main storage domains and in case of a problem you can start from a partial setup or just start using it as it is and we also don't have the scalability issue anymore. So this is an estimation of how much time it will take us to recover a setup. And you can see here the red area is the time that will take the export domain. It's mostly an estimation because you can't really know how the storage domain will act and behave underline. But you see the difference as there is a major difference. The yellow line here is actually the time that takes to import the data storage domain and register all its VMs. And you can see that the copy operation since it's not being used in the import data storage domain you can see really the difference between the two. So what are you planning in the future? Well we still have some more nice improvements that we want to add to this feature. For example we want to clone VMs and templates from storage domain. Just take the storage domain and clone it into another storage domain and try to import one VM from one storage domain and maybe clone another VM from the clone storage domain and use it in the same setup. Convert an export domain to a data domain so people should not do all this operation of copying it into another data domain. And you can also suggest other things that you are interested in or you think can be really cool in the Overture community site here. So this is a weekly link of the feature itself. You can see YouTube videos, how it can be done, the recovery for example of the local data center, you can do recovery for Blaster, how the import file storage domain really behaves and how to use it in Overt. Everything is really on the net and you can see how it's really simple to use. Ok, well, it was fast. Are there any questions? Yeah. Is it required to have a state storage or can it be done on before the storage domain doesn't provide the back end storage? Talking about the storage domain, as I showed before here, it should support all of those storage domains. So actually Overt use each storage domain as a virtualized storage domain. So though we have stored this, actually once it is being copied into one of the set of storage domain then you are actually set. Overt can work with it. Just need to import this storage domain which actually already supports the integration of this and once it's being connected with the host to use it, as a short storage for example, then it simply gets the device of the OVF store this and just register all the XML from there. So it's pretty virtual. It's already being done over and before. We just use this to register those entities. Do you think as storage domain, if there's a load called, you know, there is how can it transfer from one load called like physical conditions to another? Ah, ok. Next, you just have to have a scale of that. Yeah, so once you're creating your new setup for recovery, then you create a new data center, a new local data center and once we import this local storage domain, you have to use the same host as before because it's local and it's on the host itself. If you don't have the connection to it, you can't really use it. So once this storage domain is exist on the host server, we just use the host to mount it to itself and just take all the data from it. So the data is already there. Actually, there is an example in YouTube how it can be done, but are you asking how it can do it practically in Overview or how it's being done underline with video Sam or the host itself? Is that answered your question? Yes, I think Berkeley, I know that's Berkeley, I don't have to get answer that one. The storage domain, the storage domain would have us to make it faster and to make it not so way. It's the most real day transfer, but to be able to transfer, to recover, it's faster. And to take out to the day, but it may take two hours or a day or two hours to recover. So I'm talking especially about the local storage domain. How does that help to recover from comparing to the external one, for example? Actually, the local storage domain or any data storage domain is not really different from any other storage domain. The thing about the external storage domain is that it is a separate storage domain that we just needed to copy the data. The copy of the disks itself was a bottleneck for recovery operations. So once we are preserving these objects for this one storage domain, it doesn't really matter if you're using a local cluster manifest because all the data is there. So for example, before the SQL file, let's say that you use a local storage domain and you have a disaster and you want to recover only that. So the local storage domain is... So for example, the local storage domain didn't really have those VMs before. So you need to copy them, again, you need to attach the export domain and copy them again to the local storage. So this is the main bottleneck or the main issue that they started. So regarding... Again, regarding the local, the manifestly cluster, all the rest of the storage domains, whether they are shared or whether they are local, it should be the same process virtually where we're using the storage domain to search the domain and get the other storage disk. So I hope I answered your question. If not, we can talk again. Any other questions? Enjoy. Thank you. Thank you.