 Good afternoon, everyone. My name is Toby Chen. I'm the storage engineer from United Stack, which is the first Open Stack service provider in China. If you are interested in what we do, you can go to usethat.com or just Google United Stack to know more about what we did. So today I would like to talk about the new feature about Open Stack image service and how to leverage advantage of multiple backends in glance. This work is actually done by my colleague Leidong and me. So here are our email address. Supporting multiple backends in glance is really a pay point for all Open Slash users because Cinder could support multiple backends. Noah could support multiple backends, but not glance. So before introducing what we did, let's focus on this architecture first. So I assure that everyone is familiar with that. Under the hood of Open Stack platforms, we use safe, unified storage for both Noah, Cinder, and glance. It is quite normal white. But this architecture may change especially when customers want to add more storage backends. So now we would like to add another commercial storage for their data. It is possible because Cinder could support one or more backends in the same Open Stack platform. Be aware of that. Although in these situations, all the virtual machines are running on the safe, unified storage. So then someone could ask, can we run virtual machine in all the storage backends? So definitely yes. We could run Noah Compute to connect with the SAP cluster and other Noah Compute for commercial storage. Currently, it looks great. However, there is one problem about the images. If glance uses the SAP RBD drivers, you have to copy the compete images to commercial storage before putting the virtual machines there. So it would be much better to deploy two glance services in both safe, unified storage and commercial storage. But you know it's impossible. If you don't want to get mess with the dedicated end parts of the image service, so therefore, if you deploy the glance upon a commercial storage, the old one should be taken. If we change the commercial storage to another SAP cluster, the problem still exists. In fact, glance is unable to connect with multiple SAP clusters without our patch, which I will talk about later. So no matter which size we want to deploy, Noah have to copy the compete images to put the virtual machines from the other side. So don't question about that because we have tested it and it takes a quite a long time to export and import the images twice. So what are the problems? Why could glance not support multiple batch ends? Now, I will summarize all the problems here. So there are many restrictions for glance to support the real multiple batch ends. Firstly, we have only a few driver for glance. Secondly, there is some limitations of the feature multi-location, which I will also talk about soon. Firstly, we have multi-location and we have some strategy to select a location. But there are some difficulties for that. And the last one is that we have the limitations to config multiple SAP cluster by end. So these problems still exist when we are using the Mitaka OpenStack or the upstream glance. So that's why I'm here today to introduce our work and to address these problems. So here are our solutions. At first, I will introduce the feature of the multi-location and its limitations. And then we have improvements, the new strategy for multi-location, and improve NOVA and GANs for this feature. And last, I will talk about the future support for multiple SAP band. So someone may know about GANs multi-location feature. And it is merged as the new feature in the version 2 APIs. So take this for example. If we want GANs image source for this image, it shows up three different locations here. So NOVA may request GANs for all the location of these images, and try to along or download the images. If you try the files first, in this case, if the file does not exist, there's some problem about that animations. If the file does not exist, it will try Cinder and HTTP later. So how to enable these functions? And now we have the GANs service. And it did the configuration file and add some parameters. Then we start the GANs service. And NOVA can now we can create an image and upload a file for that. So now the image you have the first file location there. Then we want the command CUR or GANs CRI to add more locations, just like RBD locations or others. Now if we focus on this command, we will find out that we just specify RBD address but not which SAP cluster to use. Actually, the GANs configuration files we add the location of the SAP.conf, which indicated the specified SAP clusters. In this case, the SAP monitors IPs look like this. But how can we specify and use two or more SAP clusters? No way. Unfortunately, there's no way to configure and use two SAP clusters, like the images on the right corner. We can only choose only the left one or the right ones, but not both. So it starts right. There's some other limitations of the GANs multi-location features. Currently, we are not able to boot VM with the specified location. Similarly, we cannot create a bootable volume with the specified locations. And now, different SAP clusters use the same RBD store, just what I show above. And then it has the limitation to config multiple SAP bands. To solve these problems, I would like to introduce the new strategies we have in performance in our environments, which allows our other OpenStack component to get a specific location on demand. So firstly, I would like to introduce the existing strategies. There are store type strategy locations, and rotation order strategy. So first, store type strategy means that the order of locations just depends on the user defined store types. Now we can edit the configuration file and add some types in the store type preference. And GANs will return the locations of the images one by one. So in this case, they are file, then RBD, Cinder, and HTTP, one by one. And the second strategy is location order strategy. It's simpler and return the locations in its origin orders. So take this as an example. Noah could get the images from local file first. So it's the first priorities. But if the file does not exist, it's try to use the Cinder and then HTTP at last. In fact, these two strategies are quick configs. And user are not able to choose the locations dynamically. So we use the first strategy, which is already improvements in our environments. And we call it the specific type strategies. To enable strategies, we just need to edit the GANs configuration file to use the specified types from location strategy. And then you can get a wide locations when you're passing the immediate store parameter to GAN service, just like this. And for more details, I would like to want the command GANsImageSold with the parameter ImageStore as Cinder. And then I want the commands for the same images, but with the different parameter ImageStore as IBD. So now, child result. The first image is just returned a Cinder location because we pass the parameters to specify the ImageStore. And the second image is returned the IBD locations. In fact, we all know it's an image with multiple locations. But with our work, now we are able to get whatever location we want. So having the new specified type strategy is not enough. We have to improve the functionality of NOAA and Cinder so that we can choose any storage back end to put the virtual machines and create the bootable volumes correctly. And firstly, we implement the parameter ImageStore in the server side and client side of NOAA. It is super easy to use. You can specify the ImageStore. And GANs will return just the right location for that. In this case, we specify to use the image in file. Then the file location will return as expect. Well, it always work. And in a similar way, Cinder has the similar functions with when creating the bootable volume from GANs. So we can specify the ImageStore. In this case, it's IBD. So then the function will returns the IBD locations to the service. After adding this new feature for GANs, NOAA, and Cinder, we are able to choose the specified location of Image. And that's really a great improvement for GANs to support the multiple back ends. Now we can choose the storage back ends to get the image and boot virtual machines in the right way. But is that all? Of course not. We do more than that. There's still a problem I just mentioned about for GANs to support multiple step cluster. So in other words, we could not add two IBD locations in the same image or GANs could even not connect with two or more step clusters. So why? Because there's only one configuration for step.com in GANs configuration files. So now we have to change the code to read arbitrary number of step back ends just like this. And notice that now GANs could specify two step.com instead of one and actually connect with two step cluster in the same process. So it is the first step and the most important feature for GANs to support multiple step back ends. But how can we use that? So there's a famous saying, talk is chips, show me the demo. So let's watch the video we have made. This video is about how to config and upload GANs imagery into two different step clusters. Firstly, we will add step back ends. Now we edit the GANs configuration files. We add a new session for step one and step two. It looks pretty like the configuration of Cinder because it's the only way for GANs to support multiple step back ends. So now we need to change the step.com for step two and enable two back ends. Be aware of this. You can only apply this patch. You can do this after applying our patch. And now save the configuration. We attach the step back sessions and stop. And we start the process of GANs API. OK, now we can upload the imagery into step. And the first command, upload the imagery in the step one cluster. And then we run the similar commands but with the parameter image type as step two. Please notice that these two images has different step cluster ID, which means that they are stored in a different step cluster. And that's it. I'm not sure if everyone can look at the pretty tiny commands, but trust me, it's the real demos we just made. And if you are interested in this, please check it out. In YouTube, maybe tonight after they upload the videos. So it may not be a big deal, but there are many developers or users looking forward to these features. And it's reasonable. And we already have the blueprint for that GANs multi-store back end support. It would be great to contribute for that, but there's still a lot of work to do. So it's time to make the summaries. We have extended feature of GANs multi-locations, implement the new strategy, and improve the APIs of NOVA and Cinder. That makes it possible for a user to specify locations, storage back end, to put the virtual machines, and create the bootable volumes. That's what we call the GANs multiple back end support, not just the multi-location feature. At last, I would like to bring you back to this slide. Now the GAN service is still on the right side of the commercial storage. So we can move it to the left side, in the set unified storage, move it back to the right side. Anyways, it's not really good enough to boot VMs from these two storage bands. So however, now we have our patch for multi-location feature and implement the new specified type strategy. That means GANs is able to connect with two or more different storage back ends finally. And we can leverage its advantages to boot virtual machines with copy on wide feature, no matter which bands we are using. So using set unified storage, or using the commercial storage, or just both, it could not be the problem for open state users to any more. So that's it. Thanks for your attendance. And there's still some time for Q&A. So any question about that? What version of OpenStack was your patch originally designed for? We just test it in Liberty. In Liberty? Yes. Cool. Thank you. Be aware of that this patch is test, and we have the demo for that. But it's not really compete to merge back to the community because there are still a lot of work to do. No plan to backboard previous versions? Juno, Isoz? Excuse me? So you mentioned only on Liberty. So any plan for backboard on Isoz or Juno? I don't think the community with the back part are the feature for the older versions. But we could do that if this possible. OK, so that's it. Thank you for attending. Thank you.