 So, good morning, Stackers. First, a sensible tenure disperse session and sensible time. Before this presentation, just a few minutes to give a brief describe about me and the background of this cloud. Today, I'm going to bring you a wonderful story about the best practice to build a private cloud platform which based on OpenStack. So for me, my name is Charles, and I'm from H3C and I'm working as a product manager. So H3C is a Chinese ID company which can provide the customer hardware, software, and the total solution. So for the hardware, we can provide the customer the physical server storage devices and the network devices and the security devices. And for the software, we can provide the customer the visualization platform, the SDN controller cloud platform, and the big data platform. So in conclusion, what we provide the customer is that we are not a public cloud provider. What we do is just to help the customer to build their own private cloud descent by providing our product and the consultant service and the total solution. So this cloud I'm going to share you today is that we built with our customer together. So we have 14 minutes for this break session. And if you guys have a message, we'll go about 30 minutes and we will try to make a 10 or 5 or 10 minutes for the Q&A. So if you guys have any questions, just drop them down first and we'll leave them to the Q&A stage, OK? Just I'd just like to show you what we're going to try to share with you today can be separated into four different parts. First, I will give a brief description about the company. And second, I will just tell the story behind this cloud. It's about why the customer want to build this cloud and what's the problem they faced with the existing data center. So the third is that I want to share with you about the technical point. This way in controlling building this cloud from the last, you know, it's a big, it's a huge cloud data center. So we just want to offer you some tips for optimizing. First, China offshore oil group. It is one of the three largest oil group in China. I mean oil company. And it was founded in 1982. And after it was founded, you know, within a few past 30 years, it has advanced its global business across Asia, Europe, Australia, and Africa. And now it has over 10, sorry, it has over 100, sorry, 10,000 staff. So it has a total of 1,662 billion R&B assets in total. And now also they are the top 500 in a global. So I just cannot remember exactly what rank it is. I mean, maybe in 2016, it ranks 109. So what business they hire, you know, it's an oil group. So their main business is to, you know, the oil extraction that they will build the production site on the seat and will build the drilling wall to extract the crude oil and turn them into process and turn them into the secondary energy such as gasoline, diesel, and jet fuel, and some other chemical materials. And they also built, you know, the gas station across the country. And they also provide the, you know, the filling service. So besides the oil and the secondary energy product, they also provide the industrial equipment, such as, you know, the air compressor and the power generator. And they also provide the financial services. So China offshore company is a listed company. And they also provide the financial services, such as the stock service, the trust and the funding services. And right now, they have over 30 large data centers across China. You know, China is a big country. And they have their cloud data centers in the northern, eastern, and southern part of China. And now they have running about 600 coal plants on their cloud data centers. Sorry, it's not cloud data centers. I mean existing cloud, existing data centers. So I have read the core applications. There are about 600 and too many. So since the time is limited today, I just cannot tell you the applications step by step. And besides, I don't have that strong memory, right? So but we can start these applications into different categories. So they are, you know, the production services, the customer relationship management services. They are the financial system and office work for the staff. So they use these data centers to support their business. But right now, the problem they faced is that there are too many data centers. And each data center has their own devices. They recruit a lot of staff for different data centers. And the equipments in these data centers are from different vendors. So it's very hard for them to operate and to manage. And they need a unified way to manage these data centers. So just after a considerable consideration, and they have made a final decision to build a super large cloud in their data center. I mean the data center, the newly built cloud data center, first they will be built on the open stack. And we have the question is, what the cloud will be used for? So just as I said before, that's the data center, their existing data center are used for provide the results for their existing appliance. That means each data center are used for different departments. You know, the Chinese offshore company has a lot of subsidiary and has a lot of departments. And each department has a subsidiary on their own data center. So this newly built cloud platform will provide the service for different departments and the subsidiary. So that means if the staff from these departments are from this subsidiary near the new results, they don't need to build their own data center, just use the web portal and apply for their own cloud resources. So next is that how many data centers will be built. Sorry, I just cannot tell you how many data centers will be built, but these data centers will be built in China, in Singapore, Dubai, and some other parts of the world. And how large the scale is. You know, a fourth scale, there are a lot of ways to measure scale. You know, how many CPUs you have, how many disk groups, how many data you will start, and how many workloads you're going to host. So what I'm here trying to talk about is that it's a scale. What are we going to be built in fair years? That the physical CPU number will exceed about 60,000, think about it, 60,000 physical CPU. So there will be a lot of data centers across the world. And the most important thing is that what kind of services they will provide. So the cloud is going to provide a lot of services for the user. First, the infrastructure service that the user can apply for their virtual machine, and the virtual disk, virtual network, and the database. And they can also provide the service, I mean, the process service for the developer. The developer can upload their code to this cloud platform and the cloud platform will orchestrate all the workflow for them. Also, they will provide the SAS service, such as the CRM service and the financial service. And the user doesn't need to build their own applications and just log into the portal and can share the application. So how the service will be created? It's just like the public cloud data center. But we call it private cloud center because it only provides the service for their own stuff. It will not provide the service for the people from outside. It only provides the resource for its own stuff. And the same is that the administrator will prepare everything, such as the image, the software package, the script, and anything else. And then the tenant user, I mean, the user just finish a single click and they can share the resource. So that's how the service to be created. So we just talk about what will be looked like in one data center. So we have already built the cloud data center with the customer. And since there is a lot of equipment in the data center, so we separated these equipments in different rack. We have the specified rack for the management for the NFC components, computer resource, a network security device, and the same storage. So for the management rack, we will have the cloud platform, a steam controller, the visualization management platform, and some other management platform. I mean, for the whole environment, will be installed on the management rack. It's not a very rack. It's a lot of rack. Since the cloud platform will deploy it on lots of physical servers. So we will distribute it to these physical servers in different rack to guarantee its high availability. And for the NFC components, you must tell her about the NFC's virtual machine, which has the networking operating system installed that can provide the customer network service, such as the farewell, the load balancer, and some other services, such as anti-various. But the difference between each and the traditional network devices is that it was installed on the virtual machine. It's not equipment. It's what has the operating system installed on the virtual machine. And the virtual machine will provide the network devices for the end user. So the third rack we have is the computer resource. For the computer resource, we have different types. We will install the virtualization platform. We will install the virtualization platform, I mean, from different vendors, such as the vSphere, the KVM, the ZIN, and the Hyper-Ray virtualization platform installed on the host. We will also have the bare metal server on it. You know, some applications such as Oracle Database, it's not a good way to install it on the virtual machine. So we should install the database on the physical server. And also, we will have the Docker and some other container system installed on the computer resources, the computer rack. And for the network and the security device, is that we will have the core switch, the router, and the firewall, and the load balancer put on this rack. You know, what's the difference between these and NFC components is that the NFC is on the virtual machine, but they are actually physical devices. And they provide the traffic for nonce and south traffic. And NFC components usually use for to provide the traffic for the Weissman East traffic. So the send storage is a traditional send storage, which provides, you know, as SCSI, FC, and the file service for the computer resources. Also, we will build a distributed storage which is based on site on the computer resource. So today, that's the reason why we are here today, because, you know, this cloud platform is based on OpenStack. So we adopt OpenCommunity, OpenStack, but we add a lot of features on it, such as, you know, to make it more reliable, to make it, to improve the ability for the management. And as we can see from this page, that all the OpenStack service, such as Nova, Cinder, and Neutrona, other components are docker-ized. I mean, they are container-ized. They will deploy it on the docker engine, and we will use the Kubernetes to, you know, to schedule this OpenStack resources. And we, besides OpenStack service, we also have some other services, such as the database, middle wire, and the GUI. I mean, these components are all container-ized. Yeah, you know, all these components are put on the docker engine, and they use the Kubernetes to schedule them. And this is the kernel layer, and the full upper layer, that we call it, the system layer. What the system layer means, that, you know, this will be a cloud platform. So it should be easy for the administrator and the user to use it. So we add a lot to, you know, the other services, that's the OpenStack. I mean, the community version cannot provide. So we add a lot of other features, such as, you know, the charging services, the multi-tenant, and, you know, the pool policy, and some other, such as the business logical process. We add a lot of other features on this layer, and on the third layer is the service layer. That is for the under user. That means that the under user can just type the IP address you need and log in with their user password, and they can see the other services listed on the catalog and just with a single click, they can use the resources, such as, you know, the virtual machine and the storage network database and password and so on. So next, what we're going to talk to you is about the main part of this speech. It's about some technical points that we encountered when building this cloud platform with a customer. So first is integration with virtualization platform. You know, before we built this cloud platform, we talked with our customer, and they tell us that they will purchase a lot of new virtualization platform, but they also has a lot of legacy resources. You know, they have already created a lot of virtual machine on their existing vSphere platform, but they will also purchase a lot of new virtualization platform, which based on the KVM or HyperV or some other provider. So we need to find a way to, you know, use OpenState to integrate all these virtualization platform together. So what we do is that we will divide a lot of, you know, the driver, the NOAA driver, sender driver, and the Newton plug into integrated virtualization platform. Here we have an example that we take the vSphere and the KS system, which is, you know, provided by our company. It was a virtualization platform, which is based on the KVM for example. So the cloud platform is OpenStake. So we have the NOAA driver, a sender driver, and a Newton driver developed to integrate it with the vCenter and the CVK, which is the management platform for the virtualization platform. So after the integration, if you apply for cloud host, it will be a virtual machine on the ESX or on the KVM host. And if you want to, you know, apply for a virtual disk, it will be, sorry, if you want to apply for a block storage from the cloud platform, it will be actually a virtual disk on backend storage or the local drive. And if you want to apply for an internal network, or you create a VPC network, and actually it will, you know, the cloud platform will cause API or the management platform, and the management platform will create a lot of part group on the virtual switch across the vSphere and the KS system. So this is how we integrate with the virtualization platform. For the open source KVM, you know, for the open source KVM, we don't have the central management platform. So the open stack will use a nomad driver directly to talk to the KVM host. But for the solution this time, the virtualization platform have their own central management platform. So the open stack will directly cause API or vCenter or the KVM, and the KVM and the vCenter, sorry, the CVK and the vCenter will schedule the results across the host of the cluster. So how to integrate this bare metal server? So the bare metal server is a bit different from the virtualization platform. So in this cloud platform, we don't only support virtual machine, but we also have lots of physical server installed. You know, a lot of services such as the database or the ORAP system, which requires a lot of basic results. So it's not suitable to put them on the virtual machine. So the best way is to install them on the traditional physical servers. But there must be a way to schedule the results together. What I mean is that if I'm an user, I want to use the results. It should be only virtual machine, but if I want to, I need the results from the physical server. I just click the button that I need a physical server. So you don't care about what the results behind is. The cloud platform just provide an easy way for the end user, and the end user can share the results. So leave all the tasks to the cloud platform. And we take advantage of the Aronix driver. So the Aronix driver we can, you know, call the IPMI protocol to talk to the BMC on the physical server. And we can, such as the server or some other system from HP or from other vendors, they didn't use a general protocol. They have their private protocol. So we cooperate with these companies and we develop the driver to make it possible for the customer has a lot of choice for different physical server and to avoid the vendor lock-in. So in this case, a cloud host is actually a physical server. And the block storage is actually a logic line from the backend storage or local drive on the physical server. And the VPC network, you know, if we use a bare metal server, there will not be virtual switch. The bare metal server will use this physical, connect to the switch directly. So the cloud platform should has this ability to directly connect with the physical switch and sends the configuration files to the switch, such as create a V-line port and create a policy on the physical switch. And next we're gonna talk about the SDIN solution. So this is a very big cloud. You know, in the past, if you want to isolate the network from different tenant users, I mean, if I'm tenant A and he's tenant B and they want their, you know, once they are network isolated, so they need to find a way to isolate this network. And in the past, the V-line is the best solution. But the thing about this, if the scale or your cloud platform expands quickly, so you need to find a way to extend the V-line solution, you know, the number of the V-line is only 4,096. So we need to find a way to extend the number of the network, I mean, the logic number. So you hear, it adopts VX-line. Yeah, actually we have GRE and some other, some other ways to expand the network, but here we use the VX-line. So the cloud platform will, you know, will, we developed the Newton plug-in to talk with the SDIN controller, and the SDIN controller can talk with a virtual switch or the physical switch through the open flow protocol. And for example, if the VM in the same host, if they want to communicate with each other, you know, the packet or the network packet don't need to be encapsulated. But if you, the traffic from the VM on the host, one want to talk with the VM on the host too, you know, the traffic should be encapsulated into the VX-line packet. So we have two ways to encapsulate the packet. First is on the virtual switch and another way is on the physical switch. And this part of the hybrid, you know, since we have a lot of physical server connected to the switch, so we need to find a hybrid way to connect them together. So you just create a VX-line network. And the SDIN controller will control all the flow table through the open flow protocol. So that is the layer two network. So how about layer three, layer four, and layer seven? I mean advanced network such as the virtual load balancer, virtual firewall, and some other advanced network functions. So here we got two ways. The first we can use them just as I said before. We can use the traditional security devices. We can also use the NFC components. If we use the traditional devices, you know, the SDIN controller will talk with the device through the netconf protocol. And if we use the NFC components, and the SDIN controller will create a virtual machine instead. So it will be a service machine, you know. If you need a layer two network, and if you need advanced network, you just open the web portal filling the parameter you preferred. And the SDIN controller and the open stack will orchestrate everything. So it's a service machine. You know, the SDIN controller and the open stack platform will create a virtual machine and connect the NFC components of the virtual firewall, virtual balancer together, and the user can use the service. Well, it doesn't work. So the next is the application catalog and the DB service. So for the application catalog, what we mean is that, you know, some, in the past, when we talk about the cloud platform, it generally means that the cloud platform can provide a virtual machine for the end user. But now the tenant user wants to use the results in a more easier way, you know. If they need a application, it doesn't need for them to create a virtual machine first and then install the application on it. What you do is just, I need the database, I need the middle wire. So what I should do is just a single click and the platform should take care of everything. So here, we didn't use a project from the open stack. Instead, we, you know, integrated some third orchestration tools such as the soft stack and pulpit and together with the open stack. And because it can provide more functions for the customer. So what we do is that we can create a lot of private, you know, the labor, labor, and we can upload the script and the template information onto the private, private library. And we can also import the application package on the library, you know. For the Chinese offshore company, the system that they use, it's not the general one such as, you know, the MariaDB and the Massacre, it's not it. They use a lot of application from the third ISV. So we need to find a way to help them to finish the task for the application orchestration. I mean, the application deployed in an automatically way. So the customer should, you know, write the script and the provide the template first and import them to the private library. And they can drag all these things together in a visualized way and publish them in the catalog. And the tenant user can log into the portal and they can see the service that the administrator provided for them. And they can click to create application and the cloud platform will combine the script and the template together and to deploy the application system on the virtual machine or on the physical server. And in the near future, we will, you know, bring in the Docker engine. So that means next time we talk about the application catalog, there should be another way, except use Docker to work with the traditional one. So also for the database, it can be also created in such way. So what I'm talking about just moments ago is that we can orchestrate the deployment of the application. But we can also orchestrate the workflow or the deployment of the database. So since we adopted this way, that any application, any database can be installed on this platform because we have a engine to orchestrate this workflow. So since it is a large cloud, so here is some suggestion or tips from us. I mean, how to optimize, you know, what we built is a large cloud platform. So you must find a way to optimize the open stack. And you must find a way to think about your solution before you built the cloud platform. Now the first one is a multi-region cloud. So just as I said, that we will build the cloud platform on China and on some other parts, Singapore, Dubai and some other parts of the world. So this cloud platform should be managed in a unified way. So for example, data A is a data center on Beijing and data B is a data center on Singapore and data center C is on Dubai. So we need to find a way to manage them together. So what we do is that we will deploy the open stack instance on each data center. And then we will also have a central cloud platform which provides the unified management. So the central cloud platform will also have the open stack instance installed. You know, so what's the difference between the central cloud platform and the local cloud platform? So the local cloud platform will only schedule the results within this data center. But the central cloud platform will provide the authentication and some other schedule. It means that the central cloud platform can schedule the results from the local cloud platform. But the local cloud platform can also schedule their own resources. And we also find a way to extend the network. It means if you have a system, you need to install your part of your system on this data center and you also want to find a way to install the other parts on the other data center. So there should be a network connection between these visual machines. So we use a VX line to make all the data centers, you know, to create a logical network across the data center across different physical sites. So the next tips we offered is that to containerize the cloud platform, which means that you can put all your components on the container. So why we suggest the containerize cloud platform? So think about this. If the cloud data center you're building now is not large enough, but in the near future, if your business increase and you need put more resources in, I mean, you need to expand your results so the cloud platform will become the bottleneck. So you need to find a way to expand your cloud platform seamlessly. So the dockerized solution will be the best one, I mean, for the expansion, you know, if you put all the components in the docker and you can, next time, if you want to expand your results, you just put another, for example, we have a NOVA on the host one, we have NOVA on the host two and NOVA on host three, and we will have a HA proxy or some other proxy in front of these components. And the next time if you need to expand your cloud platform, just add the host four and create a new NOVA components on the host four and you know, the cloud platform will extend. So besides, you know, the NOVA and the Cindy and Neutron, what we are talking about these components is a non-active. I mean, if this service crashed, it crashed, there will not be any disruption. But if the database crashed, there are data on it. So you need to build a data in cluster and the rapid MQ, I mean the message queue in cluster. So we have two ways for the state less project, we just expand them seamlessly. And for the state project, sorry, for the state components such as database and the rapid MQ, we need to find a way to create a cluster. And if one database crashed, you know, the other parts of the other database will take over. So the next tip before the building the large scale cloud platform is how to schedule the compute results. So think about this, we have a lot of clusters in the data center. And we need separate these clusters. So what we do is that we create a tag for them. You know, if the customer has a, let's take an example, 100 clusters and each cluster has a lot of visualization hosts in it. So we create a tag for different clusters and we will combine this tag with a flavor you know, we use flavor to create the virtual machine, I mean in OpenStack. So we can combine this tag with the OpenStack flavor, so a flavor. So next time when the tenant use around to create a virtual machine, you just select the template he prefer and the combined with the flavor and the virtual machine will be created on this cluster. For example, we have a flavor one which name high performance. We have a flavor two which has a name lower performance and the higher performance was a cluster one and the lower performance was a cluster two. So when the user create a virtual machine and he needs a virtual machine which has a high performance, it just select the flavor one which you know combined with the tag tag one. So the virtual machine will be created on the cluster one. So it's an easy way to expand your computer resource. And another one is schedule the large storage source. You know we have a different visualization platform and this visualization platform will connect to the house, sorry, to the storage behind. No matter it's a fc storage, iSCSI storage or file system. So what we do is that we can create different type for different type of storage. So the previous slide we talk is about the computer results and this slide we talk about is a storage resource. If you want to create a block storage if you want to create a file system for your virtual machine then you can select the tag. I mean the type the administrator provide for the end user. You know the administrator can create a different storage and make a different type for the different storage and they can assign this tag for different tenant user. And if the tenant user want to create a block storage they just select the tags they need and the storage will be created on the backend storage. It may be a virtual disk or a logic line or local drive. It depends. So next one we're going to talk is about the large scale network. Come on, let's think about this. So let me ask a question. So for the open source community version if we want to realize a layer three or layer four network function, how we realize it? So generally you know OpenStack will have a host which called network node. And if we need a net service that OpenStack Neutron will create IP tables instance on the network node. And if I need a load balancer maybe the Neutron plugin will create a name sorry create a HAProxy instance on it. And if I need some other service they will create the results on the network node. But generally they have only two network nodes. And if the scale of the cloud is very big you know the network node will become the bottleneck because it has limitations. So what we do here is that we didn't use the network node but we use a physical device and the NF components instead. So if different tenant user needs an advanced service such as the virtual firewall, virtual load balancer that the results will be created on the security devices or on the virtual machine. So it can be extended easily. It's not only two network nodes. It's whatever you need, whatever you build. It's very easy to expand. So another thing is that let's think about this. If the two virtual machine on the same host if you want to set a security policy I mean the two virtual machine on the same host connect to the different logical network. And if you want to set a policy between these two network you must set a firewall policy between these two network. So let's take an example. If the VM one wants to talk with the VM two now the traffic should be go out the switch and to the car switch and to the security devices and the security, I mean the firewall will filter the packet and send it back to the same host. But here what we use is the security group. It's a virtual nick. It's an IP table instance between the virtual nick and the port on the virtual switch. So next if the two VM want to talk each other you know the traffic don't need to go out to the center. It doesn't need to go to the car switch and to the security device and back to the VM. You know they can talk together on the same host because the IP tables will take our that we call it a security group. So the security group is mainly used for protect the network traffic within the data center. So we call it West and East traffic. And if you know after the there's a traffic out from the I mean from the other data center or from the internet it should go to the security device such as the physical, I mean the firewall first or to the components, the components instead. So we have two different types. One is East of West and another is North and South. So the last of what we're going to talk is the optimization for the OpenStack itself. So in January solution we put all the service on three hosts but in this cloud platform we deployed the database and the middle wire on the other host. And we put all the data from the database and the middle wire to the backend storage with SSD drive. So it will increase the performance a lot. And also we added the MIME catch between the Keystone database and the Keystone client. Each time the Keystone, for example, if you want to create a virtual machine, the Keystone, the NOAA client and the Cinder client and Neutron client should get a token from the Keystone and the Keystone will retrieve the data from the backend storage. So the backend storage will become the bottleneck. So we should find the groove is that we set a MIME catch between the Keystone and the backend database. And we also set a MIME catch between the Keystone and the Keystone client. So if the Keystone needs a token, it just retrieves the data from the memory. So it will increase the performance a lot. So since the time is limited and that's what my talk today. And if you have any questions, just ask, please. Thank you.