 Hello, everyone. Welcome to our session. Today we will talk about the progress for cloud computing on ARM architecture. So first, I will talk about the Linaros work on ARM64 cloud computing upstream and then Zheng Yu from Huawei will talk about the status of applications on ARM infrastructure. In the last part, Chengbo from Edith Stack will talk about the production experience when adopting the ARM server in cloud computing area. Just a brief introduction. This is Kevin Zhao from Linaro and now I'm working on the OpenStack and Cypher and Kubernetes upstream. Linaro is an open-source organization on ARM ecosystem, which has been founded by ARM and its many partners such as Huawei, Qualcomm, and Google. Linaro mainly focuses on the upstream development and maintenance such as Linux kernel or 2-chain and also it has several groups for the different user cases. For example, we have a Linaro data center and the cloud group, which is focusing on the enterprise market such as cloud, big data, server architecture, HPC, and AI. So all our cloud work are mainly on the OpenStack, Cypher, and Kubernetes. We have based on our upstream work to set up an open-source ARM64 enterprise cloud, which has been called the Linaro developer cloud. So this cloud not only benefit to our members. We're leveraging our members hardware so it can test the hardware in cloud computing. But also it can help and organize the ARM64 resources to benefit the upstream developers. So if you are a developer and don't have the ARM64 resources, so we welcome your registration and it is totally free ARM64 cloud. And also if the upstream have the interest to set up the CI on our cloud, we are also welcome. Generally, we are based on the OpenStack and Cypher as an infrastructure address service. And on top of this, we can support the VM-based workloads, an virtual machine, and Kubernetes address service. And also you can leverage in the head template to organize your workloads. Here is our general components. We can see all our infrastructure are deployed by OpenStack caller. And on top of this are the main open-stack components for the VM provision for networking. And we have enabled the Magnum Octavian Heat to support the ARM64-based Kubernetes address service. Besides, if you have the, if you already set up the Kubernetes cluster, the Kubernetes should leverage in the different controller to talk with the infrastructure address service. Here is the OpenStack. So we have also enabled several controllers to control, to offer the volume support, the network support. Okay, so based on our OpenStack side work and the Kubernetes side work, we are now has been certified by CNCF and OpenStack. So you can either find our introduction on the CNCF landscape. We are now a CNCF certified cloud. Next, actually now the hardware automation will be a hot topic today, and especially on the ARM64 machines, because first, there are a lot of workloads needed to run on the bare metal. First is several hardware requests to CI, so they need to running on the bare metal. And also, there are several workloads need to running on the bare metal due to the virtualization limit. Some workloads are requested to KVM support, but as we know, the ARM servers do not support the nested virtualization right now. So set up the bare metal resources and offer the API to the external would be a quite important scenario. And also for the better performance request about the network and storage. So we know that a lot of companies are tending to use the Kubernetes on bare metal. So this case would be a good use case for the OpenStack hardware automation. And also on the HPC area, a lot of the machines need to be organized, so it also would be a good scenario for this. And based on the hardware automation, the diskless boot would be a very good topic to talk and to use, because the hardware on the local disk would take a lot of time. And actually, sometimes you just need to running the CI jobs on the bare metal. The user do not care about if they are running the jobs on the bare metal local disk. They can use the diskless boot to have a remote disk. This would save a lot of time when provisioning the machines. And now for achieving this on M64, there are mainly four gaps here. So first is about aeronics. Aeronics do not support diskless boot from CinderSyfe. Yeah, aeronics support boot from Wollin feature, but from CinderSyfe, this part has not been supported. So actually, due to Cinder, do not support SSE driver. We need to make sure it supports first. And then the SSE client is not stable. And also, we need to make sure that all these control pipes and the data pipes can be a suitable solution on ARM servers. We need to make sure that ARM servers are compatible. Here is just the idea of the diskless boot. We're leveraging the aeronics Cinder and call the SSE support from SSE to boot the bare metal machines. The bare metal is a diskless VM. So the disk, it should have a disk, but the disk is produced by the SSE. Many there are three components here. So many features need to support. So first in aeronics is a boot from CinderSyfe volume. So this part has been supported. So we can leverage the API. And from Cinder, Cinder need to support SSE driver. So this part, we are working with the upstream to land it to make it merge. And now we are blocking on the CI job field. And the last part is from the Cinder SSE gateway. So this is a SSE component. So we need to make sure it is stable and adopting into the production level. Now we can see the control parts and data parts. In the control parts, we need to leveraging the Cinder RVDS SCSI driver. So this one is a Cinder SSE SCSI driver. And also we need to install RVDS as a client to connect it with the SSE gateway. So the SSE gateway can be deployed alongside with the SSE. And you can use either a SSE ADM or SSE Ansible or just deploy it manually. So after deployment, it will have an RBD target API to talking with the Cinder client and then doing something to the SSE side. So on the data parts will be quite simple. The bare metal node need to use the SSE initiator to connect it to the SSE target which has been already set up on the SSE gateway. So actually we know that many of our tasks and jobs are make sure that the control parts is smooth and the data parts is compatible. So talking about our progress. So in Ironic, it supports the local disk not either by Ironic deployed by Kola or other components. We have doing a lot of patches to Kola to make it support. And also we need to make sure that the Cinder volume, Ironic both from Cinder volume has support. But we know that it's now just to support IPXE. But for ARM64, the IPXE has a problem because IPXE need to fuel a table called IBFT, SCSI both from their table. So this one is secure for booting kernel without filling SCSI info to kernel command line. SCSI need to fill this table. But on ARM64 firmware we are lacking of these tables. So we need to make sure that the firmware can support. Yeah. So actually now we have also another solution, just a workaround solution to support the PXE generally support. Now we are working on this to refund Ironic to boot from the extend to PXE support. On Cinder side, we need to base on RBD driver to reuse the RBD driver to emplacement the Cinder SCSI driver. So this is the Cinder SCSI driver. So this one is not ready actually, just send me to the upstream and waiting CI job merged. So we are working on fix the CI job error. So on self side, the Cinder SCSI driver number is due to the self SCSI client is not very stable. So we are also working on to make sure that Cinder SCSI client can be a stable and production ready one. And in the engineering, Cinder now will only suppose we want to focus currently for the SCSI driver because of the package dependency and also when reusing the ARM64 server as a self SCSI gateway, we may experience a kernel crash. We need to backport using the newly Linux kernel called 5.9 or 5.9. So this version will include the backfix. Okay, so thanks to listening on my talk and now I have a hand over to Zheng Yu. Hi, I'm Chang Bo Guo from Instac. Today, I would like to share the best practice of running cloud on ARM server. There are four parts. First, we released the customer technology preview version in the February 2020. And we released GA version in October. In the GA version, we support more types of ARM servers, more than five. Now, we also have more than 10 customers which run cloud on ARM servers. The last one is, I would like to describe the problem we meet and how to fix them. First, introduce the changes of the mental components. We upgrade the kernel from the 4.14 to 4.18. We also upgrade Kubernetes to the version 1.16. Another big change is that we replace the Docker with container D. We also upgrade SAP to a new version. We support more types of ARM servers. There are five types of ARM servers. The main work we did is that result differences of the EMC and firmware. And also test and fix the hardware issues like network cut. We support different ARM servers in the same clusters. There are some limitations and differences to X86. First is the CPU ratio. We set two. There's a little difference to the X86. We consider the CPU ratio in production. It's about Windows, yes, so we support. Also, the gas image output mode only supports UVFI and doesn't support GPU, it includes VGPU and PCIe passthrough. Another limitation is that the total number of network cut and disk less than 10. Another limitation. Also don't have a threading. And in some ARM servers, the output of FMA and disk command means some data of sensors like the CPU temperature. We also do some adoption works. First, the guest OS. We run the synthos and key link. Three versions and other guest OS based on the Ubuntu. The right list. So the middleware will run on the ARM servers. Here's a story for public cloud, CC cloud. This cloud has four regions. There are two regions ready, Beijing and Wuhan. In one region, there are five controllers and computer node and the storage compute near the 100 ARM servers. The public cloud also adding the F86 servers in the same region. Current, there are about 500 virtual machines in the public cloud. Another story company of securities is development and test environment with 10 ARM servers with CPU FT2000 and plus. The business running on the small cloud is the blockchain business. And the cost control service. The next plan is that it's part of more servers and business. Yeah, the product of ARM is ready for production. The last part is the problems we need and how to fix it. The first one is that bandwidth of 10 gigabit network card is not stable. Sometimes it's only six gigabit. How to fix it by default, the RQ rebalance will schedule the CPU to the network card queue. We need to bind the network card queue to the local CPU. The second one is some servers joined about SIOV. First, we need to turn on LMOU on BIOS and then set up some system configurations. Then we can split one network card into some virtual function. We only test at the test of TSM ARM server with the CPU 920 has passed. The ARM servers with the CPU FT2000 plus can't turn on the LMOU. They don't support SIOV. Another frequently asked question is that what's the flavor we should choose on the ARM server? We have some experience on 860. So the way to get the result is that first we get the score of the basic CPU spec test. Then calculate the flavor for ARM based on 86. Okay, that's all. Thank you. Hello everyone. Welcome back to the session. My name is Zhenyu Zhang and I'm going to briefly introduce to you the status of applications on ARM infrastructure. As Kevin introduced in the first part, now in the software stack, cloud hypervisor and container layer have very good status on ARM supports, but many users may ask how about the software and services running on top of them? I have some information to answer that question. So let's start. We have done some jobs to enable all the services, but let me firstly introduce why we are going to do, why we're doing it and how do we do it. So first, let me show you some about the current status. I think everyone knows that x86 are the dominant architecture in the current server markets. So in most open source development upstream, they use x86, they are pipelines and the pipeline will provide develop, build and test and packaging processes and the outcome of the development pipeline is on a software package. Normally it's x86 architecture. So for x86 users, the users can use it directly, but for other architectures like ARM users, they may need some extra works, which is not very convenient. So to solve this problem, we propose to various upstream communities to add ARM CI pipelines. With this pipeline, we will use the same test scripts, build scripts and test cases and after that we will provide the packaging function as the S86 pipeline. After that, we will have an ARM package, which ARM user can use directly and that will make ARM users the first class citizens. And this CI will be running in parallel with the old x86 CI pipeline in the upstream. So it can also provide the end-to-end develop pipeline and support to upstream projects. With the method I introduced in the last slide, we have already enabled a lot of areas. The first I want to talk about is the big data area. In the big data area, we have already pushed the ARM CI pipeline to the top projects, like Hadoop, Spark, HBase, Half, Flink and Kudu. We run the same test scripts with same test cases, as you can see from the numbers. And because of our stable running for quite a long time, like Hadoop, we have already running for over a year and Spark is also about a year. Those projects have officially claimed the ARM support. And Hadoop have already released the first ARM version in the July of this year. And we have also done a lot in the database area. The famous open source database projects like MariaDB, Green Plum, PostgreSQL, and ROXDB are now also have ARM CIS. And MariaDB has been tested on CentOS, Fedora, Red Hat, Enterprise Linux, and Derby. They have also released packages on those platforms. You can have a try if you are interested. Also for the new hot areas like AI and cloud native, we have also enabled quite a lot of them, like in the AI area. We have enabled the two major projects, TensorFlow and PyTorch. And in cloud native, we have cooperated with the CNSF foundation to enable the ARM support in the CNSF.CI platform. And it supported Kubernetes, container D promises, Cardinus, and FloodD and Avery projects to test our ARM architectures, which is a very good coverage for cloud native projects. And we have also covered the other most used cases, is the web services. As you can see, in this area, we enabled a lot of web services, which are dominating the markets. So we can see this full map that for all those areas, we have already enabled all the major projects, open source projects in each area, which provides a suitable selection for users in that area. So I have to say that the application stack for ARM hardware is already very good. The user can use it directly. And for users that are concerning what kind of profit can I get from using an ARM driver, we have also done some tests in the price field and the performance point of view. As you can see, we created a cluster in three different clouds that can provide ARM resources. The price is down to about 20% or 250% compared to the same x86 platform. And we didn't get very obvious degradation on the performance. The performance only got down like 3% for these Hadoop tests. And we can get some conclusion about it. The overall performance is quite identical compared to x86 clusters. And we have 20% to 50% cost cheaper than x86 resources. And we have tested that the big data workloads can be smoothly shipped to new added ARM nodes, which is very good for users that are interested. And that's all for my part. And Guoshengbo will give you some more information about how to land the ARM servers and ARM services. And here is one of our select channel. Welcome to join the select channel and discuss about your needs and requirements. Thanks a lot.