 Hello everyone, I'm William from Huawei. Today I'd like to introduce cloud native batch system volcano to you guys. So the talk basically comes up with four parts. The first one is volcano brief. Then the new scenarios and that we are exploring this year. After that, we will go through the community updates and also the release. Volcano is started in 2019 targeting on providing the extending Kubernetes and cloud native technologies to the batch computing. The project was donated to CNCF in 2019 and this year we moved to incubation level. Volcano released a feature version every three months. Now the community have around 2.6 gigahertz stars and 640 folks since joining CNCF. We received over 450 computers from all over the world. From the graph, we can also see that volcano engage deeply with upstream computing frameworks. So far, we have supported almost all of the mainstream computing framework. For the public adoption, we do get a lot of users, especially for people are running big AI, big data, genie, contrast coding workload on Kubernetes. Here are parts of adopters using volcano in production environment. Actually, you can see there's very good diversity over the industry. We have adopters from internet companies. Also, there are financial companies, cloud provider and gene computing related companies. Currently, more than 50 companies adopt volcano in their production environment. As the figure shows, volcano is not just a scheduler. First of all, volcano provides some core APIs such as job, queue and pod group on the basis of Kubernetes, which are convenient for defining batch workloads and controlling the process of resource allocation. Secondly, the volcano scheduler component provides rich batch scheduling algorithms such as fair share, topology, SRE, preempt and backfill is elastic scheduling to improve the job performance. At the same time, the plug-in scheduling framework allows users to define their own customized algorithm. The volcano controller component consists of three controllers, job controller, queue controller and pod group controller. The job controller is used to manage the job declaration and lifecycle management. And the queue controller provides resource sharing in maintenance scenario such as queue proportion, queue mean and max capacity. On the nodes, volcano provides support for heterogeneous hardware such as support such as X86, ARM, GPU, MPU, etc. As well as resource topology and acceleration capabilities. In the future, we will also support queues related capabilities. Let's take a look at several typical batch scenarios explored by the volcano community this year. The first one I'd like to introduce is the Internet audio and video transcoding scenario. Now with a large number of videos generated, how can we efficiently and accurately perform cloud transcoding and processing? To adopt to multiple terminal depletes needs and to deal with complex network conditions is the top priority of video industry developers. The basic process of video transcoding is to divide a large video file into many small video files and then perform parallel transcoding through many small tasks and then merge them into a complete video file after transcoding. In this process, three aspects of transcoding speed video code rate and video quality must be guaranteed at the same time. Mass video transcoding has very high requirements on the throughput of the system. At the same time, it consumes considerable computing power and is also very cost-sensitive. In response to this demand, we have done two things. First, through the optimization of volcano and Kubernetes, the throughput of scheduler is increased to meet the parallel processing of massive video transcoding tasks. Second, in order to reduce user costs, we collocate online business and offline workloads together in the cluster and at the same time, oversold some unused resources to improve the utilization of the cluster and achieved good results. The second scenario I want to introduce is about AI for science. In the past, new drug research and development had the characteristics of long research cycle, high cost, and low success rate. As a whole direction in the field of drug research and development, AI has been applied to all stages of drug research and development. In recent years, more and more AI drug companies have emerged and we have also investigated some of the use cases as shown on the left. This is an AI drug discovery and design platform and a multiple dependent job from a workflow. The previously job ends and the scientists analyze the results and then trigger the next job. A job runs for several days and consumes a lot of computing power. When the business reach its peak, it even breaks the upper limit of the resource capacity of a single region and is very sensitive to cost. In these scenarios, the cross region scheduling capability provided by the cloud native technology ensures the capacity demands during the peak business hours. Secondly, for the cost requirements, the basic solution is to use the cloud native platform to provide unified management of multiple computing power and even spot instance to reduce the cost. In addition, the drug companies can also use the container technology to solve the problem of version complexion and environment consistency. The third scenario is the traditional HPC. The traditional HPC software ecosystem is not so friendly. The means software such as SLRM, PBS, and HTC Condor are all GPL lessons. On the other hand, the cloud native ecosystem is very friendly, but the batch capability has many shortcomings. Many in-job management and batch scheduling strategies. In response to this situation, CNCF and Kubernetes established batch workgroup to define the general batch API for batch. In addition, volcano has a batch computing project of CNCF. It provides a very rich scheduling strategy and job management capabilities, helping the traditional HPC users to gradually migrate their business to the cloud native environment. In addition to the above three typical scenarios, we also received some of some other new scenarios this year, such as autonomous driving scenarios and IoT scenarios. We will not introduce them one by one today. Next, I will briefly introduce several typical features of volcano. The first one I'd like to introduce is the job management in volcano. As you know, the job of Kubernetes can describe one kind of pod in a job, but for most batch workloads, their job involves multiple types of pod. Take Tencent Flow Job, for example. It has PS pod for parameter server and has work type of pod executing training tasks. And there are also chief pod for management and even the evaluator pod for cross validation. So the computing framework in different domains are trying to run better on Kubernetes. They developed their operators such as TF operator, Spark operator, Flink operator, MPI operator. User have deployed different kinds of operators in their environment to support different framework. It's complex for users to maintain all these operators. People are actually looking for a deeper integration and better support for this computing framework. The multiple pod template is supported in volcano. With this feature, volcano job is able to unify and support most mainstream computing frameworks like MPI, Tencent Flow, Harvard, Petouch, etc. And also, fine grained job management is implemented in volcano. Volcano job also provides an extendable job plugin to allow user define their customer's behavior and provide several building plug-ins by default. Also, the volcano job and scheduler can coordinate to do more optimization for batch workload. For example, the job dependency scheduling and the task dependency. Next, let's go to the resource management. We add Q in volcano for resource planning and sharing. The Q is a cluster-scoped concept. It is coupled with name space. The Q is mainly used to share resource between multiple tenants. For example, one Q can map to a group in company and also you can match a specified kind of resource like GPU for this Q in the cluster. And also, user is allowed to config different policy for different Q. It's very useful. After we get to know the Q concept in volcano, let's talk about some scenarios. The first scenario is if you have two teams, how to share resource between them with Q. Here is an example. There are two Q, Q1 and Q2, which are mapping to your two teams and their VIT is two to one. There are six CPUs in cluster and there are six pods in Q1. Q2 is empty. So the pods in Q1 can borrow resources from Q2 and all pods get running. And then we submit a job to Q2. The scheduler will reclaim two CPUs from Q1 to keep the ratio as two to one. As a new job in Q2, get two CPUs and get running. The second scenario is for some users, they have routine jobs or urgent jobs. They want to reserve amount of resources for this kind of job. For this case, user can configure the guaranteeing resources to make a reservation in volcano. Next, fair share is a common requirement for elastic or streaming jobs like Spark. However, in Kubernetes, the more job submitted, the more possibility to get more resources. There's no fair share. Volcano provides the fair share between jobs and name spaces. From the graph, you can see user one and user two. Submit small job and big job to the same queue. The small job might get starving without fair share scheduling. And volcano ensures big job and small job get the resources fairly with DRF agrees. And also we add the name space fair share in volcano as the graph shows although the name space three submit more jobs, let's say job four and job five, then name space two, but they get the same resources individually. As we mentioned in the first section, there's not enough scheduling policy in Kubernetes for batch workloads. We have been spending a lot of time to fill these gaps. Here are parts of the scheduling policies provided in volcano. I will present some of them. Nowadays, the ML workloads has higher demand for GPU compared to traditionally workload. GPU is a precious resource. How to improve the GPU utilization is a hot topic and have great value for users. Elastic training can dynamically adjust the number of instances involved in the training. Greatly improve the utilization of GPU resources, especially on the public cloud. It can work with spot instance to get a lower cost and improve the training efficiency. First, let's see what elastic job is like. The left figure shows a volcano job. The mean available in first to the job has file port at least and replicas refers to the job has template at most. The job gets running while file port gets allocated. And then the job will extend more posts if there's more free GPU resources. Here is an scenario. As you know, the inference service always has a lower GPU utilization compared to the training workload. As people tends to collocate the inference services with elastic training workload into in one cluster to improve the utilization. The right figure shows an example. The inference job 2 with high priority preempt elastic port from the training job 1 to ensure the SRE. Whenever there's free resources available, the job 1 will extend more posts to accelerate its training. The next one is about the SRE. In a real production cluster, user often submit multiple kind of workload in one cluster. For example, the small job and the big job. How to avoid the big job or small job gets starving is a basic and important thing. The left figure shows an example. At the moment of T1, user submit a big job 1 with gun and small job 2. The small job get allocated and the big job 1 keep pending due to insufficient resource. At the moment T2, a new small job 3 was submitted get allocated. The big job 1 keep pending due to insufficient resources. With the time goes on, the big job will get starving. If the released resources always cannot satisfy its gun and the user submit small job continuously, the SRE plugin SRE scheduling allows user to configure the job so that it is completed on time and reduce the risk of missed data. The parameter SRE waiting time is the maximum time that one job should stay in pending. When the SRE waiting time reached, the SRE plugin moved the pending post to the next state and started to reserve resources for this job until the job request is satisfied. Spark started to provide support for Kubernetes from 2.3 version in 2017 and later Spark operator provide another way to help user to run Spark on Kubernetes as well. However, in a long time, Spark on Kubernetes lacks of batch scheduling abilities. Late last year, we started to work with Spark contributors to support customized batch scheduler for Spark on Kubernetes. Spark with volcano provides the batch scheduling abilities like job priority queue, fair share, resource reservation, etc. Now the feature is ready in Spark 3.3 version. Here is the release journey of Kino. At the very early age, we developed a set of scheduling policies to support batch workload and then integrated with ecosystem such as Kubeflow, Spark operator, Flink operator, Ago, etc. Later we found that there are a lot of gaps in the job management, so we spent quite a lot of time to enhance the job management to deeply support upstream computing framework. In the future, we are going to support several scenarios like multiple cluster scheduling for batch workload, performance enhancement for big-scale cluster, intelligent collocation for better utilization and fin-ops, etc. Here are some resources of Kino. You are welcome to join our community and give us your feedback. Thank you for attending this topic. That's all for me. Thank you.