 Hello everyone. My name is Zhongyuan Ke. I'm an A.I. algorithm engineer at Huawei. I'm honored to have this opportunity to speak to you today. And the topic of my speech is my sports foundation model platform promotes the ecological development of foundation models. I divided my topic into three parts. First, I'll share my insight of A.I. development. In the era we're living today, there are three things that are expanding rapidly. The first is the continuous upgrading of hardware equipment, which has continuously improved the computing power of various devices. The computing power of smartphone processes has reached a 10 terawatt operation per second. And the second is the explosive growth of data volume. According to statistics, the volume of unstructured data has reached 44 data bytes in 2020, and it's expected to reach 180 data bytes in 2025. The third is that the neural network model has been continuously improved, and the number of model parameters has continued to increase. Now mainstream foundation models such as GPT-3 can have hundreds of billions of parameters, and these three facts are drawing A.I. into a golden age. Now in this golden age of A.I., the mainstream direction of neural network model has gradually shifted from content understanding to generative models. In the past, traditional A.I. models were more about collecting data and training one specific model to solve specific problems, such as recondition and prediction. Now generative models have become mainstream, and more and more models are used to generate text, images, videos, music, and so on. And a recent hard event is that a painting created by A.I. won the first prize at an or fail. It can be seen that the quality of A.I.G. state is constantly improving. Traditional A.I. scenarios use one specific model to fit a certain scenario. The number of model parameters is relatively small and the generalization performance is poor. This approach usually requires training a model with a large amount of data income trust. In the foundation model scenario, a model can adapt to a variety of scenarios that have strong generalization performance. The application of the model only needs to be fine-tuned with a small amount of data on the basis of foundation model. An opportunity of foundation model means that the number of parameters of model is increasing rapidly. The number of parameters of foundation model can reach hundreds of billions, which means that training a foundation model requires more computing power and data, so training a foundation model with good results requires a high investment. However, the foundation model also brings more advantages. For example, one model can cover multiple scenarios, reducing the cost of training different models at the same time due to the larger number of parameters. The foundation model can also break through the accuracy limit and in the fine-tune scenario, owning a small amount of data is needed, which can reduce the cost of managing data labels. This advantage makes the rigorous development of foundation model inevitable and all leading technology companies are laying out their own foundation models. This also requires an AI framework with more capabilities and the importance of the AI framework is gradually increasing. Next, let's discuss the key topic of this issue, MindSport, which is an AI framework developed and open-sourced by Huawei. With the increasing importance of AI framework, the mainstream AI framework in the industry has begun to lay out the foundation model-related technology. In 2020, Huawei open-sourced a self-developed AI framework MindSport, which natively supports foundation model training from version 1.5, which launched in 2021. In November 2022, MindSport 2.0 was released at the Huawei Connect conference. It's the industry's first fusion framework that supports scientific computing. The main updates of version 2.0 are as follows. First, version 2.0 further consolidates both scenario capabilities, supports efficient federal learning, realizes cross-domain parallel training of foundation models, and realizes cross-platform deployment. In addition to existing static graph models, this version fully supports the dynamic graph mode, which can balance development flexibility and execution performance, and achieve a comprehensive upgrade in terms of ease of use. Second, this version builds a scientific computing acceleration level, which uses automatic parallelism, heterogeneous acceleration, and optimization of AI and HPC tasks. And the third thing is that version 2.0 builds three major toolkits. MindSport has an AI framework natively supports foundation model training, and the key technology of foundation model training is the parallelism capability. MindSport has multi-dimensional hybrid parallelism capabilities, including data parallelism, model parallelism, pipeline parallelism, and optimizer parallelism. For foundation model memory consumption problem, MindSport has automatic precision compression and distributed data caching module. In order to reduce the development cost of foundation model, MindSport provides high-level APIs related to foundation models, such as transformer encoder and transformer decoder, which are necessary structures for foundation models. As an open-source project, the MindSport community is constantly building and growing its ecology. Currently, the community has more than 11,000 community developers, and 28 state groups serving more than 55,000 companies, and past 13 groups in 22 cities in China and 7 foreign cities. There are already many foundation models based on MindSport, and MindSport is actively cooperating with university and research institutions to develop foundation models. There is an LLM PCL series of foundation models jointly launched by MindSport and the research laboratory, and stream-model foundation models launched by MindSport and Wuhan University. MindSport natively supports the foundation model, and there is still a lot of work to be done in terms of ecological development. Therefore, the MindSport community has built the MindSport foundation model platform to promote the ecological development of foundation models. This slide shows the architecture of the entire platform. The platform is mainly divided into three parts. The bottom layer is the computing power center. It's the foundation of the whole platform, and the middle layer is the tool layer. It provides tools for developers and upper applications. The top layer is the application layer, which directly drives ecological development. The base computing module is connected to several computing centers in different areas, and it supports processors like CPU, GPU, and CEN. In terms of the tool layer, the platform has online model library and dataset available for download and direct online use. Code lab environment is the online training and inference center where users can manage code based on gate, scheduling online files from model repository and datasets, and realize online training and inference. Also in the tool layer, we have property tools for foundation models. The foundation model online experience model and the foundation model fine-tuning components. The whole tool layer helps to lower the threshold of air development, especially for tasks that require a lot of computing power, such as training the foundation model. The upper layer includes industry providing specific cases of visible framework and foundation models in various industries, and promoting the development of community incorporation. The company module is used to hold various AI competitions, and the course module provides many technical courses related to AI and foundation models. And the AI gallery is to help developers to learn about the foundation model. Now I will introduce the platform from bottom to top. The bottom layer is the computing power center. It's the core of the whole platform, which is necessary to reduce the cost and the threshold of foundation model development. We have set up several computing centers and data centers to provide free computing powers and storage functions for users. The computing power center flexibly allocates computing power for the tasks assigned by the tool layer, including online training, online inference, and online fine-tuning. This forms the basis of the entire platform, and then we can extend many application tools on it. The core of the tool layer is the co-lab environment. It links the model repository, code library, and data sets. Users start online training tasks through the train center. For online training, we provide two ways. The first way is the interface interaction. You just select the starting files and corresponding data set and models through the interface. And open the training mission directly, then you can view the training mode on real-time. The second way is the Jupyter Notebook. The user selects the corresponding device to start a Jupyter Notebook, and the Jupyter Notebook has built-in learning, tool case, and complex tutorials of my sport, which users can train or debug. And it's recommended to use the interface for long-term training and use Jupyter Notebook for learning debugging. The online inference module is implemented based on Gradle. In this module, users can deploy their own inference mission or experience other users' public projects. The inference tasks of the platform are scheduled based on covenants, and different inference tasks are asserted from each other. For example, if I find a style transfer project I'm interested in, and I can directly upload a photo through the online inference, it will be covered into a Van Gogh style image. The foundation model experience module is a way to display the foundation model to the public. We provide several interfaces for foundation model experience. Users can directly try the foundation model inference through the interface. Taking the following three foundation models as an example. The first foundation model is called Wukong Huahua. It's based on the diffusion model, which can output a specific picture according to text description. Entered by the user. And the second model is the Dong Tai Chu. It has two missions to try text to imagine and imagine two texts. For example, the user input a photo. And the model will output a summary set explaining the photo, which is two part of there, playing the snow. And the third foundation model is Luozhanet, which can perform target detection on images. User input an image on the target and spaces in an image where we marked an output. The industry module of the platform is one of the top level applications, including related cases in fields of electricity, industry, medical, and humanities. There are all landing cases based on mines ore. Some cases are adopted the foundation model to a certain industry, such as the Yu-Ju Wukong Imagine and Text Understanding model in a field of humanities. The display of these cases provides a case reference for the implementation of foundation model, and also provides cases for the fine tool and of the foundation model, which is also promotion for ecological development of foundation model. The competition module is a core module of the application layer. It can attract many students and new develops to join the community, understand mines ore and learn about the foundation model in depth. For develops of different level, the platform provides different types of combinations. The platform mainly covers three types of combinations. First, single-stage combinations suitable for beginners, which will be held frequently. And second is double-stage combinations suitable for middle and high-level developers. These competitions are held on a regular basis. In addition, every year we customize the theme according to the cutting-edge theme of artificial intelligence. For example, this year we are holding an AI Painter competition based on the diffusion model. The co-lab environment of the platform is provided to the participants for free, which lowers the threshold of participation. The purpose of setting up the course module is to lower the threshold of learning the foundation model, which is too difficult for developers with weak period of knowledge to get started. Our course module is one-stop service of radio course with homework and tutorials. And all of them are free. For example, we have now launched courses related to foundation models, which are suitable for developers of all stages. The course content includes the basis of foundation models, key technologies of foundation models, and explanation of foundation model examples. And we also have other beginner courses and the transformer courses to attend. And AI Gallery is the place where the platform showcases the AI GC images generated by the foundation model. It's like a photo wall. You can publish your generated paintings and like and collect others' paintings. Here you can see a lot of beautiful AI GC image works. AI GC is a promising direction and the extension of AI Gallery can also promote the development of AI GC. Now the drawing force of MindSport Foundation Model Platform for the Ecology Developmental Foundation Model can be summarized as follows. The three-layer structure of the entire platform forms an ecological driving of the loop. Thank you for your listening. Here are our websites related to MindSport. Everyone is very welcome to join the MindSport community.