 My name is Jiang Yong from Alibaba. Today I will give a presentation on the image distribution in our native era, and what we have done in these aspects. But for the agenda of my introduction in the first part, I will talk about the basic knowledge of Docker. And then we'll talk about the Docker call and introduce an image, and what we have done in the image level. Well, before Docker launched, we just imagined a small sized company. We did not need to deploy many resources on the single devices. So all the resources can be deployed. We do not need to address the problems like resource conflict, or resource fighting, but with the business volume increased. With a limited resource, we have to deploy the same, but deploy some business on the same device. At the very beginning, we thought of the VM technology to address the issue, to slow, split it, or isolate the business. So they will not influence each other. But for me, I have some problems. For example, the start of time is quite too long, and it will take up additional memory and CPU. Well, for the core part, it actually has always developed more for the limited space and the sake of technology. And XRC technology, we missed technology to develop it. We can put to the, we can isolate the business in the wrong time so that different apps, so the resource grabbing of different apps will not influence with each other. So once we solve the volume process, we still have the problem of remote and conflicting. For example, for one app rely on the Python 2.7, the other app rely on Python 3.0. So for the deployment, there still are some conflicts. But for the multi-core, we actually introduce imagine technology. Or what we rely on actually, we package them into the same image package and we deliver it to the maintainer staff for them to do the deployment. The image have many benefits. For example, if one application rely on Python 2.7 and the other rely on Python 3.0 version, since they rely on different images, they will not conflict with each other. So we have one benefit that during the interaction, as they better perform much of the work. And once it runs, we can apply it into many other aspects. Once the parameters have been set, we can roll one or ten resources. There are no differences. And actually LXC have other technologies. At the beginning actually, it use LXC technology, but after that we do some optimization. Later we found that it was not that user friendly. We need additional steps to do the image or to do the business distribution. For now, LXC has addressed the... And the dark LXC has addressed the conflict issues and the isolation issues. But for most of the part, they were relying on the internal Linux code technology. If the Linux code have some bugs or failures or have some failures. And then the data may be released to other containers and it will affect the whole process and the performance. Some teams are working on this. We draw lessons from the Google technologies and we do the safety... We work on the safety to provide the safer and most secure environment. And we have the safety on the VM and we have the performance of the normal container. Well, for the Docker... I want to talk about the three core concepts of Docker. Namely, image container and respiratory. The image is quite an entity. But for the warhouse is where it restored the images. Well, for the container, it actually is evolving time for the image. We can use an image to start a container. And once we commit the container, we can generate a new image on the top of that. But we cannot start a container without the image. That is to say, if we want to start the container, we need an image. And for Docker, it actually uses the different connectors to generate a separate image. That is to say, for the first image, it is generated by the tab. And on top of that, we can perform various of jobs. And generating images that we demand. This is the image building. Plus, it's built. We need to distribute the jobs in the wrong environment. We cannot transfer the jobs with the tab package. So here comes the repository. Once the image is constructed, we will distribute it in a repository. We need it all and we can pull the images from the repository and then we can generate the container. I will give you an example for the building of the image. Here, the Docker build has a mechanism of Docker build. And we can have a Docker build. And then it converts the image. It is a problem, a basic image. And the second step is we can give some orders. And the third step is the installation. So as you can see from this process, the first from is a fundamental image. And accordingly, it comes from the tab. And on top of it, we execute a store. So it will add another layer. So this is an item where we pull all the leads. But we find out the fourth step. It should not be added to another layer. And it lists what kind of layers it needs. And what about the configuration of the image? So it is in manifest. And the building of the image, you can see this has a different bias and different steps. They are read-only. So the image is read-only, but when the container is running, it needs to read and write the documents of the system. And this is a container running situation. And the third layers, they are read-only layers. These are the images. And the top layer is read-write layer. This is the container. It is written by the container. And the data that is to be read or to be written is all in this layer. But the documents are not deleted. So here, there is another problem. Like Alibaba, it is such a large science to come. Actually, we do not write the business from the very beginning to the end. But we have some special techniques to build our own businesses. For example, when you know enterprise, our group sometimes will have special technology. That means we need to have some data in our container. And the container needs to be dragged into the container. So the image suffers a lot of pressure. For example, 1G or even 2G, 3G. And another problem is, at present, our system is a distributed system. So within this distributed system, sometimes we have some information center, the database, so for these database, they will also increase the size of the image. So finally, we will find that the image would be timed out for the businesses. Because the businesses can take up too much space. Including the fundamental data, finally we will find that the image would be even 4G to 5G. Also like Alibaba, millions of the host and every day we have thousands of audits and it has many jobs, lots of jobs. So if all the machine is in the image center, then there will be time out for the download on the image. So there will be very serious time out. Additionally, if it is serious time out, you may also encounter some other problems like the image cannot react to the data. So in the future, maybe we can also have some other techniques to deal with this problem. But it will lead to another problem. That means the download will be even slower and slower. So the situation might be even serious. So first of all these problems, the first solution is to use the P2P distribution platform. And here I will briefly introduce its architecture. This there is a configuration center that is on the bottom. So the middle part, the middle layer, it is the cluster manager. And the bottom level, it is the physical host running on the cluster and the configuration surface. We manage all those physical hosts through which nodes are most appropriate. Because we have many hosts, and each host has different configurations. So there is a certain way. At least you need to select the happiness. And for Docker, Docker has a physical host that we deploy in the process. So when Docker downloads the image, it is just like an HTTP request. And we can transfer it into another request. So that this request can download the data from the hypernodes. And on the right side, you can see the data source. This is an instance where it downloads a document to prepare to process it. For example, the node is on the layer of the image. And when it initiates the request to the hypernode, it can be typed. But this layer already has the data on it. And if it already has the data, then it can download to the physical host. But if it is without the data, then it can send the request to the image center rapidly. So normally we distribute the content there because normally it is hundreds of containers. So for the same layer, the request on the same layer will send them to the hypernode. And this hypernode would download the data and distribute the data into a different path. And maybe there will be why not already downloading the data, but it hasn't downloaded the data. So it can distribute the data to the one that hasn't downloaded. And so it can release the pressure of the node. So we can just assume that we have 100 posts. Actually there is only one hypernode to download the data. So it suffers much less pressure. And this is the effect of the drag-on fly. And let's say the drag-on fly. So without drag-on fly, with the increasing concurrency, then the average post is increasing. And when we see the orange line, although there is a slight increase on the time but it can just be ignored. So the drag-on fly is more smoothly than native. And that means it can substantially reduce the pressure that it suffers. So in fact, in very early days, we already have drag-on fly. I think drag-on fly is... Before Alidoco, we already have T4, this kind of technology. Alid's technology is it can solve the data distribution problem. So from that time, we already have drag-on fly technique. We already have drag-on fly business. So through P2P technology to solve the download problem. I mean 2015 in September. At that time, Gankun comes to Alid and it has the image distribution that allows it can support the image distribution. In 2016, through our 2011 festival, it becomes infrastructure of our document distribution. In 2017, we have an optimization of it. For example, the memory file system, the preheat and dynamic compression and open source. In 2018, it joins the CNF and becomes a sandbox project. And still, we are reconstructed and to optimize the drag-on fly. For example, previously, we now refactor with Golon and support HTTPS registry and more than 50 adopters to increase its capability. So actually, drag-on fly has solved a lot of problems. But still, we discover that there are still many problems to be solved. In our business, because Docker Hub is just randomly to write it. And that is why we continuously to communicate on how to optimize or how to reduce the layers so that it can have an even more rapid distribution. But the businesses, they have their own KPI and their own objectives. So it is quite hard to do so. So at present, currently, the best thing that we have done is to upgrade the fundamental infrastructure. Because our system is about 2, 2, 3G and we will optimize it. At least to optimize it into about more than 1G. So 1G has been reduced and the download time will be shortened by 20 or even 30 seconds. So this is even more acceptable to upgrade. Another point to make is Alibaba emphasizes a lot on the safety and the security. So the reason is to solve the size of the image and second is the security to ensure the security of the image and our businesses needs to be upgraded, updated at the same time. If without any update, there will be some security problems and we need to be responsible for this. So every 8 years we will update it and 90% of the businesses would use the fundamental image based on the updates. For the fundamental image, it is so large, like the bespokeus and many other images. It is of hundreds of Maca bits, but for all the businesses we will try to include as much as possible. So that means the fundamental image becomes even larger and larger. And at the very beginning we didn't have some normalized operations. So all these issues lead to the large scale of the image. But actually it does not need so much. This is the physical standards and the migration process is quite fast. But we need to be as much as possible to do this. And from now the speed has been lowered quite a bit. And by the end of the day, if we continue to do that, it will take up to 30 steps. First of all, it will take up to 30 steps. So if we are going to take it to a steady speed at least 30 steps, when we are at the very high speed it will take up to 30 steps. And then we will take one minute to finish. So, the whole of this will be quite a bit of a chunk in the image. When I talk in the image, I usually have the detail of the quality of the image. This is quite upgrade or extension if it's not in the peak value, and it will not be used in the effect of the user experience. For example, when launch 100 container, it can be upgraded from 10pm to 6am. If one container is upgrading and the rest of the running 99 is serving the users, then it will not cause any problems. But for capacity expansion, it's different. It's another story. If we want 100 container required for the 5 million users, and now we dramatically need 500 containers, and the response time will be timeout, so the user experience will be worse. So for this time we need a quick acidic expansion, and most of the time of this process are consumed on image downloading. Therefore, another problem is that GZip performance is too low. Actually GZip is quite ancient topic. The compression efficiency is quite low. For another reason, we believe previously when we encounter a situation when we download an image, GZip needs to take a single call to do a single thread processing. If two or three images are downloaded simultaneously, it will take up two or three calls simultaneously. It will call the JTIN of the business. Previously we also, we often receive such feedback. So for this issue we have done some minor optimization, as you can see on this slide. For the first case, we can have one GZip to compress that, we can have 14 million. And for the second case, when uploading the images, we do not compress them. We can compare the time consumed for both scenarios. For the first case, we found the time consumed only takes a half of that when the image is compressed. When the image is folding time, the process is quite time consuming. It almost the same to the image downloading time. So if we do not compress the images or do not optimize the process, it will take three more seconds. This is not very significant, but another significant improvement is that it will not affect other business. And besides, the dynamic process can be released. And for the physical machines and the hyperlinks, the transition speed will be, it's using a more, a more better process than GZip. That was optimization for the struggling. The second optimization is for the remote disk. For now we add one more image and to convert that into a remote image, which is stored on the remote disk. So the real data can be stored on the remote disk. The image record the ID of the remote disk. So in this way, when we download the image, it only need to download the ID information. And the size is quite small. And the size will be only several K. So we can achieve the second level image downloading and folding. And with all the devices deployed in the same rack, only same DC, almost we don't have any latency. And for the remote DC, we have another question. Previously, during the single day sale festival, when we do the large scale expand for the remote DC, we found a question that would the physical level check out by the business, by a large demand, and when we access to a bilingual file, to be read from a remote DC. So because it do not have enough memory to read this file, so all the data have to be uploaded to online. And we thought that about 15 level bandwidth have been taken up by the remote DC. For the second question, and if any problem, or any failure happened of the network, the content cannot handle that. It definitely will make some difficulties. Well, for the third part, the first issue is about the high cost. We will use one device, and we use a one remote DC, may have several racks, it will cost very huge cost. Well, for now we use the Dandi technology. We want to reduce the cost. And now the data are not stored in the hyper storage, and it can be stored on NASA servers. And during the transmission period, we found when we have the remote DC, the data is not compressed. So the transmission process will have a lot of pressures. And now we use a high efficiency to compress the algorithms, so the pressures will be reduced dramatically. And for now, compared with the remote DC adapter previously, we now use a local host caching. So once the container runs a business, all the data of the same images can be put onto the local device. Well, this is the slide for the compression algorithm. Previously, we always want to replace the remote DC, but for some old containers of all versions, we have already found that it's not easy to migrate into the new versions, or decompress into the new versions. So we cannot just delete the old servers or DCs. Well, for the sixth part, we find the 16 million content. Well, for the green part, we find the speed just increased dramatically. And here shows the efficiency result comparison. This is about the local caching. We found some similarity, a similar findings. If we have the files stored on local devices, we can just read them directly. If not, we'll just access to the remote devices. What is the dandy technology or other technologies we have to convert the remote images? And it will definitely take up a lot of resources and the containers. So we have to address many issues. Okay, this is my presentation. Thank you for that.