 really looking forward. It's a pleasure. It's Fabien Boucher with a talk, a Docker build chain for Python applications. So, thank you. I'm here to talk about Docker. It's our build chain for build a Docker image from Python package. But first, let's get some context. I'm working at Polyconseil. Polyconseil is a company behind Autolib information system, and Autolib is an electric card sharing services based in Paris. In fact, we have five electric card sharing services in the world, but Autolib is the first and the largest one. So, if you have heard of us, it's probably used by Autolib. The Autolib information system is composed of certain applications. And when I say applications, I do not include backend like database, ready server, and stuff like that. Those applications are mostly based on jungle applications, a bunch of open source libraries, and you run libraries to run business logic. And our problem was to deploy those applications in production. I work on developed subjects from time to time. And that's why I'm here to introduce Docker. Why we build Docker? And before using Docker to deploy your application in production, we use Debian Packaging. Debian Packaging was held in 2015 for Python application. You have to edit your Debian Package metadata by hand to, excuse me, by hand to report the versions that are already declared in your Python package. In your worst case, we took 48 hours to package your application and all these dependencies. And we, we aim to deploy your application once a week. If it took 48, sorry, if it takes two days to package your application, we have only three days left for validate your application, do the bug fix, and then revalidate the application with the bug fix. And your validation process is very long. It took about a day. It's because we have physical device like car, charge point, and rental cost. And the device has to be manipulated by humans. So we move to Docker. Docker allows us to have atomic updates. Once you want to update a system, an application with Docker, you have just to pull an image and run it. You can have a half-installed application. The image is based on another machine. So if the build process fails, your production is fine. Docker allows us to put multiple applications on the same server. Before we have only one application by server. Most of the time, the server does nothing. Yes, there are other tools that do the same thing as Docker, but in a different way. For example, OpenShift source to images do approximately the same thing, but it starts from source and not a piton package. In fact, when we start writing Docker, we do not know that source image exists. Docker comes from many approaches we try. The first one was to use pip install on the production server. In fact, we don't use this approach because it's too dangerous. With this approach, you have to build your C extension in place on the production server and it may fail. The next approach was to improve your Debian packaging tools. Like I said before, it was L. So we gave it a new switch to Docker approach. The server approach also has another issue. You have your application library linked with your infrastructure program libraries. So if you want to update one of your libraries, you have to be sure that your application and infrastructure program works with a new version. It's very difficult. You have to do two changes at the same time. And it's not a good thing. The next approach we tried was to use Docker file. But with Docker file, you have to build, install your build dependencies, install your application and remove your build dependencies in one layer, on one layer. Otherwise, the image size will be very big. More bigger than accepted. Because the current do not delete files from, sorry, it does not delete files. It just marks them as deleted. It's just a mask. And the last approach we tried was the slug approach. In fact, it's a Heroku-like approach. Heroku is a very popular platform as a service for those who do not know it. In the server approach, the build phase and the run phase are split. In the build phase, we start with a base image which includes your run time dependencies and your build time dependencies. You instantiate one instance of this image and you build your application on it. You save the result of the build process and you delete the instance. And then, once you run your application, you will start a new instance of the image, put the result of the build process on it, and your application will run in a new, it's a new instance. The approach has a problem. You have one big base image for all your application so you cannot have an application with special requirements. So you have a very big and fat image. In fact, Heroku makes the two last approach. We will use separate build and run phases. So how it works. Once you build an image with this broker, we will start by pulling a base image on the Docker hub, which is for DebianJC. On this image, we will add your run time dependencies to create the root image. This image will be, we will use this image as base for the compiler image, which is a base image extended by build time dependencies and a compiler script. This image will be run to compile the way for your application and the ease dependencies. There will be stored in a data volume. And then we will use a web server to expose them. On the web server is running, we will create the runner image, which is the final product of your Docker build chain, from the root image and we will install the way we have pre-compiled by the compiler image. In fact, it's more complex than this because we have one root image by run time dependencies set. But it does not matter. It's the best principle of how Docker works. The compiler image is here to allow to compile will and especially see extension and link them to the library that will be installed in your running image. And he avoids the Docker layer problem. Docker works fine. We use it every day to deploy your application in production. But we currently have some limitations. One of them is that the base image, the root image, sorry, is 200 megabytes for your test application. The test application uses Z bar and is a library which allows to decode the QR code and other stuff like that. And for your more cumbersome application, you have a base image of 600 megabytes. It's a very big base image, root image. Maybe it's because we use DBNJC, which base image is 125 megabytes compared to a lightweight distribution like Alpine Linux, which image is only 5 megabytes. But Alpine Linux do not have Z bar. And we use Z bar in your production application. Another problem with the current application is that it can only build packages applications. And in fact, your application has to be packaged and on a PIPI server. It can be a private PIPI server, but it has to be on a PIPI server. For the first limitation, we currently are working on it. We hope to have Alpine base in the near future. Yes. So how to use it? It's very simple. You just have to uninstall the broker, PIP uninstall the broker, and then write on your command line broker build, your application name, two EGLE signs, and your application version. And after no so long time, we will have Docker image of your application. You can run with Docker run. In fact, it's a little more complex like that. Your application has to be packaged, and it must use Python 3, or at least be compatible with Python 3, and do not have runtime dependencies. So high Python is a good example. Otherwise, we have a config file that allows you to set runtime dependencies and build dependencies to other stuff like that. Or you can use config flag. If you read carefully the example, I use one of them. I set the entrepreneur to high Python. So like you can see, is the high Python that is running once you start the Docker image. By default, Docker uses an entrepreneur named broker runner, but high Python doesn't have it. Yes. So broker was open sourced yesterday. So you can see the source on the Github document. The package is on PyPy and the doc can read the docs. If you want to contribute, you are welcome. Thanks for listening. Do you have questions? Any questions? Okay. Why is the size of your image such a big issue? Can you repeat? Why is the size of your image such a big issue? I mean, we have also Docker images, but they are way bigger than 600 megabits. Because we want to use a pass. So if we do not know where the image will build pool, if another of your pace die for some reason, you have to pull another image. If it is 1 gigabyte is a very long process. If you have smaller image, the response will be faster. Thanks for the talk. You say the PyP was dangerous. Can you give us more details? In another approach? Yes. When you will see extension, you do not know if you have good build time dependencies installed. If you don't be very meticulous of what you install on the server, you can miss header or something like that. And if you do it from your production server, if the build process fails, your application is not available anymore. So it is not a good thing. More questions? The config file you talked about, what kind of format does it support? The configuration file you can supply. What kind of format does it support? It is a YML file. Maybe I have an example somewhere. It is very small, but I don't know how to increase the size. The config file for your application is... So you can... I can read it. You can set your library, your volumes, your enterprise, your port, I think. If you go on the documentation we explain that on the... on this page. So you can say... you can put dependencies that are just run time... for dependencies you can say that it is just a run time dependencies. For example, we use... I don't know. But you do not have binding a say extension. You just use a set type or something like that. You can have one run time dependencies or a list of run time dependencies. More questions? We have time enough? Okay. Thanks again.