 Thank you. So, like he said, I'll talk to you about the PIP internals. So I guess you all know PIP, and they use it almost every day. But I guess you don't really know what it really does, at least not everything. So first, a few disclaimers. PIP has only one API, and it's not really one, but it's command-line interface. So everything I'm going to talk to you about is only true for this version of PIP and might change in the future. Obviously, the internals don't change completely from one version to the others. But clearly, you shouldn't rely on the function I'm about to show you. And so the stock corresponds to the list of PIP versions, so 7.1. I'll present you three main parts, I guess, the one you use the most corresponding to the installation of packages. So first, how does it resolve dependencies or doesn't? How does it perform an installation? And in the end, how does it select a package? So first, how do you tell PIP what you want to install? Well, there are three main ways. So either you directly pass an agreement, like my package, like Django, or a link to a file, or a pass to a directory, or a file. You also can install an editable package. So it's a .e or .editable option with a pass. This is used mostly in development to be able to change a package while you test the packages. And finally, requirement files that I think you should certainly know the three ways. And there is now a new one since 7.1 called Constraint. It's a little bit special because what it does is setting Constraint on the package versions, but this won't cause the installation. So you have to ask for, let's say, Django to be installed, and then through the Constraint files, you can say what version you want to install. So this is the structure that holds this information is called a requirement set. It's just a big bag of requirement. You can, of course, ask PIP to install several ones at the same time. So you can ask for several editable packages, also package, give it a requirement or two, and also Constraint. You can completely combine as much options as you want. So like I said, the requirement set is just a big bag of requirement. And from the command options, so the four ones I just showed you, PIP will call add requirement on all the four items beginning with Constraint. So what does add requirement do? It does a few things. First, it checks for environment markers. If you don't know what environment markers are, they are defined in the current PIP is 3, 4, 5, but will be extended in the next package metadata PEP, so 426. And it allows to specify required dependencies for specific environment. So for example, you can put in your setup.py files, something like that. And it will only require async.io for 3.3, and you will get it for free in the next versions. So you can do this kind of things, or even a little bit more complex. So this is an example from the wheel package. And it's a combination of environment markers and extras. So what it says is that if you ask it to install wheels with signatures, it will install carrying always, because it's required. And on every platform between those, it will also install pyxdg. Environment markers are certainly useful to avoid the logic in your setup.py files, and will allow you to have these optional requirements in wheel files. So this was the first thing I'd recommend those. It's checking the markers, whether we should add the requirement to the bag or not. Then it will check if the same requirement was already supplied, and will reject any double requirements. So you can't ask Django version 1, and then ask him to put Django version greater or equal than 1. It will reject it for now. Except, and this is important, if the previous requirement was a constraint, and then it will always use the constraint version. So if you have something like that, so putting Django 1.8.3 in a constraint file, and then asking pip to install 1.4.3, it will use the version specifying the constraint files. This can be really useful to force pip to install a custom version or local version with small patches, whatever the packages will ask. And then finally, and this is our currently pip reserve dependencies, if you have already one dependency, one requirement, and another package requires the same package, it will just ignore it. So this is an example. Imagine you have a simple package with an install requirement on a pip8, one version 1.2, and you ask pip to install simple on pip8.1.3. What will be called is simple, add the requirement simple, then add requirement pip8.1.3. And when we will resolve dependencies and try to put pip8.1.2, it will be completely ignored. So now you have a big bag of requirements, and pip calls what the method is, prepare files, that will, as it says, prepare the files for the installation. So it will call prepare files on all unnamed requirements. So unnamed requirements are editable ones, or when you ask him to install a pass or file, then name requirements, and after that, the newly found ones. So the dependencies of the previous requirement. So what does prepare file do? So first, it will check whether it's already installed, and if the currently installed version matches the requirement, then it will try to find a link corresponding to, if you say pip install Django, pip has to know where is the files to install Django. So this is done in the find requirement, and I will talk more about this function in the last part. And there is an interesting point here. This is the wheel caching since version 7. When it will set the link for the requirement, it will try to check whether it has already built a wheel for this file, for this link, and if so, it will use it. So giving a faster install, then it will unpack all the requirements it found in a build location and compute the metadata. So running setup.py agafo to get the requirement and compute the dependencies, eventually with extras. And that's the way it detects new dependencies and new requirements and puts it back in the requirement set. So for instance, if you pip install flakate, it will find the three requirements, pip, pyflex, and miccab. It will populate things for all. So calling the find requirement for each of these three packages. And in the build location, it will unpack the four requirements asked. So flakate, pipate, pyflex, and miccab with either setup.py to install with setup tools or if the files were wheels, just a deep directory with the metadata. So as I've already said, since the pip 7, you have a new wheel caching option that will work giving two things. First, you need to have the wheel package installed in your environment. And then you should not be using the load or no cache dear options. If you use either of the tools, the wheel caching won't happen. And then what it will do is the pip has all the requirements unpacked in the build location. And it will try for each of them to build the wheel. Obviously, it will ignore all the requirements that are already wheels. It will also ignore editable requirements or links pointing to version control system. Since there is no real... Because it could be subject to changes. And of course, if you want to change your file, you don't want to use the last wheel you've already built. The thing is, these requirements are prone to changes, so you don't want to build a wheel for something that will change. And of course, you can opt out of this option with the no binary option, either passing it all the old tag or the specific package. And there are some other special cases, but I won't talk. And then, if the wheel building was successful, it will replace the previous link with the file link to the wheel file and unpack it in the build location. Otherwise, just ignore the failure and keep the previous unpacked standard distribution file. So now, PIP has all the requirements it needs in the directory. One for its requirement. And there are basically three possibilities. Either the requirement is a wheel, so it's already unpacked and the wheels are basically the unpacked package, the way it should be installed. So all PIP has to do is move the directory, all the files in the side packages, the directory to be installed. And there are some complexity with scripts and data that I won't talk about. And it will put also a record file to ease the uninstallation of the package. So this is for the wheel files. It's quite simple. Then, yes, okay. And of course for wheels, you just need PIP. You don't need any additional setup tools. So how does PIP install setup.py files? So it will... It's using some kind of hack to make sure that you always have setup tools. Even if your setup.py works with distutils. So this is not exactly the command, but that's the idea. Basically, opening setup.py, compiling it, and executing it with the setup tools available, so that setup tools has already monkey-patched distutils. And most importantly, PIP install calls setuppy install with some option, mainly record, to record the install files, and single version externally managed to avoid some setup tools. And if you only run setup.py installs with setup tools, you will end up with invocation of ease installs that will also try to install all the requirements, which might not be what you want. And if you directly run setup.py install with distutils, you will end up without any install files.txt, meaning that PIP won't be able to uninstall the package you just installed. So always prefer PIP install. That will run the setuppy of the current directory to avoid the previous issues. And for editable requirements, it's quite simple. It's the same. It will just run setup.py develop with the option no-depth. So now we have the whole bag of requirement, requirement set, and what PIP does is first uninstall the old version, if you need to, then try to install the new package, and success always good, and the uninstall is committed. But if the install doesn't work, it will roll back uninstall. But it will only do this for the last uninstall package, not the whole install code, meaning that if you have package A and B, so A version 1 requires B version 1, A version 2, and so on, you start with A1 and B1 in your VM, and then when you run, PIP install A equal 2. PIP will uninstall B1. Well, it might uninstall B1 and install B2 successfully. Then it will try to uninstall A1, but unfortunately it will crash on A2. So A1 will be restored, but B won't change, so you might end up with A1 and B2. So it's good to know that you, PIP won't roll back the whole install code, just the last one. It also tries to respect the provided orders, and since the recent version, we'll try to install dependencies first. But there are no order guarantees, and if you really need it, it means that to run the setup.py of your package, you need to have some requirement. You should be using use requirement. You use setup requires, but of course there are some drawbacks that is currently owned by setup tools. So if... So PIP doesn't know about setup requires, and since when you will invoke the setup.py call, set setup tools, just find it, and we try to install the requirement within the install, and if the install won't honor the option you gave it. Hopefully, maybe PIP might take control of this feature in the future. So now to the biggest part, which is when you PIP install foobal, how does PIP know which file it should be using? Because you might have several pipy servers, maybe a private pipy, maybe a wheelhouse, and some other solutions. So your PIP is exposed to different versions, formats, or locations, and all this magic happens in file requirement. So you give it a requirement and whether you want an upgrade or not. And it will return the file to install. The way it does it is first calling find all versions that will basically find everything you could install in your system. So just for Django, it will find more than an Android installation candidate. And on my setup with a private pipy on a wheelhouse, it finds more than 200 possibilities. Then it will filter the requirement you might have given. And with the pre-option, it will remove all the pre-releases. No, it will keep the pre-releases. And finally, it will sort my version, so preferring the latest one. And depending on whether you told him to upgrade or not, it will choose the latest one or the currently one. So the main point is on find all versions that has three main sources, indexes, extra indexes. Indexes, yes. So they come from the option extra index or no index. You have also find links. And also dependency links that are in the way of depreciation but not really. It's complicated. And so indexes expect something like scheme some index and will try to go to package name to list all available files. But of course, you can give it all kind of names and people will try all of them because they are basically endless. So with modern indexes, what PIP does is simply goes to PyPi, simple Python data utility. And PyPi will redirect to the correct location. This is with modern indexes. With older and simpler, what you will do is go to the given name packages, package name. It will return the 404. And then it will just simply go to the simple index which on PyPi is quite big and will match all the links with a normalized form. And this is only done on the first index because you can have multiple index. And the first one will give it the correct name. So you go from simple to simple Python data utility. So now you have three main sources, index links, find links, and dependency links. They are put in a function called sortLocation that will return two lists, filesLocation and two allocations. And what is important is that for findLinks, you have here a special case for findLinks that if you give it the directory, it will get all the files out. So this is how a wheelhouse is built. Just put all the wheel files you want in the directory and give it to findLinks. Then to find the findLinks option, you can also give it direct location and also the dependency links. There is a lot of special cases for URL locations that correspond to the indexes. So to check whether it's a secure location and you can add some with trusted options. And then all these links are passed as HTML. PIP basically check all the links. There are some special cases for that. I won't talk about this. And all these links are then put in a big filter called package versions that will push some links at the end and select or reject lots of criteria. Basically, whether your system supports the format, remove some macOS files. This is really a special case I found. Whether you ask him to reject wheels or standard this for this package, it will also check the compatibility of the wheel files. Try to extract a version from the file name and reject if it doesn't. Check also on the Python version if you... This is something I did on you. And also check everything about internal or external files. Finally, if you come back here, you see that you have three main sources, files, URL, find links, and depth links. And finally, PIP will prefer always file sources, then direct find links, then URL links, and finally dependency links. And this is basically what... Find old version returns. And this is an example for PIP8. So you find all the files from the wheelhouse, then from all the files from my private PIPi, a normal PIPi. So it contains the package, the version, and the direct link. Thank you. Any questions? We do have a few minutes for questions. Is there any question around? Yes. Have you looked at Conda at all? And could you speak about the differences, what any advantage is using PIP versus Conda? No, I haven't checked. Internals of Conda. Could you explain what a wheel is? What is a wheel? It's a kind of new package format, but it's already a few years old. And what it's... It's basically... If you take a standard distribution and you install it in side packages, if you take the directory PIP as a install in the side packages and you zip it, it's basically a wheel. So a wheel to install it, you just have to unzip it and put the directory inside packages. So it provides much faster installs. And... You talk about constraints. Isn't it a bit dangerous to use constraints? Well, there is no surprise. You won't use constraints by accident. So if you tell PIP to install a specific version, it's because you mean it. Yeah, but actually, I don't see a real use case where you want to override the dependency version. I think one use case is... because you know that you don't want to use the... the version someone might require. The truth is, you might then use directly requirement files. But I think the use case of the guys that added these features, I think it's Robert Collins. It's for OpenStack where they have a lot of requirements and they want to pin all of them. So they just put everything in the constraint files, and every time they call PIP install, they just give it the constraint files and they are sure that everyone will have the same version. Because when you give constraint files, it won't install everything that is inside. It will only install it if you... on the side require the requirement. The constraint files are just to pin a version on a requirement and it won't install the... like requirements takes the requirements files. It won't install everything. Okay, hi. I have a very easy question. Why versions of your software jumps so fast? Is it incompatible each release or it's just the kind of software currently is working? It doesn't... Well, the thing... Maybe you're referring to the fact that it went from 1.5 to 6? Yes. Well, it's just because they dropped the one. But you really should think of 6 as almost like 1.6. So it's semantic versioning currently or just... I don't... Well, clearly the past from 1.5 to 6 is not semantic versioning. I don't think the developers are... plan to stick on semantic versioning. I don't know. Okay, thank you. Okay, so it's supposed to be the end of this talk. So what we can do is those who want to go to lunch, you know, and make as much noise as you want.