 Hello everyone. Yes, I'm Peter. I have a few hats on for this talk. I'm a developer of CPython, like the main Python implementation. I'm a member of Fedora's Python SIG, which is basically just the people on the mailing list trying to make Python and Fedora better. And I'm a member of the Python maintenance team at Red Hat. Python has a nice, nice position in Fedora because there's the team of paid developers, so we get to think more about how to do things better. But despite the name, the talk is not exclusively for Pythonistas and Fedora. I wanted to make it a concrete example of problems that distros and language ecosystems have. So I'll be using Fedora for the distro, Python for the ecosystem, and all the other variables like this. But if anybody is here from some other ecosystem, languages or data scientists, educators, people like that, or from other distros, I hope the talk will be useful to you as well. How many of you are there like that? Who of you doesn't use Fedora as their main operating system? Nice. I should have a better title next time to make more of you come, right? Who of you uses a different language than Python as their main language? All right. English. Yeah, I hope you use English. How many of you are packages, like distro packages? Probably could go more into how those groups overlap. That should be enough for now. So we have this distro, Fedora, and the ecosystem. And Python, the ecosystem is PyPI, the Python package index, and PIP. There are some important differences between these. On PyPI or NPN for JavaScript or whatever Latte has. The ecosystem, like package repository, is usually, you know, anybody can upload anything. Why do you reserve a name and then it's yours and you just upload there? If there's a copyright violation or something like that, you'll probably get done. But that's about it. In distros, we at least try to make it curated, make it part of a bigger home, make it a selection of the best things the ecosystem has to offer. PyPI is not exclusively for open source software. You can upload a binary there. People don't do that very much, but you can do it. The other thing this source optional means is you can upload source and you can upload binaries to PyPI, but there's no guarantee that they actually match. And if you build from different sources and upload to PyPI, well, nothing happens, right? And when you download a binary from PyPI, who knows what's in there? In Fedora, there's like technical guarantees that it is built from the source that was uploaded and nothing other than the source that is actually archived. PyP and Python is focused on package installation. DNF is focused on package management. So if you do stuff like reinstall a package or delete a package, all these things, if you do it at DNF, the right thing happens. The dependencies get tracked. The reinstalls go well and things like that. And PyP, there's a plan to do this, but it's usually easier to just throw everything out and start from scratch, which is very easy to do. Maybe it's this container-based mindset. PyPI is cross platform. You can upload Linux or Windows or Mac, Mac software there. No problem. Fedora is cross ecosystem. So, you know, we have Python software. We have Ruby software. We have Java software. There are attempts to make a cross ecosystem, cross platform repository, but it's turning out to be really hard. So we usually have either this or that. Also, the Fedora ecosystem, at least for Python, consists mostly of taking packages from PyPI and repackaging them. I know that many of you are packages. So what's a package's role? We produce RPMs by copying and pasting snippets of spec files. Right? Who feels like that? It's unreasonably hard and boring to be a package. We're trying to change that. That's probably for another talk that didn't get selected. But obviously, this is not why we're packaging. This is like the technical way how it's done and that could be improved, hopefully. I think a package's role is that it helps. The package helps projects with specialized tasks. For example, license review. You know, if we check if everything is legally correct, according to what Fedora thinks is legally correct. There is a security response team that watches out for potential security problems and reports them. And Fedora package is supposed to make sure the security problem is fixed in a timely manner. Obviously, it's a volunteer-based distro, so there's no guarantees, but that's the spirit of how things should go. And last but definitely not least, integration. This, I think, is the main thing that Fedora brings to the table. It takes a bunch of software from all over the Internet where developers go and just put their code somewhere. It works on their machine. It works on their Docker and AWS and whatever. And if it doesn't work with some other set up, they tend to not care that much. And the distro is there to make everything work together, which is a non-trivial task. And that is the task that can be automated. These three are the things that we really need people for. These are the reasons we need distro packages that can't be automated away. And I really think the integration part is the main thing why we have distros. One thing about integration that's come up lately is this too fast, too slow problem. Is there who has heard of this problem? Really? Whoa, that I didn't expect. Who has not heard of this problem? Whoa. Wow, you're even paying attention. Okay, so this problem is that if you have Linux distribution or any software distribution, some pieces of it are updated too fast for you. And some pieces of it are updated too slow for you. Right? For example, you need the kernel to be stable. And when there's an update, you know, there's 30 new security updates. Okay, let me just patch everything. And I hope it doesn't really break anything. So the kernel moves too fast. You would like it to be more stable. So you don't have to worry about it. You just want it to work and be the same and all that. While on the other hand, if you are a Python developer or then your response to the new patches and features in Python might be, you know, this commit fixes a problem I'm having in my app for the last half year, but it only comes out officially next month. So let me play the patch now. I wanted this to move faster. And the problem with making distribution and make everything work together is that there are a lot of people using it and everybody's notion of what is too fast and what is too slow is different. You know, if I'm a Fedora user, I want my apps to just always work and maybe to add new features, but mostly not to change. If I'm a web developer, I want my web framework to be stable so I can rely on it. If I'm a Python developer, I don't really want major breaking new versions of Python. If I'm a Python core developer, I want to rely on my C library staying stable. If I'm a Rust developer, I want the kernel to be stable but I don't really care about Python. Does that make sense? So there's no way to please everybody when making distribution. Let's go through this too fast, too slow with some more words. Usually, if you want something to move fast, it's exciting for you. Probably cover that. If you want something to move fast, it's part of your solution. It's part of the thing you're developing. You pick your parts, you know about them, you want the latest features. While as the things that move slowly or that you want to move slowly, you don't care about that. That's just there and you want it to work. What should happen with the things you want to move fast is you can co-maintain them. You can make sure that it is actually moving faster for you. What should not happen for the slow is you just take it for granted and you don't care. Somebody else is taking care of it and you're just using it. If you wanted to move fast, it's good to have involvement. So you kind of steer the direction it's moving in. If you wanted to move slow, then you need to trust whoever is taking care of it. If I'm using the kernel, but I don't really care about the kernel that much, I have to trust whoever is maintaining the kernel for me. Ideally, I would have some idea of what they're doing and how long they'll support it for me. Otherwise, I'm in for a nasty surprise when something unexpected happens, but I can't be involved in all the projects I use. Usually, people pick the ones or pick the ones they want to move fast to get involved in. This is not just a spectrum. You can order the things based on how fast or slow people want them to move. For example, if I'm building a Python web app with a JavaScript front end and I'm the Python developer, I want my own application and the web framework to move fast, but I don't care that much about the equivalence on the front end side. I want the JavaScript minify or whatever that is, to just be there so I can use it. And if I'm a DevOps person, maybe I want to use some custom patch in my kernel to make it work with my load balancer and just pick that part to move faster. It's not a strict spectrum. So there's that problem with making it a distribution. So the way I think we can solve this is to package everything together, to make it work as a whole and then let people substitute different parts. If I know I want my web framework to move faster, then I should have some knowledge about the web framework and I should be able to install my own version and just use that. And then I'm totally in control of it. I can update it whenever I want. And if I don't want to update it myself, then I can just fall back to the distro version. Or I can make a spin of that distro. However, spins are called these days, modules, you have containers. So you can maybe share that maintenance with more people who have a similar notion to how fast things move to yourself. But it is my opinion that the system should focus on building a cohesive whole, on making everything work together. And cater to the users that use the system as a whole and don't change anything. Because if you want to change anything, you probably have the knowledge to do it. Back to the topic of my talk, the way you change something is to install it from the ecosystem. So if I'm a Python developer, I want to change my web framework, I download it from PyPI. What that means is I lose the advantages of the distro, but since I'm involved in that ecosystem, I know the risks I'm taking, I probably know that people are developing this software. So I'm in a position to, for example, make a Python virtual environment and install the packages I trust into it that are not part of my operating system. And I think that's a nice boundary. It would be good to make it possible to move that boundary in different ways. Currently, in Python, you can make a virtual environment that isolates all the Python packages. So it's problematic to use system packages in your virtual environment. Maybe that's something we should work on. But currently, if you're a Python developer, the way to go is make a Python virtual environment, install packages from PyPI to there, and hope that those are correct. And it's working quite well. If you're not a Python developer, use the Fedora package stuff. If you know the ecosystem, use PyPI stuff. And from what I've heard, people like this. Do you like this? Possibility? Do you not like it? One person doesn't like it. Right. If you don't like it enough, then come help us change it. So, yeah, that was language ecosystem and the distribution, how they interact, what you can do with them. How Fedora loves Python. How do we advance this kind of mission of integrating Python into Fedora? And what I think other distributions, other ecosystems could get inspired by is we treat patches like distro patches like issues. If you change something, if the upstream project thinks should be one way, but you have to change it in the distribution, you can just patch it, but the work is not done. Then you have to argue with the upstream that your way is really better in your specific use case. Make them see the light and make the change even there so that everybody can benefit from it. Or they can tell you're idiots and that's not the way things should go, so you remove the patch. Or they can say, okay, you have a special need here, so it's probably better if you keep the patch, but not everybody else is better off without it. Or they add a configuration option. But it's always necessary to have this conversation upstream. I have a little graph of how far we progressed for Python 2 before we started on this mission in Fedora. We have 40 patches, we haven't really touched Python 2 in a few years now, and with Python 3 we're down to 7. You can also see how much other distros patch. Of course this is a biased metric I just using for this talk that doesn't say the other distros are bad. They would probably make bad Fedora, but that's about it. You can also see that Arch Linux is better in the number of patches, but maybe that's like having a bug tracker with just one bug on it. It's also not a good thing for the same reasons. The other thing we do is package up all the pythons so you can test your software with all the versions, and since we patch as little as possible, if you test on Fedora with the different Python, hopefully it'll work everywhere else with that Python versions, unless the other version is patched. But there's only one version that has all the libraries, all the software on top. So if you develop something for Fedora, not just for Python itself, then there's one version that we support for you. All the other ones are just for testing. And we also have the alpha of the upcoming Python 3.9. Miro here is rebuilding all the software with the new version to make sure there are no major regressions in the new Python version and that all the software works with it. This is again part of the integration role of the Fedora Packager, making sure that all the software works together and trying to make the people that use the same software talk to each other or at least be aware of the other piece moving in incompatible ways. And this is helping. The Python 3.8 release manager says that this is really quite helpful in ensuring that you can actually use the new Python version with existing software as soon as it's released because it's already tested with a large enough ecosystem. My favorite? I use Kate. So it's an editor that does syntax highlighting line numbers and can open multiple files at once. I wouldn't recommend it to you, you probably have your own favorite editor. So the question was, in Python SIG, do we maintain packages as a group or do people maintain packages personally? People maintain packages personally. And Python SIG is therefore discussing problems and there's a group of packages also confusingly called Python SIG because of technical reasons that has rights to the basic Python packages that can fix Python-related problems fast. So there are some people with special powers for that reason. But most packages are maintained by the maintainer. PIP install as root can do very magic stuff. So yes, it runs code you just downloaded from the internet as root. That's like the definition of what it does. It can break everything. I'm out of time. Sorry. I'll be around. There's a Python booth if you want to catch me. Thanks for sticking around.