 Welcome back to the ground floor auditorium. Last talk before lunch, if I've got my timings right. Daniel is an independent Python contractor. He's been a software engineer and working with Python for nearly 20 years now. So hopefully lots of insight to share with us. He's going to talk to us a little bit about how decisions are made inside the kitchen. Hi, thank you. It is working fine. OK, so welcome to my talk. Thank you to PylonDemon to organizing this great event. I'm having a lot of fun. And it's really nice to have an event like this in London, which has been my home for almost a bit more than three years now. I'm Daniel. That's what I said. Mostly working. I'm Daniel. And yes, I've been talking a lot about how the internal processes of the Python community work. I'm not really directly involved in those processes. I've been more of an observer and lurker in this community and other communities. I'm very interested by this. But there's a lot of people more involved than me in this room. So that's great for my imposter syndrome. But I'll do my best to try to help you. So let's say we want to have some Python for lunch. This is a great timing slot for this talk, ready for lunch so I can make you all hungry. The thing is, the first thing I'm going to be clarifying is that Python, unlike some of the language communities, has a very close connection between the language specification, which is Python itself, and the default implementation, which is C Python. And actually, in Python, they both are developed and come out in releases together. So the processes are really interlinked. It's hard to talk about those separately, which may be surprising for people coming for the languages and may be surprising for people in the Python community to know that other languages do these things completely separately. But we want to have some Python. So how do we get our Python? Well, our Python comes in releases. The most important part of the releases is this number, which identifies the flavor of Python we're getting. It's called a major and minor number. Some releases are already stale. You shouldn't eat them. They've gone bad because they're too old. And there are still two seven around that you can still have, but it's going to die by the end of this year. So you should really go for the freshest you can get. These releases come out, well, there's not fixed timing, but in practice, during the last 10 years, we have a new release every roughly 18 months, between one or two years between each release and the next. And they usually last for five years. Once a release is made, maintenance is done for three years, and then for a couple of extra years, security fixes and major problems are fixed. But no big changes are done during successive sub-releases of each release. For each of these, 3.3 has 3.3.0.1.2. Those are called micro-releases. The current state is that the last version to come out was 3.7, and 3.8 is about to be launched in a few months. So we are very happy about that. So we are just having this for lunch, and the thing is, well, who prepared all this? How all this was prepared in the kitchen? Well, the first person we need to know is who brought this to us and put the play together. And that person is called the release manager, or you can name it in different ways. But the release manager, which is around here, the release manager can be a different person for each minor version, the one for 3.8 and 3.9, the one's coming out. The one coming out and the future one is Lukash Lange, most likely mispronouncing his name. He's probably around here frowning. But he's the person who comes out with the final release. What does a release manager mean? The release manager does a lot of operational and boring tasks, like putting the, Lukash talked about this this morning, putting the table together and uploading to the website and uploading the documentation PDFs. He has some help from people who are called the experts. There's a Windows expert, a Mac expert, a documentation expert who help through the process of getting everything bundled together when the time is decided to come up with a release. This is also the person that makes the announcement to the mailing list and who keeps everything scheduled to say, OK, you have to come in this before this day. Otherwise, you won't go into the freeze and you will have to wait to the next version or whatever. So actually, this person also has a significant part of the decision-making process. The release manager decides what goes inside the release, not in the big sense, but mostly putting a red line, like someone is trying to promote a feature and the release manager may say, OK, this is still too immature and we're going to release this day, so this will have to wait or this can get in, this is mature and ready and you can do it. The release manager decides when the micro releases go out and it also has a decision power when a bug is discovered during the release process to say, OK, this is a blocker. We'll actually delay the release because we really have to fix this before moving forward. So that's what Lukash is doing. And there's actually, I don't remember the names from now, Active Release Manager for past releases for 3.6, 3.5, et cetera. And 2.7 even, which is still active. So, I didn't start my timer, so I need to know how I'm doing it. Yes. OK, thank you. So the release manager has to do a lot of stuff. There's a whole document called PEP 101. I will explain what these PEP documents are in a minute. But there's a document somewhere on the Python website that tells you all the stuff you have to do to be a release manager if you want to know more about the role. So the release manager makes the final cut, but it's not necessarily the person implementing all the features because that's a lot of work. The main people working together to do this are the core developers. The core developers are in each station at each end making the soup, the address, the mains, and the grill. And they are preparing the main things and making sure that they all go properly into the dishes and you have a tasty dish to eat. So what's a core developer? The main definition of a core developer is somebody who can commit into the Python repo, the people that have actually committed privileges. And they have a lot of, well, responsibilities around that. They are the people who can say when other people who are not core developers are proposing changes, they can accept or reject it, and they can work on the bug report and say, this is not a bug. This is actually a bug. And prioritize that. To be a core developer, you have to be invited by another core developer. And recently, the process has changed and there's some kind of vote about that. I haven't actually seen that way now, so I don't know much of the details. I only read about it, so I may be wrong. But it's expected from coming up core developer that they are providing quality contributions and they have been reviewed by other core developers many times in the past. And they know how everything works and they are now allowed to do everything by themselves because they've done it many times successfully. There are currently, there's not a public visible list of core developers, but if we count the last, there was a vote last year, which about later, where all core developers could vote and they were about 100 eligible voters. So they are roughly 100 core developers right now. So these people are there. Most of these people are actually volunteers or doing their work during one day of the week where they are paid to do so. But most of them are purely volunteers who love Python and who are putting a lot of effort and a lot of hard work to make this happen. There's actually someone made account of how many full-time people is actually working in Python, paid by their employers. If you count full-time people and the little bits of part-time people together and it's around 2% or so of the people are working paid to Python, which means around two developers if you are all together. There's also I think that there's some rotation around this. There's the concept of inactive core developers, someone who has been a core developer in the past, but then is not doing very regular work or has disappeared from the earth or whatever. So they are flagged as inactive unless they say, hey, wait, I'm still here, I'm doing stuff. There's a process to remove all core developers. So the active core developers are normally really people that's still doing stuff. So these are the people who commit every commit in your Python repo. So these are the people who build most of the thing you use. But there's a lot as an open source project, there's a lot of individual contributors from all over the world that are not core developers that participate in this project, which are the kitchen assistants or the general community that participates in different ways. It's not just writing code but writing documentation, discussing and proposing ideas and new changes and submitting and sometimes reviewing pull requests and issues, sometimes someone posts an issue on an issue tracker, which is a way to help because you're telling, hey, there's a bug here. And another person who's not a core developer says, okay, I can reproduce this or I cannot reproduce this and this helps the core developer, which will take the final work in hands in the end to decide if it is the actual bug or not. And sometimes a community member says, okay, here's a pull request, so now the core developers only has to review and accept and reject and so this assistant, these kitchen assistants are helping the process come through together. So this is the roughly most of the people involved. I'm missing a big part that will come up too. So let's discuss about how someone gets into the kitchen and participates into that. So you normally will start as a kitchen assistant and the main tools you will need are these three things. The first thing is that all main interactions in the Python development communities are ruled by the PSF code of conduct. This is not a long document, it's like three paragraphs which roughly say be nice to each other, please. There's the Python developers guide, which is instead a huge document. There's a lot of content there depending on what you want to do and how you're doing. There's a lot of guidance, it's a very interesting document to read about how software development community organizes itself, so there are a lot of insights here about how this all works if you have more time than a 25 minute talk. And there's of course the CPython repo which is where all the actual development is done and not just the code, the documentation is there and there are some repos close to this where you have the website and other places you could contribute to. So if you want to get inside, normally people start doing what's called minor changes. Minor changes are something like updating documentation or fixing an existing bug or reporting bug or small changes, I made this tweak that improves performance or added this small function to a library module that is very similar to the other 100 functions that are there. But generally this kind of minor changes shouldn't conflict with whatever exists, otherwise you're talking about the major change. A major change is something that doesn't happen so often in Python but happens several times per release. Things that you can find there are stuff that breaks compatibility. This is typically to provide much better functionality or fixing an important bug. Things that provide new ways of doing existing stuff. It's something can be done in Python but you provide a much better way to do it. There's an important decision there should we add this new stuff and have redundant ways to do this. Okay, this is probably a major change that you're talking about. If you're adding time to a module or overhauling the internals of the interpreter or changing the whole parser or the virtual machine or stuff how it works, it's always a major change. New syntax, no matter how small, is always considered a major change. So if you are proposing or working on a major change, even if you're a core developer, this requires a lot of discussion before getting accepted. And typically this discussion is done over mailing list but also writing a document called PEP which is a Python enhancement proposal which are documents that the Python community proposes that says, okay, I want to do this change. Let's discuss over this and let's see the pros and cons and what impact it will have and how this will be implemented and a lot of details. So the different channels where this happens is there's a Python dev list, all the core developers are there but other people can enter those lists and see how the kitchen is working inside. And in the Python dev list, you normally see actual changes or existing PEPs being discussed and approved or rejected is people saying pros and cons of the PEPs are discussing there. The Python ideas list is a more general list where people say, hey, it would be nice if this, proposals here are always more rough and unprepared and people go refining and improving the ideas here. Typically if this is a good idea and it doesn't get rejected all right, it ends up in a PEP that is discussed half in Python ideas but the discussion starts moving by the corridors to the Python developer list. And there are a lot of other informal channels. There's a forum and IRC channel and meetings at Python events where the core developers meet, that will be participate and discuss future changes. But any new features should lead to the development of a PEP and the number one PEP explains how to write a PEP because PEPs also describe internal processes within the community. So let's say you have a great idea, you wrote down, you discuss it on Python ideas and possibly on Python dev with writing a PEP and you have a PEP who decides whether that PEP goes into the language or not. And there's a new role we haven't discussed yet which is what's called the grand chef, the person who is not just in charge of the kitchen but who decides what goes in the menu. This person used to be Gido van Rossum, the benevolent dictator for life but actually that changed last year, Gido retired and told everyone okay, you core developers should decide a new way of deciding approval or rejection of PEPs and that resulted in the steering council. The steering council is an organization currently formed by Barry Rosso, Brett Cannon, Carol Willing, Nick Oglen and Gido van Rossum is still in the steering council but just as one of the five members. And these people take the final decision on PEPs. This is a very important role because this is not just approving or rejecting stuff by deciding what to approve and reject, they decide what's Pythonic and how the language evolves and where it goes to. So it has a lot of other roles connecting the core team and the software foundation and there's a system of voting it each time a new release comes out. So there's a lot of details on how this works on another PEP document that the Python, the core development team is a team that essentially produces PEPs for everything, that's what they do. And there's a PEP for describing how they organize themselves. So it's hard to get through all of this. Well, the core team has very high standards so we get the best dishes and that's really important. So there are some part of this which are really hard. There's also a lot of mechanisms of support. You have the developer guide is amazingly huge and comprehensive and detailed. There are mentoring schemes where you can, a core developer can mentor you to start contributing and if you want to be a core developer in the future there are specific mentoring programs to introduce a mentor assigned to you to get you as a core developer. So there are a lot of ways to help you if you are really willing to put the work to contribute. Something you have to be ready from the beginning and I see this a lot is you have to be really ready to defend your idea from reactions which may seem negative but they are actually trying to protect what Python is. So if your idea is good you have to be willing to discuss and defend it about different comments that you'll normally hear. For example, hey, but that breaks backwards compatibility that's a dangerous change there's more discussion about this. To give you a couple of examples of things that can break backward compatibility and seem very simple. One of the core developers here yesterday Pablo mentioned, okay, let me show you how to add the right arrow operator to Python. Okay, let's say someone says, okay, this is nice. Let's also add a left arrow operator in Python which is almost the same, right? But this actually is, if you add these syntax you will have redundancies because it's also mean A is less than than minus three. So there are tiny details that may seem very minor but can have a huge impact on all existing Python codes so everything has to be considered very carefully. If you want to add a new keyword that's also something that will always require discussion because some people may be using this keyword name as a variable name or a method name or whatever so that may be a problem. They will tell you that if you're contributing libraries or features perhaps that could be a library and live in the package index and not be part of language. This has changed that only a few people benefit with. This is something we can already solve and the solution is good enough even if yours is a little better or they can tell you your solution is worse than the existing solution. So there are a lot of things that you will hear about and you have to be ready to answer to all of this if you're proposing a major change to the Python language. But it's something that even if it's hard a lot of people do it and Python changes and evolves and there are some, let me show you some examples of cases that have gone in the history. For example, matrix multiplication was proposed in the year 2000, someone said let's add an app operator matrix multiplication operator it was slightly different than the current one and it was rejected and roughly at the same time another person say okay let's add element wise operators they added six different operators which were rejected around the same date and everything went cold but 14 years later in a very different context where data science and, okay, data science and the data science community was much more important in Python having a matrix multiplication operator was really huge so if you're willing to wait 14 years you can get some changes accepted, opinions can change. I'll skip another example because I'm a bit out of time but my main point to drive is that you can do it as long as your idea is good and as long as you're willing to do the work. I'll be around if any of you has questions or want to discuss this, I'll be happy to do it and you probably can ask the core developers which know more about this than me. But thank you for listening to me and. Thank you very much, much appreciated. As you said, we're out of time for questions now because we need to go to lunch but I'm sure stick around for discussion afterwards if you want to.