 This is Python buff, which means there is no presentation or something like that. We will have a discussion here. There are some prepared topics, but we can talk about whatever is related to Python and Debian. So, it's just a suggestion, and I guess we can start with introducing ourselves. My name is Piotr Żarowski, and I'm responsible for a few tools we use, like the H-Python, and Zigo, do you want to introduce yourself? I'm the maintainer of the most, the largest number of Python packages in Debian, and that scares me, okay, because I do oven stack. I'm new to Python, maybe I'll maintain a package or a few packages, but I plan to get more involved. I'm just a user, I think. I work, I'm a Python dev, and I use Debian for any server that I need to work with. I'm Nikola, I'm a member of the Python team. I do Python professionally, and I end up packaging the modules that we need at work. Hi, I'm Everton. I use Python on embedded systems, so I'm looking forward to see how it goes. And here's Debian, Python package maintainer, and maintainer of some modules. Hi, I'm Jonathan, or HiVoltage on RSC. Can you tell me if you leave packages in the Python team? Hi, I'm Pedro. I use Python usually to scripts for automation and also to improve to enhance I3. Hi, my name's Matt. I'm a Red Hat to Debian Convert and looking to learn more. Hello, I am Emmanuel. I'm from Argentina. But I maintain some packages. Hi, I'm Chris, I'm very much an end user of Python in Debian, but I want to see what's coming up. Hi, I'm Ola, I'm packaging the astronomy packages for Debian. My name is William, I'm a WG on IRC, and I just joined it in about two weeks ago. I'm Victor, and I've been using Python for scientific computing for three years. I'm Arthur, I'm doing a GSOC project in Python, and I'm just starting doing packaging. Thanks to Zigo, he's here. Yeah, that's it. I am Angelina, I made my first package last night, so why not Python packages for another time? Hello, I'm Elias, I'm maintaining a few Python packages, and I want to learn more. Hi, I'm Andre, I'm one of admins of Python team, and we are using Python in our company. Hi, I'm Jamani, I work with Python for embedded applications. I'm Trisha, and I work with Python. Hello, I'm Marlon, I work with various languages. One of them is Python, and I really like it. Oh yes, I'm going to pass the microphone. Hi, I'm Tomas, I'm maintaining PyCuda, PyOpenCL, and PyTools. So those two first, more in the NVIDIA team and OpenCL team, not Python team, but still those are in Python. Hi, I'm Tim, I'm a Python dev. What should I say? We are introducing ourselves. I'm Andreas Tiller, and I'm coming to the booth because I have some Python programs, and I'm coming too late because I was occupied by a method. So there's a whole bunch of empty seats at the front. Come here. Hi, I'm Stefan Orovera, I maintain some Python modules and the PyPy interpreters. Okay, so I think the Python 2 removal will dominate this meeting, so do you want to start from something else? That's a good time to do it because it will probably dominate the Python 2 removal. So should we start with the next item then? I added the last bullet point here. I like to have different environments isolated from the system libraries to provide different versions of Python. And I'm not sure if there's a sensible way to support that in the VN. And there is similar apps in tools like software collections or environment modules that we could have in the VN. So what people think about it is that something you have discussed in the past, is that something possible, is it impossible? Just to have your ideas. I usually try to have only one version of the library. Sometimes AppStream provides like in Jinja. There's a Jinja and Jinja tool, so different name for the module. So you can install both of them at the same time. But if the name is the same, then it's not really supported in Debian. You can use some tricks to switch them, but it's not really a good idea. If you really have to, then probably the best way to do it is to provide the private library and then add syspath app and this new version if your application really needs it. There's also dhvirtualm, but I never really used that, so I don't know if that works. I'm currently using dhvirtualm, dhvirtualm. But it's like software collections, I don't know if you know, but this tool, like in Ops you can have a jail with different package, Python modules, and you can just tweak the linker and have your application use those libraries instead of your system libraries. So it's not isolating the application from the system libraries. I mean, I understand system we don't want to have different versions, but there are applications that depend on different versions that are not the system ones. So that's... If one of my applications would need that, fortunately none of them currently need that, but in the past I had such situation and I just bundled it in applications private directory so not to mess with the system-wide path. But I don't know if others use the same strategy. Maybe someone has a better idea, but... I would imagine most people deploying applications on their Vienna using virtual ends for their libraries. But in the archive we obviously don't want to be going to... For applications? I don't use it. I maintain quite some applications and I just use private directory. So there's... DH Virtual Lamp or Virtual Lamp is not involved at all. When I have some kind of many applications with many different ways to work and depending on different libraries I just use Docker and work with containers for everything. It's easier than have virtual and some other virtual environments way to deal with it. I have a totally different question. I just wanted to be recorded because we have lots of Python packages in Debian which are maintained by random maintainers, not a team. And we have also in our team Python modules which could be in the Python team or not. What is the opinion? Should we try to assemble all the Python modules to be maintained by the Python modules team to get some consistency in the packaging or is it what we don't want? I think the end goal would be to have as many as possible in the team but I would focus first on making the team to work a bit better so that we are taking a look at... That was the intention of the team because I just approached today one which was a Python interval also maintained by... I don't know. But I had an issue with this package and I asked the maintainer do you want to join the team? It is the correct way to approach people also. We don't care. I think it's a good idea to encourage people to join the team if they have to maintain random Python modules as a dependency to their package. I don't know if... We currently don't have that many people who are taking care of packages other than their own. But the number of people who are taking care of them is greater than zero. So it's still better to have packages in the team because sometimes at least Andre will make a lot of mass commits with fixes. So thanks for that. So do the DPMMT administrator or owners know how many packages in these repositories are out of date or un-maintained? To be honest, I don't know that. So I don't care if a package is maintained by the team or by somebody else outside the team but for me it's very difficult to update things or to see things which are not maintained within a team and it's much easier to see if you have them well, just maintained by a single maintainer then you see that they are flagged if the maintainer is MIR. So it would be really interesting. I know the DPMMT team is very, very... a loosely coupled team but we should have a better understanding about the quality of the packages or how recent these are. So I think it would be nice if we could correctly decide that it's okay to remove any package in Debian to remove Python 2. So switching to the Python 2 topic. So yeah, switching back to the Python 2 topic. Because there's so many packages if we take the time to contact each and every maintainer of a Python 2 package to ask him if it's okay to enemy then it's going to never be finished. I was thinking about that and I think we should start with a mail to I don't know Debian Devil and proposing such a thing so that we'll get a general approval for that. And that's why we are here to discuss what would be the rules of such uploads should we allow such removal only for packages that are already for leave packages in testing. That would be my idea. So not every package, leave package in unstable should be a reason to remove Python 2. But if there's nothing that depends on given package in testing then that would be a good candidate to remove. So this process would be a bit slow but yeah. I want to come back to this question from me. The profit of a team maintenance is that I can easily do a team upload no matter what changes it is. Because if it's by a single maintainer I need file a bug report, release critical, do NMU if I see a not maintained team package I just do team upload and have very easy access to the package and you can do it, everybody can do it as a team upload without all the formalism and this is my argument why it's a positive thing if we have all the things team maintained. And then you can start removing Python 2 in the team upload because we are a team member and you can force people to do so. Team packages would be probably the easy ones that we will start with. Another question which I have completely open as well is let's say I have a package that I want to convert to Python 3 only and then it has a reverse dependency of something that I don't know and it's too hard for me to switch or remove Python 2. Then I file a bug and what's the delay until I can say oh he hasn't done it, I have no time to do it. I still upload Python 3 only for my package that has a reverse dependency. I wouldn't do that because maybe this package is important and if you remove... Then we need a delay so what's our delay until we say... Standard enemy rules apply. 15 days without response. I think it's two weeks, yes. Somebody would have to identify all packages. Didn't you want to do that? And then file bug reports and after two weeks you can start work. If I understood you correctly, you have a package, another package has a dependency on yours and you want to remove Python 2 from your package. So I wouldn't remove it until all other packages do not depend on it, at least at the beginning. At least at the beginning. At some point we will have to raise the bar a little bit lower and make it more aggressive but we are at the beginning of the cycle and we probably should start slow or maybe not, I don't know. We have more than a thousand packages in teams and it will we do one by one and it will take ages to remove Python 2 support. But we should not block package migrations either. I think it's fine to block package migration to the testing if that package still needs Python 2 support because Python 2 will not exist next year in upstream. I know, maybe in downstream, but do we really want to maintain that code for another release cycle and another one maybe? I think our users would like to use Python 2 in the next release. I think it's official that we are going to the start of the agenda now. This agenda item seems to have died in favour of this one. If we drop Python 2, that's a disservice to our users, isn't it? They use our distro so that they can get the software we provide to solve their problems. They haven't all ported to Python 3 yet. There's going to be a lot of people running on Python 2 for the next 20 years probably. But then they don't have to upgrade. We don't need to provide a lot of libraries. We just need to provide an interpreter and they can virtual env and do... I also can't maintain PyPy stack without Python 2. That blockchain is going to move to Python 3 for many years still. Do you think it is possible to identify some important packages that will definitely not get Python 3 support like Mercurial probably? Are there more packages like that? Mercurial is a Python 3 port now. I'm not sure if it's already released. Having the interpreter in the distro is fine. But what I would like to see is that we drop the Python command in the next release. We can do it right now. That's fine. That's fine. That's cool. It's only about rebuilding all the packages with a new interpreter in the Shebangs. So that users have to make the choice to use either Python 2 or the much preferred way, Python 3. I think it's not an issue. I think DH Python 2 already does that. If not, then I can change it so that Shebangs are rewritten into a user bin Python 2 and we can remove it right now. It's not a problem at all. We have all the stuff where we only use Python 2 in the packaging. That's almost all using the unversion of Python Shebang. So there's some work to do from my point of view. Do we need to discuss whether we want to do that then or we all agree we want to get rid of user bin Python? I don't care about user bin Python assembling until it doesn't point to user bin Python 3. What I don't understand is that you are saying that we need the interpreter for our users to be able to use, I don't know, a pip with Python 2. But then you don't want user bin Python and then it will break things that will be installed by pip in usual local bin. So do you suggest to our users that they do assembling? Maybe, yes. We could provide a Python-legacy package as well and get a popcorn idea of just how many people actually need this. It's maybe a cool idea because you said if you remove user bin Python, it will break a lot of things. So we need to be broken that people realize there's a problem and explicitly use Python 2 if they really want to do it. Then we have broken packages and they will not migrate to testing. I like the idea to remove this. Of course it's worth mentioning that Arch already did this and a bunch of things got updated as a result. So I think the removal is a big task. It's a bit stuck now. So I would like to have something where we keep track about progress. I mean, we are at the very beginning of the release cycle. I mean, even after two or three months you can see how far do we get with just removing leaf packages or unused libraries. Then maybe decide in, well, at the end of the year, well, that's going to slow. We have to do a little bit more. Is the release tracker thing already set up? No? No. Well, encourage people now to remove stuff which is not breaking, which is not, well, stopping migrations and things like that. And then decide, well, that's not going to work. But then we have at least something to start with. Yeah, during the DEB CONFA I plan to at least identify team packages with the leaf Python 2 binary packages and start removing them. We'll see how it goes. And then move to the other packages. So is there anybody in the DPMT team who could just keep track about the Python 2 status in packages maintained by the DPMT team? I was hoping to get this release tracker thing and use that. But I don't know if it contains maintainer. So can we... No, it doesn't. Who could grab somebody of the release team to set up that tracker? I could. Okay, one thing. Do we agree as a team that all team packages, which are leaf packages, can be removed just now or don't? Yes. That's what I want to do in the next few days. You know, I have script for everything. So be sure... Well, I think every library package can be removed. From modules, team, not from application. I agree. If we have applications which are not yet ported for 2.3, then yes, we should identify them. When we are close to the next freeze, we should evaluate do we want to keep them or do we have to remove them? But it's not something we have to do now from my point of view. Do we have some tech for the BTS for Python 2? Because I start filing bug reports about Debian-made packages which are needing Python 2 and file bug report with some title tech, but maybe we should text this Python 2 and then we could browse the BTS for some common tech that's Python 2 and we need to deal with this. Do we have this tech or should we create something? And if yes, no, is somebody knowing how to do this? Okay. Yeah, something like this. I just want to not invent something else by using it and you use one. I will use one that's not productive. So, this week I converted GPyB from Python 2 to Python 3 by just taking the latest version upstream which was deployed by its maintainer since 2 years and I did that without contacting the maintainer. Should I be burned into the public place for not doing that or should I be sensed? Shouldn't be burned, definitely not. What I'm currently doing if I spot some application with Python 2 I go to upstream write a bug report in their bug report or mostly GitHub, file a bug report in our BTS link to this and I would like to have some better way to record all these bugs and have them assembled in one place to make it published also to other people than the Debian Me team. Can we create a wiki page with all the hints what to do? So that if I as a maintainer want to suddenly find out that Debian is removing Python 2 what should I do? Create a wiki page That's a good idea. I was going to say I work with I have a colleague who is a member of the release team who will remain nameless but I'm sure they would appreciate the contact before removing Python 2 or starting to remove Python 2 and potentially making lots of packages across the archive. There's already a bug. The 931 659 is probably to the release team so they should be aware already because they will be upset if a lot breaks and and other migrations get tied up with this and so on. Release goals, should it still be the thing? Should we be declaring this as a release goal? Yeah. Anyone know the process for that? Just tell release team please. Even if they're not a thing making it very public would be a good idea to make sure that people are aware of this and not just Python people. Maybe I mail to Debian announced Action who? You? I think I will create the wiki page first so that there's a general idea what to do and what to expect and then a draft to Debian Python and then to Debian level announced. But anyway, for the team packages it's ok to remove Python to leave packages right now and expect some even uploads in maybe this week if not Andre then I will try to work on that and that would be a start but there are many more Python 2 packages outside the team so maybe just making people aware and the problem is not with removing these packages but problem is with the applications that still use Python 2 and are not ported to Python 3 and if we remove some dependencies then these applications will stop working and we don't want or maybe we want that but I think we shouldn't do that and back reports are probably already filled. Wait a second. We already have some Limpzian I'm sure I would have to talk with Chris or with the Limpzian maintainers I think they are currently just information or not yet implemented but we should talk to them to get all these Limpzian warnings about using well, Python 2 as a shebang and things like that with that we don't have that many back reports I think. I would rather prefer back reports because they have the possibility to get the feedback if someone needs that if there is no start just to make important back reports to say we want to remove Python 2 as much as possible please remove your package that we can do that and then they can feedback and then you come into that where you really but I promise you then maintainers start to remove packages which still have dependencies Yes, that happened a few days ago In any case Limpzian cannot detect if something is a leaf package or not because it doesn't know what the rest of the archive contains so we have to file bug reports or a script or tracker page or something that's not inside Limpzian because Limpzian won't be enough The nice thing about Limpzian is it's a technical way of knowing whether or not your package has dependencies on Python that you don't have to go and find out yourself it will tell you if your shebang is using an interpreter that's going away or if you've got Python 2 dependencies of some sort and it's definite you can use it while you're working on your package to make sure you've actually fixed everything you need to fix Can we make this Limpzian think an error already because I think it exists and it's a info or warning I think everything either a warning or error is fine because it shows up on the tracker pages I really don't care if it's a warning or an error maybe it's an error but it's not there are Limpzian errors which are preventing a package from migrating by FTP master so we should make it an error there's a maintainer notice but it should be possible to go to unstable maybe this is kind of compromise Also another thing I've seen a lot when moving Python 2 the binary packages for Python 2 stays in the archive so there must be something to do with the craft thing there's no way we're going to file a report for every Python 3,562 Python 2 packages that are still in the archive so how does it work? FTP masters remove them after a while there is an automated process for autodecrafting that exists I don't know how it works but it might be worth talking to someone in FTP or release teams to see If it wasn't working that's a bug with their process and it sounds unrelated to Python 2 removal I wanted to go back to the team's subject so you can have your Python 2 crap So then it becomes hard for me to know what's remaining to do because I typically would do up cache search or reverse depends or APTR depends to see what I should remove so let me give an example I have Python PBR which I would like to remove Python 3 and I look at its reverse dependencies using reverse depend and then I see a lot of Python 2 packages which I already removed Python 2 support for how do I know? Thomas I think that binary packages Python 2 only are not in unstable but they are in testing they are in unstable are you sure? Maybe there is some option to draw a graph based on UDD about the dependency that you have a tree and you can start with leaves also is it somebody who the release team will be helpful for that I think so the graph is already there Zigo created it DRAM is I see that the cruft remains because there are packages that still depend on them and Brittany will allow packages to migrate even if there are packages that will depend on the cruft yes Another issue is with documentation because a lot of Python 2 packages contain documentation and Python 3 ones do not it's not a problem if we have a separate binary package if not then we have to migrate documentation as well so that's something that once Python 2 binary packages removal maintainer has to be aware of and it seems that there are many different strategies to provide documentation and some people add some separate binary package some ship in Python 2 using the user share doc Python-foo library sometimes is Python-foo doc directory and we probably should decide one location for that so that we don't have so many options removing Python 2 so this could be done alongside the removal I think that the best location is user share doc Python dash module directory and sim link probably in Python 3 binary package but can we write a linear warning about this doc issue or can you ask for this because if we know about the problem I want to know if somebody has a better idea whatever idea we come down we should have a linear warning even at current state even if we have Python 2 inside because right now Python we should update Python policy about that and then ask for the linear warning but first we have to decide what's the right way where should we keep it I don't care so it I care to have only one place whichever it will be we should have one place and then we can put it which one is it and put it in the policy then create a linear check or whatever but currently we have too many places to look for documentation the thing is if you do not split the documentation you naturally have it in Python 3 dash foo so I would like to see a sim link at least in this place when I have split of documentation to find the documentation in Python 3 dash foo so maybe we can that's my rationale for that I think we should keep the Python foo doc directory and not rename it to Python 3 foo doc so Rebecca Palmer started a discussion about this on the mailing list on the first day I don't think there's been any replies to it yet she's sent the link to the policy I can add it to the copy for this specific issue of documentation paths so maybe we should go back to that this discussion on mailing list if I remember correctly there is Debian policy we should place it in main package so not in the doc directory right? it's a should but should or make other decision right? so if it will be without 3 it will be in the name of source package because it will be name of source package it will be the alone directory nothing else will be inside it because we will not have Python nothing dash foo after we remove Python 2 so I think the Python 3 is a better thing because the directory will not be alone but just a second there will be packages with two binaries PyPy and Python 3 so should we put it in Python 3 directory or sim link it from the other directory or put it in the doc directory and sim link it from all the others who is interested in doing that decision? Hands up so we decide you are the only one so I mean please make a proposal let's continue this discussion on mailing list it's probably and you mentioned PyPy should we start removing PyPy packages as well because there are Python 2 actually and PyPy 3 exists now exists now and reuses Python 3 modules so you don't need a separate binary package in most cases there are exceptions I don't think so we can check so we should remove PyPy binary packages at the same time looks like a couple you is there something else regarding Python? as we have 5 minutes left I'd like to I don't think so no it was 2 times 20 minutes yeah I think we have 5 minutes left only I don't know if we have enough time to start discussion about Python 3.8 because there is a new thing I don't know if we have 10 minutes left Matias told me Matias, tell us about debug packages so Python 3.8 is now unstable in the archive it's not yet supported but I wanted to have it in the archive to get a little bit of testing go ahead Matias Python 3.8 I'm not sure about things if you really need it so the release will be in September so then we can start talking about the migration from 3.7 to 3.8 currently we have two different interpreter builds for Python 3, it's a debug interpreter and the world production quality is a debug interpreter in 3.8 it's now possible to load debug extensions in the production interpreter or to run well product extensions in a debug interpreter so the question is do we want to continue with the dbg packages we could even try to extend that for example to build everything with the address sanitizer with some other sanitizers however I would like to start with that only when we have dropped support for Python 3.7 so we make the transition including the debug packages and then we can decide yes we want to drop them or to keep them because as long as we have 3.7 we cannot do that move so that's the thing with Python 3.8 what I know about what is important for the packaging and besides that we also have 3.7 I hope so the migration will be a lot easier than it was for 3.7 and with this optimistic thing we have to finish because time is not our front it's not clear if 3.8 will be the final version for the next dbg release because upstream decided that it will be available and even to get releases out every nine months so that will be a challenge for the destroyer I think so let's talk about that at next step when Python 3.9 approaches okay thank you everyone for coming we unfortunately have to leave because you can come to us and ask this is the main guy for the interpreter and I'm responsible for tools like DH, Python and stuff and feel free to ask us anything and this is the PyPy guy and thanks for coming