 Thank you. Okay. Hi, everybody. I'm Eric Dorland. I am the among other things the automate container The love is okay, all right, and I'm gonna talk Is that better Is my muted no no it's green Right, so I'm the only container among other things and today I'm going to talk about removing obsolete packages This is meant to be a buff. I only have 16 slides so I'm going to talk about my experiences in removing old versions of automake and please feel free to interrupt with questions or criticisms or Comments as we go, but hopefully a bit of discussion about how to do this better in the future or Or maybe not better. We'll see So just to set the context a little bit automake in Debian I Started maintaining it a really long time ago. It's making me feel old 2002 This was around the time that automake 1.6 came out I think So one of the problems automake has is that new releases of it are usually have Incompatibilities with preview aren't backwards compatible completely There's various reasons why this is doesn't it provides a very weird interface because You're basically embedding things inside of make files and it's hard to control the API of a make file Back in the day There was for a very long time. There was automake 1.4 and that just sort of worked and then Automake 1.5 was introduced Debian upgraded to automake 1.5 and lots of packages broke So what ended up happening is that automake 1.5 got moved into its own package and Automake 1.4 was the automake package Nowadays the older versions get moved into their own packages and The current generally the currently released version of the most up-to-date version is the automake package This has some downsides because it's not necessarily fully backwards compatible But this is usually what people want they want the latest version of automake and they don't have to install a New package anytime a new version of automake comes out all the automake packages Provide an alternative so that you can have your user bin automake be exactly which automake you want it to be the priority of that Alternative usually means that the latest version of automake is the one automatic selected, but of course you can override that and Just a note here that depending on how to make directly is like a little Risky because again, if you have fancy Automake files new versions of automake might break you potentially But anyway, this is not the automake buff. This is just a little context about why automake is what it is in Debian So there's problems with this Because these backwards of Cavaliers shoes as I said we're packaging all the versions of automake each new version is getting a new package basically and We've gone through a lot of automake packages over the years I Didn't count them up, but it looks like almost getting close to 10. I guess Which is a lot of packages to have to deal with and so people will depend on these individual packages for whatever reason and Then getting them out of the system When they become old is troublesome As of as of the wheezy release we had four different automakes in The wheezy release so we had what the old old as you can see 1.4 1.9 1.10 and 1.11 Instead at one point we were gonna have five different versions of automake just a lot of versions of automake and You can also see that the 1.4 version was last released in 2002 when I started maintaining this stuff so it's really old and No one should be using it and no one should have been using it for the last 10 years But we sort of kept it around because people were like, oh, maybe there's old software that you know still depends on a make 1.4 Yeah Not great. So this is crazy, right? This is too many versions of automake. No one really wants them So I started out on this mission to bring us down to hopefully one or two versions of automake for the next release and so I've been doing this over the last year and I was making these slides and I was thinking about This sort of had these weird parallels with some things that read about So now I present you the five stages of removing an obsolete package from Debian The first stage is denial All right So if I send mail to Debian Devel asking everyone to very nicely move off of these old versions of automake They'll just do it, right? I mean, does anyone agree that that's what will happen? Yeah, so That's what I started out by doing So I'm gonna talk a little bit of the tools and stuff that I went through this process So if you so this is sort of technical and sort of procedural you've got questions about either side just Shoot up your hand So the first thing you do or the first thing I did is That I used graffiti control To figure out which packages were billed depending on automake so automake basically has no actual binary dependencies. It's all billed dependencies So there were 169 packages source packages Okay, that's you know, that's a lot but in Debian scale is not that much should be fine, right? Then I use the DD list tool to turn that list of packages into a list of maintainers With their packages. It's a very nice tool for just generating these emails Then I sent out an email Debian Devel saying here's my plan. I want to get rid of these old versions of automake Here are the packages that are billed depending on them, you know, please do your part and fix this and I sent that out that mail out on May 27th 2013. So before last up comp, which I was not at Um Stage two anger so People aren't fixing these things. I'm gonna have to actually do something other than send an email All right, fine. So the next thing I did was To try to encourage people again to sort of Make this move Lintian has this really nice facility one of the tests is actually there's a list of obsolete packages So you if you put your package in that list Lintian will complain anytime anyone depends on you that package to try to like Forced maintainers that are paying attention that they shouldn't actually be depending on these packages. That's good And that's it's cheap to do just send a partial into the lintian I don't see any reason they wouldn't take it if it made sense and then And now I have to file bugs because you know people don't read Debbie DeVal. It's sort of it's fair, I guess So I basically just went through the list of packages I had with this simple substitution script send out mail that was like please stop depending on automake blah with a very boilerplate thing saying this is why we're getting rid of these old versions of automake and I sent that off to BTS which is really easy right because BTS is just email So you can just do that you can file a bunch of bugs that way. I Should note that my initial email was a proposal for this mass bug filing as well Which you're supposed to do as part of the procedure of sending out mass bug filings So when I did this I defile 117 bugs So there's 52 packages that got fixed Between me sending my initial email and the bugs that's I mean that's good. I shouldn't be that angry. Maybe I did wait Four months between these two events though So there's plenty of time And then the other thing I used was of course using BTS user tags to track all these bugs very easily By adding this auto make clean up 2013 tag I added the date because I knew this would not be the last auto make cleanup that I have to do So I keep these separate, you know always good to date your work if you think you might have to repeat it Yeah, no I did not finish in 2013 I Started it in 2013. I should have like been auto make clean up 2015 or something if I wanted to finish date All right, so stage three bargaining, okay So I passed the anger stage And now I'm like, okay. I've followed these bugs, but not much is happening. So let me just fix this for you I'll I'll give you the exact fix on how to like move to the newer version of auto make Here's the patch just apply it and upload your package. It's you know, I've done all the work. I've done all the hard stuff So before I even did this 34 packages are fixed without supplying a patch, all right, I began the patching in late October and of 2013 and This is the hard part this is the part that's tough to automate in many cases just switching to the newer version Automate worked and then it was easy, but that was maybe 50% of the cases I didn't I didn't keep good statistics about how how tough this was exactly, but In a lot of cases like you had to fiddle with the build system or I had to like I didn't like a lot of build a lot of build rules would like run auto make itself like instead of using DH re auto reconf and So there was a lot of actual like fiddling with packages trying to get in the build and then like just waiting to build these things Which can take a very long time To test that this worked So this was like a lot of actual tough work What else yeah, so I mean The I used of course P builder to do the builds and Then if I caught if I successfully built something with the new version auto make I would just mail off a patch to the existing bug is BTS and say here's and flip the patch tag saying here's the patch So I thought okay at this point, you know those patches out there for almost all these problems There should be a wave of uploads and everything should be great Okay, so stage four depression It's not what happened All right, I was pretty much On my own the responsible maintainers have already Fix this problem, so we're into the sort of long tail of People who don't care or people who are too busy to fix to deal with their packages or do anything here, so Now I have to upload and an amuse so I started this in late January, I think and Between January April I had to upload 63 and amuse So it's again about maybe 50% of the the remainder got fixed in the last stage But 6300 muses a lot So again, I uploaded these to delayed 10 so only seven of these were actually Like another upload came in in front of them to block them out or anything like that It was relatively easy this part was sort of fairly mechanical I just applied my patch that I'd already created in the BTS and then Using the deb change tool, which is really nice you just have it create the the change log entry for you and Then again, you use to builder to build the package It's really nice if you use this might be obvious to but if you something like a new PG agent You can save yourself a lot of typing of your password while you're signing packages for upload And then you upload it to delayed 10 And you use the NF mu diff tool to update the bug the NMU diff tool is really awesome You basically just give it the the existing package and your upload and it will do all the magic of Creating the figuring out what the patch is setting into the BTS and with the new I see mail saying I've uploaded this So that's for doing an amuse. You should be using this if you're not you're doing it wrong. I think So there's a bunch of these uploads and And mostly people were happy with them But I did get one response that was like, why are you doing this? So this is a little redaction of the so this was the the quoting of the email that got sent to me It's like I've prepared an MU for a package and I've uploaded to delay 10 And so I got this and immune for a wish list bug. Oh question mark question mark question mark There's a lot of question marks. We had must have had a lot of questions And then this is again part of the boilerplate that an immune diff sends out Peaceful feel free to tell me if I should delay any longer. Yes, please indefinitely Or at least until it's important Slash RC So I went out who there was only one person who sent me a note like this many people sent me thanks and appreciation but And this didn't really impede me because they in fact uploaded a package that fixed this About three or four days later But the reaction was kind of weird So I don't know does anyone have any opinion I guess I should ask the question Does anyone have an opinion if this is wrong to NMU a wish list bug? It seems really situational if you filed a wish list bug and there's been no activity for a while Or if in general there's no activity on the package for a while Then I don't see any reason why you couldn't upload a delayed one It's given that it can always be removed within 10 days Uploading immediately for a wish list bug seems almost always a bad idea But a delayed upload for a package that doesn't seem to be getting uploads every other day from its maintainer I don't think it's reasonable to gripe about that That'd be great There's rest there. I have a little bit. I'm sorry. I have an opinion that it's not a wish list bug. I Mean I think that you know if you're trying to get rid of old packages Then I think that it starts out as wish list fine, but when you're down to the last few maintainers who have not uploaded their package I think at that point you're climbing towards normal or important And at some point the package is going away at which point it's RC and then NMU's are of course perfectly fine Sure, there's a bit of a catch-22 here in that That I guess I'll talk about in the next slide, but it's hard to get FTP master to remove the package before all of the Dependents have been changed. So you it's actually tough to get it upgraded to RC. I mean it does possible. That's a possible solution But yeah The way to get it upgraded to RC is to talk to the release team who was to have the jurisdiction over What is RC or not? And in this situation probably we would at this point say well You're going to remove the old package that will stop things being buildable This is now RC and then you have some levers to do it that only applies to this situation and it's not a I'm not saying that's what we would have done, but right right the but but talk to us if you're getting into the sure I suppose I could have tried to petition to have this be a release goal that there were less automakes in No For completely different reasons, that's really complicated and recommend it Now I'm glad that you anonymize this but I could probably guess in three exactly who that was It's possible it's possible, please please do not speculate on who this was There's someone back there I mean What is it? You shouldn't upload for a wish list bug straight away And you shouldn't even upload to delay 10 for a wish list bug straight away I thought that delayed was specifically introduced such that you can upload packages straight away And they have a time deadline on them set such that you have 10 days to respond or to remove the package from the delay Yourself if you're if you're a maintainer and you're not happy you should know how to use decodt remove right sure Yeah, I and and and now we're saying that well Actually, you should file a bug and say that you're intending to upload it to delay 10 and then sometime later You should come back and do that actually I think should we create delay 30 right? I reading at the handbook about an amuse also seems to jive with this where it's like if this is a if this is an actual bug The severity doesn't matter you can do it you can basically upload an amute to delayed 10 at any time is What I read is that it's my interpretation of that of what it says there now Maybe other people disagree about that interpretation But I totally agree with that and yeah and I'll point out that this is after you had filed the bug Yeah, this is like three months later sent the patch. Yeah. Yeah. Yeah. Yeah, so I mean If if the patch has been available and hasn't been applied and new versions haven't been uploaded then uploading it to delayed 10 There is absolutely no reason to avoid doing that. There's sure absolutely. I mean you were you were gracious your delay of Sending it to delayed and then you were gracious into making it delayed 10 by that point so right I mean so I did I did perhaps I guess one of the questions too that I that I will come up at the last slide too, but is should Should I have waited should I have done the pet like I basically did this in two stages? I did the patch and then I waited a bunch of time and then I did delay 10 up and a mu uploads for people who didn't respond Should I've just combined those and just immediately uploaded amuse delay 10 is that is there a consensus around that? If delayed 10 isn't enough delay We could make the number bigger, but in principle delayed exists so that you can do this in one pass Upload the file the bug include the patch upload the package if somebody doesn't want it. They know how to delete things from delayed Yeah, I mean the one caveat there being is it trivially easy for people for example Who have their packages sponsored to kill something out of delayed without going and finding a sponsors answer within 10 days? But that aside you could probably find someone on Debbie and DeVal Right if you're a sponsor. I think you're responsible for the NMU is you uploading? I mean if you uploaded something to the delayed queue. Oh, I see what you're saying I mean, I would have I would have if there was a delayed 30 like I could have uploaded it there And that would have been I would have been fine with that and it would have gone faster Even though it was delayed 30 you probably I mean the person who made the upload to delayed if Somebody asks you to remove it you have the powers to remove it, right? Yeah, such that if the maintainer is non uploading to D and then He requests you to remove it or she then you just remove it because you uploaded it So I don't think there is a deadlock in there Yeah, and I mean if people were responsive on the bug I didn't immediately and amuse things like if people were like, oh, yeah, I'm gonna get to that and if you like I The grub maintainer Had a had a bug in this and he was like, oh, I've got that fixed upstream I'm gonna upload it in the next few weeks and I was like cool I'll ignore it and then I ignored it for several months And he still hadn't uploaded it and I didn't have an immune grub and then he was like whoops And then he just went over top of that because he had completely forgotten to do this So I mean there's the weird situations like that where everyone has good intentions and still there can be introduced delays But yeah, sorry I go ahead. I am gain if on IRC wanted to say that the FTP team does in fact remove packages when You're down to a few just a few packages that depend on it They wouldn't want to remove them if there's 150 of them Sure, but wanted to did didn't want to mention that that is an option and there where they are willing to do that That's good to hear it didn't quite play out that way, but it's good to hear All right, so anyone else before I go into the neck the last the final stage Okay, so the fifth stage of course is acceptance in the end To get these packages removed I I had to sort of appeal to FTP master like I did all this work They still had to remove that they still just do that final step and it was kind of out of my hands so Eventually, I don't make 1.4 and one and one ten are gone Automate 1.9 is still in the archive There's only three packages that I think there's one package that still does build depend on Automate 1.4, but it does not build from source either But there are three That depend on 1.9 that that that that failed to build from source, so I can't really fix them Yeah, yeah, I actually I Ran to Paul tag on the flight here, so he said he would look into it, so hopefully But yeah, so repeated pings on the bug hasn't really produced a removal Even though the bugs been filed for several months now But again, I mean FTP master's busy. This isn't like the biggest deal I'm hoping to get this out of here before the end of the web before the freeze, but Yeah, so These are the my last slides the open questions, which I think we kind of answered We may be answered the first one what what tools did I miss what what could I have done to automate this process better? I mean in terms of I mean some of the slowness was introduced purely from me Just kind of working on tooling on this a few packages of time in the evening But what sort of tools would I did I miss that could have made this easier? Mike Mike writer as far as tools are concerned I mean the tool that Jonathan's using here if we can still use in a dry run mode just to ask Thank If I remove this package what else has affected because that was right the way up the chain the the bill depends It's hard to get the recursiveness all the way up so that you've got the There's the full chain up to a leaf package at the top end So if you could actually run it or it's on what's the services? Whichever the developer copy of the archive is on I can't remember. Okay, I so that's that's a good point I think in this case it wouldn't actually matter because All of the things that were left were crappy leaf packages that no one cares about So I like removing them and in fact they were already moved They're all already moved from testing as far as I know it won't be in the next release So sort of mute moot, but But yeah, that's a good point that if you're if you try to figure out what the impact is over moving it Using that would be a good idea, but I guess I'm kind of looking more for How do we automatically this took me a year took me over a year of like wall time even though It certainly didn't take me a year of engineering time But it took me a long long time to get through all of this sort of procedure What is there any tools that would make this easier like it would have been really like it would have been really interesting If there was a way to just say Here's a list of packages flip out the build dependency Go build them somewhere and if they succeed then I want to upload those as an amuse right like that would have made That would have cut in that would have cut down a lot of the manual labor part of this Even though it would have made the law there's still the long tail things that actually required patches But probably like 50% of the things that were left were just me going in Replacing automaker whatever with automaker the newest and building it and then waiting around as my slow desktop machine built this thing that works well with Build dependencies like automaker things like that, but there's still other cases where you want to get rid of obsolete packages Yeah, I'm in the early stages when you're actually doing the identification of the packages. Yeah In some cases isn't it helpful just to pick out the ones that have no package in testing and just say well These these are going to become they're obviously not been looked after they've probably got all CBOB's already and get them out early Yeah, maybe I'm do we consider that a package that has Been waiting to go into testing for more than six months is by definition obsolete and Get and get them like removed from the archive more or I mean, what's the or marked somehow is being bad It's not I don't think you can blanket that you'd have to take it on a case-by-case But it's certainly a tell-tale clue that you should be looking at other factors on the package that might also Give you a big picture of this package is to go away Yeah, so it gives you the opportunity at the stage where you're preparing the DD list and that sort of thing yeah to To pick out those packages and be more aggressive with them early on And escalate right up for for those packages early and then you can get the whole cycle done much more quickly rather than waiting If you actually go through which packages were involved here Yeah, I would if it's a fair bet that the packages that were left at the tail were orphaned Not in testing. Yeah RC buggy or competition more three. Yeah Hi, so I'm I'm surprised actually you haven't pronounced the term transition because this is actually what it is and the release team has Nice tools to handle transitions. Okay, like well like you can have a dashboard of every affected package Which will happen that in like the BTS or the new tracker? Package tracker you will have every there will be a line that says hey this package is part of a transition You need to do something about it. Hmm. And also, I believe that now we had that we have auto removal from testing then well, yeah, we remove the old packages from testing which means that the Packages will get removed from testing so until they are fixed. Yeah, the right thing to do It should not want your packages to be released with the new deep end version So, I mean definitely a bunch of these tail packages were removed from testing They were all like the the three that I mentioned are all filled bill from source. They're all removed from testing, but It's not clear to me how to connect those two things up with the actual like Forcing the removal to happen, right? Like how do I make it? In how do I indicate that we shouldn't care about these packages and that it's safe now to remove automake? version X sorry No, I did I absolutely did that I found it ages ago, and I've pinged it several times with exactly these facts But there hasn't been a lot of motion on on the none of them got rectified particularly quickly and obviously there's one that's still sort of dangling Go ahead rest the statement in the audience was just you should add that you should add that information to the RM bug Couple of things one Ganoff says that he just killed automake 1.9. So you should upgrade the remaining bugs to The other one is is that The one of the things I think we could do better at is that you the the whole process of Lintian detecting that you're depending on a package that goes away Right now that's a the I think the obsolete file And I've not been involved in lindian development very much for the last couple of years So it's possible that somebody went around and automated this already, but I think that was a basically a manual File that you just add things to It seems like this is a place where we could be using Sections or something like if you uploaded the version of automaker that you want to make go away Into a different section or use the oil or to be more precise use FTP overrides to move it into a different section Lintian has a bunch of files that it auto generates Well semi auto generates in the sense that people run scripts to fill out that data before you upload lintian each time So that they could just pull down all those file all that the list of things in whatever that section is That's designated for these packages will be going away and depending on them as a bug Which may not be the same as the section we have right now for transitional packages because those are not exactly the same problem and Then that would automate some more of that process because then all you would have to do is tag the package somehow or Submit an FTP master override bug somehow to say this package is going away And then more of that process for the responsive maintainers would happen automatically without you having to babysit it Yeah, and on a similar note. There's really no reason why we couldn't automate Nagging the maintainers who depend on obsolete packages or even mass filing the bugs for maintainers who depend on automated packages I'll do that completely automatically without a human having to do anything except maybe press the button that says yeah This looks like a good idea Yeah, it's an interesting idea to do these to try to do these more automatically the bug filing Was there a question over here? Sorry, I didn't hear what you said though So if I got it right the concept that the idea is to introduce the concept of a deprecated package Which I would appreciate and the deprecated like the concept of a deprecated package. Yeah. Yeah, no I think it's a great idea like we have the old libs section Yeah, I have the deprecated section. Yeah, and then you upload into the deprecated section Which actually requires manual override by FTP master, but yeah And then lindian can automatically say you depend on a deprecated package. Yeah And you can even say block uploads on that lindian tag. That's true. That would have been a good That would be a good thing to do too. I mean that's implemented. Yeah It's a great idea. I mean and that was one of the reasons that I liked putting in the lindian warning because it would prevent new people from Uploading packages that were depending on these versions of automake, which you really don't want new problems to show up as you're Spending months and months fixing the old problems. Go ahead. Yeah, I think it's the lindian stuff and all the rest that tends to just work with maintainers who are More or less on the ball. Yeah, and I was wondering if you couldn't just take a big hammer based on lunar's idea as well And if you because if you filed an RC bug against automake 1.9 itself then it would after a week or so disappear from testing and All the oldest build dependencies would also disappear from testing and that would Yeah, the release team might not appreciate it But I guess I mean maybe the question is like what's the threshold for the Like how big does the long tail have to be before it's we can just like force it, right? Like if you know You know is if 10 if this breaks 10 packages and none of them appear to be important Can we just do it or do we have to wait? So I think one good example that this provides not just for automake But for a lot of other cases is that we should really really hesitate before we start introducing Parallel versions of anything into the archive the default state of anything should be here is the Version for the archive at which point you start getting fails to build from source bugs because hey look You don't work with the new version and the default state is not I get to use the old version and comfortably rest on it It's I got to fix my stuff Yeah, I mean that's that's another that's a little bit a little bit off topic I mean it's very relevant to automake because I think I'm going to end up with this problem the next time There's a new revision of automake. So the one of the things that I didn't mention is the automake maintainers have promised To have better versioning in the future so that like actually only major versions should break backwards compatibility Maybe hopefully So that's I mean that's promising, but Yeah, how do we how do we make sure that the new version doesn't break huge swaths of the archive before it gets uploaded? I mean, I know there's there's some people have access to large sort of build Like rebuild instances and stuff like that Which would be really useful to hook up with them But I guess you have to ask them or figure out who those people are those sorts of things as far as the long tail is concerned It does depend on the circumstances, but we have actually done some seriously long tails in one operation At a BSP we actually removed 94 packages in a single chain Wow It's so these things are possible They come you don't have to think about the tail in in in those sort of context You just look at the contents of the tail and it was all of these kinds of packages Yeah, all I see buggy none of them were in testing most of them been waiting for testing for on average average 600 days or more All right They might they weren't orphaned but on that basis they probably should have been right But there wasn't any point offering them the whole lot just got removed, but is that in a but that's a transition, right? No, no They were they had been broken So maybe my question is so was that on the release team side or the FTP master side, which is also like where do I So we happen to have both teams on at the BSP at the time. I see Yeah, we do kind of when we do UK BSPs We often tend to have FTP master release team and Neil who likes to all them things by default Okay, so trying to getting these two teams in the same room or physical or virtual room might expedite these. Yeah, absolutely, right Okay, that's good advice do an obsolete sprint So at some point before we get to the too close to the release So it's probably a bit late for it now But you know six months or so before the start of the freeze We organize a sprint and we get the relevant teams together and we go through this lot with a fine tooth comb We just throw things out. Yeah No, that's a good idea. I mean, yeah, just in general to if it's if it's six months or more before the start of the freeze They got plenty of time to put it back in if they really care about it. Yeah. Yeah, no, that's a good idea I mean, it would probably be pretty easy to have to to figure out which packages are Well, I guess there's a question of what's the difference between obsolete and un-maintained and Like how do we identify what those packages are without it being explicit? It's really great. There are all sorts of considerations you have to make and we Attendees of Cambridge BSP's tend to be a little flippant about this kind of thing And slightly trigger happy will admit but but it's We We don't Automatically go for RM, but we have a we have There's a set of telltale signs that we've picked up our Indications that things are not as they should be with a package like the popcorn is And and it's maybe it's got a couple of important bugs and an RC one Maybe somebody is reporting that they are bill-depping on it and something's going wrong And there haven't been an upload in six months and all those things together combined say to us This is probably we're better off without it than with it and but So so you have you have to analyze the packages and as an entire entity not just say are this has two RC bugs Let's get rid of it And then you also have to have to make sure that you're behaving nicely to the maintainer of the packaging question If they are still around and and not Where are I mean your package by put it back in said if you like But would something like so I mean we've got our removals from testing But I guess there is no auto removal from Sid there's no auto removal from Sid And if it gets auto removed from testing it may get manually removed from Sid later on when someone says One of my triggering factors is that this isn't in testing and hasn't been for two releases Yeah, but it won't happen automatically It seems like a long period It's an example, right I mean it would so maybe I'm just I'm just spittballing here, but Would it make sense for if something is not propagated? Literally, nothing has gone into if it's been removed from testing for a year and no uploads have happened in that time Which should we just remove it from Sid automatically should we have some removal criteria from said that's Yeah, I I'm shying away from saying yes, we should do it automatically I think there still should be some oversight because there are corner cases and that's what I mean about there being a gray Areas all over the place where you have to look at the situation right now and make a decision and think that's difficult to do Automatically the PTS does have a box at the top that gets bigger and bigger the more problems there are So if you look at if you look at the package page and the box on the PTS is the first page Then clearly you've got a problem sure The that might be a good metric how many boxes are I'm sure Neil or cook up a script that can scrape the PTS at work out who's got the biggest boxes But my second point is that you We although we're flippant about it and we are fairly trick-a-happy We always make sure that we behave properly and the maintainer gets a chance to explain why there's a problem FTP masters can look at what's going on and make sure that he's okay And we don't there are very few occasions where where it's happened But there are very few occasions where we have filed an RM bug and FTP masters gone Yeah, okay, and taking it out in the hour. Yeah, but it has happened in the right circumstances, and that's what I really want to Rather than auto removal I think it does need to be an automated process for the packages in said it just needs to be highlighting and More than just the maintainer But actually go into a list of people that's thinking this is like the double NPP report that comes up, right? You know if it is a it's a package in need of help. Yeah, probably it actually needs to be tagged that way You know we need the automated process that scans through these things. We know what the what the risk criteria are and we can be fairly General by applying them as an alert on either the bug or the package Yeah, and then look there are reasonable grounds for doubt about why this package is in this particular state Yeah, and if that tag then stays on forever and ever and nobody actually Manages to clear it. It's just like having it in a deprecated state Inside the archival is it's a flag that we need to be able to put on packages like this And then have a tool the scripts the entire archive and says right there's 600 of these packages What are we gonna do about them? Right? Yeah, that's interesting Would it not make sense to Would it not make sense to have some kind of report say listing packages by days that haven't been in testing and Number of RC. I mean come up with some rough criteria put them in order put a little Textbox so people can put notes about the package on it And that way we could actually deal with these things rather than waiting for someone to run into them. Yeah, I Was a bit afraid of that That's a good idea though. I think kind of list like that would be very helpful Okay, we've got just a couple minutes left. Is any anything else you want to talk about I think for some for packages that we expect to be removed It's also be nice if we had some way of communicating that to users of the packages I mean there are even cases where there's not necessarily a 100 RC bugs, but just a package is dead upstream and The maintainers decided that sooner or later. They're going to kill it off at the moment. We have no Very nice way for the maintainers for use of that package to know that or do anything Rico Can we have a deb tag before either deprecated or obsolete or dead upstream and then because anyone can add that The deb tags it'll eventually get listed in the apt cache and people can see it on their own systems If they're interested in the package It's also very easy to remove We can if somebody maintained it I cannot that information in Along the way the tags take to get into FTP master But it's not something you want to be edited from the depth XW net interface where everyone can do it anonymously Which means somebody needs to kind of maintain it manually Which means it hasn't happened yet It's been experimented with the security team in the past to say something like We don't guarantee security support for this thing but yeah, I Would see sections for cases like this Because what you want to use this mostly for packages and other things depend on rather than leaf packages Because leaf packages if it's dead upstream Yeah So There's both the rage involved in handling this And so you I would focus on both the rage to happen Where it actually makes a difference in in some workflow It would be nice to tell our users about it but it would be nice if it were the maintainer that told the user about it and at that point the maintainer can just remove the thing from the archive or Yeah But I would still like some annotation on the packages handled by the maintainer actually it could be tags in the control file Which is not implemented yet I'm waiting for that conf where there is an FTP master to implement it because I have a prototype It hasn't happened in the last two years So we got what type one more so I think I think you had it or Yeah Yeah, I was gonna say for users though it would be nice if the maintainer or others in an NMU had some way to Communicate you should really be using this other package instead for those leaf packages That we don't just remove it and leave the user not with the old thing installed and not knowing what's happened Yeah, I Think we're I think we're out of time. Thanks everybody. I really appreciate coming to this talk and talking through some of these issues See I'll be at the conf all week if you want to talk more about this stuff