 So this is Lucas, and well. OK, thank you. So when you ask people, is that too loud? OK, when you ask people what quality assurance is, you get answer like this. So that's not quite true. So what's really quality assurance? Sorry? Yeah, but it's funny how to blame him. So the goal of quality assurance is to improve the quality of Debian as a whole, as opposed to just working on your package. So quality assurance team is not really a team. It's more like a central place to discuss stuff. And we hang out on Debian QA and the main list of Debian QA at least, the Debian.org. So what do we do? First thing is we maintain the PTS. So the package tracking system. Everybody knows the PTS, I'm sure. DDPO, which was mentioned by Tincho. So that's the Debian Developer Package of the View. And also several other tools. So what's really important is that we aim at building tools for everybody, and not just for one team. So I find it a bit sad that Tincho built this nice tool just for the Perl team without talking to the QA team in general, because probably we could have helped him and made his life easier. So what's really important is write your tools for Debian. It's cool that you write tools that help everybody, but don't write them just for your team. We also develop tools to run archive-wide checks. And then we do the checks, and we do the mass bug filings, when it's for real issues only, most of the time. The advantage of talking to the QA people about that is that usually those people are more experienced about running such tests. So you don't start filing bugs about things that are not real issues. So that attacks the filter. One other part of what we do is that we take care of our fund and neglected packages. We are supposed to maintain our fund packages. Everybody knows what an orphan package is. And we also have a sub-team that tracks inactive maintainers. So that's the missing MIA, so missing inaction team. So when I submitted the talk, it was supposed to be above. So I'm just going to present a few new things that you might not know yet. And then the goal is really to chat about QA. So the first thing I wanted to talk about is BAPAs. The problem is that there are many package that are neglected in Debian. And it's difficult to determine which are those package amongst 10,000 packages that we have. So the idea is simply to import different informations about packages in the same script and then build a table that uses a scoring system, like a spam scoring system, to detect which package is really suspicious and should be looked into. And another part of the tool is that it allows you to say, at this time, I took this action. And for example, if you file the bug saying, this package looks neglected, should it be orphaned? Then the tool reminds you, 30 days after that, that no, you should orphan the package. So that just allows to keep track of what you are doing and to process many packages at the same time. If you have questions about that, don't hesitate to ask. So examples of searches that we have are package which are very buggy, so with many bugs or neglected and useless or found. And we use different informations such as the number, the cool stuff that is not done elsewhere, I think is the number of NMUs that the package which was already NMUs four or five times, is very likely to be totally neglected by its maintainer. Something else that's new and not totally working yet, but we are really close to getting it in production state. It's the ultimate Dibyan database or UDD. So it's a Google Summer of Code project. The student is Christian von Essen. It's a German guy. I commentate it with Stefano Zacchi-Holli or something like that, Zac. And Mark Borkschmidt. Was that right? Yeah, Zac is fine. And the goal is actually similar to what I just presented about BAPAS. The goal is to gather a lot of existing data on Dibyan in the same SQL database. So then you can run cool SQL queries combining lots of different data sources which is not at all possible at the moment. So for example, you can get a list of really critical bugs sorted by Popcon or the list of common or maintainers including co-maintenors. Rather problem? No? At least maintainers with the most bugs. So the current status is that we have a lot of data so it's already imported on a regular basis. So either using current tabs that runs frequently or the updates of the packages and the sources file triggered by SSH. So it's really fresh data. And we imported the packages and sources file for Dibyan and Ubuntu, the popularity contest for Dibyan and Ubuntu. Or why Ubuntu? Simply because you want to be able to increase such as packages which are popular in Ubuntu but are not packaged in Dibyan. We also imported testing migrations. So we know when a package last migrated how long I've been trying to migrate. The history of uploads. So we know when packages get where uploaded by womb, the bugs, everything from the BTS. I mean only the summary for each bug, not the bug log. Of the list of orphan packages, Lin Tian and Carnivore. So that's everything listed here works but it's really easy to import other stuff. Philip Kand said this morning that he will work on importing when I build. So currently this runs on UDD.Dibyan. Carnivore is a tool that is used by MIA I think. To, yeah, it allows to link different email addresses or identities to the same person. So you know that this email, that email are actually the same person. So you can access it from master.Dibyan.org using this command line. This works. And it still has to be moved to UDD.Dibyan.org machine but we work on that when everything is stable enough that it's really completely usable. You can already run queries like I will show you. And there's a wiki page with some links also. For example, one interesting query is, I blogged about this, maybe you read it. It's a popular package which are not interesting because they were removed, for example, because they were assay buggy or because they haven't migrated yet. So for this we can combine the sources table and the popcorn by source package table because popcorn usually use, not normally use binary package but we have a table with popcorn for source package using the max number of installation over all the binary package of each source package. Which I just lost my screen. So those are the results. So you can, and this kind of things it's really hard to do currently without UDD. And the query, it's the most popular orphan packages. So we have a table called orphan packages that has the list of orphan packages with a type field, which is whether the package is currently orphaned or ITAID. So someone intends to adopt it. And so again, you can link it with the popcorn source tables to get a list of popular orphan packages. So are the results. The most popular ones were ITAID but that's still, I think Imlib is going to be removed. Oh yeah. So Mark Bushmitte is not at Deppconf. Mark Bushmitte feels bored, so he wrote SQL queries. I tried to format it cleanly on the slide, but I gave up. So this gives the maintainers with the most LinkedIn errors and the winner is, so it's not the number of unique LinkedIn errors. So maybe there are just lots of duplicates, warnings or errors. So just examples, you can run lots of different stuff and if you have ideas, just talk to us. So what you can do about quality assurance first, talk to us and join Debian QA and the mailing list. Look at the list of orphan packages and adopt them if you care about them. There's a script in Dev Scripts called WNPP alat, that lists the orphan packages that you have installed on your system. So it's a good way to know if you are using something that is orphaned. You can work on improving our tools. So there's lots of bugs filed against QA.debian.org on the BTS, so just have a look at the list. File new bugs so that can fix them. And think about mass bug filings to improve the quality of Debian, but only about real issues, of course, because we don't need more useless bugs. And talk to us about them so we can help you because we are used to that. But so this is supposed to be above. So I really would like a discussion. So there's a list of topics that I would like to have your opinion about. So first, the orphan packages. We currently have more than 700 orphan packages in Debian that doesn't look really right. So what should we do about them seriously? Because I don't think that we can just continue to ignore them. Well, we could just do as if they didn't exist. Well, we could aggressively remove them. We could continue to ignore them. Or we could maybe split Debian into a main section which is supported in the universe and supported section. I don't know where I got this idea from, but... And something else that we could discuss that you might have questions about is status of MIA. So the mission and action teams that tracks inactive maintainers. And something else is making the QA team more attractive. It's a really cool team with a great atmosphere. Everybody that is in it, I think, really likes to talk with QA people. Maybe we could integrate this into NM, like Andres. So John, yes. Who's actually for you a member of the QA team? Is this a reader of the mailing list or a subscriber of the mailing list or who is... There's no, well, there's no really formal membership. You can, if you joined again QA or the mailing list, then you are a member of the QA team. So that's really, yeah, there's no... But there's a UNIX group named QA that is only used to control access to the QA website and the PTS, no, the PTS is the same group. Okay. But that's the UNIX group is just a technical thing. There's no, yeah. The usual answer to that question is that everyone is part of the QA team. Yeah. I think we all should work on QA stuff and track several things, not just in our own packages. But if we stumble upon problems in other packages, we really should get rid of that impression that there's a single maintainer for a package and do QA work in a more broader approach. Yeah. The other side of that story is, of course, that many of the tasks of the QA team are very repetitive and you have to do them on many packages, but you have to gather some experience on how to do them, but you can only do that by actually doing them. So everyone that has the courage to go to other maintainers and tell them maybe you should off on your package, it looks very bad in shape and stuff like that, is a member of the QA team, but you will notice that you have to have the courage to make errors at the beginning, errors of judgment, and you will be accused and yelled at, but you will get better at spotting which packages are probably so, yeah. So the thing is, that's the reason that everybody and nobody is a member of the QA team, because the task in itself is not, most of the tasks are not very hard. You just have to do them, but they are also very difficult on the other hand in that a lot of factors can affect the package and sometimes it's better to just leave it alone because the maintainer is working on it, even if it looks bad in the archive. I'm not sure entirely what point I'm trying to make, but I hope it was still useful. I agree that many tasks of the QA team are socially difficult because you deal with people who are already too busy and responsive, it's easy to get frustrated about that, so you have to do other things besides that not to get totally burnt out. But there are also things that are purely technical and doing them inside QA team helps you be more efficient and makes the task less predictive. For example, when I started filing my FTBF, my FTBFS bugs, I did it mostly manually and now I have scripts that make it really easy to file 60 AC bugs in one hour or half an hour. You can use it, it's written to be useful to everybody, so if you want to do mass bug filing, use those tools so that, for example, you don't have to manually copy the bug numbers to a separate list because it's just insane to do. You can just import all of them using user tags to a file and fill in the files. So, yeah, these are all small tools but really useful when doing that. It's not only talking to maintainers about when their packages might need more attention, it's also working on the infrastructure. Lucas has shown some really spectacular things that he has done in the past and has a lot of more things to do. So joining the QA team does not necessarily involve talking to other maintainers if you don't like that. It's more like you do need to talk, of course, with your fellow QA members or with whoever is working on several tools to enable maintainers for themselves to work on the packages better. If there's more tools so that maintainers can see what the state of the packages and what would be the best investments of working on their packages, that is also QA. And the point that Frank was going to make was, of course, that alcohol hurt your brain. Yeah, part of the point that Frank made was that it's really sort of repetitive work and it's getting annoying and boring at times. But I would like to really encourage new maintainers or people who try to get involved into Debian to join the QA task force because usually it's not hard work but it will get you very much visibility and a good track in the history for your new maintainer application or things like that. Yeah, I wanted to come back to the first point on the slide, the orphan packages. I really think we need to get more aggressive in removing them. I mean, we have got more aggressive over the last few years but it's still a very slow and tedious process. But I mean, there's, of course, the human side of the problem that will not go away, that people actually care about these packages. They just don't have the time or don't feel that they have the time to actually maintain them. So if you remove an orphan package, it can be that you annoy many users, some of them might even be Debian developers. But it is just an unmanageable task at the moment. So it can't continue like that. And I think the sooner people realize that, the sooner they will stop complaining when we remove packages. And I mean, there are tools out there so that if you really care about the package, you can reselect it. I mean, you have the versions in the stable release it if it was ever released. And you have things like Snapshot, DebianNet, and hopefully at some point in the near future that will become an official Debian service that would make it easier to justify removing packages because they would still be available. At the moment we only have this Morg on FTP Master, but it's a bit difficult to get there and stuff like that. So I think having Snapshot, DebianNet being an official service would make life easier for tasks that involve removing packages because we can just say if you want to reselect it, it's all there, just download it and upload it as maintainer. But the current situation just doesn't work. What sort of threshold are we going to put on for that sort of thing? Are we just going to say that the orphan packages with the most bugs, the lowest popcorn score, just get dropped at every release? Well, the problem is that orphan packages sometimes are reversed dependencies and you can't just remove them without breaking maintained packages elsewhere. So it's really a tricky problem, but starting by not releasing all orphan packages without reverse dependencies, which have been orphaned for a long time and have low popcorn? Historically it's always been one of the maintainers of the reverse dependencies who's had to take on the orphan package. I know Thomas had to do that with NuCache. NuCache took on a long list of reverse dependencies just to maintain NuCache itself. It's a big burden for those who have to use it, but it creates an incentive to see whether there's better code that they can convince upstream to use. Yeah, someone has to do that. Someone has to maintain them. This may sound a bit a weird question, but what's the problem with having orphan packages? I mean, surely it's not a good thing to have, it's better to have a maintained package than an orphan package, but how is it better not to have it at all than having an orphaned? Well, that's the point. We could just continue to ignore them and consider that it's fine. But the problem is that we are supposed to support them. Sometimes they are buggy, but nobody knows until a user installs them and that's still a problem. I think if we are supporting them and saying, why are we releasing them? No, I'm not saying we should include them in stable releases, but just have them in stable, why not? We're not supporting them in any official way, but they're just there until someone comes up and maintains them and puts them to testing. Currently they are released. They should be removed from testing, but why should we remove them from the archive? If you remove a package from stable, you just basically remove it for the users, which means you remove it. It's not available for the user. There has been raised a valid point on ISE discussion channel about that popcorn stats shouldn't be the only criteria for these packages, especially for some corner cases like accessibility or other special areas with special impact. You won't get ever any great popcorn stats for that, but that doesn't mean that those packages aren't important. Especially with respect to accessibility, they are very important to some specific users. It's hard to get people... It's very few people really interested in those because they probably needed themselves, and it's hard to get people involved into working on those, unfortunately. So just popcorn stats shouldn't be the only criteria to keep packages out. Yeah, sure, but it's really politically incorrect to say that you can't value accessibility more than science, for example, and if people care about science and people don't care about accessibility, that's life, and maybe that's another social issue to address, and I'm not sure that the right way to address it is just to make as if those packages were maintained when they are not. Well, when talking about removing packages, I can see a problem here with this situation because also they are still available at Snapshots. People usually don't know that they were removed because of some reason, like they were found and maintained, and nothing very great with them, so they can actually get them back and restore them. So I think we need to create a list of those packages that are removed so that our users can actually see that the packages are still somewhere and they don't need to package them right from scratch. So it's basically that. Actually now, if you are subscribed to the PTS, you receive a mail when the removal of a package is requested, so if a user is subscribed to packages that he cares about, he knows about it. I think he meant that if you want to package a software, that you have an easy way to look if it was ever packaged before so that you can actually reselect it. At the moment, you have to go to... You can first check if it was ever in a stable release and you have to check whether a snapshot maybe has it from some testing or unstable version, but it's not very retrieval. Actually, I think that the PTS lists packages that were ever in Debian. No, only that are currently somewhere. Well, but sometimes there's some missing information and also another problem is that the PTS only knows about source packages. So it is not something pretty easy. I'm sorry for repeating my point, but we have unstable and the bug tracking system for that. Just leave them there, we have the bug tracking system and if someone wants to work on the package and make it to fit for a stable release, then he can look at the bug tracking system and fix the RC bugs and take care of the package. Yeah, but then you run into problems like people not caring about releasing Debian and that's another issue because lots of people are just happy with using unstable because they will never work on getting them into testing or into a stable release. In that case, what does it hurt to have it there? Well, it's usually stable, but there is a package that's orphaned. If it's removed, you wouldn't have it at all. So what's the difference? Actually, this is a bit incoherent with the release practices which says that if a package is not meant to be released, it should not be in unstable. So this is what the release manager is saying and keeping packages in unstable which are not meant to be released is a problem for release purposes because it can get in between transitions and stuff like that. What is the problem? It adds noise to the list of RC bugs so you look at the list of RC bugs and RC bugs like this package isn't stable but it's not meant to be in a stable release so you have to scheme all those bugs. Does that mean that every orphaned package is per default RC buggy? We could introduce that policy that every orphaned bug is per default RC and it looks like the only way to get people to maintain orphan packages is to threaten to remove the package. So that could be a way to fix the problem. Just to even underline that, even just keeping the packages in unstable still has a problem because you're sapping Huey people's time dealing with packages that nobody ever is going to have any interest in maintaining or at least for the foreseeable future. It may be useful to when we remove archive those packages somewhere like we currently do with snapshot.devion.net and maybe perhaps make something official for those types of packages so that if somebody later decides they want to come back and do it and the archive is just wasting people's time and again when you remove it makes people sit up and notice oh, I actually use this package maybe I should be maintaining it perhaps even announcing to users that hey, these packages are going to be removed because nobody's maintaining them they've been orphaned for N days, whatever if you want to keep them around instead of packaging yet another DNS or IP over DNS thing that you should consider taking over an orphan package. Yeah, that's... I mean, as was said before the current problem is that the whole infrastructure is targeted at so that every package should eventually go to testing. There are a few exceptions which we accept but the whole infrastructure assumes that so that's not an unsolvable problem if we want to change... if we want to change that but it is a current problem so the question would be do we want to change that and I don't think we want because even if we change the infrastructure so that those packages don't... clutter... don't... yeah need so many resources like they do at the moment in terms of that they show up in all these lists of packages that need work that Brittany tries to handle them and stuff like that I don't think even if we change all that that we want to have them in the normal archive because every package needs resources and if we just let them pile up like they will need ever more resources and I think we can spend those better and if you really care about the package then you should provide those resources and that means maintaining them and taking care of them Well I guess I guess we can agree if we aggressively remove them from unstable they should be somehow accessible and actually there is a source mark on FTP master but to my knowledge it's not yet mirrored somewhere so if this would be mirrored I didn't see it on Michael when I just looked but maybe there in the FTP tree so if the packages would be actually accessible to reintroduce them into Debian and not to redo all the work I think it would be easier to just remove them or lower the barrier instead of relying on snapshots Debian or DebianNet which is not even an official service I guess and which stopped mirroring currently So not a problem I see with keeping orphaned or in general buggy packages in Debian even only in unstable is that it lowers the quality judgment that users have of Debian if one packages even out of ten is problematic for them is buggy they see the buggy packages much more than the non-buggy ones they feel them more an impression they have of the general quality of Debian goes down very fast Well in some media another way around is to for example keep all the orphaned packages out of testing and during the release cycle one can try to adopt them or either adopt them, release them or or try to decide whether in that package release should be in stable or not but make sure and then unblock the package so it can enrich testing and be prepared for the release that way we could remove several packages reduce the workload and in the same way keep the packages around for some time so that they can still be adopted or whatever I mean nobody is suggesting that we just remove orphan packages after a month or so I mean okay I've suggested that I think but for special cases it is important to find the balance between keeping them long enough so that potential adopters can notice them and keeping them short enough so that they don't cause too much work for the QA team but it is my experience that successful ITAs are very fast and everything that is a bit slow and most of the ITAs that are just a little bit slower like more than a week or so never get anywhere I mean there are exceptions for some packages that need really much work or need a new upstream version that is completely different stuff like that but you can see if packages are often there are two general ways how it goes the one way is there is an ITA in less than 24 hours and upload in less than a week and then there is the other case where it just drags on and drags on and maybe there are ITAs from some people that aren't developers and then nothing becomes of that and stuff like that but most packages can be classified as one of those two cases so we shouldn't wait too long I think because it doesn't actually help the package to be adopted so the consensus is that we are going to remove often packages aggressively and I think the point Raphael made was that we should compile a list of packages that have been previously and Debian somewhere on QA Debian or maybe with the latest description latest version and tell people if you want to have this package back in Debian go to SnapshotsDebian.net or ask somewhat to get you the latest package from somewhere that should be easy and the list should really be useful and if it's visible so people can say it really was removed at some point and that's why and maybe even a pointer to the removal bug and that would be a nice service for QA to provide and basically that's only FTP master. This removal stock TXT is just too long and it's not readable so I really like the idea of making a policy of making an orphan package marked as RC buggy and I think that will resolve a lot of problems it will preclude from propagating to testing and if it affects the reverse dependencies then it's up to the person who is maintaining the reverse dependent packages to fix it or find an alternative solution and that actually it doesn't matter if it drags for a long time and it's sitting in unstable it will now propagate to testing until its status changes I was actually if people want to further discuss the orphan issue I was more planning to talk about UDD I think it's a very good idea what I started just thinking about it and potential uses I don't know how much of a package metadata you are importing but it seems like by importing all metadata you can come up with very exciting new checks like right now Lintian as I understand it does only static checks but you could do dynamic checks for example if you import information about what files are in the package check for duplicate files or you could check that the version which people are building the library version people are building against is actually available in unstable eliminating cases like people building in a dirty truth or something like that you could probably even do an automated check for the cases where there is a freezing effect and people wouldn't be would be warned to not upload the stuff right now linking against the particular library version and things like that so are you doing it or is it something you are planning to do currently it's a Google sum of code project that started 2 months ago so it was a bit hidden currently not really public we are going public with this no because the Google sum of code ends no if you are interested in importing more data just contribute and join the GNQA talk about us and we can give you access to work on new import scripts we are going to import a lot more data there is a goal for the Google sum of code project was to build a strong base if we can build know that we can build on it for example Raphael discussed with Raphael the idea of using the UDD for DHS the DebianWatch checking system because we could import some files from each package into the DB like DebianWatch or DebianChangeLog and then have a look at those files automatically you won't need any file backend for DHS anymore basically I have an idea when I have seen that there are 700 packages QA maintained I will not browse this list regularly but if there would be a very simple tool that just scans the list of packages which are installed on your computer or so whatever and sends you a monthly email using package ABC which are not I am very sorry it should be default because I am also using a package and I have got from the release maintainer there was no interest in this package since 2004 and this was just not true but nobody noticed sorry for repeating there was mentioned that packages that get removed will put into some snapshot archive and people have the possibility to revive it from there this hopefully will also include all the bug reports that were filed against the packages before it got removed well the bug reports are archived and could be unarchived yes but it should be linked with that information so that people will be made aware when they revive the packages it probably should reopen all the bug reports that were open at the time the package was removed and should check if it still apply I think we could just document that something like resurrecting packages in developer reference saying unarchive all the bugs look at archived bugs for this package when you resurrect the package I am not sure that we need a script to do that automatically yeah in the interest of not duplicating work we should probably talk a bit about now that we have Jeroen here how to use the stuff that he wrote in mole with the UDD because I think for some of the imports for data that requires a lot of work to import like like file contents and stuff like that it would be very useful to use the mole stuff because it is actually designed to do that and we could then import the data from mole and not directly so that we can reuse that code and me and Christopher have looked a bit at it and are getting familiar with it so that mole doesn't have a single maintainer anymore so that we can hopefully improve it in the future well I don't know what things but why don't we schedule another talk or a meeting actually so that we can continue talking about all these topics because now that we have still 5 more minutes well we won't finish anything so I don't know if we can use the other room or what not right now okay I take care of that asking for another script even if we don't choose it for one hour it doesn't matter we haven't talked about MIA maintainers yet do we think we have a problem there I'm not really familiar with MIA so I'm not sure how well it works how well incoming mail get processed I don't know well there's two sides of the issue one side is processing the known cases on the other hand I have the impression that we have lots and lots of maintainers who just maintain one single package which in some cases doesn't need another upload but in some cases could use another upload and do we have too many maintainers is that a problem do you want it like that or what's the general impression there I agree that we have quite a lot of maintainers and usually it's really painful to deal with with them because sometimes they are active they answer to emails when you mail them but they don't do anything about their package and they think that it's okay and they are not aware of Debian policies they just do it in their corner and that's really on one hand it's nice to have them participate in Debian on the other hand it's difficult to deal with so I'm not sure I'm not saying at all that it's a problem for people just maintaining a single package it's a lot harder to actually tell if they are active or not if someone has 10 packages it's very easy to tell if they are active or not just by looking at the bug count but for MIA stuff it's probably always if someone else is working on pinging maintainers and asking them what they are going to do actually maybe we could make a difference between maintainers that maintain a lot of packages and maintainers that just maintain one or two packages because if they just maintain one or two packages then we can orphan them using BAPAs for example we don't need to go through the MIA process for them because it's easier to just say okay this package looks neglected I will orphan it in 30 days if you don't fix the problems I was going to suggest that I think we need to be careful to focus on what the actual problems are and address the actual problems I think for example that we could make much more use of NMUs into the delayed queues and assertions of this is not well maintained we'll orphan it in 30 days or whatever the right threshold is because I think sometimes we have people who are actually reasonably engaged and are good people and we don't really want to drive them away from the project but we shouldn't allow their periods of inactivity or their attention to other things instead of that package to keep us from fixing the package and so we need to be a little bit careful sometimes the problem isn't necessarily that we have to maintain only one package it's that we have packages that are poorly maintained and we should not be afraid to do the things that are required to fix the packages whether we can fix the people is a whole separate issue I think we are done out of time I will take care of scheduling another slot for this I think Andrea has the notes thank you