 This next talk is going to be about quality assurance in Debian. It's going to be held by Lukas Nusbaum. He told me he's been a Debian developer for like three or four years and I hope you enjoy this, Boff. Have fun. Thank you. So my laptop doesn't work with this BMR, apparently there's a bug with KMS. So if someone wants to help me fix it, I will be interested. So using Zach's laptop, I will talk about first a short introduction about Debian quality assurance. I have some numbers in the current state of the Debian archive, which are interesting. And then I will outline some ideas of discussion topics, because this is supposed to be above, so the point is to discuss things. So first, quality assurance. So the goal of the QA team is to improve the quality of Debian as a whole, as opposed to working on a small set of packages inside a more specific set of packages. The QA team is not really a team. It's more like a central place inside which people do work on QA, but there's no strict notion of being a member of the QA team. So we hang out on ISC on Debian QA and on Debian QA mailing list. So what does the QA team does? First, we maintain the PTS. That's mainly Zach's work nowadays. So everybody knows the PTS. That's the thing that recently broke yesterday. DDPO is a pair developer overview about one's packages. Since last year we have UDD as well, which I will talk about later, and also many other tools. So we also develop and use tools to run AFK-wide checks and do mass filings about the problems that we find in the archive, like rebuilds of all packages in Debian, and a few parts of Debian.org, so we'll just talk later. We also take care of orphaned and neglected packages, and we try to track inactive maintainers in the MIA team for missing inaction. So for those who, since last year or the year before, there are new things in QA. First, we've got some new contributors to the team. So first, Barry Duffries, who isn't there today, who has done a lot of grant work in QA. Raphael Geiser, who is here today, can you stand up so everybody knows you? Okay, thank you. Marco Rodriguez, who is not a DD neither. Also has done a lot of work on, for example, closing bugs that were filed against packages that were removed from Debian, which allows to archive them. So we have less bugs which are unarchived and makes everything run faster. And finally, Sandro Tosi, who is currently the only person really working on MIA, and who isn't here as well. So we have new tools as well. So who was in the PewPads talk just before this one? Okay, so if you missed it, there's no website called PewPadsDebian.org where all packages are checked on a regular basis by PewPads. That's work by Olga and Liv. So that's really good because for a long time before, nobody was really using PewPads and reporting bugs about PewPads. There's Ultimate Debian Database. There's a talk about UDD tomorrow evening. So come tomorrow for details. So basically the idea... Don't remember if I have a slide about it. No, I don't. So the idea is to gather all the information in all the different places where we can find information in Debian in a single SQL database so we can run queries which combine all the data from those different places. And, well, I will show some examples later. There's also the PTS who got redesigned yesterday. So if the PTS doesn't work for you, use Control Shift R and then ask Zack if it doesn't work. And finally, there's BAPAS. So BAPAS stands for Bad Packages Search. So the idea of this is to use the various sources of info about packages that we can find in Debian and the scoring system to get a list of packages that are interesting for some reason. For example, we can, using BAPAS, we can get the list of packages which are really buggy, which has a lot of open bugs, which are neglected by the maintainer or which look useless because their popcorn is really low, for example, or which are orphaned, and then process them in a systematic way. So here is an example of a table and I kept another... Well, so, window 5. Yes. What's the mapping of this keyboard? Yeah, but it's... OK. Waze Plus, the equity keyboard. Shift. No, that's French. OK, so, and it's missing. OK. Well, so I will just click on one of them. So this is, for example, the list of orphaned packages. There's orphaned written on the left of the beamer. So it's... Today I worked on rewriting BAPAS to use UDD instead of ad hoc scripts to import the data. So this is generated in real time by a huge SQL query. So on the left, there's a list of packages which you can see, but I can. There are various columns about the status of the package. For example, this is a list of orphaned packages. So you see the type of orphaned bugs that the package has. The number of days since it was orphaned, so the list is sorted with the oldest orphaned packages first. The number of days since the package didn't migrate to testing, or, yeah, that's about the same. The top score for the package, number of bugs, the number of days since the last upload of the package, number of NMUs, and then commands that you can see as well. So using this, we can easily get an idea of what's the appropriate action for a package versus another list about packages that are maintained only with NMUs. So it takes a few seconds because of the huge SQL query that is behind that. So for example, if you take the first package there, it has been maintained using 13 NMUs in a row. The last, yeah, that's quite nice. Actually, there's a bug because it's displaced 10 there, so it's a bug in UDD, not in there. So, yeah, and there are lots of packages which are maintained only using NMUs, which is quite impressive. Or do you know which one is that one? Okay, well. So now some numbers about the state of the archive. So first we have a bit more than 14,000 source packages in seed currently, including contrib and non-free. 4% of them are orphaned, which is quite a lot, more than nearly 600 orphan packages. That's the list of the most popular ones. So all those numbers are generated using UDD. So that's the kind of things that you can do using UDD. So come tomorrow if you want more details. There's also more than 2,000 packages which are not maintained or co-maintained by UDD or a team. That means that all the maintainers or uploaders are not, none of the maintainer or uploaders are DDs. That's 15% of our packages. So this can be either a good thing if you consider that, that means that we can attract new people, or a bad thing since that means that those packages, well, the state might be more questionable than the one of those maintained by DDs. So that's the list. So there are no really major packages, but quite a lot of well-known packages anyway, like TCSH. 6% of our packages are RC buggy, with a list with many major packages this time. So that's the number of popcorn installations. And finally, we have more than 1,000 packages that are in Ubuntu but not packaged in Debian. So the list can be seen live using UDD. Lots of them are Ubuntu-specific, even if I included some of them. There are some minor things that we don't want in Debian, like small shell scripts that someone thought was a good idea to upload to Ubuntu, but also some things that we really should have in Debian and that we don't. There are some topics for discussion since this is above. So first, orphan packages, which is a usual topic that we like to discuss during QA buffs. So orphan packages are a real problem. First, they are often of bad quality compared to the other packages. The problem with that is that sometimes users just look for a package that does something and install them by mistake while it should be better if they install another maintained package. Before the release, usually we have lots of RC bugs in the orphan packages and people try to fix them when sometimes it's not a good idea since the package is totally unmaintained anyway. So last year, during DeepConf, during the QA buff, the proposal to not release orphan packages was raised. So what was suggested, what would be avoidable by the release team when needed? For example, if an orphan package is a dependency of another package that we want to release with, that could be an automated process, which is a good point. Another good point is it will increase the visibility of orphan packages because often the problem is that people don't know that the package that they are using is orphaned. But on the other hand, it doesn't really solve the many orphan packages in an unstable problem because even if we don't have them in testing, we still have them unstable. And the worst problem is that people might adopt packages without maintaining them. Instead of, well, to prevent them from being removed, people would just adopt them and that doesn't solve any problem regarding quality. So the other solutions would be to remove some orphan packages to work actively on finding the ones that we should remove, remove them and try to, then we would get a shorter list and then can work on finding maintainers for the other ones. So if you look at the current list of orphan packages, there are more than 200 packages which are orphaned for more than one year and with popcorn, lower than 500. So those are probably, lots of them are good candidates for being removed from Debian instead of just keeping them in without doing anything. Also, most of them are really easy to find alternatives. So it's really quite stupid to have five orphaned image-viewer or music players while they are good ones which are maintained. And the other final solution is to find ways to increase the awareness of possible maintainers about orphan packages because the main problem is that people don't maintain them because they don't know that they need a maintainer. So if we had ways so that possible maintainers would know that some of the things they use are orphaned, but probably they would adopt them. Other things that we could discuss is the status of MIA, so that's a team that tries to detect inactive maintainers. And also, of course, all the QA checks or tests of mass bug filings that you would like to do for squeeze. And of course, the idea there is not to just throw ideas saying, oh, you should do that because we would need help as well on doing them. Okay, thank you. So let's discuss. I have got a question. Who is deciding which package will be removed or which Debian developer who is missing in action for a long time will be forced to leave the project? Well, the QA team sort of has authority to do that because nobody complains strongly about them doing that. But there's no official delegation from the DPR to the QA team to allow it to do that. I think it has been like that for years and everybody accepts that, I think. For the packages, what I recognize, there are many packages that are orphaned since a long time, not maintained. I ask myself why aren't they removed already? Shouldn't we set the level that the maintainers should work on their packages a little bit higher so just remove them and not wait another one or two years? Well, the problem is that our priority is our users. Sometimes there are users of some packages which are completely obscure. But for example, yesterday I looked at a package called Rhyme which chooses a dictionary to find rhymes about words. And I pinged the last person who filed a bug about this package and he was still using it. And the package has a popcorn of 50. And nobody has been maintaining it for four years, I think, or five years. So what do we want to do with that? Just a quick reply to this. In terms of missing in action maintainers, they don't get asked to leave the project, but we have a special status for them. They're inactive, so if they come back, they can be quickly reactivated. Now with packages, I think we should have something similar to that. I think it should be... I've been personally involved for the last two weeks now or three weeks in a discussion to remove Rolo, which is unmaintained upstream, and so on, but there's someone that uses it and wants to keep it. So I was like, well, then go and maintain it. And he said, well, I can't. I was like, okay, we'll just orphan it, we'll remove it until somebody actually can maintain it. But he obviously didn't like that very much because he uses the software, and also because I think that he doesn't like the prospect of removing it, meaning that it has to go again through the new queue, that it might take four weeks for the package to get back in. So maybe this is something we could streamline. A package can be removed, but if somebody actually then decides to pick up a previously removed package, put it straight back in because it has previously been in there, but I don't think we have that. Well, that's an implementation detail about the new queue, I think. A small comment about Martin's suggests that we should handle packages and developers the same way. Our users are using packages. They are not using developers. So there are completely different requirements on how to handle these two sets of resources in the project. I was going to say one way you could implement a package not showing up in unstable is just making an upload to experimental and leaving it in there until someone picks it up and uploads it back to unstable. But then it doesn't get auto-built. I don't know if that's... At least it's sort of there, but it doesn't have to go through the new queue again just to get back into unstable. I think it's a good correlation that an orphan package, a package that has been orphaned for a long time is actually un-maintained, but it's not a perfect correlation. And I think it would be good to find approaches for finding packages that are nominally maintained, but not actually. And I suggested somewhere earlier that we could, for example, decide that any package that hasn't been uploaded since the previous release is un-maintained and should be removed probably. Well, there are some quite old packages in Debian that work perfectly well, that have not been uploaded for two releases and... I know, I maintain one. But frankly, if someone... If there's no change to a package for three, four, five years, I'm sure there are exceptions, but mostly I think it's a package that isn't being maintained. In response to, I think it was marking, I think we could create something that would be the opposite of volatile, static.debian.org, to which we move these packages that people are still using but don't have a maintainer. And whoever actually really cares could then pick up the package from there. Just a question. I remember there was a previous discussion and they would say that the package that has been orphaned for a long time and what is supported that has a lot of NMUs that are badly maintained will rise up as a release critical bug. And to be this package out of testing at least, and then it actually will be removing a release cycle and the next release cycle, the package won't be stable. And to do that, somebody push up and maintain the package again. I don't know if that discussion moved up, so I didn't follow it. Well, actually, the discussion that took place last year at DeepConf didn't go anywhere, partly because we were supposed to release Leni soon after DeepConf, so it wasn't the right time to start trying to remove lots of packages from testing. Now, whether we want to remove them using RCBugs or using release team inns or another way, that's just an implementation detail, I think. The question is, do we want to remove them from testing or not? Yes. If we move the package out of testing at least, raising up the orphan status to a release critical bug, the package should be removed in the next release cycle. The problem is if we remove them from testing, then users might not notice if they use app spinning. So it's not that easy to see if the package that you're installing actually are found or not. So I'm not sure, well, it's absolutely a solution to the fact that we release with them without knowing if they are properly maintained or not. But that's not a prong term solution to finding maintainers for those packages. Maybe it would be possible to add a DeepConf warning message to all these packages. So if the user is installing or upgrading his system, he will get the message. This is an orphan package for a long time. It may be possible or we like to remove this package for the next release. And if you really need this package, please mail to our file a bug report. I need this package. So I think then we could get a lot of packages removed during the next release. So what we have to do is make a new version with the DeepConf informal message and then we can inform the users or maybe we will not remove the packages but move them from main to another repository like Volatile as somebody else just suggested. Well, there's already a WNPP alert that lists the package that you have installed and that are orphaned but almost nobody uses it to find packages to maintain. So I'm not sure that adding a DeepConf warning but the problem is that we need to find maintainers who want to maintain them, not users. We might not want to remove them if they are useful to some users for good reasons. There are two kinds of orphan packages, the upstream orphan packages and the un-maintained or without a deviant maintainer. So probably for those one who has not a deviant maintainer, we should keep it and find a maintainer and for the other ones we should remove it. But we should put a difference between both of them. I don't know how. As a user rather than a developer, when I did some work with a package and I didn't know that it was un-maintained and there's no way of finding that out as a user and it sort of horrified me in a Debian stable and everything's maintained and actually as a consequence of finding out that this package was not maintained I've now worked with the package to take it on. So I think raising the awareness of it is definitely important. I think it's important to mention that effectively we can always find users for other packages and it's always difficult to go to... It's easy to go straight to remove the package and don't care about the user but we really should focus on the long term and the right thing to do is try to find maintainers and not try to put the package outside if possible. I know it's not easy but we really have to develop something you've told about a dev conf warning. I'm not sure it's enough. We could have something more integrated sort of, I don't know, a sort of demon or applet that you can have in your task bar that tells you news about your Debian system. So you have two package editor often for statistics and stuff like that and I think we could have users which are not yet really contributors but which would like to follow and use this kind of stuff and maybe try to contribute this way. It's... Well, I... But I think we already had some processes for those packages that are often where our users that like to have this package and if there's maintainers who says, oh, yes, I will maintain this package we do not have problems with those packages. I think the package that have some problems are where we do not find a maintainer for a long time and maybe we do also do not know if there are users that are using this and if those packages have some bugs I think we should remove them. If we could find a maintainer for a package because there are many users these are not the packages we are talking about now. What's the question in the back? I just wanted to say I agree with that but I don't think we do enough to find maintainers and we should improve on that. We need time for that mainly but it's not a difficult task to go on the upstream communities to see if there are maintainers or another user would like to maintain it. Well, when there are more than 500 packages for the first time we do that it's quite... It's difficult for us but technically difficult so it's a way for some user to start contributing. No, because the problem with that is that you have to make an assessment about the usefulness of the package and it's really hard to say to someone who you don't know just come and see if this package is useful and if it isn't remove it because you need to trust him at least a bit. Some mistakes have been made in the past about... Sure, I don't think that those contributors should ask for the removal but they can get in touch with in forums or in web forums related to the software or mailing list upstream or in user groups somewhere else where this software has been mentioned. That's... Well, we need a deviant maintainer for it If they find nobody and it's one more information that we have well, maybe it's really time to remove it but currently we don't do that at all. I have a question as a Debian user as a commercial user if I have demand for a certain package which I do not want to maintain which options do I have to sponsor the maintenance of such a package? I think you can always pay someone for that. Yeah, that's the obvious solution. In Skull Linux we paid some developers to work on the packages we cared about for a year or two and that's definitely one alternative you can also pay companies that hire developers that will take care of packages you care about if you want that. Of course, you can also do the community way where we have organized developer gatherings for areas where we wanted to have progress we have partly funded DebCamp, DebConf, that kind of thing the Extremadura government is funding a lot of developer gatherings on the topics that they find interesting and Debian find interesting so if you have money to spend there are many ways to do it to make sure your areas are well covered. So where would I especially ask if I want to find somebody with the expertise in the area I'm looking for at the package? I mean it kind of depends on the package but usually you can find on Debian Consultants mailing list or if worse comes to worst you can ask on Debian Devil saying hey. Thanks. Sorry, Debian Project and ask if there's somebody who's interested in maintaining the package and those would be good places to ask. Yeah, I mean the worst case you ask on one of those places and someone will tell you the best place to ask so don't be afraid of asking the wrong place. Just a short idea to raise awareness about off-front packages, here I am. Okay, thank you. We guess all people are reading the mails for you so we can implement a crown job or anything like that to inform the route that any package is off-front. And for desktop user maybe it can be done by update manager or something like that so that we have a pop-up box for the desktop user hey your package is off-front maybe some people will jump into the train and take it over maybe. So this may be a way to raise awareness for the people that are interested in using the package that are off-front. Does this work? I've actually written a program for Ubuntu that is included in Ubuntu called computer janitor for the desktop user which among other things looks for off-front packages and always finds Skype. So it needs some improvement still but tools exist for this for telling users. I have been lagging in uploading it to Debian but that will eventually happen. Please kick me if you haven't seen it yet. The problem is also the level of well how many users do we want to reach with such information because if we only reach the people who have the specific package that provides the script then we won't reach many people and I'm not sure that even amongst DD would install such a package. I think at some point too we have to decide that we've tried our very best to reach users and people who wish to see more stringent controls before removing packages I think it moves them to put in the effort to try to find maintainers for those packages so I think at some level the QA team has to decide perhaps internally exactly what level they want to do and request removals after they've identified an orphan package and people would think that those packages should more should be done they should participate in QA and spend the time to find maintainers for those packages so I mean it's easy to tell QA to spend more time to find maintainers it's much more difficult to jump in and actually do that work so I mean if you think QA makes a decision and a procedure that's not rigorous enough jump in and do the work I think that's the most important point Raphael's saying that this is not a technical difficult problem no it's not a technical problem at all it's a social and resource allocation problem is this the best way to spend an hour trying to track down potential developers and maintainers of packages if they are interested they should show up when the problem arrives if they don't then they are not interested enough and my hours are better spent working on packages that I do maintain and I suspect some probably most developers feel the same way it's how do we best spend the resources in Debian if people want to do this I will not stop them I will actually reward them and send them on their way and hope they will do a good job finding new maintainers but you cannot expect anyone to actually drop what they are doing at the moment because we do it because it's fun we believe it's fun what we are doing tracking down potential maintainers doesn't sound like fun to me so one idea that was raised last year was to have a current job to reach potential maintainers that would report newly or found packages who would think that well, installed by default when you upgrade Dev Scripts because developers obviously run Dev Scripts on the machines they use to develop Debian who would think that this is appropriate and so who doesn't okay so we have two people who doesn't I have because well okay one, two, three, four something like eight and who doesn't and who is reading ISE or email maybe leave you can explain any more email to people who get thousands of emails per week it's not to work very well you shouldn't do it probably won't work very well unfortunately I don't have a more constructive suggestion maybe sending Debian user these emails I want to say something back to maybe we should not call it that we want to remove the package at all but maybe we want to move it out of the main part of Debian because users are our priority but also a level of quality and if the package do not have a certain amount of quality they should be moved from main to something like volatile or something else so our users also have the the wish to have packages that are non free or multimedia package also do not have them in our main repository and there's this Debian multimedia repository what I think nearly everyone uses and so I think we could move those package to a new repository which is knit which is not in the official part of Debian but if there are some users that like to use these very old not maintained maybe buggy packages they could do this and I think users are our priority but also and I think that's very important the quality of the packages in Debian main that's Andreas I've heard the suggestion to add new ways of keeping packages more than once I think the first time I run across this suggestion was on Saturday morning in Vancouver discussing with Elmore how to structure packages and from that time on I've heard a lot of suggestions but why doesn't somebody just do it I think Debian is not about saying somebody needs to because I've not seen somebody as Debian developer until now but only people who are doing stuff so if you want it just do it if you need help in things like setting up auto builders of course I'm happy to help you there but if you think it's useful enough then do it if not well that won't happen no but I think the current question is do we want to remove those packages from the main Debian repository or do we like to keep them inside the official main repository well if a package is too crappy we need to remove it what else should we do of course there is something like how should I say we have these packages that were ever in Debian so we didn't lost any package at least during the last five years we can always cover any intermediate version so yes of course we should keep that but if we can't support it we should remove it what else should we do I don't think it's good to say well we have these packages that you can work with and we have here a bunch of crap that you might want to install otherwise as well and what you should also do is that every package which goes into the main archive and is not removed needs to be supported by the security team and that's it might be worse to put more effort into just saying okay this package is really too crap just remove it from the main archive because every crappy package adds load to the security team and to other teams like the QA team so I think we should encourage the QA team to remove packages I kind of like the suggestion to use I like the suggestion to use experimental to move packages that are and maintain because that's quite easy and still inside Debian so all users who use experimental know what they are doing who would agree with the fact that often packages would be moved to experimental after some individual examination but not really not very throughout but yeah please comment before me so I like the idea is nice but using experimental like after if a new maintainer gets interested in the package but first one to release something to experimental it's like we are using something which is supposed to do something else to solve the problem that we don't have a place where to boot those packages one issue that's not been mentioned about removing packages or moving to experimental is what about often packages with maintained adepts oh yeah yeah yeah of course the short examination about each package would include that test to see if of course so just can you raise your hand again if you agree with the idea of moving them to experimental or to another place who doesn't agree with that okay Raphael explain well I don't think that moving packages to experimental is a good reason maybe if snapshots will be back will be working then we could just remove the packages and anyone who's interested in the package can look at the snapshots and get it back of course by fixing it I mean because packages are not removed just because we want to remove them I mean there's always a reason either they are maintained they have some sort of problem so just moving them to experimental doesn't make any sense to me I mean we are just temporarily moving the crap to some other place and then we will close experimental with more crap I mean that doesn't really make any sense yeah but the point of doing that is to have an intermediate solution between keeping them in testing and stable and stable and having them removed yes I agree it's a problem if it's filled with crap but on the other side well while the package is in experimental it's in debian and it has a page on the package tracking system and it will be clearly identified as well it's not in unstable so you need a maintainer to push it over there and so it's a good I think if we use this together with the fact that we remove package early from testing we keep it visible within debian over a longer time period while still showing through the user that the package is in danger because it has been removed from testing so it's a good intermediate solution I think yeah is there any particular reason it will show up in the orphan package list in aptitude well some do we don't need all users to notice we just need those who care is there any particular reason why we don't just remove the package and use the snapshot system that is planning on tracking all versions of packages to debian ever uploaded and a user who desires to use such an orphan package can then point themselves at this snapshot system but then you have to build the package yourself the snapshot system my understanding of it is that it will also archive every single binary package that we've done so we just remove it completely as once the snapshot system exists and the binary packages will be there and a user who wants to use it will still be able to use it it will be no different than moving it to experimental but we won't have it cluttering experimental as well so I think that may be the ultimate solution just remove it once this all done and it will resolve the issue for users who still have a use case for it while telling them completely that it's not supported I don't think we have many users going to look into snap together package the few users that would complain about the package being removed already so they have it on their system and it shows up in the orphan packages the idea is to try to find a way to streamline the process so that we can orphan the package and hopefully find a maintainer and remove it only as a last resort and not the D4K immediately well, removing it immediately is not really interesting but it is may I just answer to Don the one thing which is still missing to get the snapshot archive working is getting a machine shipped to Sanger which Stephen Gran is willing to do but was just too busy directly in front of Depconf doing so so that will happen a few weeks after Depconf and I hope we can get the snapshots up by something like beginning of October so there is some progress made from the DSA side I just wasn't aware with the current status because I knew that it was in progress that's awesome to hear but really I think there's a point of having something like experimental or another archive that user can directly use is that it provides an intermediate solution between removing and keeping and moving to experimental and that's really the point because often when you look at one package that's been orphaned for a long time you don't want to spend two hours doing that so after some time you think okay I need to take a decision which one will I take and if you have something in the middle I think they're in the middle once snapshot is working there's no difference between a user having to add an experimental source and a user having to add a source that is in the snapshot, it's the same they just point it to a particular package that they want, they write the source sources list entry and it's done but then the package wouldn't show up easily on Google for example if you Google for the package name that all can be I mean if that's something that's a serious problem then it can be added we can have web pages okay what do I do if my package has been removed from Debian that I can no longer install well okay this is what you do and so that's all stuff that can be done I mean it'll archive at least the it'll see the removal request for the bug and in the standard QA request for removal you can say have a boilerplate text that says if you want to install this package which we've removed for these reasons add this line to your sources.list and install by the way last year I said maybe the best idea will be to create a page where we mentioned all the packages that were removed because they were un-maintained or something because currently we cannot just point the users to FTP master removals file because it is cluttered by other removals because of some other reasons so if I remember I think perhaps was working on something like that or I may be wrong but the idea will be to create such a page so that we could point users to that page or even people interested in those packages so that for example because there used to be some at least some people trying to get those packages back and it will be easier if we just said go look for that on that page one thing that we could do to make that already easier in the FTP master there is a user tag for the FTP master user called ROQA which does the removal by request of QA so you can add another user tag and just have a search request to the BTS that just displays those bugs and it will display all of them and that will be easy and there is no extra effort needed one other reason is that the user is against moving it to experimentalist if the user want to use an old package and adds a line for the experimental repository to his sources list file then he will get the old package that he likes to use but also brand new package that are not stable at all sorry what was discussed is the fact that experimental is always given to the app preferences so there is no problem but I'm not sure if the users are that good in changing the pinning numbers for getting only the one old package that they like to use it's done by default by the releases file so the user doesn't have to do anything they have to know enough about app preferences to change it from the default okay we have two minutes remaining so I originally asked for the first slot for the QA box because I knew that it would probably take more time so there is another part on Wednesday or Thursday I think so if you maybe you could gather and try to summarize different proposals that popped up during the discussion because we don't want to start again from scratch so talk to me if you want to help to summarize this well maybe we can take another comment because we have two minutes left no okay just one question snapshot, storage the source of the package also the binaries well if we storage the source it's easy to retake the package for a newbie it's easier when you have the old source both are stored in snapshots sorry I didn't