 I would like to announce GANF presenting the FTP team and enjoy. Thank you. Thank you everyone for showing up. And welcome to this year's FTP master talk. My current plan, even if you can't read it on this beam projector, is to talk about the history of the FTP team going back to when Devin started. After that I will describe the roles of the team. We have three of them currently. I follow up with the current job we are having and then I'm going to describe how you could help the team or join it if you want. So let's go to the history. We are starting with the scripts. It started with scripts that Ayn Murdoch and Ayn Jackson later on just wrote on the fly and got it somehow. At the time when Guy Mauer joins the team, he rewards the whole stuff in Perl and then it's called deinstall. A term we still know even if it's now just a shell script. The thing at that time wasn't exactly secure. You could upload it and it got shipped. There was no signature, nothing on it. The trust level at that time was if you could upload, go on. When James Troupe joined the team, he started a rewrite of this whole thing going from Perl to Pison and that's still what we have currently and it's known as DAG. The hosting of the machine, it started under a desk somewhere at Zimich EDU and then got the first master Debian org which was hosted by BDEL somewhere at HP. The first machine, I'm sure you can't read it down there because it's too small, it was an HP, I even can't read it because it's small here too, yeah. It's an HP Vectra 486 with 66 megahertz and had some ISAs, GASI controller and two disk drives with a size of 330 megabytes and 600 megabytes and there was room to spare with the archive which is pretty nice. It had two 5.4 full head drives in Z and when it got out of away from BDEL and into the first hosting it got upgraded and had some 1.3 gigabyte drives in them which had lots of room to spare at that point and it was the first machine to host a .debian org system on the net using Debian itself. The machine then moved to iConnect and from there FTP master got to Nuware for a few years then to ReasingNet where it was known as FTP master Debian org for the first time got relocated to a buffnet and then had a short break at the HP data center in Fort Collins which is currently dissolved and the machines that are still left there are going elsewhere and current FTP master machine sits at Brown EDU and they are happy to continue the hosting of it. Luckily a bit history about the archive the Bo release had only the Intel architecture Ham had added M68K which we later on dropped in Edge a few years and we have a few more releases between them continuously adding architectures so that currently with Lenny we are up to 12 architectures for Streets we already added two new ones those K3BSD ones let's see what we get later on we currently have two proposed architectures from which at least the AVR32 one looks pretty good to get included in one of the next releases let's see what happens to them the archive size is also interesting going up bus with some 230 megabyte size slowly getting larger the first gigabyte release was Bo with 1.2 gigabyte and current Lenny all architectures requires already 125 gigabytes on the machines and I want to see when we get the terabyte per release, Streets plus X it's interesting to define how big X is I guess it's not more than two or three already the daily pushes which is all the new packages and start going out to the mirrors in the year 2005 when I joined the team we had some 200 megabyte up to 2 gigabyte in average one gigabyte pushes once a day currently we are pushing between one and six gigabyte with an average of two four times a day so quite a lot of traffic on the mirrors I've taken a week or something to write down a complete history of the team whenever someone joined as usual not readable from far about we are started with Ion Murdoch of course somehow and later Ion Jackson Bruce parents and B. Delgabi joined the team back then it wasn't clearly separated between the archive maintainers and the whole administration crew so all of them just did whatever they could we had Guy Mario taking over the job and moving to his Perl Deinstall somewhere in the years 95-96 unfortunately no one could tell me yet when exactly he joined and started working if some old-timer is hearing that I do want those information because it's the only one I'm missing we had James Troupe and also Richard Brackmann joining in July 98 and they later got Auntie Yuhany however you spell this name Kayyan Naho, EBIT and also Gagli Madaris joining the team in 1990 later on we had Michael Beatty, Omnik going in in 2000 and work on Katie started also in the year 2000 Anthony Torrance joined January 2001 months later Guy Mario became inactive and then we had Ryan Murray joining in June 2001 the last FTP master to quit currently Wendell Donner in November 2001 joined and was followed by Daniel Silverstone in the year 2003 and then we had a two-year period of nothing until the team got restructure and the current assistance role got added that was the time when I joined in March 2005 we had another period of nothing for three years until Kalle Kivima and Thomas Fiemann joined as assistants none of those two is still in the team unfortunately we had myself getting up to FTP master in April 2008 shortly after that Anthony Torrance and James True left and the other current FTP master Mark Reimers came in as assistant in July 2008 followed by Frank Lichtenfeld, DJ Pick in December shortly after that Thomas Fiemann resigned unfortunately he went out of complete debbie and not only FTP master we added another assistant in January of this year and a little later Ryan Murray left the team about the time when we promoted Mark Reimers to FTP master and Kalle Kivima recently at the beginning of Debconf left the team the current team as such is made up of Mark Reimers and myself as FTP master we have currently have two assistants which is Frank Lichtenfeld and Michael Conner both of them unfortunately very busy and we currently have one FTP 20 which is the role I will explain a little later Barry D. Friesel and since April 2008 we almost always had people in the FTP 20 role so the current FTP master role the easiest way to describe it is to quote a site from Matthew Garrett we are responsible for maintaining the infrastructure required to support the archive this takes the form of the scripts used for processing uploaded packages but also the flow of packages between the distributions as such our job is to keep the archive running keep it legal and support the teams that depend on it like the release teams, the security team soon also the back ports and volatile teams because it gets integrated into FTP master now and we have to keep the archive up to date with the requirements of the project which is for example the source v3 stuff coming from Buxy but also t-dabs and d-dabs and I go to that later also the FTP assistant role created in the year 2005 adding myself and Jeroen van Wolffener at that time that allows additions of people to the FTP team without having to give them the full FTP master rights those people in the FTP assistant role they can access the new queue they can handle overwrites and they can remove packages which is most of the day-to-day work besides coding and they can do all the common actions but can't do too much harm to the archive so it's a good step between nothing and the full wide set and now the FTP training role we created that in 2008 to make it even easier for people to join because getting someone into the new queue to run any kind of work in that area isn't that nice to them and to the project it was initially called slave new but somehow they didn't like the name can't imagine why and we currently had six people trained so far two of them promoted to assistants and one of them got up to FTP master later they can take a look at new they can look at everything that is in there and they can leave comments the comments are then read by an FTP master and act it on either they are right and we just do whatever they want or we tell them what they missed out the current archive we support is once the main archive with unstable experimental testing and all the stuff we have there we also have the security archive since Woody it's running with Duck and the security team has multiple suites in there we are currently going to integrate the volatile archive it will stop being a separate service but being just another suite on FTP master and the backpots archive it's currently run by me but we want to integrate it proper into FTP master having a backpots debian org somewhere running so let's describe the job a little bit more for removals and overwrites removals are one part of them are semi-automatic we are craft report that's a tool which is running the install one sending us a mail about packages that we shouldn't have in debian anymore mostly because they are superseded like library packages where new upload is going in and the package is renamed that gets not built from those anymore for the old ones there are packages in experimental where newer version is in unstable and various other checks and then there is manual where we have a nice html page summarizing up all our removal bugs we have on FTP debian org whatever the maintenance wish to remove we get that in a nice list and can just copy and paste the comments over if we agree with the removal like if there is no reverse dependency left on the package we shouldn't remove one that has a few hundred build dependencies or something the other job with overwrites is currently still the priority and section stuff where sections is something that should go away the override disparity mail people get whenever they upload a package and then find out that FTP master thought something different that an application isn't in xf anymore but in utils or something they get a mail and can reply to us and ask us that's currently by mail but we want to have it in the bts to make handling of that easier and also easier to follow what's up with them new the holy core of debian the checking there is about three things and the order of priorities that is trying to keep the archive legal we usually don't want stuff in the archive where there is a big fat header in there if you distribute that anywhere we are going to sue you we had people uploading that and we don't like that it's also trying to keep the package namespace in which is hard with 20 something thousand packages but we usually try that non-random maintainer and upload some x- or kde- which doesn't fit in there and we are trying to reduce the number of bugs in debian but that's very low priority we can't do full quality checks on new packages that would make new queue ten times the size of it the most common checks we are doing is checking the whole source for license problems for new uploads and for license incompatibilities or non-free license terms just mentioned one we are trying to get packages that are splitted having that in the same way like recently we had one package that tried to create 30 binary packages each with one file and one sim link back for the user doc we don't like that very much especially as the dependencies didn't change we try to keep policy violations out of the archive as far as possible we have lintian running for that whenever we are going into new and also an important point is the source for everything included most of the licenses requires us to redistribute the source together with the binaries unfortunately some of the upstreams just forget that so the code we currently have it's in a git archive it's pyson and a bit of shell and a bit of pearl and honestly also a bit of sea it's at least nine years old at least nine years ago we have the first commit message for it and we have currently about 40 42 different sources for the whole code base so you can imagine that it looks different from file to file and it's not easy to maintain so what you can do to join the team well first you can try to join the qa team our army running again for all packages that are no longer of use in the archive trying to find out what needs to be done like all the reverse dependency change to a newer package or something and generally just do qa work if a package doesn't become that un-maintained that it needs to get out of the archive it's something that will never get to us you can just go on to our bug page and maintain the bugs if you want like many people just don't get it with the right format of the bug so it never shows up on our removal list and might get lost we name it or close bugs that don't apply anymore because someone requested package change and stuff and it was done years ago and we just missed the bug and make sure that all the stuff on the bug page has some kind of sense we never ever bite anyone who is modifying our bugs have fun with it it's open do it you can only help the right code this page intentionally left blank please write code so we can put it in here yes you I know but I didn't want to put it in here was a bit too much and then join the team mail us ask us that you want to join the team usually we get people in here unless they already have way too much roles we have multiple people already with multiple roles in deviant and we don't help much but if you have a bit of free time mail us and ask to join the team our current plans for the future that are only a few points right now we are replacing the database code in duck with some based on SQL alchemy currently the database code is wherever we need database access it's done with an SQL statement within the code and then using some old PG Python library to get that via the database we are trying to get that a little bit more modern using objects and stuff the source v3 stuff it's on our to-do list to do it's mangles up with the database code changes because they are huge like the 5000 lines patch currently and we are stuck in there somehow we also want to split out the long description of packages files and making English nothing else than another translation which makes packages files a whole bit smaller to download which could probably get us rid of the pdiff stuff we don't like that it's very flaky and we want to integrate depth text and in squeeze plus two or something get rid of sections maybe have four sections of stuff left but all the rest out of it and all in depth text we have the t depth proposal Neil Williams did some time ago we had a meeting about it already that's still waiting we have d depth which is currently running in some google summer of code project and preparing the whole archive for you build from the reaction I think you are that one or not well okay we have some mails exchanged there already we are currently writing the code to get lint in auto rejects so even before you enter new and then just get it back again we have already checked already right at the upload stage for a set of lint in text which should be carefully selected so it only applies to the most several texts you can have which should never apply but unfortunately do apply sometimes for the build d folks we want to have auto signing done currently all the build d stuff gets mailed the whole lock to the build d maintenance they have to sign it and do the work for all those and with the 1000 packages uploads a day we have 11 architectures 11 build d maintenance who all have to sign 1000 build blocks and that's just tedious work and we could make it better somehow the proposal is already written down and we are currently in a discussion between dsa build d maintenance and fdp master getting it all done it's pretty far already and we want to throw the binaries away that maintain this upload we still want to keep binary uploads and require them pass them through lint but then throw them all away and have them all built on devian machines and build these which has few things that we need to change we need changes in the d package because we need a new control file stanza for that for architecture all packages when we throw them away not all of them can be built on one architecture on all architectures only be built on one of them and we need to mark that somehow it's also part of the multi arch proposal the same field so that helps pretty much so also thank you for skipping through the talk as usual I'm pretty fast through those slides so we can take the rest for questions and answers if there are any I guess you can you can guess my question sorry yes I can guess your question can you give us a time frame your patch is on our list and it should be done within the next two maximum three months we are nearly done with SQL alchemy and right after that is your patch okay because well the problem is the time is going on and it's quite a large transition and we need to put it rather sooner than later otherwise we will the thing is we do want this patch somehow in the archive but we do not I don't want to go against mark's work your patch is changing database stuff he is having a 5000 line patch already and if we mingle that up it's pretty bad but we are nearly done it's only I think one file left and then we are in the testing phase of that SQL alchemy stuff if it all works in the archive as expected it should but well let's test and after that your patch is coming in okay there are not many SQL queries in my part of the patch so it shouldn't be too difficult to merge it I guess I can try it if you want yes mark should have put it up somewhere contact me later and we can see where the gateway is someone else yeah no one no questions everyone likes the ftp team okay let's make it something different I want everyone there is a question yes but until he has a microphone I want everyone that has a bit of free time to join the ftp team please apply we want more people we want more people going so new we want more people doing your removals but still we want more people everyone please apply Steve okay couple of questions I'll do one first and then let other people dive in no couple of questions I'll say I'll do one first in terms of the current mess with Linux modules extra and friends I know you already know about this has anybody looked into doing the extra coding work needed to replace it with something that works the current mess with what? Linux modules extra I think mark looked at it at some point and didn't have a very nice comment in IRC about it yes we should well Linux modules extra is the best example but there's actually quite a few other things like the problem of other packages depending on the source of something else which we currently can't handle in the database so we might lose the source of it we need a general solution for it and it's somewhere on our to-do list it's just not so nice task sure I mean as far as I know mark has ideas on how to do it we need database changes and stuff to link between and yes we already thought about it we aren't yet but we dislike those packages and we want something better for them sure it's a mess whenever we get to a release and need to have another suite within the archive which no one sees but we have like such R0 Edge R0, Lenny R0 which just have links of sources so they never get removed and of course it also breaks other people like Debian CD who have to do their own parsing of the archive yeah it's fun isn't it possible to simply do BNMU right after or just before the release so that we have really use the latest sources that we're going to ship? the problem with that is that you can have you can break the package with it if you've just been in NMU at the way and right before the release you don't want to do that okay nobody else has another question then I'll dive in with my second one I've been prodded by a couple of people specifically looking for more guidance over what will be accepted by the FTP team I know that you're busy please a little bit more feedback to people when they ask for help would be really nice okay get more people in the team then we have more people to answer speaking of getting more people in the team could you explain a bit more about getting from nothing to FTP trainee to FTP assistant from nothing to FTP trainee it's a mail to us and we need to like you because we need to work with you later on which is usually a given task except for a few people I'm honest I'm not polite in the FTP trainee role we had people for like four or five months and then leaving because they either had no time or well and we had people going from FTP trainee to FTP team within a week the usual maximum we want to set for the FTP trainee role is six months after that we and you also should know if it's work you can and want to do and have time for it and so after six months you should probably get FTP team unless you are very dumb and when you have FTP team there is no hard timeline where we want to get you up to FTP master or not for Mark I think it was a year or something and Thomas would also have been promoted after a year if he didn't resign a month before that so a year is probably a good guideline after which one can look at it do you have requirements for FTP master more than FTP assistant I guess if you don't write that code have you a chance to be FTP master unlikely I guess requirements basically build up a link of trust between us and we should know you and we should not fight against each other and the team and well looking at the work you are doing and if you get lots of complaints about the works and it's probably not going up to FTP master but if we are just going together and working together nicely and having exchanges of code and stuff and all that it's usually easy to go up at least it was for Mark so first thank you for all the work you are doing inside the FTP master team because it's quite clear that a lot of this work is not really fun to do but necessary and now currently the new queue is at about 200 or 300 packages around 200 bit yes yeah between have you what do you think of also the way to help you with new queue ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?