 Yn ymchwil, byddwn ni swydd o'r stethos ym Mhwyl Gweithreid. Felly, ddim ffawr i'n gweithio i gweithio i'n gweithio i Gwyrddol. Rwy'n gallu'n gweithio i'n gweithio i'r mhwyl Gweithreid. Diolch yn fawr, eich cyflwyno'r Gwyrddol yn rhan o'r PPA, a dddechrau'n gofyn nhw i gweithio i'r ffordd gweithreid, a dwi'n gweithio i'r ffordd. A dyna'r bod yn y gweithreid, wedi'n gweld. Felly, fel y gweithio y gweithio y cyfnod gwahanol, mae'r PPAs yn gallu gweithio'r cyfnod yn gweithio. Mae'n gweithio, mae'n gweithio. Mae'r pwysig o'r rhagleniol yn ei gweithio. Mae'r PPAs yn gallu bod yn gwneud o'r rhagleniol ar y cyfnod. Rwy'n gweithio'r Debcon 11 yn Banyluka. Mae'r cyfnod yn gweithio ar y cyfnod, ond we've decided, yes, we're going to do this. This is how we're going to do it. It's going to be excellent. And then roll on a few years because everyone gets busy. And in 2013, Joerg posted a lovely posting to Debian Development Outs, that's the URL. And we said we're going to have them by the end of the year. And now it's 2015. So this has been something that has been a long time coming and it's something that's appreciated. But hopefully, by the end of this talk, we'll have a much better idea of where we are and what's left to do. So for, I mean, hook your hands up if you don't know what PPAs are about and never heard about any of this. BDL and other troublemakers around. OK, right. So PPAs are the sort of why of it is it's their way to create, build and distribute packages really easily. So ideally, there is less constraints on having to upload to unstable or experimental. And it ensures that the main suites, which in our terms is suites are bits of the archive that you upload to, and it makes sure that they are consistent and that the integrity of them is maintained. Additionally, for one of the sort of PPAs we're looking at, it helps manage our legal liability risk. So if Debian is distributing something, then we need to make sure as a project that we're OK to do so, that we're not going to infringer on anyone or breaking licences, etc. So it's a way to reduce the load on the project in making sure that happens. So most importantly, when I say PPAs, it's not like Ubuntu PPAs. We want to make that quite clear. This is something that is different, but we've also tried calling them developer repositories, but I'm not sure. Bike sheds. Apparently we should call them bike sheds. So these are now personal bike shed archives, something like that. Yeah, bike sheds. Or we can just call them bike sheds. What do we think about that? Right. Excellent. So this is why I like a buff. So the new name for PPA is just bike sheds now. So you can store your own bike sheds. You can put bikes in that bike sheds or just gardening equipment. Yep. Excellent. So these are now called bike sheds. Fantastic. And so these developer repositories are something different. A lot of the comments I've had when I've been talking about PPAs has been, oh please don't do that. That's going to be horrible. You'll be full of all sorts of random stuff by people who don't know how to make Debian packages, and it'll be a disaster. But we're going to approach it differently. So there's two main streams we're looking at. One is PPA main and PPA exed. PPA main is essentially your own little version of experimental, but it could be built against different suites. So things have to go through new checks. They have to really much fit the same standards as they do for the main archive. So yes, that includes new processing, but it's a way to just make it a lot easier for you to do various things. PPA exed is a slightly broader version of that. So they're sort of external to the archive, and it's not integrated in the same way. They don't need to follow that full set of rules and new processing, etc. Importantly here with PPA exed, the maintainer who creates it have to agree to terms of service, terms of conditions, and any content is the responsibility of the maintainer, not of Debian. That's where it lies here, because we're not going to check it, we're not going to do new checking, then Debian is not going to take the liability for license compliance. Let's see what the next slide is. So, yeah, as mentioned, this is for Debian developers only. So uploading and possibly Debian maintainers. So working on that on how we can do it. But it's for people who already know how to make packages, it's people who've been accepted into the project in one form or another, and it's for them not just a wide anyone could create anything in the world. And the main people involved with creating these PPA systems have been FTP masters, WannaBuild and DSA. So I've done very little work, in fact almost none, apart from creating a set of slides for everyone and creating a huge buff to try and push this. So any sort of credit definitely goes towards the FTP masters and these teams up here. So having a look of where we are now really, I'm basically going to hand over to the FTP masters and they can sort of let us know what the state is, what we have next and if anyone can help and where we can go and sort of timescales and what's coming up. So I'm sure there's a mic around for your work somewhere. Okay, should all have just been in the buff one hour before where we talked about DAX, there was this pretty big topic too. The basic status is that we mainly need an interface for the developers to actually create all of them. We have infrastructure in DAX already for dealing with multiple archives which all can have different suites inside. So they can be stored wherever on FTP master. So every PPA could have their own space where they are so the files don't override each other and stuff. What we need is a command interface for the developers to actually work, which will end up being DAX command files to upload similar to D-PAD or D-CAD AM or something. Where you have a signed command file to upload and inside that command file you say create a PPA based on stable, put those list of packages with those versions inside and the fingerprints A, B and C are allowed to upload into that PPA. Then possibly having a command to copy the PPA back into the suite like if you have created one. A command to copy the contents back into the original base suite where you have been based on like if you have something from unstable to prepare a transition inside, you had an own suite for it and now you are finished with your transition and it's already built and stuff. Then you just go and copy it back into unstable. It's doing a few version checks and stuff so that now a newer version of your libraries have been uploaded or something where you would need to build again and then it takes it over and then some way to remove those. So what we need there is the exact definitions how the command should look and then some duct tool to actually implement then, which is mainly a coding work. In case you want to help, it's Python programming. That's very accurate. When we have that in the archive, we need DSA and the vulnerable people to work together because every PPA should probably be built on the series of architectures. So want to build and then all the different buildies need to learn about the new PPA and need to be set up so that they have change routes and all the triggering from duct into want to build and the buildies goes automatically. I can't say anything on the buildies so I have no idea there. Okay. So about, I will stand up because otherwise I'm standing with my back to most people which is not so nice for the build demons. There are two different things to be said or done. The first is a bit hard to get the code really in place if we don't have a suite we could try on so I'm hoping to see first bike sets very soon so that we can try on them. It's much easier to try instead of doing it theoretically. The other is we already have change routes in Debian where we build packages on and only takes a new version when necessary. We, for example, do that for back ports so if you have back ports of experimental we always build from the base suite and just add additional packages which are needed from the specific sources. So we have dynamic sources which gets updated so we have one change route or one tar file which contains a change route for experimental it gets unpacked it gets additional source list added and then it gets uploaded packages get built and then uploaded. So I think we have lots or most things in place however there will be some very interesting things how to get it in the database how to get the data from WannaBuild to the build demons and I think we will deal with that when we have examples to work on. But it would be good, I think, from a WannaBuild site to have one or two of those archives available even if they are created by hand by you so we can play with them. Currently there would be the data archive which we are setting up for data Debian packages and the new debug stuff which would be a play archive. It's a bit different than the actual usage but the bases are the same. Otherwise, yeah, we will tell you as soon as we have something. I think it's important that we really have the same let's say data structure so paths are identical or the variables in the paths are the same place as they will be in the future so that we could get it into WannaBuild. And yeah, about how we do that we will also be in the next interesting discussion. By the way, if someone wants to help there WannaBuild is pearl based so. But both are actually quote. Yeah, we are getting less ugly over time but same historical code you always add and add and add so at some point it gets interesting. Yeah, we have a few more coding things than just the comment files. We also need ways to only rebuild the indices for only those archives and threads that actually got touched. Currently we are just rebuilding everything where it's possible something could have been done which means not the actual release archie not the actual release treats like stable or stable and stuff we don't touch them unless we force it everything else is rebuilt which takes a lot of time and if we add it doesn't know something PPAs that would be bad there's another point where we could need some help if you want to program a bit resounding silence and one question that I got earlier outside here so let me repeat it is the PPAs are just for unstable or experimental no they are not you can base them on any existing suite in our main archive so you can have something for all stable for stable for whatever and so you could do your own back port suite for your own package and have unstable blah all stable blah and whatever have the libraries put in and then just build it all against that. Is that recursive? Can you do bike shed on bike shed? I hope not. If we define Sure just makes I don't see big problem technically from it because in the creation area you just say take from that so we could actually make it free to take from wherever. I think the interesting question is in that case how to configure the sources line because then you could have basically a more or less indefinite list of options so basically which package version do you take I mean we have working code for apt of aptitude to decide with two different sources that takes a best package from so if you want to make it indefinite or at least more than two versions I'm happy to take in apt code some of those writes. So I think there was a question at the back there I think there's a microphone behind you oh no that's the audience one So the main stuff hit also the murals then the public ones, the official ones or will it be hosted somewhere else on Debian infrastructure The plan was to put it on a different network and outside because many murals are already loaded with our 1.5 terabytes which we currently have So when PPO main is created you're saying based on unstable does that mean this unstable state at the moment of creation is the basis of the PPA or the new uploads to unstable will get reflected in that PPA bike shed I think we can just make that an option it would be an option that you can have your packages that you have inside updated whenever they are updated and unstable have it as an option in the sweet table and just say yeah I want that and then whenever a new library whatever is uploaded you get it in your and then you need to see if your stuff is still working if you don't want that you say I don't want that because I want the state of it as of now Ansgar I want to come here What about binningham use is there anything planned to allow regular developers to binningham packages in their own PPAs is that possible at all or will we have to do no change source uploads I will first I would like to go a bit back on what I wanted to say about your comment about automatically building the packages because I think at least at the current moment we don't have any code in place which could do the same in unstable which I'm quite pretty sure a few teams would really prefer it if we only build but automatically build the right packages again if something has changed I think about the the greatest load of some of the teams the other question is well let it put it this way currently we control the possibility to write to wanna build basically by per architecture permissions however if you have right access you have right access to all which means you could see for example what is currently going on in security streets so we don't want to have that for everybody I think for good reasons I could imagine to go via comments that we got from DAX that we got some mill somewhere and then we could work on that so that would however be a later stage so I don't think we start the set for the beginning because I'm happy to get features implemented step by step and not wait until everything is completed I see that Ein has a question for some time oh ok someone is hi the question would be how do you do do PPAs expire at some point or will they be forever on the ftp master or whatever servers located I think we should offer a way that you can say I need that PPA for so much time frame and then it goes away automatically or you can renew it otherwise PPAs that are unused for time x whatever you may get a warning and then it's cleaned up we cannot keep them forever we don't have the space for it after a while it will get big ok Ein I think I saw you so are you using or expecting people to use the same are you expecting people to use the same version number namespaces in the main archive or are people expecting so will it be expected to upload a version to the PPA you might want to upload the same version to the main archive with different with different actual files you currently cannot upload the same version that would be rejected if you want the version from the PPA to go over that would be the copy way get my contents of the PPA back over duck does not allow two times the same version in different suites by upload and I actually think it's really a good feature we should keep because it would make things much more complicated if you have bugged the ports with the same version means with the different things we shouldn't do that I think if we would go and we with a really big mail we have something oh no not now it's really big we have something written about the versioning with the PPAs and have some way how we could do it already written inside but it's a bit big in total so pulling in yet another team is Weasel around are we going to also hook this into Snapshot I've talked to him later and I think I think he said that's a possibility to put it into Snapshot and as he is planning currently to possibly put the debug archive into Snapshot too and doesn't think that's a big issue so I don't think PPAs are an issue in sizes because the debug archive is probably getting much bigger and also while I've still got the mic before I pass it on which architectures are we thinking about building for? the source that we have the PPAs will support all those that we have and the billing part is over here but I don't see it should be in the priority lower than the main archive but whenever there is some resources free I think it should be able to build I fully agree on that I think we need to perhaps discuss how many machines we need Exactly, we're going to have to massively build all power if this is going to take a while In that case the main archive should be the absolute priority before something else is coming Well actually I've concerned to build this PPA on the same build elements and the one using for the archive because we control a bit less what's going to be uploaded there and it's a way that we can take control of the build elements Now actually for PPA mains we are controlling exactly what's uploaded there because it's only that which already went through new and all of those people that have keys in the key ring PPA X we will never put onto the normal infrastructure so the external repositories where other people are taking the liability and stuff they are not going on onto the normal autobilder network or anything normal but the PPA mains are basically what you have in the main archive looked at by the FTP masters and stuff just different versions or different ways of combining and giving it out to people I think the concern is that you can upload when you have a stolen key you can upload malicious packages to a PPA and it is less likely to get noticed but it can take over the build d-demon and inject code into packages that end up in unstable so that could be avoided if you use throw away VMs for building but we don't do that currently and not all architects are supported I think and also of course if you have the right to acknowledge nobody says that VMs are not as you can't just care for them if PPA X is not going to be on any of the standard infrastructure which I agree with will it also have the version number restrictions I think we lowered them I would need to review that in the part Hey, we made it easy we don't say anything about versions for PPA X so we can define it from whatever I would really try to encourage you to keep the restrictions the same that we don't have double version numbers at least that we can't have something on PPA X which is in Debian the other way around I'm just thinking about whether we are testing a text path to it but yeah is this also meant to be used as a general software distribution point where up streams could build their packages like could I move my Postgres packages there if you are a Debian developer then you can move them over there yes if you provide a service for Debian users with Debian packages with yours of them then I don't see a big problem moving them there and build a tour of Postgres and XFCE whatever different versions like doing your own baked parts of them and have them on the main infrastructure that's connected to the question how long the repositories are going to be there for that we would want to have it indefinitely and the question is if you want to use it as a development tool or as a release tool at the least it will die whenever we put the old suite over on archive all the PPS based on that will go away so whenever we remove old old whatever stable and put it on archive then the PPS based on that will go away too also to archive can do I think as long as DSA is giving us a space then we can put that there and for that ask him to just buy more disk so while the mic is running around we've also got some other updates that we'll need to do obviously like the BTS will need to know potentially about PPS and there's like the tracking dot deviant dot org and a few of our other bits of infrastructure around so we're getting there quite quite well with some of this but for it to essentially have the same sort of experience that everyone's used to when they deal with their packages and all the extra infrastructure around is still something that we're doing so when we say we want to the versioning sensible which makes sense are we just talking about source versions are we talking about binary versions as well the use case being for example so if you've got a source package providing an alternate version of the binaries in deviant for example across compilers at the moment if you build it the other way you get the same binary packages from a different source and I just wonder whether that would work in a PPA main or not because that would be a sensible place to build them if it would all explode then we can't do that there so how that currently works is that if you have a file with a given name it has to be identical and we cannot change that as we want to be able to copy files from PPA to the main archive and we clearly don't want them to differ in that case okay so it would have to be synchronised somehow to be different so what I'm currently thinking about the version I'm not sure if this is a good idea or not is to enforce that's a bike shit name that's actually part of the deviant version of the package which of course would make sure that we can't have conflict but of course makes it a bit more ugly if we copy that back to unstable but I'm not sure if it really needs that so just as an idea but that doesn't look nice for copying and you still can have conflicts if you for the upstream tarbol because that doesn't include the deviant version yes we could have conflicts for the tarbol but not for the deviant version which is actually built so just as an idea yes thought about that we'll see maybe we'll try to enforce it probably for the external repositories because there's more likely to be conflicts if we have non-debian developers doing whatever they want there so I'm not correct and assuming that this eventually just completely removes the utility of backports I'm not standing up, sorry well backports is centrally maintained and you have a central policy behind it but yes in theory everyone could just go and make their own I have my package, I want a backport for it and I have a suite here for because it seems to me like the usage model on backports is already pretty much that people are picking at the individual package level which seems possibly even finer granularity than the bike shed level well actually I think it doesn't remove the set of backports because on a lot of servers I have backports included backports out automatically not installed unless you explicitly say I want this from backports and so yes it's quite a very easy way to do it at least take a look at security so I think it's better than individual bike sheds so I agree with Bdale that it could easily be used to implement backports in theory but I definitely think having a single repository serving the function of backports is a useful thing because for most users who are not sitting in this room or of similar background it's a lot easier to follow instructions to install something from backports than it is to add a new repository and it's one command instead of a bunch it's one flag to a command even so I think the ability is worthwhile We have a backports mate and I want to give a comment To some degree the name PPA contains something that backports does to somewhat better because a PPA is something personal and it's usually attached to a certain user and if they are not willing to maintain it anymore or don't have the interest or motivation to continue with it a handover to a different PPA from a different person would mean every user would have to change the sources list entry for it and with backports you just hand over the maintenance of the package and it's still in the same place so there are definitely different use cases for that Depending on how we lay out the UAS you could actually hand it over in Davion Who can upload into a PPA main is defined by the list of fingerprints that are allowed for this PPA main which could be just a whole key ring or a set of a set of them and then you could just say I upload and I take myself out and other people have it and then you have a PPA master someone who has created it and giving that over to another one shouldn't be actually hard to do So I was just thinking back to the comment that you made about how you were worried about the number of bike sheds and how much load this was going to put constructing index files Are you intending that each bike shed is a full index of any suite that it came from even if it's effectively a dynamically updating suite or are you expecting that bike sheds indices will only contain the packages that have been explicitly pushed to that bike shed So I think it should behave like experimental in relation to unstable so it's basically just a set of small packages otherwise we'll just have to keep too much stuff around because I mean then creating a repository means copying the entirety of unstable huge but presumably if you've snapshotted the suite that you're based on then you're going to have to migrate things that change in your base suite you need to migrate the old version into the bike shed before you complete the domination in the base suite It depends on how far you go you should probably look at the dependencies to keep them straight if you have a straight dependency I need that version and nothing else If we have a PPA we can either just ignore that and whoever is using the PPA has lost and the one who set up the PPA needs to make a better list of packages or we could actually be helpful and look at that and then copy the few needed packages over When you create it you have a list of packages anyways that you should give and then we probably can go from the dependencies So now I have a question about you said at the very beginning that if you create a bike shed you say and these people should upload too So is the bike shed owned by someone or does is it not related to an owner or how is it done and also is there other names with policies on the bike sheds I think we had something written there but yeah the one who created is the owner of it master of it whatever and he can also add fingerprints and stuff would be bad if everyone can just add fingerprints everywhere and he can also say there is another one doing master for it but there is only one master at a time so we can't say there are four masters why not just ask him about the implementation I don't see a reason why we shouldn't have multiple masters it's a list of fingerprints like a list of fingerprints we can upload I think it's a good idea it's a namespace policy it will indicate that it is a PPA probably will use PPA-something or similar so PPA- PPA-Tor PPA-Tor-Stable so basically whatever comes after the dash is just after the person who creates it if it's wrong the FTP master will fix it yeah okay any more questions oh yes there is one so I want to support this for DGIT push I think that's probably straightforward talk to you about it later yep anyone else or are we done there's food coming very soon so not too long here possibly off topic but if it's so easy to support PPAs in the BTS can we have back port support in the BTS don't we support it nope talk to the BTS maintenance by the way I would never so no we cannot do it we are not modifying the BTS we are only doing duck it's somehow from the BTS team here now otherwise from the experience I had with playing with BTS time version tracking was added I wouldn't support the statement that it's so easy or easy it could be easier these days so are we expecting then to see support for this well through other sections of the project are we expecting to say we might make CDs or live images or anything with like sheds included is that it would well we are hoping for the black magic that those people maintaining those are deciding if it's actually worse for the parts of the projects they are doing I don't see much use in creating CDs with for example Postgres on them just to have 10,000 CDs with 10,000 different Postgres variants something like that so I think it's up to the different people so it probably makes lots of sense to have said in the tracker Debian org or something and in the BTS of course the tracker might get pretty long CDs are a worse example so we've got about 10 minutes left so Ronda then Andy at the back from the point of view with the BTS I talked with them when it last popped up getting backports into the BTS and the biggest blocking issue was the view of different maintainers for different branches like the person taking care of the backports not being part of the regular maintenance team from unstable and I see the same issue even more an issue for having PPAs in the BTS so it definitely would need to be adjusted to have multiple different maintainers for different branches in the view for the BTS so it sounds like a lot of work for the BTS yeah I want to add a word of caution you seem to be advocating PPAs or bikesheds for a whole lot of users just be cautious that you don't end up fracturing main so much that people wanting package X always pull it from PPAX people wanting package Y always pull it from PPAY and as a result main just gets so fractured so you don't bother to push back in how's that? excellent don't speak for your microphone muted so there's a few different because this is being done by Debian developers as well when there is an understanding that there's a few different scenarios you could use so you could have those aggressive backports type things but you could use it for preparing transitions for example so you'd upload your bits you'd make sure everything builds and then only when you're happy that the entire transition is ready you can then pull it into unstable that way so it should make things a lot easier in that way I think basically we need to trust DDs to do that at the moment they can build their own random stuff on people.debian.org but putting it into the main archive now we will definitely have a few PPAs that are long running and never go back into the main archive but mostly from people I've heard about that already that are maintaining the package in the normal archive anyways they are using those PPAs to provide backports for more versions than it's possible to provide on backports.debian.org in a way and with the current policies so they can always have stable or unstable with the latest version of unstable or something those will never go back into the main suite ever and will split out but the main package will be still in unstable in those cases about the maintenance thing we just discussed before I think whenever someone uploads something to his bike sheds he should of course have the maintenance thing to himself because otherwise it's a wrong person all those maintenance doesn't agree of course maintenance is just uploaded as normal otherwise I think the maintenance should be the set the other remark we had before about the question could we use from B-Dale could we use bike sheds instead of let's say backports I would say from a use perspective I don't think so from a taking perspective I wouldn't wonder if we have one time use the same mechanics behind but that's a totally different question OK just a couple more questions now as we got only a few minutes left what keys are going to be used to sign bike sheds particularly exed bike sheds the exed one we haven't decided at all on it's probably a new key or something for that for the main stuff as we are creating it from the archive I don't see much of a problem doing the main key or worst case we could create another key that is just signing the PPAs so I was wondering about this transition scenario if you're having multiple transitions at the same time you need to NMU stuff to pass that transition it's going to be in name conflict I guess I mean the archive apparently doesn't allow it so you're going to rebuild it in one PPA and you're going to rebuild it in the other one how you're going to merge that you're copying one transition into the archive and then the other transition has to rebuild again so that's what happens when transitions get entangled basically please don't do that talk to the release team so if you're planning some of the transitions and you think there might be a solved problem in fact if you're planning a transition go talk to the release team basically so I think that how does the PPA help exactly why if you what you now say you can just do it in the main archive so basically means that if there's a if there's one of the libraries that's changing in an incompatible way in the PPA you can make sure that all the reverse dependencies are all managed together and you don't end up with things that are uninstallable at some times and some not not another it provides a stable place to try and manage that transition basically when you do transitions it's often but not always the case that some things are changing and then you want to see does it still work with the new change stuff so there it will help of course it doesn't directly help as a build time is unstable and one thing which definitely will happen with bike shits with bike shits we will compile packages and not use them because way more packages which are compiled never used than we have with unstable experimental I mean that's obvious side effect if you don't want that we can't do the bike shits we have one more question and then it's time out normally most time of the day I'm an ordinary Debian user and sometimes I have other people to use Debian and when I heard about Debian PPR I say oh no and after that talk I will sleep very better no more nightmares anymore and I think the time between the first idea and now I think it was used very well one final question for me when do we get them then pretty soon though we have all the comments ready we could actually create PPS by using completely manual comments but it doesn't help if FTP does everything do everything manual so we need the dark comments interface to implement them and then we need the policies on removing them and stuff so it's writing a few tools to implement the dark comments files to read them so we got literally one minute left this is better be about 10 seconds did you actually study the technical problems problems from the one build side if you are going to say it's soonish ready if it's soonish ready it's soonish ready from FTP master side after that comes the one build and build implementation so I bet it's sure we could have another talk in two years we cannot talk for the one build people they of course need to do their work but they can only start their work when we are done so we are real soonish now and then the one build people will have fun and probably yell a bit at us and then we will yell back and at some point we have it sorted all out whoever wants that to be last help is always accepted and so we are out of time as I have the wonderful video team waving at me furiously at the back just a huge thanks to FTP masters and everyone else has been involved with this it's really good to see that this is something that can finally come along so thank you very much