 Thank you. So welcome to the derivatives buff everybody This is mainly going to be an open discussion And I'll start with a few minutes describing Devin's initiatives towards our derivatives So that means the basis for a lot of distributions Notably a bunch of and some other ones. I'm sure everyone here knows much of those So, yeah, we'll share some experiences and that sort of thing during the buff So the agenda Initially, I thought this was going to be in more of a buff room around a table like And so we could do some introductions, but maybe people can do that when they speak so Okay Okay, cool. So I'm Paul and I initiated the Devin derivative census and Have been working on derivatives for a while. I mean Integrating Devin and derivatives I guess I'm representing him Debian mostly and in Leonardo as far as it's a distro Which it's trying not to be but not doing very good job on the whole Sure, I'm Colin representing Ubuntu. Hello, I'm Raphael Erzog and I'm representing Kali as well Flogen Haftmann. I'm actually representing the city administration of Munich here who runs on 14,000 workstation Ubuntu like operating system and On the technical back and so to speak we use a lot of Debian infrastructure Anyone else? Okay, I'll introduce Derbians Work around derivatives So this is the derivatives wiki page and there's a bunch of links at the bottom Our main initiative is the derivatives front desk Which is a place for derivatives to come together and talk to each other and ask questions from Debian people. I See channel mailing lists Yeah It's has been pretty active initially, but it seems to have taken off and Mostly me sending mails about sense the derivative census these days The next initiative we have is the derivatives guidelines how to be a good derivative and The different things that you need to have to create a derivative and How to do some of those some of the things that need to be done? When working on a derivative The next thing the next thing is the derivative census this is a project I initiated a couple of years ago now I think It aims to integrate information about derivatives Back into Debian so that they've been to developers are exposed to the ideas that derivatives can bring to the table So it basically consists of a bunch of wiki pages describing the derivatives and Then we use that information to feed it and we feed that information back into Debian infrastructure So as you can see there's about 30 or 40 derivatives represented there. There are a lot more derivatives out there, but Yeah, I need to do some more outreach So the integration of the information about census from the census so far we have Planet Debian derivatives, which aggregates a bunch of blogs from all the different derivatives in the census and I'm working on Generating patches from all the derivatives against Debian packages and then presenting that information on the package tracking system The patch generation is all done. I just need to work on the display of that information And our other initiative is decks which is aims to Get groups of developers from Debian and from derivatives to specifically push Particular sets of patches back to Debian. It's not particularly active at the moment We need to identify some areas we can work on and do that Yeah So that's Debian's initiatives with distributions. Let's open it for discussion if someone could help take notes of this open discussion that would be good and That's the gobby details if you want to join that. I guess I'll start you you mentioned that Debian derivatives is pretty quiet, which is which is true. I would argue that that's kind of a good thing the from my point of view the the value of Debian derivatives is helping Drivers who are less familiar with with the rest of Debian. I Wouldn't generally direct Ubuntu developers at that if they were trying to figure out what to do I think it's generally more productive for them to go straight to The particular places they need to work if they if they need to work on packages They should be going to the the containers of those packages if they're trying to discuss something that's This release process relevant. They should be going to the release team, etc. So it seems Kind of unnecessary to indirect through Debian derivatives for that Yeah, I can definitely see your point there So one thing I'm interested in from the Ubuntu perspective Ubuntu does have a policy that when when they derive packages Patches are meant to be submitted upstream to Debian either immediately or you know next time the package gets merged or whatever In order to keep the the technical debt from patches low I wonder if if if the the DEX initiative is at all interested in Tracking the status of that although you mentioned the DEX is clearly inactive at the moment But so I don't know what I don't have any visibility on like how many patches get upstreamed How how effective are we getting those into a state that the Debian maintainers will integrate them? Is there a large volume of patches? They're just sitting in the BTS on addressed or How that relationship looks writ large across the distribution because I know at the microscopic level at the individual Package your level. I know there's lots of good relationships, but I don't know how we're doing overall I Seem to remember a discussion recently about this and I think the answer was that most package it most patches get adopted by Debian And I think Lucas there are some stats about that and UDD maybe for the user tags For for Ubuntu where the user tags they are supposed to use I think it's still being used For other Sometimes for other The the thing that we don't have stats on in an organized way is how many of the So we have stats and how many bugs total have been filed ever without user tag There are no stats and how many of those remain open as far as I know. I remember at the At last but one debconf I did a sort of ad hoc search and I think there were something like a thousand Ubuntu patches In the in the BTS that had not been applied I don't know I don't have anything that that's a vague memory and I don't have anything more current than that So but certainly at various points in time We have we have indeed been in situations where we have had very large numbers of patches Outstanding in the BTS with very little activity But these are not all patches So currently when there are some specific topic I prefer to file a bug in Debian for the specific topic like Wookie to set up a user tag for for arm 60 arm 64 and So I file back with these user tags and you cannot file a Bug in Debian with two different user tags So Yes Yeah, yeah, but sorry, but it's 10 years then too and so Yeah, I've done that knowing as well. Just on that. You actually can I think I've done that to myself Really? I thought it did but okay Yeah, yeah, it seems to me that the patching part of this patches stuff is relatively Organized the thing that seems to get stuck is new packages that start in a bun-two And then just kind of sit there for years before somebody and sometimes there's USB creator got it still stuck after about three years of we really ought to move this because Usually it's down to slightly higher stricter standards of Packaging someone saying well, it's not quite good enough We really ought to change the name to something less generic or more generic before I go in and then whoever was enthused goes Oh, well, that sounds difficult. I'd have to change it in launchpad and I can't really be bothered So it doesn't happen. Yeah So one thing I've encouraged the Ubuntu community to do and I haven't haven't been as rigorous about this as I was meant to This is meant to be announced more widely is is from the Ubuntu side We've proposed okay if a package exists in Ubuntu and it should go into Debian. Well, the logical thing to do is to Request an adopter and just use the existing WNPP system and and RFA it because the packaging exists It just needs the maintainer in Debian in most cases. Do people in the room think that's a reasonable approach for that and is that something that Anybody here would actually be interested in adopting those packages. I Think that works fine so long as It's in good enough. You know the packaging is in good enough condition But sometimes it was a bit sort of haphazard and somebody will complain that really it should be done a bit better than that And that becomes a harder problem As I said in my previous talk, we have some connection to some Ubuntu derivative and Debian It's called BioLinux and they are using the common VCS with us and we are We are working the same code. We are just releasing it in Debian and working together And so if you have some In this field, yes, I will adapt it But I don't know if maybe we need some some kind of shared interest between these people So we started with this when we made a common sprint with these people. So if if you have Some kind of common field and whatever sense just let's meet to Ubuntu people and Debian people to some sprint and then I could See increased chances for this My trollish rebuttal who wants to come to the Ubuntu mirror and unity sprint Fair enough So sometimes when there are new packages in Ubuntu I'm explicitly not not going to upload these two to Debian unstable to Debian at all Because I do know that I don't have the time to maintain them and Two examples Open JDK requires work for for every architecture every new architecture. So it's extra work for every architecture To to maintain that in Debian and for open JDK I explicitly use my Ubuntu address to do the maintain maintenance in Debian another example, so so I Wouldn't add a new architecture now to To open JDK In Debian unless I have some somebody with in Debian to maintain it and I think the other thing are the cost compilation environments Which are not yet in Debian and but I think we will have a talk on Thursday or Friday about that Ultimately those packages need maintainers and if they come with maintainers that would be even better True. Yeah, so I have a question For people in the room, maybe I can have a show of hands who reads Planet Debian derivatives Okay Is it because you don't have time or yeah So probably something more useful for me to work on is the patches stuff do people think that's a good thing to have patches from all the derivatives I have a question regarding the source of the packages Some derivative are using testing and also are using unstable Sometime both depending of the long term for Ubuntu for example and I'm a bit concerned that Testing is not a rolling distribution as it should be because people are Quite happy when the package isn't unstable only and some some people are well if it's unstable it's in Ubuntu and then that's okay and They're kind of competition and well, I'm always looking at the empty part of the glass. It's a bit negative, but It might become an issue or if Ubuntu continue to grow So we we do use Generally unstable we have flirted with using testing in the past as a source during Debian freezes It mostly hasn't worked out for us for one reason or another I'm going to go into this in a fair bit more depth in There's a there's a session on I think it's entitled Ubuntu daily quality improvements on Thursday afternoon And if people are interested in that and encourage you to come to that the I I do I share your concerns about the division of The division of labor between testing and unstable in Debian and I think we could do a much better job It's not letting not letting testing language And I think there are things that that have been to contribute to that What about the other distribution do you have some statistics about what source are using was that is that testing so the Census wiki pages lists Which Debian suite that all the derivatives that we have there a little derived from There are some who use old stable some who use Previous versions of old stable some who use stable some who's testing some he is unstable and Some who use some packages from Ubuntu some packages from other places So there's a whole range of different ways you can make a derivative distribution But what I'm interested in if you say people are happy if packages are in unstable and don't call for care for testing Do you think they are actively? working against migration by filing Reese critical box or what what are they doing to do to stop people from going to testing I I haven't seen cases of malicious That's that's not not care. That's not that's not no. No. No, that's not not caring. That's that's caring not to That's kind of different the it's it's more much more common is neglect Just it's not something that scratches your particular it's having things in testing this sort of attitude is very very common It and it's understandable right if if if you're not doing something that relies on testing working and many people many developing developers aren't then Why do you care? And you you kind of should care I I think that I think they should but No, in many cases, you know if you don't sort out your Build failures on some architecture you used to build on or if you're stuck in some big complicated transition or something It can actually be a non-trivial amount of work to get everything sorted out I think it's something that Debian developers in general have a duty to take care of but not every recently But that's just an issue of lazy maintenance. Okay, that is not because testing is not a rolling distribution Yeah, yeah, whatever whatever where to take here, but it it's I had the impression from the contribution that People generally don't care about packages if they are unstable. It's fine so if you want To to to look at current issues you could look at the proposed migration script Or web pages which common maintains so if you look for example for your team in a moon to yeah, but it does give it give a hint on Well, which package currently built and which one doesn't Huh, I didn't want to to a taxi Debian made team No, no, but well No, but but What I do want to say so so the proposed migration Detects these kind of issues a bit earlier then then they are detected in Debian because Debian well only cares once Parities or once they shouldn't migrate or be removed from testing and would do does this check I think now continuously The the checks are there in place in Debian as well the the Brittany reports are published everywhere and Responsible maintainers certainly can be proactive about looking for the status of their packages and knowing what's keeping them from migrating or not Right the the reports are not substantially different between Debian until again I'll go into this much more detail in Thursday, but the There may be there may be some differences that Ubuntu's archive cycle is much quicker than Debian's I mean, I'm interested in what What are there just what other Debian derivatives do by way of continuous integration work? Doing ongoing testing of their Of their tip branch Is is there anything? Do we have any sort of general widespread information not or does anybody else want to want to share the sort of the sort of things that they're doing? Yeah, just to move this away from Ubuntu specifically generalize a little bit more I was well, we actually wanted to say pabs well done for all this work, right? It may seem like a bit of an uphill struggle sometimes and nobody ever Nobody ever says anything and you ask a lot of questions and and so on But actually I think it is useful important and and as I said the benefit, you know Isn't obvious because in fact it is mostly done directly with in the right place by people with some clue and so So I mean, I guess part of the problem is that a lot of these things are worn off Kind of they get done once and then they just kind of sit there So some of them there's not surprising that we're not getting much response back They did the thing and it's done and it just sits there and as you say it's based on old stable There aren't that many active things I mean and M Debian has spent the last ten years Integrating everything we did separately back into Debian So actually there's hardly anything left to do, you know, we don't really do integration anymore Nearly everything we wanted is in Debian apart from the cross-stall chain stuff that doc is having to go at me for because I haven't done it yet is the crush thing where the Repacked the devs is that integrated yet? So in practice the things that were fit too different from Debian We just stopped doing because it was too much work. So really small Debian. We decided wasn't Enough value for the amount of work it was, you know, you're better off doing that in OE to be honest So we just stopped and we kept the things that were easy to do and easy to automate and easy to push back into Debian So Yes, pretty much So I mean there's some separate processing of a subset of Debian about 3,000 packages Which are just de-bloated and they actually now live back in the the main Debian archives So we send it all back again after some sub-processing I think it's probably fair to say that in Debian is more or less morphed into the cross-building efforts in Debian proper Debian cross mailing lists and the huge pile of bugs with user tags for that. Exactly. There's two parts There's there's making cross-building in general work and there's making de-bloated versions of Debian binaries and Both of those have been are increasingly integrated and You know, that was all working reasonably well, but yeah, you know, we don't do anything on the decks So they might have filled a page in once, you know, but there isn't really much to report I don't know. I don't blog but Neil does and maybe that appears in the thing. I don't know. Yeah So speaking of Mdebian, I'm curious to know. Is there anybody? Raspbian-ish in the room Raspbian? Paul Green plug wash. I don't know. I don't think so sadly Yeah, that would be very interesting because they've done a lot of good work But I think mostly they're just they intend to be a straight rebuild of Debian, which is similar to what the Mdebian Mission is. Yeah, but of only not unstable only testing I think or only stable or something There's a fairly small subset that actually gets done What is the problem that you can't run Mdebian on Raspbian? So this is all about different arm ISA versions of the chipset. So we support arm v5 Well v4t in and v7 with floating point in Debian, but Raspbian, the Raspberry Pi falls squarely between those stools by being v6 with floating point So they could run our v4t old RML image But then they wouldn't get all their shiny floating point and it would be slow Yes, yes, and it's an RMHF Rb6 port is what Raspbian does. So it would be in principle a matter of just adding another architecture to Debian or So yeah, you could add another architect. Well, it's not another architecture. It's a different ISA Architectures are strictly speaking ABI's not ISAs, I mean tricks and sets We've never previously added we've never previously added an architecture for us like a flavor of the architecture So and we talked about this like two or three years ago We could mark packages with which ISA they were built for and then we could have tools do all this sort of magically But we've never done that work in d-package and by the time we got that work done the Raspberry Pi would be obsolete Yeah, but this problem may arise again in other contexts I mean it would have been nice if we decided it was worth the effort three years ago and The Raspbian thing would have been smoother, but at least more easily integrated, but Just to make it shorter and more Likely to be done now. Isn't it correctly understood that what we simple people would call an architecture would be the Would be doable now And we're likely to do with things like Raspberry Pi And I know technically it's not the same as an architecture you would you really it is something else, but we could do Like with it with our matef Well, I mean it's all it is a new architecture But that involves multiplying up to the number of bills that we need to support a number of bill failures Debbie and Mutenors here about right It's Right exactly, so it is genuine. It is genuinely a significant amount of work And it's it's not something we should brush off with we could just do a new architecture Raffael, do you think you could say something about Kali and arm stuff? Could I just Yeah, could I so There's no I see there's no there's no reason why You couldn't add a arm 6hf or whatever to to do package but then that would require Raspberry and now to rebuild all that stuff with arm 6hf and we could never we could never sports with that still It wouldn't mean we could support the Raspberry Pi in Debian because You can't you can't even you can't boot it without non-free Okay, because it's really it's a graphics processor with a with an arm on the side So and the graphics processor is completely non-open so It's probably not much to be done to improve this Yeah, it's probably not worth putting too much effort into that particular device And actually I disagree quite strongly with the idea of adding new architecture just for a new isa I don't think we should do that I think we should fix it properly if we're going to support that sort of stuff, right and architecture is an ABI is not an isa Well, and but we could do that you're right it would work. I think but I just think it's a really bad plan right the It is the way we implement architecture in Debian defines And I said I'm in a version of the isa We could just turn the minimum version of the isa down and just start building v6 Probably doesn't gather you never know if you have an old package If you have a random dev, you don't know whether it was built for v7 You need to put that information into the packages So then all the all the all the stuff built for Raspins will work on everything else it works works one way It's compatible. It's like I 386 I 486 I 586 I 666 exactly the same problem Right, I mean I was I was going to point out that we had the exact the same problem on I 36 where I 36 for a while meant I 486 then I 586 and I think today it means I 686, but there was Is it still for it? The kernel may think it's 486 the user space disagrees Yeah, I believe the GCC maintainer disagrees with you about what's supported Well, that means I 386 is it's built for requires 586 Ubuntu's I 386 require 686 Anyway, the point I was meaning to get to there was that this is a problem that we we Failed to deal with throughout the lifetime of 32 bit x86 where There was there was certainly lots of demand for a 66 optimized just based on volume of hardware and the benefits to the platform And we simply never dealt with that and Yeah Sure, but I think the fact that we the fact that at the time it required us to spin up a new architecture And there was a very clear Incentive to do that and we never did it I think is proof that we need a different solution to move forward or we need to you know get a new consensus around Bringing up architectures for that, but I do think that what he's plan is is actually we're way off topic aren't we I Think when we had this discussion in Ubuntu about LP I a so we decided to to have a completely Okay So the alternative which we just did discard at this point of time was just to reuse an Existing architecture and have something in the binary package marked. So this is built for for 686 So at least then DPKG or APT would be able to tell you not to install this package on a system Which just is for 86? So I guess to try to bring this back around with question of derivatives we've kind of gone on this this digression here, which is about Eliminating the need for derivatives in certain cases. So is that a useful thing to talk about how we could make more derivatives into flavors? Is that is that conversations is entering interesting to anybody in this room? I mean Ubuntu is certainly not doing this anytime soon. Yes, I think that's it's definitely interesting and I think When I'm coming from the branch side and Paul to do the derivatives We are working on the same thing from different sides and my plan is to to create all the best precondition to to hand over something to people to doing some support and if it's called derivative or whatever I Think the the best thing would be do them the main preparation inside even because it's the least work for everybody And so this is definitely goal. I'm explicitly to all the time Yes, so you're working inside Debian to help people to create groups of factors. I'm working to bring Debian derivatives existing derivatives closer to Debian and try to Avoid creating new derivatives by improving Debian So that the things that derivatives need are in Debian So who here maintains a package in Debian and Okay, so if this is the Debian derivatives patches page, this is the one for boot chart So you have blank on Ubuntu phoenix tails So a bunch of different distributions are patching it and there are the patches The way this works is that it runs on the snapshot.debian.org site and it gets the Packages from the source packages from the derivatives. It does some fancy matching and tries to find out which package which Debian source package each derivative package was derived from and Then does a derivative and that's where they are It's on I don't know if you can see it, but it's dex.alioth.org slash sensors slash patches Is there some meta information in UDD about this? But this would be cool because then you can easily find this. Yeah, the code that generates it is Not quite ready for Outputting enough metadata for UDD. There is some there, but it's it's not quite enough and also we need some information from the BTS about source package renaming so that we can Present the patches on the correct BTS page Yeah This is great work really but the problem is that it's made on the Debian derivative site So it's not really used by derivatives themselves and not advertised by them So what I would like to see further is that you Help me to integrate this in the pts so that we can have pts Wait a second pts installation for derivatives where we can grab this information directly from the derivatives and Show it on their side as well because as you know the pts revites that we have been working on during this summer We'll make it non-debian specific and I fully intend to install pts for Kali in the next few weeks and use the pts on the Kali side to manage the the difference between Debian and the tech package which needs to be merged again and updated and At the same time we should generate this patch and keep them on the derivative side Of one so that's an interesting idea. It's not one that I thought of because Well, the the goal here is to expose Information about well, this is the goal of the census that I had been working on was to expose information about derivatives to Debian people But the other way is also interesting and I think at least for the patches we could certainly do that the metadata that it produces Has both sides of the equation I'm not sure if Yeah, so this is the metadata that produces show one of the Debian source package that it thinks the package is derived from Debian version names and stuff and Yeah, sometimes the heuristics that it uses to figure out which Which packets pick a package it's derived from and not always the best so you can see in this case that it thinks that Gfx boot themes up to see it which sounds like a theme package for that thing is derived from the Gfx boot themes package, which is possible, but Yeah It's astonishingly difficult to write a gfx, but it seemed entirely from scratch I find it extremely unlikely that they would have done that so it's much more likely that they derived from from that Whether it's structured as a as a patch that could actually be applied to that package I obviously don't know so yeah, the point is that we have the metadata to do what you wanted to do Raphael Yeah, I'll just add that from my my traveling a couple of years ago in India and Indonesia and Discussing with the boss Linux and a few other things in those places I can really see a need and interest from their point of view to see Well as a deviant developer, I look at the quality assurance pages to see How far have others drifted away from us and how could I cherry pick from those and get that into Debian? But in from their perspective from the derivatives from your perspective the derivatives perspective Both us in and as in how how much cooler am I than the Debian? But also as like like a warning sign of well, how much have I drifted away that I need to then maintain myself? As a warning sign of how much should I maybe try to push it back again. So it's a Great way to see this as quality assurance pages for for the derivatives to serve them too lovely idea, but I Would like to Well, I do not agree if you say they are much cooler I think Yeah, I know They just realized Debian as a finished product And I do not notice that they are able to change something and those are only way to change something is Well, I derive and do something else, which is perfectly possible, but it's only in my opinion is the second best way when when you're merging back, I I've tried to do this a few times with well, obviously it's been too little also with patches from things like MIMO and generally The ones that the ones that derive from unstable you have some hope They're likely hopefully to be reasonably current in terms of what the patch base is You have some chance that if you merge some set of patches then You will actually be able to see their delta getting smaller because they'll probably we base at some point But for the the ones based off stable It's it's really very difficult for a Debian antenna to do anything The patch is probably against, you know, 19 upstream versions back Doesn't have doesn't apply anymore. Even if you do apply it your to-do list doesn't go away because the The derivative in question is unlikely to actually re-sync until Debian's next stable release It's it's generally quite an unsatisfying thing as a as a Debian developer to do I find Yeah, but I think it's it's not the first place when people think should I derive from unstable stable the people want to find a solution and they Have no better idea to find this solution than doing this or that and I think it's it's maybe also a matter of documentation Just I think this text is doing a really good job to to explain people that you can change something inside Debian and this is Yeah, I think there could be maybe half of the Derivatives if they people would notice I am I have another question regarding them The depth tree implementation in the patches. We have a applied upstream not needed Or does that work with derivatives? Is there some standardized way to say a that that is about a new logo that is derivative specific and should not be applied upstream or upstream upstream for derivatives Anybody thought about that? Yeah, I think you referring to this standard, but there's no specific information about whether upstream as a Software product or upstream of the distribution So you can't say you should not upstream this to the absolute author, but you should upstream it to Debian There is no distinction I don't think it matters really because if it's specific to the distribution, it's really specific to you That's Rally useful for the absolute distribution, but not to the upstream authors So one thing I wanted to point out to you Colin is in the case of Kali Linux We're based off stable, but the package where we diverge usually it's make more Backport stuff. So we have more recent package which are closer to unstable testing and the The changes we we make are potentially interesting Another thing that I would like to say inside. It's a bit of pity, but often We find problems too late. I mean even in Debian, we relax some of the QA things that we have done in Ubuntu because well, we were already late in the freeze and we Kali at least fix a few of the important problems in Life build Another tool which is called Well, the other one who makes small Simple CDD and tools like that which are not widely used within Debian, but quite used in derivatives and The big there is a way big issue that we are not identified on the Debian side until Derivatives start using it. It's a bit too late So Raphael if you plan to deploy the pts for for other derivatives one thing I really do like is To know when was a package the last time merged from Your main distribution so that that would be merges.wuntu.com for wuntu So it would be good. Maybe to have such a status for for other derivatives as well. So you can see How old is it or how long is it wasn't taken care? So that might be useful Yeah Main Yeah, choose me. Yeah, right. So actually that's an interesting point If if we're saying that one of the concerns about derivatives is when they're basing on on old stable that it means there's It's not rewarding for package maintainers to merge having that kind of statistic about how good the derivative is doing about About actually merging from Debian It Could tend to create a virtual virtuous cycle there in that the maintainers are incentivized to they have the information to know which ones they should focus on which encourages the derivatives to actually keep up with Debian unstable and and yeah So if somebody wanted to do that to make that information available I can see how that would be potentially useful if people were going to use it. So we're reaching the end one minute left Do you want something? I just want to know whether Just brings back slightly more general thing from all this research you've done Are we missing any mechanism which would make it easier to integrate changes? Is there something we need to do to make or is it just a matter of actually integrating the stuff using the existing mechanisms? I'm not really sure how to answer that. I think it's just a matter of integrating the patches And the people as well, you know as a maintainer there's basically only two places I look in Debian, which is I look at incoming bug reports and I look at the PTS and Normally, I only look at bugs and I occasionally look at the PTS So if it's not and if it's not showing up in one of those two places It's basically not gonna rise to the level of getting my attention and getting any action So Eventually the patches at least will be on the PTS so there are some ideas about Directions we could go for this derivative census and integration of derivatives information into Debian so Which areas do you think that I should focus on? These are some ideas here or if you've got any other ideas. I would be happy to hear about them I guess after because we've now finished