 I think you should have said it a lot earlier. Good morning. And welcome to this presentation of the DRI, which stands for Debian Don't Review Yourself. It's a tool I made for semi-assisted automatic Debian packaging, which sounds really cool. The idea is that Debian maintainers should focus on not on paperwork, but on actual maintenance, talking with upstream, fixing bugs, replying to people on bug reports, and that sort of thing. Deb Helper did that for Debian rules, in that Debian rules is just do what I mean, accept, do what's usually done, except for these two overrides because there's something wrong in upstream packages that does not cover everything. And the idea is to do exactly the same, but for the whole Debian directory. So the DRI is the Don't Repeat Yourself principle, which is the opposite of wet, which is write everything twice. And when I package Debian packages, I do write everything twice, especially because I'm off in upstream. And so I need to put dependencies in setup.py and then in Debian control, I put a description in setup.py and then in Debian control, website in both. So yeah, I got bored. And so I wrote that DRI, which works like this. So you maintain the Debian directory as usual, but you only write the important things on it, like package name or change log or copyright file, unless you maintain Askel packages, in which case you don't have them, because they can write their own copyright file. So it takes what you written and moves it out of the way. It runs some automatic Debianization tool, depending on what's the source package written on. If there's a setup.py file, it will run Python STDep. If there's Ruby, Gem, things, it will run DH make Ruby and same with Perl and so on. And then so you get a standard baseline packaging. Some things are obviously wrong, like the maintainer name unless you are upstream. So it takes what you've written in your original Debian thing and puts it on top to make the final source package. And the original Debian things are saved in a sub directory, so you can undo the process, because otherwise, then you need to re-simplify the package every time. I looked around at what tools are available for each package. Actually, I know Haskell. It's ready supported. If there's anyone at the conference that knows DH make Perl and NPM to Debian or that knows tools for some other kind of upstream code, we should meet for a moment. And I need to write two or three lines of Python to add support for it. So yeah, that's the idea. It would be nice. Last year, at last year's DebConf, one of the things that came out in the unofficial lightning talkbuff is that one thing that a Debian developer could do and would be really nice is to have dinner with upstream once a year. And that, I think, should be way more important than copying the home page from setup.py to Debian control. So I would like that Debian maintenance goes towards actual interaction with upstream, actual interaction with users, taking responsibility for the package in Debian, testing, all that sort of things. And I know from my experience that when I maintain a package, I end up spending way more time than I want in, oh my god, this weak Python switch from PySupport to PyCentral. No, actually, it's both. But which should I choose? I don't know. Depends who I ask. But now it's PyBuild. Don't to, but then PyBuild is coming. And then there's a policy change that means I can wrap my fields. And there's a lot of bookkeeping. So I would like to focus on the things on top rather than the bookkeeping. And also, the situation with upstreams has changed since Debian packaging was introduced, which was about 20 years ago. Upstreams are now quite, many upstreams are metadata rich, so especially for the new programming languages like Perl and from that point on, there's a lot of metadata that we can use in upstream packages. And yeah, it would be good to make sense of it. There's a lot of cases in which making a Debian package is essentially auto-debianizing and then changing the maintainer field and writing the copyright file. So yes, that's what all Debian work should do for those packages, which also means that we could think of being able to maintain a lot of trivial packages so that, I don't know, if this could be applied to browser plugins, it means that we wouldn't need to go to some full bar market for whatever full bar browser that we have, but we could package in Debian the plugins and have them with our own cryptographical signature verification and everything. But obviously, if it's trivial to package such a thing, then it can be done. If it's not trivial, then it won't be done. And I still have calibre that wants to download plugins probably over HTTP and run it on my system, or VIM plugins, which usually get pulled from GitHub from random branches maintained by random people over HTTP and run as my user whenever I run an editor, which is always. So yeah, let's make it simple to maintain things so that we can maintain more things. And we have Debian patches. So if upstream got packaging wrong, we can patch upstream and send the patch to upstream and give back to the free software community. So that dry works on the assumption that upstream is well-made. OK, I hear giggles, but we can help upstream being well-made, which is, I think, a worthwhile goal. And if upstream is not cooperative, we can patch. And if we patch, then we document what upstream is doing wrong. And we give it back. And also trivial things should not be outcoded. The Belper 7 made it right. We need to add a compile switch to all Debian packages. Then we just implement it in the Belper and VH auto build. We'll know how to add it, pass it to dot slash, configure, cmake, and what have you. So it's a bit of a proof of concept, more like to show the idea. But it kind of works. I did not want to introduce a new syntax. I was not interested in creating the Debian spec file, although it would have been fun. But it's like just write Debian slash things with the syntax we already know only omit the obvious. It's not about handling corner cases. If anything is out of the ordinary, we've been having a way to deal with that for 20 years, and that's still good. But yeah, the point is to weed away the routine. I'll show you how it works. So this is Debian directory of Debian and that's Debian control, which whatever. OK. That's Debian control. Seems a little redundant. I have missed the pens and Python events. We know it's a Python package. I know. I have not implemented partial merging of the pen lines. I'd like to. But that is introducing new syntax. So I would like to be able to say the pens colon dots comma this. But those dots is new syntax. And I'd like to have some peer review before that. Yeah. Seems like the source and package lines could potentially be inferred as well from the fact that you're in a directory named DevDry. You could override them if you need to change the name of the package. But if the source package could default to source package and source package could default to, while I'm in a directory, food-version number. Agreed. At the moment, I'm having source and package there because they are identifiers for the Debian control standards. And if I remove this package DevDry, then it's not obvious to which binary package it applies. But then we only have one. So we could be smarter. Yeah. Martin, Joey, Gunnar? Just quickly, please don't rely on the directory. The current directory would be called DevDry minus 1.0. And then infer from that. Because if you check it out from gate, if you check it out from somewhere else, you might not want to have that sort of parent information. Yeah, but I can rely on this. So upstream has the package name. You don't actually need new syntax for the depends. What you can do is introduce a new control file field like x, DevDry depends. You can put the Python 3 app in there. And then it's automatically pulled in. Along with this depends. And Python depends. As long as you don't mind those being always added. I like that. Or the other hand, going back to this control you were showing, well, if DevDry understands semantically what is with each thing, source packages build depend. Binary packages depend. So finding the depends line means it's in the package field. Sorry, yeah. Yeah. The source part of it is the only part that build depends. The binary part of it never build depends. It only depends. Good point. Yes. Nice. And I like the x DevDry to add. There could also be some x DevDry remove things to remove fields. But it goes into a territory I'm not comfortable with, because it tends to start to introduce more abstraction. On the topic of dependencies, if you were to come up with a reasonable proposal in some namespace that was meaningful to upstreams, for example, package config files or something like this, I would love to put that information in my packages, speaking as an upstream maintainer. We have dope files, for example. It's an XML format. Maybe you could come up with a reasonable way to extend that. And I'd be very happy to give that information to you. Oh, yeah. I would like DOAP upstream, whatever else, whatever other kinds of metadata upstream has to be brought in. Although I think that would be a job for the auto debianization tools, rather than DevDry, because those are the ones that give the baseline Debian. But what I would like to happen is, since this gives more dignity to the auto debianization tools, then auto debianization tools starts to have a motivation to be smarter, which I think is a good thing. For example, the DH make Ruby is a bit insecure. It will get the version control system headers correctly. But it will comment them because it would like you to double check them. And I understand that that makes sense in a system where you generate a baseline for somebody to edit by hand. But we can be bolder in this case, because there's a way to override them if they are detected wrong, or patched upstream if they are wrong upstream. Anyway, let's run DebDry. And we have DebianSource. This DebianSource option is something that I would regularly forget to add to my packages. So yeah. And if this gets uploaded, there's also a Debian.DebDry directory, which I get the source package from Debian, run debdry-r, and get the Debian maintainer maintained simplified Debian directory back. And I can work on this. So essentially what DebDry is is also what DebDry is. It's an auto-detector of auto-debianization tools. So it looks at the upstream directory. If it finds any file.gamespec.gen file, then it's a Ruby. And that's how it's auto-debianized, simple enough. There's a proposal when Joachim yesterday pointed out that it would be nice to add a Debian slash DebDry as a configuration file where one could say auto-debianize with this command. So one could pass extra options to the auto-debianizer, which at least in the case of Cabal-Debian for Haskell, makes a lot of sense. Because it does support a lot of things. In the case of Python ST-Deb, it's quite nice because there's an ST-Deb.cfg, which could be included upstream, that customizes the auto-debianization, which is an interesting idea. A lot of Debianization could actually happen in the auto-debianizer configuration file. So that ST-Deb CFG could actually be the way Python packages are Debianized and the Debian directory is essentially a changelog, microphone. So the changelog seems like another example of write everything twice, especially if you're upstream. Could you just auto-generate the changelog directly from, for example, a git changelog or worst case scenario, a changelog file from upstream? And then you don't need to write that either. And ideally, upstream changelog is different than Debian changelog. Less so if you're upstream and doing most of your work in upstream. But I'm more thinking of the git history of the Debian directory, for example, where that should tell you everything you've changed. And putting it in a file is just automatic. The way I work is the other way round. I write the changelog in Debian changelog, and then I use that comment, which will. But yeah, definitely. Good point. There's already tools that do that. Build package comes with a git dhc command, which will do exactly, basically exactly that. Use your git commits and generate the changelog file from them. So you could integrate that into something like this if you want. Yeah, the one of the problems that you run into is that those are two files that have different formatting conventions. So there's kind of a way that we all sort of write changelogs, particularly if we read Aplis changes output and tend to copy the way everyone else writes changelogs. And then there's a convention for how you write git commits. And those two things are not as compatible as you would like, mostly because the changelog file is intermediate in descriptiveness between the first line of the git commit on message, which is usually not enough information, and the entire git commit message, which is usually too much information. So it's this weird intermediate thing, which has always been my problem with automating and creating the changelog entries. One really quick comment, though, on something else. I spend about, out of the total effort of 10 to create a new Debian package, I usually spend about one or two of that on most of the Debian directory and eight of that on the copyright file. So that's the hardest problem here, I think, for a lot of, I mean, for simple packages where everything's under the same license, then you can hopefully pull that out of the upstream metadata, create a very simple copyright file, and move on with your life. Unfortunately, I find that that's often not the case, even if upstream thinks that's the case. There's some random files under some other license that's compatible or something. One of the things that I'm wondering is if we could, I'd been wondering this for a while, and it's sort of ancillary to this, but it would help this process a lot, is if we could start getting upstream to adopt copyright format 1.0, and actually put that in their own license files, then it becomes really, really easy to write copyright files. And if we could get, for example, the FSF to do that as the upstream that cares the most about this stuff already, that would drastically increase the number of things where we could completely automatically create a copyright file. As an upstream, if somebody told me that I could write one Debian copyright file, Debian-style copyright file, and not put license headers in my files, I would do it now for everything. Can I? Is it kind of legally OK and stuff? It's legal in my opinion. I've done it. Ah! Then we may have an innovation here. What I was going to say is, I think the obvious question here is, what happens if your package is being an emu'd or something, and somebody goes in and edits your control file or something, the auto-generated one? Then that being an emu would work well. It'll work well, but then you have to go manually read. Yeah. I can't think. Well, one would review the emu patch, and. You might want to put something in the files where you can put it that says this file was generated by DevDry. You might want to go edit the .devdry file instead. On every single file. Like a comment in the control file, if there is a way to do that. Right. But then the Debian combat. Obviously, the other files you can't. Yeah. Readme.source, like readme.devdry, yeah. It's actually convenient the microphone came to me, because I didn't have a question. From the point of view of reproducibility, are you recording the versions of DevDry and the auto-debianization tools that you're using when you run it? So I think that beyond just recording the version, I think if we think about what happens with DevDry, suddenly these people who are maintaining various things that do automatic devianization have a new problem, it's basically the same problem that I've faced with DH and complained about the last DevConf, which is any change that I make can go off and break some random package. Now, it's only going to break it when, in your case, when a maintainer is running DevDry to update their package. But it's still, I think, something that they might have to start worrying about. They have to formalize their interface more or their output to make it more stable. I completely agree. And I don't think it's a bad thing. I would like them to do the work for me. That's kind of the point. And for everybody else in Devian, so possibly getting help from other Devian developers as well. Just not me, yeah. You showed how you're running today where you just auto-detect which Debianizer tool you should run and run it. How do you think DevDry should handle cases where you need to run four different helper tools because you have different pieces of your package? And what about the common elements that they currently all tend to handle themselves? Do you need a pluggable system similar to DHS, drop something in and handle that? No, I would think such a situation maintains Devian directors by hand. Please don't. I'm mostly thinking of things like, well, I need to run this tool because it's a Python package and this tool to auto-generate the copyright file and this tool to auto-generate the change log. So more like a bunch of helper tools and a core Debianizer. Then I would say let's change Python as DDEB so that it runs fair enough. So I think you had earlier mentioned maybe allowing people who use DevDry to have a config file that specifies what program runs or what options are passed to it. You could always extend that to having it be some kind of a little program that uses the DevDry as a library and then says, oh, run this here, run that there, whatever, to put it together. And I think this is kind of a core tension with what you're doing is that you're using declarative files but you also want the escape patch of being able to run arbitrary code. Well, not too arbitrary code, but useful code just as we have in the rules file. So maybe you should think about that and I'll also put in a plug for my talk tomorrow which is gonna touch on this in a slightly more abstract way. Okay, I'm not sure I can visualize the problem you have in mind because so far I intentionally limited myself to trivial packaging and everything that is non-trivial bail out to maintain Debian directories by hand. Yeah, and at that point, yeah, I'll look at what's the difference from 70 to 90, see what can be added. Luca? Yeah, have you been testing upgrading packages from DH, make Ruby version 0.6 to DH, make Ruby version 0.9 or ZBAs? That would be really, I think that's the main use case that I see for this tool. It is, we tend to have BitRot in our Debian slash directory and it would be really useful to have a way to automatically upgrade packages in a reliable way. For example, we are working on adding automated auto package test and we need to go through every package for it. If we had such a tool to help with that, it's really nice. I have not tested anything like that. My general intention is that this would hopefully move from a hacky patchy thing to something that essentially says the maintainer's this person, not upstream, the rest is fine. It's kind of a more abstract standard and at that point, migrations should kind of be ideally trivial because I'm representing what I mean and there are tools that implemented in what is run on the system. Only I've tried to not create tools that are too smart introducing new layers of abstraction but just ditching bits of Debianization that are usually redundant. That reminds me of something that might happen. For example, you start adding auto package tools to the output. Then as a maintainer of the, what was it Ruby package? I don't change anything, but I get a different packaging. So what do I put in a change lock? Do I put in, didn't do anything but lots of things have changed or do I review the diff every time and manually figure out what has changed to put in there? Or do we expect maybe the automation tool to have some kind of change lock fragments and then you record what was the previous version of the automation tool that was recording it and you pick all the additional points that was added there added to the change lock so that you can basically get the effective changes there. Open question mark, I guess it's, don't need to keep in mind. If I have to think about what I would do right now today and like making a new, thinking of a new version of that dry is about to be released, I would look at the depth diff and figure out what happened because ultimately I am responsible for what goes in the archive and not the, it's me responsible for what I upload and not the upstream of Python STDep. So it's something that at the moment I would still feel like I need to do. The other, only it will be easier for me to review, A, this change looks all right and maybe add a few lines to that in change lock rather than having to figure out what are the trivial changes that are needed this week. But yeah, the responsibility is still on the developer. If tomorrow or in a few months, we can move to a model in which like happened with the developer, there's tools to do the right thing and I don't need to double check the sequence of commands, sequence by the developer, then yay, it's another step. Did I see how? Yeah, actually just have a thought about this, is that, so what you are doing is you are using the output of auto-debianization tools as a basis. I wonder if that's not a bit backward and if the right thing to do would not have been to have some kind of preceding for auto-debianization tools like a config file for DHMAC Ruby that makes it output what you really want in the end and by default it would put a package that is broken but if you configure it correctly, it just works. Yeah, a bit like the STDep CFG. In that case, you will have an empty Debian directory or maybe with a change lock and then some configuration there. Yeah, but still, I think the main problem currently is that each Debianization tool does different things like the Ruby one, generate stuff for tests. I suspect that each one is doing something different in that regard. Yeah, it seems that if right now we could have most of the Debianization happen in a five line script language specific to control the language specific Debianization tool and that gives us empty Debian directories, I would like under sign that plan any day. It would be a gigantic step forward. If further down the line, we end up with a single kind of control script for auto-debianization tools, then yay, even better. I think that's already quite a nice step forward. Yeah, there's another question. While the microphone moves, there's a readme file which is actually documentation for that dry including the description of the merging process. So it has some documentation which is unfortunately not auto-generated. Have you thought much about how this might work with packages that are of the package name dash X dot Y variety that or Emacs 24 versus Emacs 23? For me anyway, there are a lot of the files that I automatically generate from essentially foo.in files. I have a lot of Emacs XY whatever files that get automatically transformed into the right major version whenever that changes upstream. I don't know, I mean, for me, I just treat it as a pre-processing step which would be a little cleaner if we had a pre-processing step more formally, but I didn't know how that might fit in with what you're talking about, if at all. Right, okay, the Debian slash foo.in files, I would not know how to, I've never used them and I would not know much about the actual use cases to do much planning on it. It probably makes sense to add the pre-hook to that drive so that there's one command to run the whole thing. Yeah, but I don't know. Yeah, more generally, I think that the common use case when modifying packages that because package name, end codes, version, SO name or something that you have to modify it at many different points in the packaging scripts and having something that automates this would be really nice. For example, you also have it sometimes in a list of files and I agree that yeah, you have the STD type thing but that's not everything. I feel like taking up this slide again. My point was that I should do less work, not more, but those are all great ideas for tools that we should have in Debian. Just don't expect me to write them because I'm busy maintaining an MWN org and WN contributors and a bunch of other things. But yes, I like this discussion. I like the fact that we are discussing ways of making Debian maintenance more focused on actual maintenance and not trivialities. Great, I'm really happy about that. Let's keep it. Speaking about the pain of writing copyright files, what if I have a perfect upstream so I don't have to change anything in the Debian? Am I allowed to skip adding a files Debian slash star stanza to the copyright file which I find mostly useless? A Debian slash what? In the copyright file, often you find the files colon Debian slash star and then this is me, but I have the same license as upstream or some other thing. If I don't have to write anything in Debian anymore, there's nothing I can copyright, so I can just skip that, can I? I mean, the change lock is not copyrightable if you see that there's names anyway. I would hope. So in the new copyright format, you have globs which apply and essentially override one another. So if in the end, we're gonna have a Debian copyright file which has a stanza which is file star, it will cover Debian stuff anyhow. Right, but there's nothing in Debian that... What I'm saying is that hopefully at the end, there's nothing in Debian that is copyrightable so we can stop worrying about it. And yeah, okay. Well, and there's also the default seems to be that the Debianization license is the same as upstream. So if upstream has a copyright file that we can use as Debian copyright file, then we can also trivially add a line to extend it to Debian if needed. One thing that could help with this is Debian 11 which enables us to map between upstream package names and Debian package names. So you could tell that install Python module foo and it would install Python three dash foo, for example. Another thing is that tomorrow there's the upstream guide bof. So it would be helpful if you could come to that and tell us about what Deb driver lies on in upstream so we can recommend that to our upstreams. I don't know if that conflicts with another talk of mine but otherwise I can do my best to be there. The remaining, well, if there's no copyrightable material in Debian at all, then obviously you don't need to add anything in Debian copyright but even if there is copyrightable material in the Debian directory, one thing that I've wanted to finish doing for a while is just to automate adding the statement about that because if you're happy for the Debian packaging to be under the same license as upstream which most people are, then the only thing that's left that would go into the copyright file would be the copyright statements for the people who worked on it and whether or not you really care about that is arguable but the thing about that, of course, right is that we can generate that automatically because we have that information it's in the changelog. So it does all it would require is parse through the changelog, find all the names that are in the changelog, associate them with years, generate a bunch of copyright statements, throw them into Debian copyright and you're done and you could completely automate that process. Okay, that requires also changing the habit of using only first and last name in square brackets in Debian changelog because we do have Debian developers that have the same first and last name. Well, so remember that your goal here is to generate copyright notices that you may or may not really need anyway. Then being ambiguous between multiple people in Debian is now we're into problems that I don't think anyone's ever really gonna care about. Anyway, I've already submitted a wishlist bug against Debian change. I will promise that my next upload to new you will contain the line. The contents of the Debian directory is copyrighted by the people mentioned in the Debian changelog and if it goes through new you, then I'll let you know. Because I don't see any point in copying the data out of one place in Debian to another place in Debian just for the sake of nobody using it at all. Did you think how to maintain a package like that in VCS because what do you keep in the VCS? The clean directory or the generated one? Because if you have to create the generated one before building then git build package will complain because you have uncommitted change and I think all other VCS helpers will complain as well. I agree. When building Devdry at the moment, like when playing with it, I did hit that and I created like a branching it where I committed the Devdry generated thing so I could run git build package on it to get the source package and it's annoying. Or it could be automated by Devdry, I don't know. I'm not the best in thinking about package maintenance workflows. I would welcome somebody to fix that problem for me. So this sounds kind of like the old auto generated files problem all over again except we're inflicting on ourselves. And I wonder if you've thought, well, maybe if we get 90% of the archive using Devdry, would it then make sense to say de-package dev should go run Devdry in using this interface if it is used and then we can just throw away all the auto generated stuff as far as the source package is concerned? I believe, yeah, that's entirely among the scope of possible future developments that this goes. I'm being told it's minus one seconds to the end of the talk. So my only concern on that one would be that now to unpack an arbitrary source package, I need all the auto generation tools because I have no idea what's gonna happen until I try to unpack it. The. Okay. Well, I guess time to wrap it up. So further development, slides, there you have it. I hope I don't need to explain it. And thank you very much. I hope this kind of direction will keep being pursued after the end of this talk because I'm excited that we can even talk about something like this now. Thank you very much.