 Good evening. The next talk is about Debleon Technical Policy Update by Manoj Fribassava. And if you want to make any question, please be sure to have a microphone in your hand. Thank you. Hi, guys. When I first submitted this talk, I was thinking it was going to be a report on progress and work done. Unfortunately, the last few months, nothing much has been done on the policy side by me. So this talk is going to be where I think policy should be heading. And I want to make it a little bit more interactive than I had wanted it to be initially. And solicit ideas about where people think policy should be heading. So I'll make a few remarks telling you what the state is and where the policy people think we ought to be going. And then I'll open the floor for discussion. So let's talk about people. The last year or so, Russ Albury has been doing great work with the policy. He has essentially taken over all this lack, and he was the one who committed the last release. And he's still the point person on the next update for policy. Policy has always, well, I think the first time I made an appeal to, for more people, was sometime in the 2000 timeframe. So for the past eight years, we have been looking for more people. At various times, there have been about a dozen or 14 people who have been authorized to commit to the policy version control system. As far as I'm aware, only three of us have actually ever committed anything. There's Julian, me and Russ. Part of the problem might be that policy is a bit of a black hole. People come in with the best of interest. They find it is highly unstructured, often fairly contentious, and very thankless, really, for the most part. Russ and I have been trying to improve this. A couple of things I wanted. We have put in the user tags the policy update process, how we move from somebody saying that there is a need for policy to a particular area, to discussion about that, to wording proposed, and finally getting into policy. Streamlined is based on around a Git repository, which is open to everybody. And hopefully, it is less obscure than it was before we did the user tag in the open repository schema. We do need more people, so as soon as I get back to my GPG key, I'll send out another announcement on Debbie and Devil's soliciting help. I acknowledge that we can't just give everybody the right to just commit random rule changes to the policy. That would be far too disruptive for the rest of us. So, maybe we need to have a well-publicized process by which you can become part of the policy team. I thought that perhaps leading discussion in the Debian Policy mailing list, looking at the open reports on policy and trying to bring them to a conclusion and helping triage bugs and answering questions, is going to be a good place for a policy intern to make their name known and getting on to the team. I am also ambushing anybody that I already know and who might show any hint of interest of maybe wanting to work with policy. I already grabbed Zach and had him put in the Debian Policy group by the DSA, so he can't back out now. Any of the rest of you who do want to be in policy, please raise your hands and we'll see what we can do to get you in. So, where do we want to go? One of the things that we have always said about policy is that it should be conservative. Policy is not a place to make design decisions. I don't think anybody wants to have every developer looking up a new policy update checklist every week which changes because the design from last week wasn't quite right and we need a new tech this week only to see a new policy version come out next week that causes all your packages to change again. So, there is something to be said about making policy conservative, so you don't have a new policy every week or even every month. Ideally, I would like to see one policy update a quarter at most. Two policy updates a year is not too bad either. That also helps us define the other parts, characteristics of policy. Policy should be minimal. It's already too long, I think. It's already hard to read partially because it started with Ian Jackson trying to write documentation for de-package. Part of it was thrown out, but part of it wasn't. And then over the last 12 years, various people have come in and added a paragraph here and a paragraph there. And so the policy does not read consistently. The grammar has been, let us say, interestingly diverse. It would be nice if we did a hollow editorial review of policy throughout pieces that don't actually belong there. Make sure that the severities are consistent at least. Make sure that policy doesn't contradict itself. I believe there are still a couple of places where policy says one thing in one place and another thing in a second. And often it is very hard for anybody to find out exactly what policy says about it. Any particular thing without having to go through all of policy to make sure that we haven't missed anything. In order to make policy conservative, I think we might have gone too far, there is a belief that policy should never ever say anything which is not already accepted practice. I think that might be standing in the way of development because sometimes you do need at least a draft policy or a tentative policy in order to bring development to the point where we can consolidate and solidify that policy into the final one. Currently policy has no concept of time. So doing transitions is particularly hard. So if you want to say that today we want to do something but it's not a bug, Lenny plus one it will become a regular bug and Lenny plus two it will be an RC bug. There's no way to state that sentiment in policy the way it is written now. Currently policy just is, you know, you either are compliant or you're not. There's no such concept that policy might change over time even though it actually does change. But we don't codify that change in policy. I would like to change that. So, all right, let me back up and let me say that these are the characteristics which we do not currently have in policy which I believe we need to have in policy. One is the ability to draft policy and bring new policy rules into the main technical policy. Second, it ought to be possible if I'm writing say a X windows package. I ought to be able to say that here is the technical policy please present to me only the rules that would apply to every package and every rule that also applies to X packages but don't give me all the other extraneous stuff about say how to package Python or Ruby on Rails or web applications or libraries. If I want to package a X library then I might want to have general plus X plus library rules. This will make it easier for people to be able to read what part of policy is relevant to their packages as long as we can do this filtering reliably and without false positives or false negatives. Actually, false negatives are worse. We should also and this is meant for things like derived CDDs and derivatives. Right now, Debian technical policy is one giant monolith. It's not easy to slice and dice and it's not easy to add rules or subtract rules. Debian edu or Debian med might want to add additional rules that they want to check their packages against. Not being familiar with CDDs, I'm not really sure what these rules could be but I can readily imagine that there could be additional rules that are relevant only for the CDD. Ubuntu might want not to have certain rules that we follow in Debian. They might want to put in something else. Right now the only way they can do it is patch the technical policy. We certainly don't give any of our derivatives any help in slicing and dicing our policy. The release team might also want to review policy based on the severity or the priority of the rules. The release team might want to make sure that everything that is a must directive actually is consistent with the release policy. There is no easy way to do that right now. You have to essentially read all of policy in order to do this. I've also noticed that especially people who are new to Debian tend to treat the policy document as some kind of a holy writ. It is in policy therefore it is gospel and it must be so. But you see when we make policy it's rarely that cut and ride. Somebody comes up with we need policy, some ruling in this area. And then various and sundry people jump up with 50 different alternatives and we argue back and forth. And what actually goes into policy is a compromise. And that compromise how it is selected depends on assumptions and certain criteria. At one point for example policy had this hard coded rule that if your document was more than 4K in size it had to be GZipped. HTML not HTML didn't matter 4K you GZipped. Because when the rule was written you know we used to transfer install Debian using floppy disks. 4K was a big deal in those days. You know it made the difference between having 20 floppies or 40 floppies that you had to carry around. That rule is long gone. It was actually long gone before I took over policy. But the point I'm trying to make is criteria change. If we do not carry the rational, the alternative and the actual why we put the rule in in the first place. Along with policy which in a large part we do not today. Then it is very easy to fall into the rut of hey this is policy so therefore it must be so. Instead of reasoning about why we made the rule in the first place. So I would like to change policy to always carry this meta information which is not policy in itself. Around with the policy document which also make it easy to either include all this extraneous material. Or exclude these extraneous material when you are viewing policy as a whole. Also there is this bit about there's so many documents that one has to read. In order to figure out what are all the policies. In fact I'm pretty sure that there are very few people in this room or watching this talk. Who actually know what all the policy documents there are in Debian. Pearl policy, menu policy, python policy, java policy, web app policy which very few people have had the privilege to see so far. Debian hardened policy which I wasn't aware of until a couple of days ago. And I'm just getting started. It would be nice if we could create a meta document which is not. Which doesn't have any one particular editor or point of control which basically says here's a framework this is a book of books. And within that we'll have developers reference a technical policy menu policy draft policies. So it would also be nice if you could link lintian to the policy that we are currently trying to follow. If policy were made modular and if policy module could point to a specific lintian test or a suite of tests in lintian. And you could tell if you could tell lintian hey here is my policy that I'm currently trying to check my package against. And this policy happens to be Debian technical policy plus developers reference plus let us say Debian edu policy supplement. And ignore the web app policy supplement which is not yet standardized. Then lintian can go figure out which tests have to be run and only report violations of the particular subset of policy that you want to check against. That would then also help not only help CDDs it will have derivatives. It might help somebody trying to create policy which is not yet published and say that OK I'm trying to do create a policy about how we package Ruby on Rails packages. And make sure that all my Ruby on Rails packages actually conform to policy. If not whether I have to change the Rails package or I have to change the draft Ruby policy. It would greatly help in making sure that the policy that we are trying to create is same. I'm kind of ahead of my schedule. So how does one go about allowing policy to do all these nice things? Actually since I'm ahead of time are there any suggestions any other things that people want policy to do that I have missed out upon? Surely you guys must have something to rant about policy. I mean there's no subsystem in Debian that is anywhere close to perfect. Policy probably being one of the top few. Hello my name is Giafred Fuchs. And there's one specific thing in policy that I stumbled with one of my packages upon a while ago. And that I rather think is that effect in policy than a proper approach. We have this thing with symbolic links. They should be relative in various sub directories. But absolutely if they go through various top directories. Now it is that the user and var hierarchy is quite a big split up. And I would expect policy to allow absolute symlinks within different sub directories from user or from var. Do you think that might be a good idea? I actually alright. I think that probably I haven't looked at the issue. But my opinion per se is not really relevant. If you bring up a proposal on the policy list. I'm sure that there are domain experts who will come and tell it weigh in on that proposal. And if it is we need to have pros and cons about why we need to have symlinks versus I mean relative symlinks versus absolute symlinks. The one reason that I remember from when the rule was written if my memory serves me correctly was because some people were mounting slash user over NFS so the links were actually slash it was AFS something something user Linux user. And in order to cater to that major university in the northeastern United States that rule was put in that you know if it is within user it was within their AFS mount point then you could have relative links. But if you wanted to link from that NFS mounted to somewhere outside on and and the NFS volume it had to be absolute. So we just kind of ask questions and ask everybody how do you mount these directories and when we found that usually people only mounted top level directories that rule was put in. If this has changed now and if very likely has changed then maybe we should revisit that rule. But we can only know that if we put it up for discussion we solicit more comments from all our users and the deployment. I have no comment on this specific issue but one thing more in general I think at the moment is a problem is the fact that due to the historical reason that it came from a deep package language manual history duplicates or defines large part of deep package behavior and it's often not clear whether how the relationship is between policy and deep package. So who defines what and when deep package changes has policy to change or has how far has deep package to go to remain. Compatible with policy so there are a lot of gray areas where the process is not really defined and yeah that should if we really go about reorganizing policy it would be good to especially look at these parts and see which don't really need to be defined by policy and can just left for the deep package documentation. Right. I mean this is a good question. There is no clear cut answer. The general principle is that policy should be minimal. There should be nothing in policy that doesn't have to be there. Policy is not best practices it is not in how to. Because if you try to be those things it will just mushroom and how to and best practices are often very subjective. There's more than one way to do it. We need to allow the flexibility in order to come up with new approaches and new software. In fact I don't I'm not happy with the fact that policy kind of says deep package the program is what is the only thing that we should do. But it should be in theory possible for somebody to come up with a better packaging thing as long as it is compatible. The interfaces are compatible with what we have. And this is where the distinction between what is deep package and what is policy comes in. There are some things which ought to be standard. This is the standard Debian packaging interface. Deep package is the reference implementation. There are some things which ought to be left up to the implementation to decide the exact options or the ordering of options or things like that. And yes we do. I have been threatening to rewrite policy from scratch for at least the last two debcons and I assure you that is still on my to do list. I want to get to it. I haven't had time when we do do that. Each everything that goes into the new policy manual would have to pass the test of does it really belong here or is it documentation. Hopefully we'll have more clarity once the new policy rewrite starts happening. In order for that to happen we need you guys to come in on the policy team. So if anybody has having second thoughts about not volunteering this is the time to stand up and become part of. You mentioned that policy stands in the way of development and we should work out the ways to use draft policies. Why can't we use the current must should and may and start things off as may and then increase almost their severity to should and eventually must. Sure. But this is you can it becomes hard to change. Right now policy is a mess of things which are much which should not be and things which I should we should be met must and so on. Once you start adding in rules into add it into the policy as a whole reviewing bits and pieces become slightly harder. Let me take a analogy. Think in terms of the version control system which is in high fashion nowadays get has objects. Each object is essentially a file. Then it has trees which are basically just a collection of these objects. Now suppose we had policy which was structured that way each policy rule is an XML entity. It has a certain structure in metadata associated with it. I'll come to that in a bit. You just start adding policy rule objects. The technical policy is a tree that links to certain one of these policy rules in a particular order. Just like in get you have you do a branch. Your branch is your own little draft policy. It has all the must should maze consistently. You can add these objects into the general policy repository. You know those XML rule entities. It wouldn't change the technical policy in any way. You can have your own little metadata that says that add whatever is in the current technical policy add whatever is in my little addendum. And you make up a composite document and lintian can check against that composite document. So you can develop your branch of policy totally separately from the main branch and can merge it back in. Even in software development sure you can instead of having a branch off you can add little patches in line with everybody else adding their own little patches into the main line. But we a lot of people have realized that doing development off to the side letting it come to a point where you can run tests you know lintian against your thing. And then merging back is slightly more productive than adding may rules and then later converting the may rules to sure. So we can do it this way. But if you're going to rewrite policy and if we can cater to the other use cases about slicing and dicing policies and long CDs to add rules and allowing lintian to more closely reflect the current policy or the current policy variant and provide testing I think is a slightly more productive way to go. Two question. The first one is a really simple one which a popular feature request in the PTS is adding the list of items which need to be addressed from one version of standards version to the other. And currently upgrade checklist is just plain text. So I guess it's really simple to implement but would you have anything against making it machine possible. That's the first one. The second one is about deriving policy but policies for more specific topics like Pearl or whatever. I for example am one of the author of the OCaml policy which has existed since I think 2002. And one of the reason not to including it in the official policy is that we would like to have the freedom to change it when we want. So I fear that if I push it to the official policy it will be more hard for us sub authors of that policy to to change it. So do you have an idea how a workflow would be possible to let people which are ordering the sub policy interact with the main policy. All right. Two responses to that. Firstly the new policy update process we have provision for allowing for domain experts that the people who manage policy should pay more attention to domain experts. So it's not like somebody who is not an expert can veto a domain expert. We pay more importance to that. So people who create these policies are often de facto domain experts because otherwise they wouldn't have bothered. So if you are a domain expert you will be paid attention to. Secondly comes back to metapolicy policy about how policy should be changed. Every time policy changes even if it is in a small area it affects every package maintainer who has a package in that area. So we I guess we should recognize that changing policy is very disruptive. So I don't think there's going to be much of a objection to changing parts of policy. If the domain expert think that it ought to be changed at roughly the same frequency as the general policy is changed. So if you're OK with changing even the sub policy every four months or six months then I don't think there's going to be an issue. But if your policy is in a state where it requires weekly or monthly changes still then perhaps it is not yet mature enough to push into. But I do want to bring it into the fold so that LinkedIn can check against even that policy. And this was Bedel's suggestion by putting some kind of a tag saying that this is draft policy yet. So there's not real policy even though it says must it is still has draft status. So if you violate a must roll in a draft policy your package is not going to be thrown out of testing. But we definitely don't want to have inflexible policies stand in the way of moving sub policies into the main policy. If you find that there is currently some problem in changing parts of policy please let me know offline. I apologize for it. I'm not aware that we have been stalling any changes. I'm just not really question more of a comment that this would be really helpful for for stuff I've been thinking about as far as developing LinkedIn rules for testing the results of tool chain hardening for the hardened compiler options. So I like the idea. Okay. So the next thing that I wanted to talk about was flesh out a little bit about what this modular policy thing that I've been talking about. I've talked about this last depth cons. I have made a little bit of progress in that not enough for me to be able to publish it. We were thinking of rewriting policy because current policy copyright is a mess. It's it's hard to read policy end to end because of changes in grammar and style and the kind of detail things go into. So if you are going to rewrite policy I would also like to move away from Dubby and Docs HTML to something which has a little bit more wider tool support. So I was thinking of a subset of Doc book. I think I've got my subset right but I would like Doc book experts to review my work before we go forward with it. So I was thinking that a technical the technical policy becomes a meta document. It all it does it it sets up sections and it includes individual policy rules and individual policy rule is a XML entity. It needs to have a name. It needs to have the default severity which may or may not be overridden by the including document. It needs to have a normative part which is the actual policy rule that you're trying to put in. It has to have the rationale. Why do we want this and that has to be informative part which is why did we choose the alternative that was selected. What are the criteria it is based on. There should be a little bit which says that oh by the way this is the UID of the lintian test that applies to this rule. I have tentatively talked to Frank about whether lintian can support this and it ought not to be that hard to implement to be able to see that this particular test has a tag associated with it. And that tag will tie in lintian with the policy rule. I think that we this would be useful for T-debs or Debian Harden for web app policy for the Ruby and Rails policy. And this is just talking about stuff that I have heard of in the last two or three days. If it is written in so everybody can have their own. This is kind of going back to the Git analogy. Debian technical policy just is like the Linus tree. There is a tree of policy objects that's maintained by the Debian technical policy team. Everybody has their own tree and they can ask the Debian technical policy to pull from their tree. We use Git, clone from the Debian technical policy, write your own rules, objects and ask us to pull when you are ready to have the thing go into formal policy. This is actually all that I have. So now I would like to, we are half way through. I would like to open the floor for discussion about what else you guys think needs changing in policy in the policy process. Whether you like this whole modular XML based approach. Are there any alternatives that we should be looking at instead? Whether the Debian technical policy, and this is for Clint, has become the fastest overbearing overlords who are not listening to the proletariat. The floor is yours. I might point out that Clint has helped us out in bulk triages. So we are trying to pull in the proletariat when we can. Yeah, I think it would be interesting to know what would need to happen so that your proposal happens. So, I mean, as you said, you talked about it the last two webcoms already. So have somebody buy my house in Tennessee so that I can move to DC and have more time. Volunteer to be part of the technical policy team so that some of the routine work can get done. If you know anything about DocBook, please help us review the subset of tags that we ought to be using. Because throwing all of DocBook open to the uninitiated is not a wise idea. Help with the discussions on the bugs that are open against policy. We need to have more input. Look, the people who are in the technical policy team might know a little bit about the packages that they maintain in their technical area. But technical policy covers everything. So somebody comes and says we need a rule in such and such area, like we need a rule for packaging Ruby. Well, heck, I don't know. Just saying that we need a ruling and somebody proposing some set of wording is often makes me scared. I don't know what I'm putting into policy. I need input from everybody to make sure that whatever goes into policy is actually sane. I want the domain experts to actually pay attention and help us out. I think with, I haven't been doing much with policy, but Russ has been doing a wonderful job. I don't think that anybody who has made comments has been ignored. If you can get more help for Russ to be able to do the job he's been doing, that would free up time to explore all these new areas. In other words, we need help. If our DPL is around, we reiterate the technical policy team needs help. I think we said that in the review. If you feel like there was a joy, I had put up this thing where the DPL could nominate people onto teams. Please, please nominate people to the policy team. Any suggestions on who you'd like? Any one body which can write a string together, technical policy legally in English will be acceptable. So, doc book expertise helps. No, really, I'm open to anybody. I acknowledge that we probably want to see people demonstrate some judgment and be able to lead discussion in the policy list, but our threshold is fairly low. Surely you guys must have something that you find objectionable about the way we are doing things. What are we doing wrong? I think in general, it's sort of moving in the right direction. I think the main thing that's not happening is it's not moving fast enough in terms of making policy, machine readable, and directly integrating it with Lentian and some of the other tools. But of course, that's the age old problem in Debian. Everybody can say that, oh, it's not moving fast enough, but there's almost never enough people doing the work on the parts that needs to happen. I don't have a solution for you in that regard. I'll be happy to take whatever I have. I've got this little doc book template that I've been using to write my own tiny little policy and G. I'll be happy to hand over the template and all my notes to anybody who thinks I have the time and the competence to carry it through. So, I'm definitely not standing in the way of progress here. So, anybody who wants to stand up and take this over, please send me an email. If that's it, maybe we can break off early and go and get some coffee. I like this thing about not having to write slides.