 Okay, is everyone ready? Should we start? So, why are we here? Enrico came to me on Wednesday or Tuesday, I don't know, and told me about Git and Call of Main Donario. We have a problem. Basically, our main problem is Call of Main, because we have file-based inaccess, and if you add a hook, the hook is executed under your own UID. It basically means everyone who can commit to Git and Call of Main is able to kill your home if you have a stage of forwarding, maybe connect other homes and other stuff. So, that's also good for any team of audience? Yeah, but Call of Main is our biggest problem. So, it's basically a global problem for all of you, but at least for most other teams, you are able to control who is on the team, which makes the situation less worse than on Call of Main. I don't know by hand how many people are able to commit to Call of Main, but it should be a few hundred. So, we thought about replacing our account Git setup. It's more than a thousand. It's more than a thousand. Maybe. Okay, okay. Because RDD is for us a few hundred. Yeah, sure. I should add an easy-to-get solution. Sorry? Like, easy-to-get solution. I don't know if you're streaming, and now for the disclosure we want to. But, the situation is pretty bad. Yeah. So, we have to change something. Currently, we have 22,000 Git Reapers on Adios, which makes 150 Git space. And, yeah, we have five base Git sets, which is a problem. So, we thought about replacing the whole stuff with a dedicated machine, which also helps getting some load away from Adios. Adios performance improved a lot in the last year where I fixed many of the performance problems. But, it still can be better. So, I already talked to DSA, and they're basically open in getting us another system as Git leaving or replacement. So, the target of this bot is to get a list of our requirements, how we should do it, who should do what, and how to proceed next. I had opened some Detang Pet. That's who you are. Everyone is free to add commands. And, I will do the live too. I'm sure it will reconnect. Yeah, I am connected. Switch. Okay. The basic idea is currently to move Git to live free. So, if anyone has other ideas for software, please add it there and speak up. We also evaluated GitLab, which is basically a security nightmare because it bundles just everything that is available. And, it's basically not meant to be. What we wanted to have is some kind of Adios integration. I don't want another ID provider. So, there's age keys and user scoops and so on. It should be reused from Alioff. Git to live, that wouldn't be such a big problem because I already have some kind of Adop exporter. And, another exporter that just generates Git to live. It wouldn't be that hard. There is also plug-in. Yeah, I still won't use this to create Git to live. Git to live has some shell like interface where you can create windows and so on, which I want to use. What we don't want is arbitrary hoops. Not everyone should just be able to add some hoop. What we want to have is to provide predefined hoops that you can just customize for email, web hoops and so on. Yeah, we want something like Siege it, some content and yeah, that's it for my side. So, we now need your input about how to proceed. What do you need for Git? I have no idea what people are really doing. There are a lot of it. And, what do you expect from a Git replacement? We have some mic. Yes, streaming, probably we have. The mics are on the... The room mics are very directional. Okay, so I'll ask loudly then. One nice feature of Git to live is the ability to create a repose with a push within some hierarchy. It's so called wild repose. And I mean, I haven't thought through all the security issues, but it seems like for a team this could be rather nice rather than going through a shell or some other... Well, you need some way to access this host and you don't want to give people shell accounts. Yeah, I guess. Git shell isn't a shell where you connect to SSH tool, it's some kind of command interface of Git to live where you can just issue commands towards Git to live. We also want that everyone is able to create repose under its username space and probably team admins should be able to create repose in their team name space to get push or some dedicated command. What I don't want to be right or provide is some web interface where it's fancy repose or stuff like that. I think some shell like interface, Git like interface should be enough or it's a hope so. Remember? Git to live shell is not really anything that you use except for setting tags and other stuff. It's actually not very interesting. But what is very cool about Git to live and what I think we definitely should go for no matter what the tool that we choose is that the team admin, the owner of a repository hierarchy can create their own authentication so I can pull in an alien group and I can add these three people that I just met at that point to meet immediate access. I can do that all of myself. Mention Git. You can amend authorization. It's a little bit louder. Authentication is still handled by Git Alive. Yes. You just set the authorization rules. I agree it's good and important for the team admin to be able to add and remove people easily and even create repositories and also be able to do it via Git or most of that via Git at least. I wanted to suggest considering Garrett which could either possibly be used with one of the other servers or without but I think it can support you working with at least Git tiles and maybe Git Alive. The nice thing there is you can actually configure it via Git pushes if you want that's supported and you can integrate with things like LDAP from Alioth or what have you at Google I use integrations with their authentication right and it supports code review which is a big win that we don't currently have where if you want somebody else to review your code whether they're the maintainer or you're an enemy or you're a sponsor or you're just a maintainer and you have your own flaws in code just like everyone has one someone else for double check. I was going to say I think this sort of depends on what you think the boundary conditions are around what we're trying to accomplish because when I think about Git Alive it's because I mean when I think about Garrett there are really sort of three fundamental and then a couple of variations on those workflows that people use around Git and the part that's the most important to me is that we have really solid Git provision control infrastructure underneath and then the ability to run any of those workflows that make sense so would having Garrett available on top of this be good? Absolutely. Do I want everyone to have to deal with Garrett? Probably not. The thing that I was going to say is that I don't currently use this on any because we're not used to it but one of the things I've really enjoyed about using Git Alive on other projects hosted elsewhere is the curb branch permission mechanism maybe this is what we're talking about when we talk about a group or things within but the notion that you can do things like constrain who has the ability to push to the master branch but anybody who's got permissions within it can pop feature branches to go do work on that kind of stuff even if you're not using Garrett. I was going to say that even if you're not using a Garrett flow that kind of ability to sort of structure who has what roles the given team can be useful. I was just saying Garrett doesn't even require code of view and you can totally ignore that part of it if you want to that's the main purpose of it. Right. So the main reason why CollabMain is so bad is partly because it's running on any of which is doing far too many other things that's really bad for security and the other is that the access control for that is insane. It gets deliberately insane. Right. So if we are expecting teams within Debian to do their work in some repo on whatever server that's not our office new server maybe that we're thinking of we're talking here about maybe allowing that team to modify their access control if we are to allow uploaders who are authorized to upload that package to Debian if those people are not expected to do like a security review on all the get branches that they pull which if you practice I don't think anybody does even though there are people here saying oh absolutely they must in practice that's not really a usable workflow then the set of people who can push to whatever branch the uploader is pulling from have effective right access to the package even if they can't immediately upload it because their work is not going to be checked and so I don't I'm not entirely sure there are probably reasons why we want a set of repos with different access control but it might be that most of the repos we're talking about should have the same access control as the archive in that DDs or DMs who have a particular application should be able to push those repos or to those branches or something so then I guess the question I have Ian is in that context how do we preserve the useful part of the thing that is different about Alioth which was that we had a lightweight mechanism for allowing folks to collaborate with us at the source code level and if we weren't willing to give them their car public load or are you trying to suggest that you think that the actual practice and behaviors is so scurrilous that it's just not different if you give if somebody's got access to to collab maintenance then they've got delayed archive access upload access at the moment and I don't have any and there's very little auditing on that but I imagine that it wouldn't be very nice get someone into the archive if you wanted to because that would be the reason why we're here if you do want to fix that I'm not thinking Garrett I just really think it's a good choice so I guess I am I want to not have any more Garrett plugging because Garrett is not going to solve these problems because it can only solve these problems if we can persuade everybody to use it can I clarify what I mean I don't mean mandatory code review for DD I do not mean that you mean something where everybody has to deal with Garrett what I'm saying is what I'm saying is Garrett can be configured so that those people you don't have to use the web UI much you can push directly with Garrett you don't have to push to the refs4 branch you don't have to push to that magic branch you can push normally if you have the permissions and so we can configure it so that everyone who's a DD or a DM will authorize in certain ways can just push normally if they feel like it and if you are someone who is a random other person who just creates an alley off account you can just go through the code review and have somebody do the security review but you don't have to force anyone to use the web UI much if they just want to push with Garrett and they don't have to use a special change ID graph and so forth that's not going to work that's exactly doable in almost every other GIT server as well it's not a Garrett unique GIT or light could do as well as far as I understand it GIT or light for example I spoke to Thomas who was who has been thinking about doing a bit of work on packaging Garrett for Debian and he's already gone through a few Java libraries he's gone about he said the next thing on my list is such and such which is a thing I didn't even recognize and he said I've only got another 30 Java libraries to package for that we know what package and Java libraries are like whereas this is like it of course is in Debian right now and actually works I absolutely think that we need to be picking something that is already present with security support which is the exact reason I'm not pushing my GIT server here despite the fact that I think it's better than GIT because I am not prepared to put my name on the line for security support for it on a platform that thing yes so I think it will be useful to have places where people can push their stuff early off is quite a good place to push your stuff if you don't have authority and you're expecting yourself to be reviewed and it's kind of okay for that if not the problem we want to solve is the problem where the team are using early off as their primary authoritative server so under that circumstance I think the best thing that we can do is to have not a default instantiated set of rules but really good documentation of how teams should set the rules for their branch policies or even template for those template exactly so that you can go okay I've created a team repository and these are the people who have upload rights to the archive therefore they're the only ones who are allowed to push to whichever branch it is that we're saying they're going to upload from and they need to be able to define that for themselves and everyone else can only push the branches that are prefixed by their username for other things so I think we have two problems here we have a security program and we have a social problem another one we are currently talking about is the social problem that we talked about and I'm saying this is a diagnosis bot to solve this social problem yeah we need to create some best practices for working with Git maybe you can get it or whatever too but what we currently, what I currently want to do is to solve a technical problem another social problem but the point is though that the social problem exposes a technical problem which is that whichever solution we go with needs to be able to have branch policy type ruling constructible per repository by the owner of the repository rather than by some cutitive Git server administrator but I don't think it makes sense to restrict for every package who can upload there to the Git repository the same as in the repository we could go this way we didn't have the technical problem because collaboration would be disallowed by this policy anyway because it's the whole purpose to allow easier collaboration of certain teams so yeah if we wanted only that we could only just restrict alias anymore collaboration is solving a technical problem no it's a it was solving a technical problem but we didn't have a mechanism for easy construction of teams and repositories everyone was user repositories the whole point of collaboration is that you can allow other people to push into your stuff so it was a technical problem that we did not have easy project generation that was solved well it actually solved two separate problems of one completed solution one of them was we didn't have links for lightweight creation of teams within the project and the second one was allowing collaboration of people who were not DD, DN, not interior and I argued those are actually separate problems because even if we had an easy mechanism for creating teams constrained by identity that you know DD or DN kind of identity that wouldn't necessarily have solved the how do we let the random person be the world's leading expert on this thing and help us with it and that's a different problem if they have an alley of account they can be added to a team the pain cost was that non DD still is that non DDs can't get it auto creation of project teams I've always been sufficiently confused about what the point of a lot of alley off was other than the identification model and support for non DD participation and things that when we talk about the word alley off I don't mean anything negative about that I sometimes find myself struggling with what it is we're actually talking about so all of this has boiled down to that technical solution of being able to configure these things so that individual projects potentially can choose the policy that they wish to apply which means that the easy collaboration facilities of collab mate can be maintained in a way that doesn't cause certain people who have interest in security like myself and in conniption fits about there being anybody who doesn't currently have upload rights the ability to push to a branch that might put it in the end up as an upload okay thank you just to remind I'd like to remind about the goal of the discussion here is that we currently have developers I guess most people in that being not really understanding how vulnerable the whole idea of this setup is as in arbitrary code execution sort of things the attack surface is everyone can get an account using the social engineer the system so that people can get commit access to it it can get escalated by having hoaxed that right to give request to be controlled by that person and not in collab mate so features are nice and well if you want to maybe so that from this doesn't have to process collab mate other quest will be only happy but the point is that to make other people now and replace it with something now even in a week with all the development stopped because that's currently that bad and I would like to start committing around code in that in control in repositories in collab mate and others to see like adding headers like this developer didn't verify their upload to have a look if they show up in the archive and then close account so so ok I get that and I hear you I guess based on that I haven't heard anyone saying anything yet that suggested that getalight3 is an underlying technology was a bad choice I was going to say this to Enrico and I'm glad that the front of this point we've been looking at bitalight for about a year now as a serious replacement so far it's been locked on the LDAP integration but we have obviously been willing to work on that and I would like to ask the question what sort of things do you think italight wouldn't be able to do that we need because that's exactly what I was about to do we can talk endlessly about Garrett and all these things but italight seems like a perfect match it's really good software I think it does everything that we need is there anything that we need that it doesn't have we already have a getalight instance that DSA is using we already have a getalight instance that DSA is using for our repositories and some other ones as well yeah it would be really nice to be able to speak in a loudness at the end of the day if you could hear it it would help me really much keep it in the sky was that did you hear what I said just now yes and then there was something else which I didn't understand so there's what is it the server that the DSA people use for our proper NADJOS factory repositories there is also some other things on there that aren't to say I think I can't remember the other thing I want to say do other version control systems have this problem or not I don't know the version control system is the same the version control system is the same issue on alias but we always think about Githium right the question why do you have the right to treat it as a hook directory in the current domain because it is explicitly a feature of that system that you can write arbitrary hooks well no I could make it I do not agree we're really interested in getting to answer the question if you're only looking at Githium or not because it has an impact on the discussion here so are we looking at other things except Githium as well or are we mostly focusing on Githium as well as we can discuss later at least because of my problem this is a Githium only problem but in the demo Githium it's tough it's tough in that way does Svn have server side hooks yes but at least in one directory if we are going to talk about Svn the whole conversation is going to get totally mixed up and we're going to talk about Svn servers and Gith servers and it's just going to be a possible we have to like focus here on Gith because that was in the boff type I have another boff for your buzzer I don't know okay but sorry also at the same time if the scope is bigger then maybe going the other way around like restricting hooks and doing like maybe doing the chute into the Gith repository or the Svn repository regardless that server is far too overloaded if nothing else making a separate Gith server will reduce the load on the rest of that we should therefore do it regardless of hook access so what you want to do for this new Gith server is keeping alia for authentication but severing the link for authorization and keeping the authorization inside the new Gith server okay has that been discussed with the FusionForge upstream no I have another idea about the FusionForge upstream I am accidentally at the root access in aliaf and I haven't seen other aliaf admins for years so I don't agree that that's what we want to do like the group structures still need to come from aliaf I want users and hooks from aliaf so everyone can add another one to the hook and that's it I don't agree with that as another life principle because that needs us still with using SSH key based push so the authorization of push is based on SSH connection authentication and that leaves no audit trail of who you want that's not true you can still have some HTTP smart server where you can authenticate with a password that could come from aliaf I have work in SSO also I don't want aliaf to use the Apache I don't want aliaf on my gcb trusted computing base aliaf is doing far far too many things even if we dealt with this hooks problem aliaf has got an enormous number of shell account users an enormous number of strange pieces of software running on it I have no confidence that the machine isn't controlled by gomos whoever and that can't really change without stopping aliaf being aliaf I have a fully accurate statement however I think what we should do is break pieces out of alias break the gitges out so I think that's a good thing however aliaf agrees that basically we should do SSH infrastructure also from alias that's another topic so this discussion is really whatever we do it should be able to use a central debian authorization whether it's alias or should we move debi.org which I would prefer is something else but it should be able to use something which is an held up as authorization source so at this point the proposal for the Git server change does not increase your trust in the Git server but it does remove load from the machine called alias does it decrease your trust in Git does it reduce your trust in that Git server probably not okay good so it's not a net loss well okay so long as you know after this has been done nobody is telling me how the problem is fixed now it's partial fixed it's an improvement no it's not an improvement it's a partial improvement it's a huge improvement because you have much better audit trail Git align actually then you have Gitania it's a really good audit track Git align can produce a net loss you can basically report the person who owns the SSH identity and people have multiple persons even like per project reports that time of course the commit that was made for every single project and I do expect that SCM from alias will remove a lot of the needs for shell access to that machine and that in a certain future shell access could be done on demand and not by default absolutely the only time where don't show up like access on alias in years is just to the Git management right what kind of books are needed for example call up main and is it possible to have some templates there or even some ready scripts switch on and off I think Git align has a management piece of functionality in it if I do one of the contrary modules but it's definitely got something and it shouldn't be possible for users to inject arbitrary hooks it depends of course it depends on what you want but you can define for each hook whether it's mandatory for every single repository whether it's mandatory and according to a wildcard whether it's optional and usually the optional ones are the ones that if you have parameterization and you leave those empty they don't run or what you could do is just use things which are in the Git config in the local and say okay if there's an email that sends then I make it send out I think about providing a standard set of hooks I will list some on to downtown and users should be able to edit the Git config to add some variables like a main address for a mailing list or some webhook or some other stuff for those hooks we need one where we are on this if you're going to be consequent about this then you should not have users have access to the config file of the Git but with Git Alight as well you can specify specific keys that the user is allowed to set and this is where the Git is like shell comes because only through the shell can you then set these things and configure the hooks and so on and it's as quick as possible I'm sure that some people won't like that but I don't think that's a way around it You can change it for everything or a repository per person You can say it's a person I want to have in two years such a list of exceptions where someone is able to edit on hooks and so on and we want to get the rows back to sanity The thing is no matter what we do with the default set of hooks someone's going to go I don't like that commit mail script I want to use my own and as long as we have an arbitrary ultimately it can make an external HTTP call they can write whatever the heck they like on their own servers under their own security domain and just have an HTTP call come out to go by the way there's a new commit on that gets given as a trick I don't like it so Git or Git have where they only get arbitrary HTTP yes So DSA will probably give us a separate mirror server anyway that's what they did for the DGit Git server so the CGit does not run on the master wrapper Right So one comment I don't know if you I don't know if this is immediately useful but just to keep in mind at some point in the prototype in Gitalite 2 that scan the Debian key ring for authentication keys and transform those into the Gitalite access keys and it works well enough so if that was a desired feature you could have authentication based directly on the key ring of course most people don't actually know how to deal with authentication key ring authentication sub keys so it would be a traumatic transition if you did it tomorrow We already have all of the authentication data in Allioff at the moment So the short term quickest route to separate our Git server is to have the export mechanism which we've already discussed and that includes then all of the people that have Allioff account in order to be able to collaborate via that service rather than using the GT authentication keys Ian's idea of actually requiring signed commits and using the signed pushes What's the sign push Sign push is where you say Git push and your Git client signs an instruction to the server that's the list of rep updates and the server verifies Is that the mainline already? Yeah, it's in Git mainline it's in SID I think it's in stretch already the needs of actual This is better than signed commits because signed commits are signed at the wrong time they're signed when you don't want to be and then at the last moment the server needs this commit and they get the signed commit and they've got no idea which branch was supposed to be on whereas the signed push it's exactly what the user says Git push this and the cryptographic stuff is a declaration of exactly what the user is saying they want it done So it has to be this advice that you need another interface and people another key for using this thing GPG I'm not developing a machine that has access to my IBM key You only need to do the git push on the machine and if you are pushing for the repo that's going to be uploaded from you are in fact pushing to deviant you're just doing it with a bit of a lack Maybe make a subkey I need another Make this too complicated Actually I think for the moment we started what we had I really think that's a very good idea to do with that However, I don't think we should have a precondition before we can fix this part of the git problem We should add it later and have it optionally and perhaps after some time say it's working so fine now but not for me required from start Even so I'm happy if we get through it as soon as possible One pretty simple thing that could work was if we wanted to still accept drive-by contributions by random people is to just disallow pushing to call up main master by anyone except Elise and EM so everyone else needs to use whatever branch that someone That only works if merge and it would fix 95% of the problem Except that only works if everyone that uploads packages only takes stuff from master The problem is you can't control the reference space like that I'm sorry, excuse me I'm suggesting that it's okay to have somebody on an uploader's list who's not confident Let's fix the actual problem The problem is that somebody is uploading packages, doing merges or pulls from other branches without looking at the code to just reference the wave and they shouldn't be on the uploader's list Yes, and one cutting that is what you could do with Clip to Light which I think is totally harmless is you could say everybody has a subspace user slash look in name and could push to that whatever he wants I don't feel about bad about it at all because everybody could push him in any of my packages and use a namespace on the uploader's list Right, all of this is nice optional add-ons I have little said we shouldn't let the basic things bring that getting home by setback we shouldn't lose so they could do it and that I do it as soon as possible I haven't heard anything that suggests that using GitLight 3 as the base definitely is really a problem Does anybody object to just doing that right now? No Right, so GitLight 3 is the chosen one thing that is bugging me slightly because this means that I'm intending to the Digit Git server to continue to exist indefinitely I don't want to block this improvement by saying it should be the same as my Git server but ultimately I think these two Git servers won't be the same Git server with one tree and one Seagit instance this will require some care taken about the ref namespace because Digit has a ref namespace and I think this might cause trouble Is Digit's ref namespace under refs tags and refs heads? The worst part is the tag namespace which is the defaulting tag namespace Right I'm not sure that it's a very good idea to have those together and to have very strict opinions about what should go to Digit and how that it's mostly equivalent to pushing packages to IBM and I think for Avios we want something that is practical But that's the same if you push and commit to a master branch here it is uploaded to Debian No It will answer Not necessarily If I have a repository that might have a Debian branch that's where I push to Debian then it's not master That's why I said we can't control the ref namespace It's a blind branch If there's a branch that's made to regular pools for most packages if a push doesn't make a branch it's basically the same as uploading with a delay and hopefully a review I'm sorry The conceptual notion that people merge or pull without reading the deltas just as glom and on It's because they don't have another It's because they don't have another yet server, right? What you do is you set up a team and there's four of you and each of you has got like two machines and you need to get server, you need somewhere to push your changes to so that the others can look at it and what you think is this is our personal branch and nobody but people in the team is supposed to push to this but you put it on collagmate you don't understand the seniority properties of collagmate because once in a while a collagmate was theoretically only pushed to by dds, right? You've heard about my mental circuit breakers four times without assertions I see where the problem comes from Hence the whole I don't know how other people work If I ever see some commits with a push that are not mine I do reviews of them Every push Do you ever forget Do you ever make a mistake No of course not, never I just do What I mean is I think most of us do try to do that and I think most of us are pretty good about it most of the time but there's a lot of exceptions because we're all human if it's not automated we're going to be fallible and I don't need to put, I'll stop, I'll stop with Garrett, right? but like if we didn't get get a like three or some other solution it would be good if there's a way to allow automating code review again because if you're allowed to forget it'll happen regards to people being classical of code I see that a general hygiene of code going around practices are going around I know people who run the plugins of github clones and they regularly get pulled to get them up to date and it's the stuff that's run every time when it affects the editor so the situation is in general, in the general phase of the environment, ecosystem bad I don't know how it is in Debian I think it's probably a different problem than we're having here but I would totally like have to have a second ball at another depth conf or online in which we figure out ways to run white hats to do some estimate on how it is to actually get stuff into Debian and I wouldn't here to be proud of that thing so what you said before about you would like to get the github merged one time you said you don't see it but actually what I think we should conclude for this discussion we should like to set up the github light in a way that doesn't obstruct the possible immersion of the future I don't think that we need to discuss massing or the side massing what the moment has we should try to keep the way open if we say that we don't do it anyway for other reasons it's fine but we shouldn't close the way now unnecessarily so I think that the general agreement would get like 3 when can we turn off collaborate and it's called 4 by accident maybe giving vcs something like that where you can discuss all the stuff to get a list of expectors maybe enhancements to the skid server like github and so on to get everything in line there's also vcs pickage there's already a list called vcs pickage which is not used for very much else and probably want to just use that what's the status of the right bit on all of the books on alia for right now we can't change that because the directories are the whole directory path up to the hooks is writable by any being in the lab but if it's a hook the directories are made read only you couldn't change that without making people more able to do the rate management that they currently do we can't change that we can't change that we can't change that we can't change that this is the rate management we can turn it off and chat chat to alia unfortunately our time is over yeah so we could make a call up main to it only but i don't think that's what we want to do now so watch all the next steps watch all the next steps next step is somebody goes to dsa and says can i have the vm for there's no fuck there's already no fuck for getting a vm for that so that's already done so as soon as we have a vm we can start to do it so you start to set up the group server i think we're not going to get it right now so i think the basic conclusion is that you will coordinate the further efforts well okay so let me have a status thanks i still suggest that the hooks directories and the hooks will be set read only as soon as possible because i think this is streaming and i think there's a bunch of scripted kids that do watch those streams and they might be just right now trying to figure out how it works absolutely this is now open are you sure about the idea i don't know if you want to have me i don't know are you certain we can stop before we call up main request so i was thinking about getting those hooks read only as soon as possible or limit us to another group i think we have a dg group definitely because you have to also look from the you put that as a dm group so i would probably write for every dg okay but remember that i can go to the dolphin directories remain hooks in full and make the hooks