 Okay, welcome back to the second session of the day and it's going to be Alex and I will talk now about salsa debian dog Thank you. Good morning. Um, I usually really don't give talk in English. Talks in English. So please be nice to me However I'm here. I want to take today about our journey from alioff, which is still running but not long anymore to our new service salsa I Want to get a little bit into the history of all things and what we have already achieved what we still need to achieve and What are our plans for the future? So let's start with the basic things. Who am I? I am the guy you react rejects your males on list see been Ork, I'm a list master. I am the guy that we rejects your backpots. I'm the backpots FTP master And I am the guy that will destroy alioff debian ork So I'm for the last 10 years I Was the admin by accident of alioff debian ork. This is not a history another story. I will tell in a few minutes and beside from that I Work as an open source consultant at creditiff which is a small company in Germany Which is specialized in open source. We only do open source consulting in Germany. We do Is what you could what today is called def ops we do Every kind of consulting if you do something with open source. We are probably the one you can't talk with our father of two wonderful girls They're not here unfortunately, but otherwise I wouldn't be able to work and In my little bit spare time I do world playing games and tabletops tabletop games so In theory there should be a picture now There's a picture missing. I don't know why Which should tell we need you if so a little bit of advertisement if you want to do open source work in Germany paid And you need a drop. Please talk to me We are looking we are always looking for good people especially in C development kernel development But also of course consulting so please talk to me Some steps in history Some years ago it was around 2028 29. I told the alley of channel Hey, if you need help, I can help you with system administrations or not the GeForge stuff which is running above, but if you need help tell me Yeah one or two years went by and Step-by-step all remaining alley of admins left. We were alone in the channel and Around that time I detected. Hey, I have Zulu permissions And I'm admin Somebody made me an admin So I had to decide That I will be the person that is the future alley of admin and I stepped in So this was a big the beginning of our alley of journey Then in depth con 15. We had a longer birth of the visa where we talked about several security problems and Collab meant Some of where some of you are maybe not aware of it, but since we use git At file system level on a love We are introducing a number of interesting security problems Like if someone writes a hook that who gets executed every time someone pushes. So you have basically Shell access and of course you executed as your own you ID So if some dm or if not dm nearly the whole world has wide access to cool up main It drops some hooks in It can make you execute code on alioff as at your you ID, which is a problem we did some things to Solve the problem, but the main problem remained So around that time we decided that we will need a successor for get even orc at that point We are talking about V2 light Which we evaluated in that time however As it's somebody some things happened two years went into the land and nothing will happen We just played with it Then mays 2015 17 Sweat come up moving a moving away from fusion forge What nobody was really aware of is Is that alioff is an easy vision machine and we see is running out of security sport end of the month So time was running up The sweat was pretty long as user and the end of the end devil and We decided to do a few steps like evaluating things and In June 2015 17. I did a survey about Our new idea of services It was clear at that point that I wouldn't be able to Maintain all the things alioff had in the future So we decided to just bring over the important things What is important for everyone everything else is important. So we decide I decided to do a survey Which was pretty successful with a few hundred submissions Then in Then we Evaluated me's probably me We is probably me evaluated a few solutions named page who which is the good solution Fedora is using which is an Python thing based on good to light I'm good lap which was the biggest github sex component Competitive thanks Gox dash Githia, which is some go long based small get service Pedro turned out to be Not stable enough for our needs and we would have to do too many too much coding inside Pedro to Use it in our infrastructure because Pedro is very strong bundled with the fedora infrastructure, especially its user Authentication and user management stuff Githlap has another had another problem called open core and computers license agreement which means I And others were not very happy with contributing code to githlap Which is something that will happen will always happen if you maintain such a service and Gox and Githia is nice, but it's small. It's not it will not be able to Manage 10,000 of repositories Next step a step happened in August 2017 where we had a sprint here in Hamburg and at the hacker labs the ccc and the other side of the building Where we talked about it After long discussions we decided to go with githlap Because githlap at that point was the most The best solution, but it was already ready. We had didn't had to adapt too much. We don't need to patch it Which turned out it isn't true, but that's another problem And It had features like continuous integration ready. It had features like code review ready Vicky pretty work pretty good working and one of stuff and it is very scalable in all directions every component is scalable Which is good for us Yeah, this is to two point. I wanted to add a list an image About the restaurant. We decided on the name salsa Somebody may ask of you may ask you yourself what we what the name is coming from There's a small Mexican restaurant a few hundred meters from here where you can get great burritos and They have a panting in the back Terms salsa wouldn't on and we were deciding on the name which is Which does not describe type of service on it. So we wanted Yeah, it's it's also a sauce and So salsa at sauce, but I wanted to call it Klaus But we decided against it. So somebody come up in that restaurant with the name salsa and so It's called salsa In the meanwhile, we talked a lot with the Hitler people which were very kind and help us for our problems and we also talked with them about the CLA problem and after some discussions The layer the law of SPI was also involved We made them to remove the CLA and replace it with something better So I'm computing patches to get lab is no much easier and better it's just something we are very proud of and Between November and 25th December we implemented salsa two times first time on GoddardDB and net where we had root but After more discussions, we decided having these a maintained box at dot org dot org box would be better Which made us sawing away our Ansible stuff and developed a new to be able to install salsa as a non privileged Hitler as a non privileged user which we did then so in Christmas we was able to release salsa into public beta things went well Which allowed us at the end of January to lead salsa to leave the beta so since then it's official It's our official git repos successor What will happen in the future? I know it's it's already passed On May we disabled user and project creation on Adios Still in May we disabled Not so much used any virtual control systems like you named bazaar my career in the darks on Thursday I disabled project websites and This is future at the end of the month all other remaining Virtual control systems on area off will be we'll get disabled So if you have anything running on area still running on area of be it Contraps are also disabled. So you don't have corn drops enabled anymore Be it whatever you Think of remove it With the first June Ali of will be off. You won't be able to get any data anymore from any of you can get the dance We are DSA to get something from the backups. That's up to you But I don't recommend it and they won't like it Yeah, in June Ali of will come to an end It's served as well for what I had 10 15 years, but it's time. It's over some numbers. Where are we now? Upon yesterday we had said 23,700 retort stories on GitLab with two three thousand two hundred users 400 groups Which sums up around 90 gigabytes on disk Which is nice for service running for More more less six months. It's a pretty nice number something What are our future plans? You know The one to docker registry by now you can use external registries which is working You can use the GitLab registry for Docker content images But it would be more nice to have our own registry. It's Pretty high on my to-do list after Ali of is gone. We want more runners. So if you are able to sponsor runners so if you have machines or Some some money you want to spend on runners, please tell us what are runners Runners are the things that are used by a GitLab CI to build code or test code or do things You can use it to build your packages You can use it your auto package or pet auto package test your packages You can use it to build bath sites Or whatever you like It's pretty useful and I think using CI more will be a big step forward for DBM And We should really get more into it There are already some projects like the reproducibles are out of this depth CI guys That are working on such stuff and now we have the infrastructure that every DD Every developer or a package maintain or a package maintainer can use it There's also another feature called DevOps which is based on Kubernetes which allows you to even deploy and test Things properly, so if you have a web package which Implements a web service you can even one come up with a Kubernetes pod which runs a web server You can test it you can even record it can QQA tests and so on all based on the step ops feature Which would also be a nice thing by now we have we don't have a Kubernetes instance we can use for it So if you have a spare Kubernetes instance you want to offer DBM, please talk to us and integration with SSO DBM org, which is another side project of me and my summer of code student sitting there We want to build a successor for the current SSO DBM org Which has a problem that it doesn't have an user backend the user backend is alias. You see the problem Which is the case for all guest users the official DBM developers come from UD LDAP which Will still work But we have a problem with guest users, so we have currently don't have an alternative source for Managing those guest users especially give additional groups like hey, it's used as a DM I would love To give all the M's access to the DBM group white access But I can't currently because I'm not able to identify them Which is something we want to solve with the new SSO DBM org feature as a DBM org should also develop New authentication protocols like OO to which we use use use for Salsa Which new web services can also rely on so? that we come away from this Certificate stuff, which is somewhat nice, but it's not that good integrated in most browsers anymore and Yeah, it doesn't work that well We hope to have we only we already have a prototype and we have hoped to have it live into the end of the summer What we left behind We don't have shelves anymore so you won't be able to run any corn drops or other stuff on salsa and Please don't ask we want we won't give anyone a shell on Salsa DBM org or Goddard, which is a host hosting it We have APIs several of them over shows of them And please use them we want to run any corn drops or custom stuff on a good lab it was a nightmare on alias to maintain and to administrate and I will never never want to get into this again But we also don't have custom domains, which is a feature GitLab has but DSA decided against it. So you will have to live with project name dot pages dot DBM dot net and Someone decides for that feature We also left behind Old version also not so much used anymore version control systems like dark spaza subversion Which isn't a problem But we also don't have CVS anymore, which come to support is maybe in surprise for someone But DBM is still a heavy user of CVS especially for our websites and translations But maybe they will now faster Migrating away from CVS. They are working on it. I know they're working on it for 10 years but Things are getting things are getting faster and they are making progress and migrating away from CVS Yeah, that's right. We also left Mercuria Whatever people had in their home directory Yeah, we also had RCS on a layoff. They were as RCS we pose. Yes What we got instead We got a bunch of new features we didn't had before so this is Such maybe start of new ways of working in DBM. We got a bunch of collaboration features in the past Collaboration often means finding the right mailing list sending a patch and hoping Now we can use merge requests which allows people to easily fork and Modify packages or repositories and after they are done. They can just hit a button or whatever And can create a nice Merge request which is already heavily used by some projects like apt or duck Or my own redirector That allows the up in the straighters or the admins of those Positories dash projects to review code easily. They can add comments. They can Discuss with those people all without mailing list if people updates a bunch and that and hit committed Those if pull requests get updated Which is a workflow we are also using very heavily in our company Which is pretty nice in my eyes This also allows contribution to packages from from from outside people it allows the barrier for people to Collaborate with DBM, which is my eyes a good feature Something I already liked always liked on git lab github and I'm happy we are having it too now good web has a nice feature of a good well-designed Webfront and some things could be better. That's but that's always the case, but in most cases github is still blazingly fast Except if you hit some of the bugs in the API, but that's another problem And You can work with it if you don't like the web front end use the API Nearly everything the web front end supports is exposed via API. That's later topic And they are also a bunch of command line clients which can integrate into git to allow things like feature merge request Allow you to pose as merge request from the command line if you don't like web front-end You can also open merge requests by mail If you still like it, you can just hit the right buttons You get a mail address that you can use and if you send a patch that mail address you will create a merge request Some of the not so known features issues You can track to do items or bugs Please this is not intended for deep-in packages. So please don't replace the BTS But using it as an issue tracker or to-do list is great You're using it all the time We're also having some upstream projects on salsa like sane or our own code, which is not packaged So they're using issues as fine, too Issues are disabled by default for our projects But every project is fine to just to enable it and to use it you can have Boards where you can organize your work. You can add sprints you can add Milestones and other things those all the basic stuff you need with an issue tracker is included And we also enable to reply by mail. So you don't have to use The web front end you can just use your mail client to reply and your answers in It's a good lap. You can also close issues by much requests So similar to our BTS Good lap has this closest features Well, this is Below is all the same. So closed closes closed closing closes and so on is all the same and we close your Your issues You can even close issues in other projects so if you have projects related together and you fix something in Another project you can even close it with that syntax. You can also create issues per mail, which is basically the same As for much request You have that email a new issue button where you get the custom mail address You can use and then you can use that meta dress from the future To submit bugs if you don't want to use the issue tracker What we also got are web hooks Custom hooks are not anymore able possible because we you don't have access to yaki to deposit always directly But what you can use our web hooks? web hooks Come in standard in the web world. You can use them to react to events in your repository Events may things like someone created an issue or someone created a pull request someone pushed something someone decked something things like that and You can use those events To use to create IRC notifications We have two IRC bots available for you to use which is KGB and Eirka my own Eirka instance You can automatically close or tag bugs If you look into our documentation Wikipedia and org you find a small pergola for about it where you can adjust as we did before If you close the bug and you enable the tag pending Take pending web hook your bug will be attacked automatically as pending Like before if you used the appropriate hooks on any of and you can also trigger external CIRQI systems like Jenkins or Sonya cube or whatever you like to test your code In the future, we will also use it for a cool up for the collaboration stuff from tinchot Where we will just forward every push happened on the whole salsa system So you don't have to Configure that money ready. It will be happen automatically so if you contribute something to DBN it will Shine come up on collaboration. We are net a collaborative in it Contribute us was it what contribute us? Yeah If you want to write web hooks, but you don't want to run your own web server You can come to us Which means you have to code Ruby We have our own web hook Server implementation for salsa DB and org which is currently also running on salsa, but that Must be the case in the future So if you want to run a web hook Provided as a patch for our web hook implementation, which is pluggable So why to plug in which listens to your web hooks? Provide in provide a patch a merge request and we'll happily add it to our web hook implementation So it once for can be used for everybody Documentation isn't the wiki currently provided hooks are as already mentioned tech pending which allows you to To tech bugs expanding if you mentioned them in a trench lock and some project directly working with commits Using the closed web hook which allows you to directly close a bug with a commit Which is used by some web services and other stuff directly used in DBN One of the most powerful features features we got is get lips. See I Get lips see I system that allows a continuous integration and continuous development on salsa and That allows you to build test and eventually deploy software and packages from within github You can nearly do whatever you want in this see I stuff you can yeah, I said compile your drops run linda Run auto package has whatever you can imagine you can do We have two runners Provided Which one of it is running as an instant on Google Cloud the other one is hardware sponsored by a sponsor and We relaunch forever we see I one relaunch a docker container in it You can even provide image you want to use as This one for the runner for the one And then you can do whatever you want with it But please don't do bit mining or something like that Be kind to them we all have to use them and we have only two of them So please if you want to do something bigger talk to us like the KDE people already did how to use it Using github see I is surprisingly easy There is this github see I yaml file Which is usually in the root of your repository But you can add configuration to your repository for example to add it to your stash db and repository which works better for Gbp packages or Whatever you have if you don't want to clutter the upstream directories with your github see I file We already have a bug open. It's a github issue tracker that allows us to Change the default name of the github see I file because currently if you import an external repository Which has a github see I file which can happen? It will happily run on our infrastructure and for example, you know It's it's Ansible or some some other project which has it so for every upstream and commit results that will happily run our runners and Build the pipeline Yeah, after you added your file, that's it From the from then on you can watch every commit Happening on your pipeline This is a simple github see I file github see I files are yaml based documentation is in the github repository documentation repository and As you can see it's pretty easy you have a pre-step which allows you to do things like install dependencies, which is what's happening here since Github see I is running a detached hat and If you want to will also use git build package We will have to check out master to for it to properly work Then you can do a good pull good build package and after that you have Build your package That's basically all You can also use artifacts artifacts allows you to add to Keep a bit artifacts For downloading so if you want to use a packet Someone will use a package you can just an artifact Stance are here and that allows you to later download your depth files now as it doesn't allow you to Create repositories with apt But it's another problem if it's too much you can also use this thing which popped up yesterday this is a prepared Docker container which is prepared for github package and The oil oil you have to do is to execute this After that you have good left see I I don't know who provided it But it popped up in the wiki yesterday or something like that But we also have our github pages the pages pages are like github pages and allow you to host websites static websites from within github Internally they also use Github see I so you provide a github see I drop that just deploys your website So it's nothing to do and here's our build artifact feature All we all we do here is just add those public files in the public directory to our pages and We only do this on the master bunch and basically, that's it The magic is happening here. They're pages. It's a pages step and if you correctly configure pages in your repository configuration, you have a github page after that You can also do more fancy things like a hugo website just depend on a docker image which has to go hugo installed and Then execute a script which builds hugo Adds some artifacts After that you have a working hugo website. You can also use it for blocks You can use it in personal repositories. Then for example for your own website or blocks Of course, it's not intend it's not intended to serve big websites, but Providing blocks for planet even orc is perfectly fine for example or websites of superb project or Whatever is deviant related This comes brings me to another topic not mentioned in my slides Some people asked us what is fine to host on salsa as long as it's open source as long as it's Intended to be debian related or to open source related or can be included in debian It's perfectly fine to host it on salsa So we invite every app steam which is looking for a home like the same guys To host them on salsa which we what we got with the latest major version is web editor which was pretty new Probably buggy, but Yeah, it works. So if you don't want to clone a repository Or you have just to have simple changes you can add your file in the web editor You get a web editor with syntax highlighting You get even mark your preview if you just do documentation So that's great for everyone just doing documentation without that was that what doesn't want to hazard with git or the code inside You can even preview it Then you can write a commit message And that it what we also have is two-factor authentication Which is security feature and allows you to add a second factor to your github login. I Don't that I can only recommend to use it. It's good integrated and adds a lot of security Hmm It works with ube keys or any u2f compatible key and also with software solutions that implement a tot p time or time based Time based one time passwords So every tot p compatibility generator also works for example the Google authenticator, but there are also others That's which are open source that all works Adding it is easy. Yeah Well as Jordan puts would say I had the bar was for-bereided. It's easy What you can't see now is me getting my hand my smartphone out scanning the barcode generating a pin code and In two seconds, I will enter the pin code Was it what to see here? I recover your codes. I will mention them in a few minutes You can use them to recover your account if you lost your One-time password generator So if I log in now, I have to use my smartphone to generate authentication code edit and now I'm in that's it So it's pretty easy Some people say, ah, what is well? What is if I lost my token that is such much much such work No, it's easy. If you want to recover your token. I do that all the time You can just you just use SSH and Do SSH get it's either DBN arc and to command to FA recovery codes, which will generate you a number of new recovery codes and You can use it to log in So if I don't have my authentication the token with me, I just just use this every time It works and SSH is also another token No, it works. It's the second factor. So it's fine So here we are the API. I have to get faster Get up exposes a powerful JSON West API That allows you to query and to manipulate nearly all aspects of good lab before I say all I really mean all nearly Everything you can do with the word front end is covered by the API. So API has an extensive documentation in that link and Ensure that you use the CE for a community edition in the docs Otherwise, we will get a see features that that aren't available in our edition You can use it for everything small hint You often see the term namespace ID in Nearly all cases you can replace the namespace ID with the path of a project of the thing mostly projects But if your projects include slashes, which is nearly always the case You can adjust you have to replace a slash with a person to F And you can use it as the namespace ID So you have don't have to look up every Every time your IDs you can just just use names Some examples get data about the user if you want to know what start in your user You can just ask the API for every use of authenticate things that need authentication You need to generate a private token, which is easy Just go to a profile and there's a link to access tokens and generate one for your use case You can also add expiration dates Even when I use curl here, I can only recommend to use a proper library Don't use curl. Please do me a favor and don't use curl Yeah There are a bunch of skits flying around as you scroll and send then they hit the first time page pagination or something like that And they start wondering why it doesn't work There are Python libraries, Ruby libraries, Perl libraries, forget lab, so we have everything in place that you need What you can do too Create a repo in your namespace. If you do it that way the repo will be private Of course there are parameters to create a public repo, but that means you have to read the documentation We had this List open merge request of a project also easy Just go to projects, which is a namespace for our projects related stuff at your project It's in my case. It's an audio free writer at your merge request Ask for a specific state and you get a nice format adjacent to Get nice jason you can format to list all your merge requests, so I have to get to an end How to get support if you have any problems with idea of you can of course talk to me or to Ganev Huh this channel this channel is still alias and will probably stay for some time You find us an ISC You can drop us an email at salsa atman at Debian orc We also have an issue tracker So if you have a problem open an issue, but please don't open up stream delayed stuff in our issue tracker If you have problem with salsa fine create a ticket if you have some problem with How githlab is doing things or I found a bucket githlab Please do us a favor and open those bug in the githlab issue tracker because that's the same thing We would do I have to do and we want to track your bugs How we do it if you're interested how we did those stuff We are all doing it. I'm privileged and get on a god at Debian orc and everything we do is automated We are Ansible and you find it in our Ansible repository in the salsa group on salsa Debian orc Where are we? Who are we? Salsa currently has three up in the strait us. It's Bastia. That's Valdi York, Yaspard, Ganev here and me What you would have to see here is a thanks pitcher sanctify attention and We have maybe one or two minutes left for questions. I Have a quick question about SSO going down with SSO going down with Elioth No, it's of June, but As mentioned about announcement, I usually do write announcements and I hope people read it I will we have I will Backups the authentication database of Elioth to the a belly Debian orc, which is so the host running SSO and I will maintain those users by hand until we have The new SSO back and already quick So if there's people going to the register for depth con which are not debbing developers in Taiwan do they can they still register after first of June or not? They can't they can't use SSO any more for registration, but they did implement it a second authentication but It's been any further questions. It's been three years since depth count doesn't need SSO Any further questions on salsa? I have a little question about salsa at all There is somehow tools How to create an account and salsa how to manipulate it that account Create the reporters for example. I mean just for people who Never came to you to just to get used not At the tone of documentation except from user registration. There's nothing specific on such a debing arc So you can just use the git lab documentation which has videos webcasts Tutorials and so on just use it. It's good documentation nearly everything is covered by it And if you have still questions after that ask us if you have Debian specific stuff Look at the wiki wiki Debian doc slash salsa slash doc has nearly everything I hope most things covered if not tell us and in the end it's a wiki So if you find something out, I think it's interesting for other people. Please please add it to the wiki Any other questions? Okay, everything's answered. Thanks a lot again