 Hi, everyone, and welcome to the first Manage 201. So one of the things we wanted to do within the Manage team is to share our knowledge more widely, because a lot of the things we do are really expertise or really narrow focuses, which one person shares. So myself, I've been working on SAML a lot, but Imra has been working on SmartCode authentication. James Lopez is the expert on import, export, and Rubens, the expert on billing. And on a lot of these things, we don't have much visibility into what other members of the team do, and even less for people who are not on the team. So we thought we'd create these 201 sessions where we kind of go into quite a bit of depth about how we work on these areas, and also that will help other people who want to contribute or to review the things we're working on. So today I'll go over quite a lot of things, but starting with what is SAML, why it's important, a description of the protocol, how to actually get started working on this, and then I'll go on to some other things like the security implications of some of these things. So what is SAML? It's a single login solution. So I'm sure you're all familiar with things like Twitter sign-in or Google sign-in, which we use ourselves at GitLab. And what these systems help us do is not have to remember lots of passwords for all these different services we use. So it means you can just click one button and you're signed into the service. But there's another side to this, which is for the company, they get a lot more control over how a user authenticates. So they might enforce that everyone uses two-factor authentication, or they might control which services you can access, or even remove access when you leave the company. So you go to sign-in and it no longer works, and the company doesn't have to worry too much about that because they set up their system and everything just works. So SAML stands for Security Assertion Markup Protocol, sorry, Markup Language, be some, otherwise. And the keyword here is assertion. So this is a confident and forceful statement of fact or belief. So SAML lets the organization make a confident statement that this user is James, his user ID is 27418, his email address is jadewasjames.gitlab.com, and he's a member of the Backend Developers Group, and he's on the managed team. So we kind of, so that's what an assertion is, and that's kind of the key to understanding how SAML works. Getting more into the detail, it's an XML-based protocol. So you see here we've got an assertion, it's got lots of different sections. The kind of interesting ones are these attributes. So we've got a UID attribute to user one. Here we've got an attribute that maps to group one and an email attribute. Another really key thing here is the name ID, because that's what we use to link a user. So when the identity provider, which is the service you put your password into, returns its response, it says, this is user one or this is user james.example.com. And so that means which user this is, that's the name ID. So if you're kind of flavor of how the protocol works, people normally would show you this kind of diagram. So you've got the user, you've got the identity provider, and you've got the service provider, which is, in our case, GitLab. But that's way too complicated to follow. So instead, I'll give you a quick demo. So at a very high level, we have this kind of My SAML button, and that's, in this case, I've called it My SAML, but that'll be the organization name. And so a user who wants to authenticate, they click that button, they get redirected to the identity provider. So here I'll put in my username, user one, my password, which is user one, password. Identity provider, you're running it locally on your machine, right? Simple SAML, is that what it looks like? Yeah, so I'll go into that a bit more later. But in this case, it's one I'm running locally on my machine. But in other cases, it'll be Azure, act ADFS, ping identity, one login, and behind the scenes, the company will have their own system where they've got all their users. And so it will, to the user, it's just the end of their password, but behind the scenes, it could be a number of different things. Yeah, I'm just curious, is it in the GTK to install this? So I'll go for that later. Okay, thanks. But yeah, feel free to jump in with questions anymore because there's a lot to take in. Sorry, yeah, so click sign in and you've ended up on GitLab. And I'll show you that again, and we're signed in as user one. I'll show you that again, but using a tool to show you what's actually going on underneath. So this tool is called SAML ChromePanel, but there's also SAML Tracer for Firefox, and I'll have linked to those later. So, and this time, of course, we're already signed in on the identity provider. So when I click this button, I'll immediately be signed in and redirected back, but there'll be a lot going on underneath, which the user won't see. So we've got these kind of two key parts. First, the request was made to, in this case, simple SAML to the identity provider, and we've sent them a request saying, please tell us who this user is, a bit about them and are they signed in. One of the things here is we've said the issuer, so that's who we are, so they can look up which service we are and respond appropriately. And another thing we've sent in this request is where we want the user to be sent back after this is all over. And that's because this is a browser-based protocol, so everything's happened in the browser in these requests. One other thing to note is that this request actually looks like this, because it's base 64 encoded, so that's why it's really helpful to have a tool that kind of decodes that for you on the fly and gives you this nicer thing instead. Where's the user insertion in there? Or... Yep, perfect question, because that leads on to the next part. So the identity provider responds with its SAML response. And back to that key part, it's the security assertion markup language. It replies with an assertion. Can't find it in here, so I can see the end of the assertion, but the stuff is all in here. So one of the things is the name ID. So in this case, it's a really simple thing. It said, this is user one. And it's also replied with some other stuff that I screenshotted before, so it's replied with the user and with the group. One other thing that is important for security is that this is a signed response. So it's replied with a signature value and also a copy of its certificate, so we can check that it's signed correctly. Just to go over some of those concepts again. So you've got the identity provider, so that's where you sign in. That's often called the IDP. So if you see that, it just means the identity provider. The service provider is us at GitLab or any other service like Office that you might be signing into. You've got the request, the response, the name ID, the issuer, the attributes, the response. Cool. And I just put some of these in here, but in case anyone wasn't watching the demo. The next thing is how we link accounts. So when a user's signed in and they click that button, we link their account using that name ID. So we say, the user currently signed in is the same person as name ID one, and we remember that link for future sign ins. With Instant Samal, users can also automatically be created. So if user two signs in, we can create an account on GitLab for them. And things are a bit more complicated on gitlab.com. And so I'll describe that. And any questions so far? Cool. So on gitlab.com, we've got another problem, which is that instead of configuring these, your Samal provider in the config, there's one, there's a Samal provider for every organization. So every single company that wants to sign up and use this needs to configure their own identity provider. So instead of having this kind of button on the left, which says my Samal, when you click it, when you go to user sign in, you have a page for each group. And so you get something like this on the right, saying sign in to my org. And so the configuration page currently looks something like this. And so instead of having it in config, it's configured per group. And that's an important difference because we then need to look up which group someone's signing in for and then respond appropriately. So just to go over that again. So Instant Samal, we've had that for a long time. It's in C and thanks to CERN for doing the initial implementation. But this group Samal for gitlab.com is something we've been working on this year. And it's very different. One of the things is the trust model because that identity provider that someone configures could actually be a malicious one, a one we don't trust. So I'll come back to that a bit later. And yeah, so I'm gonna have a sip of water and I don't think there's any questions. Go ahead. So how do you get started with all of this? Six months ago, I just said you need to set up Active Directory. You need to configure ADFS. You need to configure some users on that. You need to add certificates and then you need to set up some LDAP transform rules. But you don't have to do any of that because we have a Docker container in GDK. And so I'll run you through how to configure it with that. So the main thing or the first thing, sorry, is adding it to the actual config. So with instant SAML, it would have looked something like this. So you're saying what the fingerprint for the certificate is, you're saying where you want the user to be redirected back to to process that SAML response, where you actually want the users to be sent to for the identity provider and who we are. There's also this name ID identifier format which is less important, but we often want that to be persistent, but we'll leave it like that. But for gitlab.com SAML, it's just add the name group SAML and then you configure it through the UI. The next thing is to run this simple SAML PHP service. So we'll do that with Docker and I've forked, someone had a Docker image all set up and ready to go. And so we've made a fork that was minor changes that aren't too important. And we configure it like this. But first we'll go in and we'll create a group and I'll just sign in as root to do that. In reality, you wouldn't sign in as group. Sorry, you wouldn't sign in as root, you'd sign in as a user on the organization. So we'll just create a group and we'll call that. And so when we're configuring the identity provider we'll need those bits of information. Sorry, that path. And I will just stop the one I was running before because I'm gonna reuse the same port but in theory, you could have lots of these running. Then we're configuring the two things, same as before, where we want to be sent back. So that's sometimes called the assertion consumer service URL, which is a mouthful, sometimes called the callback URL. And we're configuring the issuer which here is called the entity ID. So there can be a lot of confusing terminology like sometimes assertions are called claims but it all starts to blend in and make sense eventually. So next we'll go in and we'll configure SAML for this group. And going back to the GDK docs, it tells us the certificate fingerprint for that Docker container and the URL we need to use. So I'll fill that in here. So we've got the certificate fingerprint and here we provide the information that we used when configuring the Docker container but our users will often be configuring that on Azure or on ADFS or on any of a number of services. Now, one of the differences I mentioned with group SAML is users don't sign in from the kind of global GitLab.com page. They sign in on a different page for each group. So we go to this one and we've got a warning saying, you're about to link this account and it will grant you access so it can sign you in to GitLab. So these will click authorize and be redirected to simple SAML and I'll just use user one again. This Docker container only has user one and user two. So, oh, what have I done? The thing is I demoed this earlier when it worked. No, really. Yeah, actually working. I've seen this error when like my web worker wasn't quite working right. See, I'll just restart that. One thing that I do that not everyone does is I run a worker in a separate process to the rest of GDK and that means if I'm debugging I can jump straight in there. You know what I've done there. This isn't a problem with UDK. This is the way I've configured the Docker container. I've put its callback URL to my demo group but it should actually be groups, my demo group, SAML, callback. So we'll go ahead and we'll, so down. So here it should have been callback and here it should have been my demo group. So it does show that this is the URL that the identity provider will send users back to. And so, yeah, configuration mistake. Let me just go back to the SSO page and double check that we're listening on the right socket again. And so user one, user one pass. If I were to inspect this with the SAML panel we'd see very similar activity to with the instant SAML. I think I'll click a bit late. Cool. And so it's saying we've now connected it to our account. So the user does not need an account before that or do they have to have an existing GitLab account? So allowing this on gitlab.com with sign in is a work in progress merge request. So I'll just describe how it is at the moment but if when you visit that page, you don't have an account it'll redirect you to the identity provider because it doesn't know if you have an account yet. They'll redirect back. We'll show the registration page and if, or the sign in or registration page if someone signs in then it'll redirect them back and they'll be the account will be linked for the first time and if they register when they do that it'll be linked. So we don't have a automatic registration process but because of the way we're doing the redirects they'll get shown the sign in or sign or register the page if they haven't linked an account before. But one of the things we're looking to add in the future is something called SCIM and so that allows organizations to add and remove users automatically so that they don't need to worry about this as much. And just to include a link to the tool that I was using and there's also SAML Tracer for Firefox. Any questions? Again, cool. So now I'm gonna go through just some areas of the code so that if you're reviewing a merge request or trying to look for security flaws you have a vague idea of where to look for things. There's quite a lot to go through so I'm just gonna start jumping into the code and kind of showing you different areas. So feel free to jump in with any questions. So we'll start with the identity model. So this is used by LDAP and OAuth sign in but one thing we do differently with Group SAML is we also store which SAML provider has been used. So when we're looking up who someone is we can say they've signed in with this SAML provider and they've got this UID, sorry, this extern UID which is the name ID. And so with that combination of we know who it is from the name ID and we know which identity provider saying who that is we can identify the user to look up. The SAML provider itself is just those things that we configured in the user interface. So that's the certificate fingerprint with a link to the group. And through those identities it lets us find all the users for that SAML provider. And you can see that we've got this name ID format that would have been configured in the configuration. Here that's configured in the SAML provider. So one of the key things about how this works within our code base is that we use Omni-Auth. And so that comes in before we kind of hit the rail stack. We, Omni-Auth does it stuff in rack middleware and it has its own kind of way of doing things. So it has a setup phase and a request phase. So that's where we create that auth request we saw earlier and then it has a callback phase. The thing we do differently for Group SAML is we, in that setup phase, we look up which SAML provider this is, do some verifications on that and then use the appropriate configuration for that provider. For instant SAML, we just use the Omni-Auth SAML library as it is. So just to be clear, it looks like the Group SAML stuff is only in EE, right? Yeah, so with C, it just uses the plain library itself. And just to give you, like when I'm developing this, a lot of the time I do have to come in to the actual library and have a look at what's happening here. And that's all configured here. So we've got the request phase. It uses the configured settings. It does a bunch of stuff with the fingerprint. And it also calls through to Ruby SAML, which is a library that does the kind of lower level validations and things like that. In both cases, the user then comes after it's been validated that that's a valid response of a correct fingerprint and everything's correct about it. Only then do we get through to the Rails stack and to the callback controller. And we kind of have, so it's, we kind of have two different paths that can be taken here, ones when the users are already signed in and ones when they're not signed in. And so it's a bit weird because we do two different things and everything's in two different places. But the kind of key thing to wrap your head around is that we have an identity linker. And in here, we say if the user's signed in, then we either link their account, if it's the first time or if it's already linked, then we do that check in here as well. And this code here is actually shared across different OAuth things, across held app. And so it's something that we override specifically for Group SAML there. If the user is not signed in, then there's this whole other bit of code. But it also handles things like logging, audit events, setting, remember me tokens, dealing with two-factor authentication. And by having that shared with the other ways users can log in, it means we don't end up accidentally forgetting to do two-factor authentication with SAML or anything like that. The OAuth user classes are where that's then handled. And so we have one for SAML and one for Group SAML. I want a branch where I don't have that implemented, but it does very simple stuff. Looks at the user and just returns if they're signed in or not. So as you can see there. So that's a very high level run through. Look, those are the key things to look for. Cool. Security. It's in the name. It's the security assertion markup language, which means that there's a whole lot of things that could go wrong. The main kind of ways that SAML helps protect us is it's got this protocol that's very well thought out, very well specified, and then it's signed with that certificate. And so you've got XML signing and you've got all these validations that different parts of that XML are correct and as we expect it. But that also means that those are all the places where it can go wrong. So attackers can bypass the XML signing. Sometimes they'll do this by creating a second element with the same name. And depending on how you pass the XML, you might get the one that's actually valid or the one that they've inserted. There was an attack where they use XML comments to change a user's ID. So instead of being user ID one, they can make it user ID one, XML comment thousand. And then using that, you can have it so that when you look at the XML one way, you're user one, and when you look at it another way, you're user 1000. And by doing that, they can choose which user that they want to be. And so there's all sorts of very clever attacks based around this. And so, yeah, I've got a slide here showing all the different validations that Ruby SAML does. And so you can see it validates the response state. It validates the signature, it validates that the issuer is the correct thing because if it wasn't GitLab but Office, they'd send that SAML request, they don't be completely invalid. It validates the time on the response. So when we get back a response, it only has a short window in which it's valid and that prevents replay attacks. But then it has another time that says this user is definitely this user for this many years or months or weeks. And so they can, so the identity provider can make these assertions with different time periods as well. Yeah, every single one of these is something that an attacker can work around. So we have to be careful thinking about those. Luckily, Ruby SAML takes a little of that away and does it all for us, but there's a lot of things we also need to be careful of. And I'll just jump to this slide for a bit. One of the complications with this multi-tenant SAML is that the identity provider might not be trustworthy. So when we're configuring in the config, our own identity provider, we know it's trustworthy, but when we allow any user on gitlab.com to add their own identity provider, they could then trick users into visiting some site where randomly on the internet being redirected to their malicious identity provider, which could then send back a valid response all by itself and then, hey, that identity provider can now sign its own users into GitLab so they can then go on their computer and pretend to be you and hijack your account. So although we haven't changed much, we have to be extra careful and we also need to protect against social engineering because we've got that warning page saying, are you sure you want to link this account? But if someone doesn't know what that means, they might give permission to someone else to sign them in. And some of those attacks I've been talking about, they've allowed uncontrolled access to Office 365 and to GitHub, so they're not just theoretical attacks, they're ones that people have actually gone out and found ways to, for research purposes, enact. But yeah, so we need to make sure we're very thorough when reviewing this and think about all these different possible scenarios. Questions? James, is there anything we can do to kind of approve or like pre-check the identity provider? I guess not because it could potentially be self-hosted and something that we're completely unaware of upfront. Yeah, so it's something, we could place an organizational process around this and saying whenever someone enables SAML, we'll check their identity provider. But another kind of attack that maybe kind of not like that as much is that someone might set up an identity provider just to play around with things and then that might have a vulnerability that can then be, if someone can get access to any one of these valid organizations, they can start attacking users as well. So it still wouldn't completely protect us and it would add a lot of process overhead. So it's not perfect. So just to back up here a little bit. So, you know, with LDAP Group Sync, basically you've got groups in your LDAP provider and then as soon as you log in, we automatically sync users that belong to the group. Is the same thing happening with SAML here? So it's something we'll plan to do but at the moment, they configured the identity provider for the top level group and we don't do anything other than adding the users to the group. But because we've got those assertions and they can say this uses in Group X, we've got plans to then use that and say they've asserted their own Group X, let's add them to Group X. But that's something we need a bit of design around and also which permission level they have because at the moment, they'll be added as guests but we might want to allow them to add people directly as developer or maintainer. Okay, so the user flow right now. So let's say this gets deployed to GitLab.com next week or whatnot. People set up there, they have a group, a GitLab group. They set up their own SAML provider, right? And what does that give them now? That gives them ability to just people to log in and get access to that group or they start to manually add users to that group? So what we've already got on GitLab.com is that ability that when someone signs in with SAML, it automatically adds them to the group. So if I sign into that MyOrg SAML provider, I automatically get added to that top level group. What we don't have on GitLab.com yet is the bit that actually signs you into GitLab.com. So they need to enter their username and password first. So it's kind of halfway there. Ah, got it. So it syncs to the existing group but it doesn't automatically log them in. Yep. Got it, okay. Yep. And that kind of leads us on to the things that we want to do next. There are also things that we're going to need security review on, things that we're going to need UX review on and things that I'd like to be able to sign to maintain as well and having to say, I don't know what this is. I'm not too sure. So these are all things that are relevant. So as I mentioned, the actual sign-in bit is something that's work-in-progress merge request. And as part of that, there were these kind of three other merge requests allowing users to unlink their accounts and providing that warning up front. There's also this metadata configuration because one of the things that SAML administrators like to do is just get a URL for some XML, auto-configures everything for them. And the next stage is providing administrators with more power over enforcement because they want to be able to say, all the users on this group all use SAML so that no one can be added or accidentally get added. That's just a normal getLab.com user. Another thing that we see people wanting is the ability to say that that user is currently signed in with SAML. They haven't signed in with their getLab.com username and password. And so that lets them for audit reasons say, yes, they've definitely used all our policies. They've used two-factor authentication and we haven't removed them from our system because they're signing in with the right provide with SAML. And then two other things are making those memberships expire if someone hasn't signed in recently and then allowing them to mass-add users and remove users when they leave the company. And to do that, we can't really use SAML because it's a browser-based protocol. So if someone's left the company, they can't exactly log in and go through the flow and be trusted to do that. So we need to look at another technology, SCIM. And then kind of next area for improvement is the stuff we're doing up with setting the access level. So whether they're guest, report, maintainer and deciding which subgroups to add the user to and also making sure that those notifications go to the email address they are provided in that assertion rather than just their personal email address. Finally, just some of the area we want to work on after that is making things easier for the user. So things like sign out a better registration flow and just items which are polished in the UI. So things like displaying the active sessions on the user profile correctly. To recap, SAML is great for organizations, managing users, great for individuals, only having to click one thing to sign in. Group SAML is the thing I've been working on. And we've got a lot more to go, but it's going to be very powerful for organizations which want to move to GitLab.com instead of self-hosting. And so that's something that for sales is really important and something that's really important for a lot of organizations. And so it's a, yeah. Just some resources that are linked to. You've got the full SAML specifications in all their detailed glory. Some links to the doc. And you can reach out if you have any questions after in GManage on Slack or myself or Jeremy Watson's the PM for this. Any final questions? So I'm just curious in terms of like people, why would someone choose using a SAML provider over like an LDAP provider? So they're kind of a different beast in some ways. So for GitLab.com, one of the reasons not to look at LDAP is it sends passwords over the internet. So someone using LDAP, they put their password into the UI and we don't really want organizations having their users give us their password each time and sending that over the internet to their server and it's a very different approach. For self-hosted things, when it's all within a contained network, sometimes it's just the convenience for the users. It's one click. If they're already signed in to their office account or their gyro account or anything else, it's just one click and they don't even need to notice it. They don't need to put in a password each time. Yeah, it solves a very similar problem to LDAP. Cool. If you have any other questions that you think of straight after we finish, just as I said, ping on gmanage. It takes a lot, James. Thank you. That was awesome, James. Thank you. Thank you. Thanks, James. No worries. See ya.