 OK, well, it's probably should get started. I have a lot of slides to go through here. So I'm Nathan Kinder. I am an engineering manager at Red Hat for our identity and security OpenStack engineering team, as well as our Red Hat directory server product and Red Hat certificate system products. I'm a contributor to Keystone. I'm on the OpenStack security group. And I've been a long time contributor to 3D9 directory server project upstream. So I'm going to talk about leveraging existing identity sources in OpenStack today. Sort of a review of what's possible just as a kind of primer, as well as new things that have come in Keystone in Juneau and kind of where things are heading that will help in integrating with existing sources of identity. So first, just a quick overview. Keystone's general role within OpenStack, which can be interpreted in a lot of different ways. But its primary purpose is to provide an authorization context for OpenStack. It's not designed for being an identity management solution or doing user and group management. It does have some limited capabilities there. But at the core of it, its responsibility is handing out a token that basically defines what your authorization should be based off of roles. Typically, in most scenarios, you have an existing identity store that has your users in groups. You don't want to set up a brand new database of users in groups just for OpenStack. You just want to allow your existing users to consume resources with an OpenStack. Along with that, being centralized already for other applications, you quite often find that there are existing tools in place for managing those users in groups and controlling authorization. And from an auditing perspective, that usually is centralized and everything needs to go through one place there. So for Keystone, there are a number of ways to integrate. And it's expanding more and more each release, each cycle. So we'll go through some of that. So the scenarios that I'm going to outline here, especially with LDAP, there are a few things that we should go through up front. So Keystone is basically divided at this point. And it has been for a number of releases now between an identity side and an assignment side. And you can have different databases for each side. So an identity side are your users in groups and projects. And we'll go through that in more detail. The assignment side is really OpenStack specific. This is your definition of roles within OpenStack and who those roles are assigned to, hence the name assignment. So the scenarios that we go through here, and particularly with LDAP, are assuming that you are leveraging LDAP purely for identity and that assignment will remain within Keystone's SQL database. And in most cases, definitely in the private cloud case, that's really what people are looking for. They aren't allowed to add additional things into LDAP. The team that's deploying and setting up OpenStack may not even be allowed to write to LDAP. And if they do need a write, they have to go through a process and a change request and such. So this is assuming that identity and assignment are split like that. Consuming LDAP is also what the scenarios are tailored towards. So the read-only case, like I was talking about. If you have centralized management of your users and groups already, you're not going to go through Keystone to also manage those users. It's a very limited set of what you could do anyways through the REST APIs. So Keystone shouldn't be an additional management interface. Management should be centralized. Benefits to using LDAP over the alternative within Keystone, which is just a SQL identity back end, is you have centralized authentication identity info that other applications can use. Keystone's not designed to be a central store for non-OpenStack applications to use. Some people I've talked with, there is a misunderstanding about that. And they want to find out, how do I get my other applications to leverage Keystone? Really, the answer is you shouldn't. And typically with LDAP servers, you have things like password policies, account lockout, and activity, all sorts of things that Keystone itself does not have at all. You have no password complexity. You have no lockout after X number of failed attempts or anything like that right now. And then replication as well. Most LDAP implementations have a pretty solid replication scheme. And it's usually preferable for a lot of folks to use that instead of trying to synchronize a database. So we'll go through a few scenarios. Basic LDAP identity, and this has kind of been there for some time within Keystone. So this is assuming that you have a single LDAP identity source that contains your users in groups. And of course, you want to just leverage that for authentication and authorization. So this is a single namespace. So this is not using Keystone domains. This is kind of like the V2 Keystone scenario. It's a single namespace for your users and groups and projects. So it's not like a multi-tenant environment or anything where you would have customers in a public cloud case. So this really fits the private cloud model. I'm standing up an internal OpenStack instance, and all my users are in one database, and I just want to use them. Although it also could work for a public cloud case and does already in certain scenarios where you don't have a need for tenant isolation in LDAP. So it's fine that they all go in one database. And that's really no different than SQL in those regards without Keystone domains. And so basically, I mean, this is showing how your users go through Keystone, and it just leverages LDAP like any other application that you might have deployed to use centralized authentication. So in terms of how Keystone uses LDAP, it's fairly basic in what it does. So it does search operations for your user and group lookup when you're doing things like an authentication or just user list operations or user show from the command line utilities. And then it does bind operations for authentication of your actual users. And so it's actually binding as the user to check the password versus doing something like expecting to get the password hashback or something non-ideal. So the rights that Keystone needs to do those things are fairly limited, which is sometimes misunderstood. You don't need to have some sort of administrator account set up for Keystone to bind to LDAP and do things. All you need is read access to your user and group attributes. And in many LDAP servers, you either have anonymous access or you create an unprivileged user that can only do reads. So in the case where you have separate groups within IT that run LDAP and run OpenStack, you don't need to request some sort of special privileged account that can do things and mess with data. And then, well, yeah, that's pretty much it there. So in terms of the attributes that Keystone uses, that's also fairly limited. And in most situations, you would have quite a bit more in your user and group entries with typical schemas that you see in an LDAP server. So for a Keystone user object, you have an ID, name, description, enabled, and mail attributes. So it's really pretty limited. I mean, mail and description are just kind of useful things, I guess, to show, but they're also optional. You don't even need them. So we really get down to having an ID and a name and an enabled attribute. The enabled attribute is sort of interesting because not all LDAP servers have an enabled attribute or it's expressed in very different ways. So if we talk about Active Directory, you have user account control and it's a bit field. And then you might have, like in our Red Hat Directory server, you have an attribute called NSAccountLock and it's true or false. So Keystone's pretty flexible around that attribute and it's not a direct mapping. If you don't even have an attribute at all, you can control, if an account is enabled, it's simply by group membership. So you have an enabled group, if the user is a member of that, they're enabled, if they're not, they're not. So that adds some flexibility. It's also, though, not very performant. It adds a lot of extra operations. So it's something to use with caution and it's really preferable to use an enabled attribute if there is one available. On the group sides, it's even more simpler. ID and name, description, and of course, member attribute, so you can see group membership and do things like role assignment there. ID and name are a little bit confusing because they're really both considered to be unique identifiers in the way that Keystone has used them. So we'll get to Keystone domains, but within a Keystone domain or in this case of the basic LDAP case where you're not using domains, the name has to be unique within the domain. But so does the ID, so it's a little bit pointless. They can be mapped to the same attribute and that's actually preferable in a lot of cases. If you look at it on the SQL side of things, typically you see the ID as a UUID and name is sort of like a UID that we'd use to log in. So for LDAP, you usually don't have a UUID and you can map them both to UID or SAM account name or something similar. And we'll go through some mappings. So here's a pretty basic example. I need a person, very common object class to use for user objects and just showing kind of what the mappings would typically be configured like here. So as I said, ID and name from Keystone, you could just point them to the same attribute. There's no reason that you need to create some additional attribute and extend your schema to a store UUID or do something odd like point the name at CN because nobody wants to type their first name, space, last name to log in to anything. As I said, mail and description are optional but there's pretty direct mapping there. So that's kind of your more standard LDAP what you'd see with OpenLDAP or 3D9 directory server. The other very common case of course is Active Directory. So with Active Directory, things are just done a bit differently. It's mostly LDAP compliant but not in some areas. So the user object class is what's used for users there and SAM account name is a pretty common one to be able to use for your login. And so ID and name can go to SAM account name which is essentially the equivalent of a UID in a POSIX sort of user. And then as I was saying, the enabled attribute is done differently. User account control is a bit field. There are settings in Keystone to essentially set a mask against that bit field to check it so it's aware of AD isms here and knows how to handle it. For a group, it's also very basic ID and name for CN. You usually don't have anything like the UID attribute, the CN's typically the identifier for a group, description, and the member attribute. An important thing to point out here is currently Keystone is not capable of using the member of attribute which would be a nice enhancement for it. But it actually looks up group membership by searching the group and looking at the members there versus seeing a member of attribute that's directly in the user entry. So it does expect the member attribute to be a full user DN. So if you're using a schema where you had member UID and it's just the UID of the user, so if just nkinder was there as a value, Keystone wouldn't work with that. So it doesn't have flexibility today there. So it's an important thing to be aware of if you're using that sort of schema. So the directory information tree structures vary quite a bit from LDAP deployment to LDAP deployment. It's not specific to any one server. Active Directory can be set up in a very standard way where you have more of a flat tree of your users and groups in one container. And then you look at other deployments and they heavily use organizational units to have more of a hierarchy. And so there's some flexibility in Keystone that allows you to deal with these different sorts of structures. So just for some review, a flat structure basically is a case where entries of the same type are kept in the same container. So you might have a users container, all your users are there. You might have a groups container, all your groups are there. They can't even be in the same one and that's still considered flat. What a more hierarchical nested setup is like is you have lots of organizational units which could map to things like geography, like you might have an OU, EU and OU, US. It could map to departments within a company. This was much more common in the past. People used to essentially have their tree follow the structure of their company and that was a hassle because reorgs and people moving entries around. So you see flat trees are much more common today. But OUs are also still frequently used in like AD cases where you might do like group policy and things of that nature. So just a visual example of these. So flat bit, you have all your users under CN users, all your groups under CN equals groups. The flexibility for dealing with this and kind of how you would configure Keystone. So to tell Keystone how to recognize a particular object like a user, it's all done based off of object class and this works well when you have a completely flat tree where you have groups and users and you may have other entry types there. You basically just specify the object class that represents a user and Keystone's going to use that as part of its search filter to find users. And then you can add more flexibility where it might be that you have your LDAP server for your whole company, but not everyone needs to be an OpenStack user. You don't need to have a separate tree for your OpenStack users. You basically can just have a filter and filter out some of those users. So a really easy way of doing this is to create a group like an OpenStack users group and add the user filter there. So basically a user will only be recognized if they have the right object class and if they match that filter. So Keystone essentially makes an ANDed filter and does a complex filter for that and combines them. And then the search scope, Keystone has a setting query scope, you set it to one so it won't do a subtree search. And so with our nested dit, and you can see in this case we have a couple different OUs. These might represent different sections of the company, for example. And so to deal with this, it's pretty similar. The object class part's the same, the filter's the same, but the scope you could just set to sub and you set your user DN tree, or user tree DN setting higher up within the tree so it can just go throughout the hierarchy. It doesn't matter how deeply nested it is, it just does a subtree search. So I think the filter is pretty flexible here and it doesn't have to be based off of a membership attribute, there could be some other attribute in there, maybe you have employee type or a department number or something like that. So utilize the filters essentially to select out your real users there. And that works the same with groups, it's essentially the same exact settings. So for authentication, Keystone does an LDAP simple bind operation as the user being authenticated. LDAP standards have support for more secure mechanisms. So essentially through a sassel bind and things like DigestMD5 or you start to get in GSAPI and other mechanisms there that are more secure, Keystone doesn't support that right now, unfortunately. So it's just plain LDAP simple bind. It's not doing something like what you might've seen a dozen years ago where it expects to actually read the user password attribute, so that's good, shouldn't be doing that. But a really important thing to be aware of is LDAP simple bind is taking the clear text password that the user provides to Keystone and it's sending it right over the wire to the LDAP server in the clear, the LDAP server hashes it and does a compare. So using TLS is just critical and a lot of people I think might miss doing that. So that's really, really important to set up because your passwords are essentially going around there on your network to your LDAP server. People could sniff them. So for authorization, and this gets more into the assignment side of things, but you can control that via LDAP as well. So role assignment could be done at two levels with the Identity V3 API. So you can assign a role at the user level, which is great for one-off cases or a very small set of users for some proof of concept deployment. But if you have thousands of users, you're not going to do individual assignments. Keystone has a concept of groups in V3 and that's really what should be leveraged for the bulk of your assignments. So you essentially assign your role to the group and then you control authorization by adjusting group membership. So this is really natural in the LDAP world and if you set things up in Keystone where you have your group assignments defined, you can control everything from your existing tooling that you use for LDAP user and group management. So you don't have to go off to Keystone and create a brand new assignment unless you're creating a brand new group. So there are various ways you can go about it so you're not switching between two different systems to control authorization. Try and get your groups established well in a logical way and get your assignments set and then control everything through LDAP and group membership. But groups are only available with the V3 API so that's something to be aware of if you're running the Keystone CLI which only does V2, you're not gonna see anything for groups. So use like the OpenStack common client and you'll be able to see all of the various commands that are possible there. So the downsides with the basic LDAP setup are only a single LDAP server supported so if you have multiple identity sources, you can't use them. You would have to set up a whole other Keystone and then kind of have this isolated setup. Also service users end up having to be stored in LDAP and with what we talked about before about read-only access to LDAP, a lot of places don't like storing service users. So you might have AD that the AD team runs and they're gonna tell you to go away, we're not gonna add service users for you. So that's a downside for a lot of people. And then lack of administration granularity, this gets more into Keystone domains which we'll talk about a bit more but if you look at the default policy with Keystone right now, you kind of have one top level admin and they're responsible for everything and they can't delegate privileges down. So domains are what helps us with that. So quick review of domains in Keystone. So a domain is essentially a grouping of users, groups, and projects. An object of one of those types essentially can be thought to belong to a domain. They don't belong to two domains. This allows for tenant isolation within Keystone. So the classic multi-tenancy example, you know, Coke and Pepsi sharing a same database. That's sort of what this allows. You would set up a domain for Coke, you would set up a domain for Pepsi. They're completely isolated from each other. So domains have been around for some time in the V3 API but V2 is default right now and most people are using V2 still in a lot of places. V2 has some fallback support which we'll talk about a bit but in terms of domains, it's important to be aware that you have to use V3 and with LDAP and domains in particular, it's only a Juno thing which we'll go into. So this kind of shows you what I was talking about with the domains. They're completely isolated from each other and so there might be an end kinder user in domain A and there might be an end kinder user in domain B and that's fine, they're completely separate users. So authorization with domains, Keystone has a separate policy file that's domain aware, the policy V3 cloud sample file. What this allows is for more fine grained administration so you can delegate things down to various levels so it's not one top level admin that has to do everything and always has to be bugged. So in this case, you would have a cloud admin. They're responsible for managing the domain objects or the creation of a brand new domain, managing the end points that end up in the service catalog, managing identity providers if you're using Federation, managing policy objects, managing regions, services and roles. So things that apply to the entire cloud are managed by the cloud admin. The domain admin is responsible for management of users, groups, projects and role assignment within their domain only and with this sort of setup, you don't have to go off to the cloud admin and have a manage all role assignment for you, you just delegate that down. So if this was a case of more of a public cloud or where you have partners coming into your system or even different departments within your company that you wanted to have different domains, you just hand it over to them, you assign your quotas there and you say, go ahead, manage your sandbox. Then you can go down one level further which is the project admin. So when you create a project, you can give it an admin and they can be responsible for role assignment within their project, but they don't manage users or groups, they can't create other projects. And so that's what the policy V3 cloud sample will allow when you're using domains. And so this shows the kind of separation of roles where you have the cloud admin that is responsible for everything. They set up a domain, they establish a domain admin for that domain and then they can go off and manage their other things within there and then the users are isolated. So domains in LDAP is new in Juno. It was in Ice House for SQL only, but still not really widely used it seems. So in Juno, the interesting thing is you have domain specific drivers that can be set up. What you do in this case, if you enable the option, is keystone.conf has an LDAP section where you can figure all the mappings and such and where the LDAP server is that we went through earlier and you can have a completely separate config file with that section for each domain. So this allows you to point to multiple LDAP servers and map them to different domains, essentially. You could point separate domains to a single LDAP server. There's no reason to have to be separate LDAP servers, it's separate config. So maybe it's just a filter that's different or maybe one domain points to one organizational unit and another points to a different organizational unit. So there's a lot of power there if you look at how I'm using filters and object classes to choose different subsets of entries that should represent different domains. And then the other really nice thing that this adds is you can still have domains in SQL alongside of LDAP and this solves a number of problems for us. The main one that it solves for us is the service accounts. Service accounts can still live in SQL in their own domain and all of your regular users can be consumed from your LDAP server that you already have in a read-only fashion. So now you no longer have to ask for those to be added. So I think a lot of people have been waiting for this and that's been a source of frustration for some folks in previous releases. And so this sort of shows that concept of the multi-backend approach. The assignment side just stays with SQL identity. We can have multiple domains there. So as I said before, domains is a V3 only thing but there is some fallback support. The big difference with domains that clients need to be aware of is you need to specify a domain to really identify your user uniquely in some cases which the V2 auth request in the API has no provision for doing that. So if a V2 authentication happens and you're using domains, Keystone automatically maps all V2 clients to what we call a default domain. It doesn't have to be named default although we set one up like that out of the box but it's just a regular domain and you say V2 clients all go into this domain. The V3 API has the ability in the auth request in JSON where you can specify a domain for the user. It's not needed when you are using IDs. So if you're doing your auth request with a user ID that uniquely identifies the user. If you're using a name, a name could exist in two domains at once. So we need to know which one to look in so you have to specify a domain there. And there are environment variables if you're using the CLI utilities or options to specify like OS user domain ID. But that's an important thing to be aware of because it does affect how the clients interact with domains. A common case if you just had a SQL domain for service users and you had your regular users from LDAP and another domain, you could just set your default domain to be your user domain and that way they can keep using V2. There needs to be some work done in the other projects to be able to consume domains for service accounts but I think that will be a pretty common approach at least for the private cloud case. The other really important thing to note is that the authorization scope of a token can be at the domain or the project level. So you can request a token at the domain level and this is what the domain admin would do. And that would allow them to perform the higher level domain admin operations. Or when you request a token, you can specify the project ID which is just like you can do in V2 and you'd get it scoped at that level. But if you're a user that's the domain admin and you get a project scope token, you're not going to be able to perform your domain admin operations. Which can be a bit confusing at first if you're trying to use the command line utilities for this. So roles, going along with that, roles can also be assigned at the domain level. So when you go in to do an assignment instead of giving user X, roll Y on project Z, just domain Z instead. So that's how you can actually grant things at the domain level. And there's also a concept of an unscoped token as well which has no scope and provides no authorization but proves identity. So that's domains, federated identity. This is also something new in Juneau. So Keystone has generic federation capabilities which allow external users to be recognized. It's typically SAML that we see with it but it's designed in such a way that other schemes can be used. So with federation, users are not persisted in any database that Keystone's pointed at. LDAP or SQL doesn't really come into play. So when a user comes to OpenStack, we basically are able to synthesize the user in memory and be able to figure out what roles would map to them. So you set up basically a trusted identity provider relationship and that gets utilized and we'll go through how that happens. So the use cases here are there might be an existing internal IDP within your company and you have a stronger authentication to that. Maybe you're using Kerberos to get a SAML assertion and then you can just take that SAML assertion to Keystone. You no longer are giving Keystone your actual password credential which is a great thing to do there. So that's one common use case where you have that SAML provider or whatever it is for single sign-on within your company. So you already have it, it just gets passed around and you have a nicer user experience. There might be a case where you have an inaccessible LDAP identity source or maybe it's like a partner and they have their IDP publicly available but LDAP is internal and their IDP might use it. That's another good case for Federation. Or maybe it's a non-LDAP identity source. So you're not using SQL, you don't have LDAP, you have something else. This is another route to get into Keystone and use your existing users. So we'll go through the SAML case but again, there's nothing in Keystone itself that's SAML specific. So we can do other things like OpenID Connect and things of that nature using this as well which we'll talk about a bit. So SAML concepts to understand though are there's the IDP which the identity provider, that's the trusted provider identity information and SAML, that's who's going to issue a SAML assertion. Your service provider is basically Keystone in this case. It's the service that's consuming the identity information and it needs to trust the IDP and then you establish that trust when you set it up. The assertion is a trusted representation of your identity attributes that's issued by the IDP. So it'll say what the user's name or ID is, it might say groups that they're in, other attributes about the user which are going to be trusted by the service provider and that's the information that we can use to map roles to the user. So this is basically how federated identity is set up. Keystone has to be run in Apache which is now the kind of standard and recommended way as of Juno. And the actual SAML work all happens outside of Keystone. So we're using ModShib, the Shibboleth module here. You could use things like ModOthMelon or other SAML providers or like the OpenID Connect module. So the user comes into Keystone into Apache there and they actually hit the module. The module does a redirect to your SAML IDP, they authenticate with that, they get back a SAML assertion and the Apache module passes it through to Keystone as a set of environment variables. So all the SAML heavy lifting isn't in Keystone at all which allows different scenarios to be plugged in there. So in this case we get a username, maybe some group member attributes which list our group membership. So Keystone has to basically consume those and figure out what to do with it. So we'll get into how that works here. The OS Federation extension is the extension within Keystone which is used for this. So you have a couple different resources that you have to create when you set this up. You have an IDP and this basically ties a special Keystone URI to an IDP and we have an example here. So whatever you name your IDP, this is an URL that your users are going to do when they're doing a federated authentication to Keystone. You create a protocol, so this would be like SAML or OIDC or you name it whatever you want but essentially the protocol lives underneath your IDP and so it's basically assigned to it that way and that's where you can do your authentication at that auth URL for that particular Federation type. And then you create a mapping object and the mapping is actually assigned to the protocol when you create it and we'll go into how the mapping works. So when you're using Federation, assignments done at the group level, so groups are created in Keystone, you do your assignment there and all the mapping is really responsible for is taking your users and figuring out what Keystone groups that they're in. And so we'll show an example mapping that does that. The Federation-specific auth URLs are what the users have to use. They don't go to the normal auth URL. They have to go to a special Federation one so it knows to go through the mapping code. And when you authenticate, you get an unscote Federation token which you can then turn around and use to get a scope token. And so here's an example of we have our values from assertion come into the mapping rules and what gets spit out there. You know, we have the name and the ID that end up getting filled in and we had a mapping that says, oh, as admins goes to the groups and we'll show you a mapping too. And so here's an example mapping rule. We look for things on the remote side which is the SAML assertion and if that rule gets met, then the top thing gets set within the user object. So you have to create groups within Keystone in this case to hang your assignment off of. And so external LDAP identity is basically just a layer on top of Federation that is kind of a more up and coming thing as a new possibility. Keystone has external auth plugins which allow you to have the web server upfront and do authentication much like ModShib. You could use things like ModSSL for X509 client cert auth or ModOthCurb to do Kerberos authentication and it sets a remote user variable which Keystone can then recognize. It also can set remote domain so you can have users mapped into specific domains there. So if you combine the Federation extension with the external auth plugin, you can do some interesting things here with this mod lookup identity module. So mod lookup identity calls down into the underlying platform and into the SSSD daemons. SSSD is basically for system auth but also application auth and can talk to LDAP or Active Directory. So we can call down into that to get all of our identity information from LDAP and Keystone doesn't have to manage LDAP connections or any of that itself. And then it exposes that as environment variables which the Federation mapping can use. And SSSD provides some nice benefits that we aren't able to do within Keystone itself or are just starting to be able to do such as connection pooling, being able to use multiple identity sources in a single backend and being able to cache results. And so this is essentially how that model would look as well. And so you start to get strong authentication into Keystone which is much better than what we have today and you take out all of the need for doing schema, knowing the schema logic within Keystone, having to manage the connection side of things, having to do paging or anything like that and you offload it to the underlying platform. And so that's it. We're pretty much out of time too. But there are a lot of different ways there that we can start to integrate. And I think building off of some of those are really going to allow us to get another level of security in terms of our authentication to Keystone and throughout the rest of the services, so. All right, well, thanks everyone. Thank you.