 So, hi. Hello, everyone. My name is Martin, and I'm here today to explain how to integrate Active Directory with Keystone. First a bit about my background. I'm a tech writer at Red Hat, where I work on the OpenStack and Keystone and Neutron documentation. But before becoming a tech writer, I was actually a sys admin in Active Directory and VMware. So I imagine many of you are here today because you have an OpenStack environment and an Active Directory environment and you want to be able to integrate them together in some kind of meaningful way. And specifically, you want your 80 users to be able to use their username and password to log into OpenStack. Well, if you haven't done that kind of integration before, I might understand why, because under Keystone version two, you might have been expected to do a few things that you weren't exactly comfortable with. And well, I have an entire slide to explain those things, and so I'll get to that. But I will say that first I need to describe that this procedure is looking at the here and now today. So this is what customers are doing in Keylo and Liberty with Red Hat OpenStack seven and eight. This is how people are today in the market integrating with Active Directory. So first I'll explain what's being discussed in this talk. I'm going to give a very quick overview of Active Directory and Keystone, how they relate to each other for the purposes of this talk. I'm going to describe exactly what you needed to do under Keystone v2 and why you might have been unhappy with that. I'm going to talk about how Keystone version three solves all of those problems. I'm going to make a big deal about encryption using LDAPS, and then I'm going to also go over the steps that Windows admins would need to do, and then also what the OpenStack admins need to do to their Keystone and NOVA. I have a demonstration prepared in real time, and then we're going to look at just the high availability options to make this a bit more resilient. There is also a landing page for this presentation. You'll see the URL over there. So that's where I have a link to this guide that's the partner guide, have a link to the slides, and whatever else that I feel is relevant I'll throw in there as well. So if you feel like you need to take notes and everything, well, some more background is that there is a whole integration guide that we've created that talks how you can do this integration with IDM and Active Directory and just generic LDAP. So if you go to that page, you'll see a link to that guide, and everything I'm saying here gets unpacked in full detail right there. Cool. So a very short overview of Active Directory for purposes of this talk. It is, in fact, a Microsoft product, and it uses an LDAP back end to hold a central repository of user accounts, and these accounts get replicated in a database to a number of servers. These are known as domain controllers, described here as DCs. And so any changemaker on DC will get replicated across to another one. That's pretty handy. It is considered LDAP-compatible, and you can encrypt your LDAP lookup traffic using LDAP as encryption. So I just make a note there specifically to say that tested in my personal lab, but also if customers are running, I've seen customers using Server 2012 R2 for this integration, 2008 R2, and anecdotally, I've tested this myself with SAMBA4's domain controller emulation, and it actually works identically to what I've done before, but I suppose whether that's actually supported by your vendor or not, that's going to be a question for your TAM or SRM. So a quick overview then also of how Keystone fits into OpenStack. Keystone is essential to anything working in OpenStack. It performs AuthN and AuthZ services, so the AuthZ is using credentials and, oh no, sorry, I always get this mixed up. The AuthZ is with, no, I was right. The authentication is the user's name and password, and the authorization is how the Keystone validates it. You actually have access to the permissions that you claim to have. It's also the way that tenants are managed, so you can keep users segregated from each other and end points for all these services, how they would integrate with Keystone, it manages the end points, and specifically this validation is done with Keystone version 3 multi-domain. So if you're running Keystone version 2, you're going to need to use Keystone version 3 for this to work in order to get access to the multi-domain compatibility. So the old way that you might have seen on the Internet and a bunch of documentation out there is for Keystone version 2, and this is a long list of reasons why you might not have wanted to do this, so it required you to create a bunch of AD user accounts and instead use those service accounts instead of your local accounts that you are using for each component. So this, as you can imagine, resulted in a lot of concern for people because if you lose a lot of control over your IPv2 environment, if something happens to your service account while it's in AD, it could become disabled or some security policy gets applied that disables it or the password expires, you're going to have a bad time because that service could get impacted and not be able to start properly. I suppose also for political reasons, OpenSec admins are going to lose control of service accounts, you might not always like that, but then there are also things to make the AD admins uncomfortable because it required writing to LDAP and if you can avoid that, that should be the way to go. Also, yes, you'll see steps that require you to actually make changes to the AD schema, which is a major task that you can do to your AD environment. If something goes wrong, all authentication could potentially fail and to recover from that, you'd have to take the major step of shutting down all your domain controllers, powering one up, restoring from backup, then letting the good one replicate all its goodness back to the bad ones. Active Directory is unavailable for the duration of that, so if you can't avoid making changes to your AD schema, that's really what you want to do. Keystone version 3 solves all those issues that I've just described. Again, this is using the multi-domain support that is what enables this. You'll service accounts state locally in your Keystone database. There are no AD schema changes required and LDAP read only. These are all things that are going to make a lot of people feel more comfortable about this integration. Also, yes, if your domain controller becomes unavailable, it's going to not affect services. The service is going to keep running. Who will be impacted will be those number of users who have their user accounts in Active Directory and are expecting to be able to log into your OpenStack while they're not going to be able to. I'd describe later on how to make that configuration more resilient. Yes, encryption is important. Let me go back to the previous slide. You'll see, I'll just highlight there, user logs into Horizon and they authenticate with the username and password, and that goes through to Keystone. That's great, but what's going to happen is that Keystone is going to query the domain controller and that connection under vanilla default 389 lookup is not encrypted. So it's going to look like what I've highlighted here, clear text, SBC LDAP, password there and clear text for anyone to go and grab. This, yes, the result of just a TCP dump running on F1. So I think I've made that point really clear. I'm going to describe next how you can solve this and how you configure LDAP's encryption on both the 80 side and on the OpenStack side. So I think I've made that point very clear, encrypt. So here is all in one page, everything you need to do for LDAP's encryption. I will first give you a tour of, let's go over there. So here is the Active Directory domain controller. There's going to be a certificate. So your Windows admins are going to be very first step, step one, they have to go and export the certificate that they've configured. It's going to be the local and the local computer for that service. They're going to need to export out the certificate and they'll know it's the right one because they'll have set this up previously and it will be the one that's using server authentication. So they export that out as a, let's go back, as an X509.CER file that can use the GUI just to do that and then they securely send that to you. You can use whatever secure means are available to get that onto your OpenStack server and then you're going to convert it to a .pam file from the previously used CER. And then, I suppose for REL, you're going to copy it into that location and then update CHRUS and that'll make REL aware of it. If you're using a different distribution of Linux, you'll have equivalent steps. And then in order to get Keystone to use it, you're going to copy it, you're going to convert it again and then copy it over into that SSL search directory. So there's a later step where you configure Keystone to refer to that location. So I'll get to that. But here is the LDPS configuration on one page so that what I'm describing here isn't going to happen to you. Oh, yeah. So here on one page as well is everything that your Active Directory admins are going to have to do. There is another little step that happens later on but it's pretty optional. So this is PowerShell code. So they're going to create a new AD user called SVCLDAP. Now, they can choose whatever name they want but I've just used SVCLDAP to give it more of a service account name. And this account does not need to have any special privileges. It doesn't need to be an admin account. Just normal user will do. Create it, set a password for it, enable it and then you would then create a new group called GRP-OpenStack. And that group is going to be the group where all the users that need access to this Active Directory, sorry, to the OpenStack environment, they're going to need to be members of that group. So create the group, then add the user that you just created into that group and then that will be all they have to do except for a little bit later. And then for the ongoing BAU of continuously adding more users to that group and removing them according to your business practices. So here I talk about what needs to be done on your controller. So if you're not using controller concepts, this would be on your Keystone server. You're going to configure SC Linux because obviously you're running SC Linux or you might have an equivalent but that's what you need to set on it. And then you need to create a domains sub folder. So this is a new Keystone v3 specific step. You create a folder called domains and then you give the Keystone service ownership of that. And then you go into your regular Keystone.conf, enable domain specific drivers and then you point it also to that folder that you just created. And also what you see there is the back end is looking at my SQL. That's the bulletin for authorization. Yes. So some more steps you have to do on your controller. You're going to configure the local settings for dashboard. So what this is going to do is on your dashboard, it's going to look a bit different. I'll take you out here. So yeah, what's going to happen, what's going to appear there now is a new field called domain. And so when your user logs in, they're going to have to type in that domain. That field is not case sensitive, so they can use some variation that they're familiar with. And then as a result, they'll be able to log in with their username and password. I suppose once you've completed the rest of the steps, but that's going to be the change to the user experience. And so returning. So a step for the Windows admins, if they don't really know it, they're going to need to get the net bias name of their Active Directory domain. In this example, it's lab. And then you're going to create a corresponding keystone domain also called lab. It doesn't strictly have to be lab, but it doesn't strictly have to match what's in the AD domain, but that's going to be the field that gets entered here. So that's going to be useful for the users to keep that consistent. Okay, so you've created that domain. And so now on this page, these are all the steps that are going to go into this new keystone.conf file. So this is in the new domains folder that you've just created. You're going to create this new file called lab.keystone.domainname.conf. And here is everything on one page of what that file is going to look like. What I've highlighted and read over there are the settings you're going to need to think about that you're going to need to change. Everything else is underlined. You'd most likely be able to keep it as is. So the URL is LDAPS. As you can see on port 636, that's for the encrypted LDAPS port, not the 389. For user, you can see SVC LDAP is in there. So this is using the distinguished name. If you're not sure how to get your distinguished name in AD, I'll show you right here. Let's go out of that. So you can use tool called add see edit. And in add see edit, it gives you your same view that you have of your actor directory but with just more of the guts revealed. And you locate your SVC LDAP account and properties. So you find distinguished name. That is the value that you're going to paste in. So that easy. There's also power shell ways to do it. But I thought the GUI would be more representative because you're going to need to populate a few more distinguished name fields in there. Let's see. Okay. So here's what's cool is this is the user filter. So what this means is that you're giving the distinguished name of the GRP openset group that you created earlier. And the result of this is going to be that that is going to limit the visibility that Keystone is going to have to your actor directory. So when it does lookups, it's not going to pull down the entire database. It's only going to look for members of that GRP openset group. Much more efficient. And just gives you more control over who's getting exposed. Also, yes, this is what I mentioned before. You're going to specify the path to that TLS CA cert file that you converted from all those various formats before. Next slide. Yeah, you're going to give the Keystone service ownership of that of that file. And then so here I'm showing how you give the built in admin account access to that domain. So you get the UID of the of the lab that you just created of the of the lab domain that you just created. You get the UID of the admin account. That's the built in admin account because we're specifying domain default. And then you're pulling down a list of all the available roles. And then in that in that bottom step, you're tying this all together and creating a command where for the domain in lab and for user matching that UID admin, you're going to give them admin permissions. Yes, to that tenant. Yeah. So, oh yeah, so some some final configuration that you also going to have to do for your compute nodes, just kind of set them to use auth version v3 so that they're aware of Keystone v3 and then restart the necessary services in order for that to take effect. Oh yeah, so as a result of all that configuration, you can now go to your Keystone server and retrieve a list of all the active directory users. And what you're going to get back is a list of everyone that's in the member of the GRP OpenStack group. You're going to pull down a list of roles. And then in this example, user one who has that UID is going to be added to the demo project as a regular member and that's going to just make them a general user. So this is what this would have to be a BAU task for you. Anytime a user needs access to that, you just follow this step. Yeah. So as a result of all these changes, authentication is going to be limited to certain users in GRP OpenStack group. They need to be members of that group, of course. And the crucial authorization, what I just demonstrated over here, that is still managed by your OpenStack admins. Yeah. And by example, they need to be explicitly added to a tenant. I'll demonstrate all of that now. So I will just put the notes. So first of all, let me show you. So this is my my pack stack OpenStack environment. And here is my Active Directory server. This is server 2008 R2. And over here you can see a list of all of all the users that I've created. And you can see the GRP OpenStack group. You can see the members of that group is this bunch of users here, including SVCLDAP and Kate and Lisa and Timmy. So in theory now, when I run the OpenStack user list command for the domain lab, I can expect to see them come back. And yes, there they are. Lisa, Kate and Timmy, SVCLDAP. Also, also another way to pull down a list of users is using the LDAP search command. Let's see. So this bypasses, oh wait, I don't need to put in the correct magic word, which I have here. So this is just a raw dump from LDAP. This is a good way to test that you have that initial connectivity, because this bypasses what you've entered into the keystone.conf. So in your LDAP search, you're just going to let you know that there's like no firewalls in the way and that you're using your SVCLDAP account as correctly configured. But you can see there's Timmy, there's Kate, Lisa and the GRP OpenStack group. Okay, so to demonstrate the workflow a bit, I'll create a new user, Dorothy. And then Dorothy needs to be a member of GRP OpenStack. So we'll get her in there. Okay. So now, when I do the OpenStack user list group, we should expect to see Dorothy in that list. And so there she is. So now, though, before we've done any kind of authorization, let's see what it looks like when she tries to log in. Let's see what happens because now the OpenStack admin hasn't gone and actually given her access to any tenants. Let's make that camel case. So you can see there, you're not authorized for any projects. So even though she can log in and it authenticates her, she's not getting anything because the OpenStack admin hasn't done that yet. So now let's demonstrate how we're going to give her that access. So we have Dorothy's UID. Okay. Let's pull down the list of the roles and then we're also going to get the list of domains. A product, project role? Yeah. Let's check the OpenStack. We're putting in Dorothy's UID and then making her an admin. So now, when Dorothy tries to log in again, she is now authenticating and she's able to, she has administrative access to the demo project. And she's able to do all the usual things you might expect. Yeah. Also, I'll show, so I mentioned all the service accounts before. There are all still within the default domain. So that's, I guess, the default domain is the default domain. That's where the service account remain untouched and not an AD. That is all I'm going to demonstrate. Quick talk about the high availability options available to you. Yes, you can do general DNS aliasing. So you can create a DNS aliases for one domain controller and then have it intermediary between the other so that you have to go in manually each time the one domain controller goes down and update that alias to the other IP address or the other domain controller. It's not really super ideal. Because there's going to be an outage while the admins realize that the server is unavailable and then they have to go in manually update the field and then also wait for it to propagate out. But it's an option I've seen people use. You can also have a hardware load balancer in between. But here we go, the comma separated list of domain controllers. So what I mean by that is in your keystone.domain name.conf file, just have a comma separated list here. So comma and then the full LDF string of another domain controller. So what's going to happen is that authentication is going to continue going to the domain controller until it becomes unavailable. And then keystone will send this queries on to another one. A few things about that sort of setup though is that you don't want the firewall on DC1 or an intermediary firewall to silently reject traffic from the keystone server. Because that can affect the way that keystone automatically detects the unavailability of that service. So just something to keep in mind. Yeah, as I say all the time, test these in your environment first. You don't want to wait until it's actually gone down before you know what's going on. Yeah, so getting near the end. So just to find a link again to the resources page, it looks like this. So I have the slides, I link to the guide and the event page and where I am on LinkedIn. So just everything on one. I'll update this with probably a link to the YouTube of this talk when that becomes available. And then also just to talk more about what's in the keystone integration guide. So it includes, yes, full detail of everything I described. I haven't covered absolutely every step. I've just highlighted the ones that you might have questions about. There are still a few things here and there that are going to need to be done differently. Also talk about how you can troubleshoot these things, steps along the way. You know, if you're doing something, you're doing an Avaldap search and it's not going the way you expect. I talk about options for how you can do that. And yeah, because I'm a former SysEdman, I went and included things that you might get asked during the CAD meeting when your ITAL guys are going to ask you tough questions. Specifically, the impact. So just like the one liner that they need to say, you know, what exactly are you doing? Well, that's in there. And also talk a bit about the outage requirements. Yeah, this is where I've mentioned before, you know, these configurations are very similar for Red Hat IDM and for generic LDAP. And they are there in the guide. Let me actually pull up this guide I'm talking about all the time. So this is what it looks like. Active Directory integration. Key terms. I'll describe assumptions. There's the impact statement. All these steps in detail. And then yes, some troubleshooting. So then I do this all over again for IDM and for generic LDAP. So if you're integrating these things, I could suggest you look here as well. Cool. Oh, that's everything. That's the end. Thank you. Is there someone there with the question? Yes. Yeah. Okay. So that it works. So my first question is, since as you mentioned with the version three of Keystone service, you do support the case when LDAP server itself is read only? Yes. So yeah, in that case, it will cause issue because the internal OpenStack service account cannot be created in the LDAP server. So I know looks like we have the solution uses multi domain. So we keep the existing local service account with local domain. We create new domain for LDAP account. Yeah. So in those case, we would have secure the issues in case OpenStack service need to access some something across the domain to LDAP domain. No, no, but I can let Adam Young answer that one. He's he's got to. So it works. Yeah. Oh, I can, I can. You have a service user. I don't know. Here you go. There's actually a rule that says don't give Adam a microphone. Ask the Keystone guys are pretty strong about it. So what you have with thanks, Adam. Okay, like I said, so for those of you do know, I'm Adam Young Keystone core. I also work for Red Hat. I did the original LDAP implementation, which is why he's deferring to me here. But if what we say is different, he's done it more recently. And as I tell everybody, I lie. So verify everything that I tell you. Because I also don't remember which versions things work in which way it as far as that goes. There is no security issue because it only has access to OpenStack resources. Okay, so if you have, say a Nova user that stored in the sequel back end, which is where these local users go, it cannot get access to active directory type stuff for outside of Keystone. So I don't know how there would be a security issue there. Thought is in case the service account for no one, right? If there's any chance Nova service need to access user information, but those user information are in different domain. The domain is different from the Nova service account. So that's kind of like cross domain access, right? So this is domain as Keystone understands it, not a domain as in active directory domain. Yes, yes, the Keystone domain. So that's allowed. You can set that up in Keystone. When you do a role assignment, a role assignment can be across domains. You can say user in domain one has this role on domain two, stuff like that. So that's Keystone's job now, is to do these kind of role assignments, which is exactly what he was showing when he was doing the assignments there for project or for domain specific operations. Okay, one more question is I saw you demo the whole thing with the redhead version of OpenStack. So is there any compatibility issue, you know, if we test everything with the standard OpenStack version, when we really start to go to production, which we also use the OpenStack, your version, right? So something I took out of this talk, which is really cool, is I did not know that this could work with SON before. And so what I think the answer is going to be, we've been talking about how to do continuous integration for the LDAP stuff. We don't even have continuous integration for straight LDAP, which the active directory stuff is a subset of. We could potentially use SON before as the CI approach to making sure the active directory specific stuff works. And I think we're going to do that. So thank you. So basically, basically that means the redhead version of OpenStack is compatible with the standard version. Okay, sorry, one last question. I saw the demo and you use LDAP, but in your demo, you always have very small user list, but in reality, in big enterprise, right, we always have huge user list. So when you create, you know, user manager or the first step, you want to list the user, but with enterprise situation, that will not work. Oh yeah, I'd expect they'd be using OpenStack, sorry, PowerShell for access to that list. Yeah, we already had this issue because when we test this scenario, it doesn't work. We cannot list the user because it's huge. So what's the planning OpenStack to fix this issue? You're saying you can turn on paging. I'm going to say don't do that. I'm going to say don't list users in general. With Federation, you're not going to have access to the user list anyway. So the goal is to get away from list users as being something that you do. Now, it makes it really painful in order to do role assignments that we've got problems to solve there. None of this is actually directory specific. Also, your when you list users as some company who will remain nameless found out, if you don't limit the number of results that come back, your queries could take a really long time like hours to come back. So you probably on the AD and in LDAP side in general should limit the number of results that come on back. So there's a lot to be done there. Let's look at other people. Okay, I wanted to, I'm over here. Oh, wait. Oh, there you go. Yes. Sorry. So I know I think it's with Kilo that it's implementing some group based permissions, I guess. So my question is, you know, in order to grant when you created Dorothy or whoever the name was, you had to go in and grant them access to as an admin to a project. So is there ability to specify a LDAP group to have certain access into a project? I do know of people doing that. I actually just this last week I was attempting to replicate it in my lab. It does seem like that there could be customer demand for that because that's going to ease the workload. Absolutely done and it works now. When he added her to that, what would you call the group? A GRP. He could have created a role assignment on that group. So that as soon as you add Dorothy to that group, she could have logged in. He just created, he created an individual role assignment, but a role assignment in Keystone can be on either a user or on a group. And so long as the user has that group, there's return when they do the LDAP query, they will get a role assignment automatically. So yes, that works now. Is that a user or only Keystone? Essex? That's been there since LDAP support's been there. Okay. Thank you. So great demo. Thank you. I see from the demo you are doing authentication with LDAP and authorization locally. Yes. Is authorization also supported with LDAP like if we want user to be assigned a particular way? We removed it. What's the reason for that? Because LDAP is read-only. Yeah. Yeah. That people seem to really want their LDAP to be read-only as much as possible. You don't have LDAP schema that maps to things like roles very well. A group is essentially the same concept. So instead of trying to force custom schema into AD or whatever your LDAP server is, they use read-only to group-based role assignment. Thank you. Okay. So I saw that you used a kind of proxy account for the LDAP. So you had some user to log in and have an LDAP search in a given container for users and as a way to do the authentication. So for a general LDAP, you should have, give certain permissions for that user to read the password attribute of the object. Do you have to do this kind of thing for AD as well? No. So it's automatically granted, right? The reason he did it that way is because of most AD setups, you don't have anonymous. But if you had anonymous browsing there, then everything could be done anonymous. The only thing that's done as an explicit user is that when you log in as that user, it does a simple bind and that's how it authenticates the users. So that's not done as that admin account. That's done as that user's account and then that's dropped. And then the admin account that he set up there is strictly for like the list users and general capabilities. Yeah. Okay. So no extra ACLs needed for that? No. No. That was a general vanilla user account. Okay. Yeah. This game you laid out required you to create a group and active directory that was dedicated to the OpenStack users. Yeah. If you have a campus group that doesn't want to mess with groups like that, they don't care if you just use active directory for authentication. And they're ready to let you see everything. Is that doable? That will work too? I don't know. Yes. Or why wouldn't you do that? Is there a good reason not to do that? Just to limit who can get access to OpenStack. Yeah. Well, but aren't you controlling access to OpenStack on the OpenStack side? Yes. You have to assign them to a project. Yeah. Authorization can be in OpenStack. Yeah. You could, but there are some business practices that prefer having multiple layers of control over a single transaction. Can you go back to your config slide? Yes. That one. So you can see in here there's nothing that actually filters. You don't see group in there anywhere at all. A member of user filter. That's the one place you could drop that out. We don't have this right? It's in my national lab. Okay. Yeah. So my question is if you have 10,000 users is there something I should be looking out for? I heard less users. Is there anything else? Well, not from the AD side. Well, I suppose what's something to consider is that, you know, these lookups are going to be hitting a single domain controller. And so just having that second one in the list isn't going to have like round robin lookups to them, right? So yeah, you want to at least have some monitoring, performance monitoring of your DC. You might have to scale it up just adding additional resources. On the OpenStack side, I think. Oh, on the active directory side. Yeah. On the OpenStack side, I wouldn't imagine so. Oh, answer from paging. Paging is you need to consider everybody thinks it should be good. I will try it out. Thank you. Cool. I have two questions. First one, if you can go back to your config file for a moment. Yeah. Okay. So on their user filter, actually, sorry, on their user 3DM, say I specify our organization on unit because that's how pretty much will it be recursive? So will it search for the O use underneath it? Yes. Oh, okay. Yeah, yeah, because in V2 you couldn't search underneath. It was only in the static. Okay. All right. And the other question is, can I use global catalogs? So if I have three domains, and they all share the same parent, I can search from the parent to the global through each three. Like big companies where you have a search separated by continents. So I will use the global catalog. So I don't have to specify three domains. I set up the parent through a global catalog so I can say, look everywhere. Is that supposedly work? Oh, okay. Right, the only problem later is going to be if you pull 100,000 users. Right, then you have to do pages, pulling whatever. Right. That's massive. Yeah. Let me answer a slightly different question from what you asked though. Okay. Because if you don't have that single tree for all of those, some people probably in that case with the M&A, you can set up multiple domains where they're all looking at the same AD server, where the difference is the tree that you have there too. So it may make sense to say, if we have Bank of America just bought Chase, and so we still have Chase's after-directory out there, put that in its own domain specific back end like this, everything else is going to be roughly the same, but the trees that you use for those. Okay, thank you. I just want to jump in with a couple things. You've only said one thing here that's actually incorrect. Oh, yes. Which is you do not, even under the old stuff, need to change the schema. That was like the one rule I had in Violet that is, we had to work with the default schemas. So everything is in place that you could work with after-directory without doing the schema, even with the old way of doing things. I mentioned that because I actually saw docs that, multiple docs that actually prescribe schema changes. Yeah, I wanted to put that to race straight away. So, bullet in the brain, no matter what, you don't need schema changes. Why don't you ask your question and I'll get with the rest of the stuff. So, two questions really. The first one is, if I only have one directory regardless of its AD or not, do I have to specify a domain? Okay, so that was in the, my notion of what you had there. It's only on the horizon config where you do that. You can actually do two things. One, you can actually change the default domain. Now, so long as all your service users are using v3 as easy pointed there, those can still, the default, the ID will still be default, but Keystone will say, if you're asking for a v2 token, you can give it out that way. And so for a lot of backwards compatibility issues, that's an issue. So you can say that, you know, domain ID, big long UID is the default domain. In which case you don't have to make that change in horizon. The other thing that was in horizon, there's another config option that you don't know about yet, obviously, which says, here's the domain to you. So even though if I'm doing v3, I don't have that value there, and it pre-fills out the domain. Oh yeah, yeah, you can do that. So it's in the local settings file, an option to say, this is the domain to use for people logging in. I know there was once a time when I couldn't get that working, so I left it out, but sounds good. And then, just to, not to be the dead horse, but just to make sure I understand, going back to the authorization discussion earlier, so if I have a directory, again, not totally just AD, but if I have a directory with groups that I can use to manage authorizations, can I use those groups to manage authorizations for different roles inside a keystone? Absolutely, and I was almost tempted to have them pull up a demo again and show that that works. You can do a group based role assignment. Okay. So anything that comes through as a group will be available for role assignments. The reason why we don't push that that hard is because typically we say AD and LDAP is is read only and you're not going to have access to that information. If you do have that degree of power, then yes, you absolutely can use it. Now, the other thing you can do is you can assign users to groups in other domains and do all sorts of walking stuff where you can actually manage your groups within keystone, but that's beyond the scope and that's again, nothing AD specific. Yeah. Thank you. Thank you. The recording, so if you can. Oh, yeah. Yeah, thanks. To expand on that question, you were saying that in order to your question about the role assignment, that's if you change a schema within actual... No, no, no. I'm tempted to have you just show that. But no, it's the groups. Remember how explicitly I'm not used to the group? He was doing it because of this filter right here, but it's also going to show up in the list of groups that's returned when you do the query. The general pattern is this. You go to keystone and request a token as the end user. The first thing that's going to do is take the password you did and the user you have build it into the end and do a simple bind just to authenticate you. Then as that admin user that he added there, the user line they have in second, it's going to do list groups for users. Okay. And that's going to come back as a second list and you can do any group assignments based on what's returned there. So it doesn't show it here, but all of the user underscore attributes, there are a whole bunch of group ones too. So whatever type of group you're using, posits groups, whatever, you just tell it what attributes to do. So what's the membership attribute? It figures all of that out. One thing when using groups is to do group-based role assignment. Groups didn't exist in Keystone v2. So if you connect to do a role assignment over v2, you won't see any of the group commands with like the open stack client. So be sure to connect with v3 and you could do group lists from LDAP and you could do group-based role assignment. Another thing I want to clarify, v2 versus v3 is the API version. Every version of Keystone from I want to say Folsom or Grizzly onward supports both. You don't have to redeploy Keystone, you don't have to change anything there. The only difference is the auth URL that you use to talk to it. The standard auth URL ends with slash v2.0 or the ones in all the documentation. There's discovery that's supposed to work that you don't even have to add the version onto it. But if you want to force it to v3, it just ends in slash v3. Of course we need to v3.0 because, you know, why be consistent? But it's all, it's not a different version of Keystone. It's the versions of the Keystone API and they are both supported everywhere. Very few people know how to actually remove them. And those people avoided their warranty. Just to expand on the group thing again, does all this, the group assignments, does that work with AD nested groups as well? Yeah, we don't have the config up here for it, but there's query options for saying, what is the query that you do to get back the groups? So as long as it comes back in the query that you say, here's the groups that the user has, then yes, you will get those. Okay, just a quick question. Responding to a previous saying you had, so that you said that the first thing that what the user does, although the authentication is good, is that you bind to the LDA if it was a user? Is it? Yes. So that's the way to authenticate that. Don't you need to have a plain text password for that? Say that again? A plain text, a password is needed for that. The user does pass their plain text password in the token request. Yes. Is that an issue? Okay, but you, you don't have it in your hand. I mean. It's not stored anywhere. It's not, that's just so. So the way it works is it uses an LDAP simple bind. So the user passes their plain text password to Keystone. Keystone looks up the user, finds the DN, and then it attempts to actually do an LDAP bind as that user and passing the password. The hash comparison is done by Active Directory or whatever LDAP server it is. You don't need any, Keystone doesn't need any read access to the user password attribute. So the LDAP server does all of that. I, well maybe I'm wrong, but when I debug, let's say the communication to Keystone. So when I'm talking to a port of 5,000, I see some SH1 hashed password traveling over there. So it's not clear text. So could it be decrypted on the Keystone side? What is that? Well, this is the password. What I'm just authenticating to the, how do you answer it from a security perspective? Well, it's, the clear text password definitely is passed all the way through. So I mean, for what you're seeing with a hashed password, I'd be curious to know what that is, but you can't, you can't authenticate to AD or another LDAP server with a hashed password. You might be thinking of maybe Keystone doing some of its own hash in comparison, but LDAP, you have to use a clear text password. Yeah, yeah, yeah, definitely. Yeah, yeah, yeah, yeah. Okay. Okay folks, I think we're low on time. All right. Thanks very much. Thanks for attending. You're great.