 Good morning. Good afternoon. Good evening and welcome to another episode of ask an open shift admin I am Chris short executive producer of open shift TV and autofocusing cameras I'm joined by the one and only Andrew Sullivan Andrew, how are you doing this morning? In focus and ready to go. I guess there we go Apparently my camera just decided it needed to refocus right as we went live. Good stuff, you know technology technology is hard So, you know, you wanted to talk about Authentication and authorization and I think it's good to point out that it's like the idea behind today is that it's not that hard Yeah, definitely so I want to yes authentication authorization is today's topic and My goal here and Chris and I were talking about this before the stream went live is More or less to convey that this is pretty straightforward, you know, I've I've got some things planned You'll notice that it's just Chris and I you know the first time in a while that we haven't guess So my goal is to just kind of walk through some things, you know, you all our audience You're welcome to ask us questions at any time. That's why we're here This is you know, this is office hours So ask questions ask us to dive into things, you know, if if we if I can if I have the resources I will go through and try and explore that But yeah, I want to look at Setting up both HT password HTTPD authentication as well as LDAP with active directory And I chose those two simply because HD password is usually my default because it's just super duper simple like it's you know, basically one command two commands to To add the config into the cluster and then you know LDAP isn't really any harder, you know active directory You just basically have to have active directory, which I normally don't have I actually set up an active directory instance for this So, wow. Yeah Well, you know, it's gotten dramatically easier. I used a server 2019 the last time I set up and The last time I did a domain controller like a production one Yeah, like server 2008 maybe server 2012. Yeah, 2008 I think it was my last time that I was like maintaining 80 instances or yeah So, I guess yeah, I forget in the terminology now. Yeah, it's yeah, there's no more DC promo. It's all done Really? Yeah, that's awesome. Yeah, it's all done from this from the fancy management console I mean you can use PowerShell and all right, but yeah So yeah, it's it's changed pretty dramatically in the last or a decade Wow, I might have to actually get an AV server running here in the house Maybe I don't know we'll see I know well and it's funny because again you and I were talking before the show of I think a lot Of organizations are starting to adopt Azure active directory. Mm-hmm You know kind of goes along with the whole office online office 365, you know A lot of folks are adopting the software as a service aspect and you know, why not offload something else? That's just You know active directory is not exciting You know group group policies are You know RSOP, you know resultant set of policy is still my favorite tool in all of windows land So I'm especially as a workstation administrator like why is it doing this? Anyways, yeah, I'm getting distracted. I'm getting distracted. Anyway, yeah So yeah today is where this is an office hour show, which means we encourage you We we encourage you to and want you to ask questions about anything and everything that's on your mind However, today's topic is authentication authorization So with that being said, we do generally have we start off this show talking about some things that are top of mind and Let's see while lead I see your your question there. We'll we'll talk a little bit about that later on I knew key clue come up. So We'll talk about that in a few minutes but Chris, I hope you have your your thinking cap on because what one of these today is a little bit of a It's a conversation that I think that we should have and this spawned out of an internal question around Kind of how do we recommend or how do we position? Management philosophies for lack of a better term around clusters and more specifically Red Hat and OpenShift gives multiple Options right get ops Get ops the concept or the the philosophy is basically you the human don't do anything directly in the cluster Right, you have yammels You have these things that define what the cluster should look like and you rely on Argo CD and the get ops operator to make it look that way that is In many ways contradictory to the type of experience that you know, I think Many people are used to right of I deploy a cluster. My cluster is you know, mine I log into it right. I Administer it I go in and I edit things and manage things and configure things as needed Maybe we have some sort of change control process, right? The infamous itil for anybody who's working in and around especially government, you know, government loves itil Yeah, so You know, I think it's important to have this conversation around Is there a recommended way? Is there a best way? Is there something that Red Hat recommends, you know for an OpenShift cluster, right? So, you know Chris, of course, I would love to have your opinion audience You know, please contribute your opinion as well, but from my perspective Andrew is very much the Switzerland of these things, right? I take the approach of you do what works best for you and My response to the Red Hatter who had who was asking this question was more or less along the lines of You know, every organization has different skill sets has different personalities and those personalities You know, some of them are big some of them are small. We're all affected by What I call the burn factor So how badly were you burned by something in the past and now you refuse or how, you know How much do you avoid doing that in the future type of thing and all of that shapes? Your Willingness your tolerance for adopting, you know, different different things So OpenShift is flexible enough that you can do Many of these multiple of these at the same time, right? You can tell for example if you're using get ups for application deployment You can have it ignore aspects of cluster configuration You know, you can temporarily disable it while you figure out, you know We implemented the fix now we need to figure out why that happens now We can go back and apply all all that through the get ups philosophy so on and so forth. So Chris any any thoughts? I mean, you know, I like we were talking about I've been burned by actor directory more times And I wish to admit But I am much in the same vein as you right like It was funny I worked with John Willis for a while as You know like DevOps consultants kind of deal and we tried to stay away from like specific tooling because it was always like Every organization had their thing, right? Yeah, and they wanted to use that thing no matter what Because it was already in place. I didn't want to reinvent the wheel So I'm very much in the same vein as you right like the personalities the talent the systems you have in place already Like don't reinvent the wheel Utilize them if you can but in that same vein I've seen or have worked at companies where it's like yes, we have a massive ad infrastructure But for like our web-facing applications everything's authenticated through our github org With, you know, some third-party thing like octa or you know, one of those yeah types of tools and And that worked well too, right like because integrating ad with a bunch of AWS stuff was really hard And we eventually did end up putting ad instance or managed ad instance in a AWS and just kind of synced everything up to that from our on-prem data center and then migrated away from that on-prem data center, but Yeah, I mean it was cumbersome to use ad not in the cloud like on-prem with some of our AWS stuff So we just kind of said all right. Well, how about a github org and okay security team You can manage, you know, who's in that who's out of that, you know, the administrative functions of you know Person leaves company they get we had dealt that automated, right? Yeah, it was just one of those things where it's like HR did this that user ad doesn't exist anymore in ad All right, go remove them from the github org. So like I said, whatever tool works best for your organization is the right one Yeah, and it's interesting you bring up, you know, philosophies like DevOps and GitOps is the same way and chat ops and You know, all of these other things, right? There's a lot of folks who believe that in order to be successful, I have to adopt this 100% It's either all GitOps or it's none It's either all DevOps or it's none But you know far better than I do that DevOps is this huge spectrum of things and And it really comes down to what works best for you, you know, I used to you know, I used to work for a storage vendor and Storage admins. I don't know if you know this. They're kind of known for being Stodgy and reluctant to change You know, wouldn't know what you're talking about. Yeah Storage admins and and DBAs, right? No, yeah You know, it's obviously yeah, and I get it. Yeah Well, there's reasons for that and I used to have this conversation all the time of you know Why why is the storage admin? Reluctant to change reluctant to do these things. Well, it's because we've all gotten those angry middle of the night emails or phone calls Because something broke. Why did it break? Well because of X or Y or Z? Well, now I'm going to be cautious of remember that burn factor. We're gonna be cautious of affecting that in the future So funny enough, I used to blame virtualization for why storage admins are so grumpy So maybe so the rationale is Pre-call it 2008 right storage arrays were responsible basically only for Application data right out of physical server in Iraq. It had a fiber channel connection over to the storage array The only thing on that storage array was whatever the application was using and then almost overnight, you know, global recession You know, all of these other things almost overnight It was suddenly we're adopting virtualization and now the storage arrays are responsible for every IOP in the entire data center Which is scary all of those operating system and that is very scary And you know, it takes storage systems, especially the big frame arrays and stuff like that They have life life spans of 10 plus years. Mm-hmm So, you know, suddenly they became responsible for all this and it's you know, why is it so slow? Well storage latency because we're running, you know X number of Drives at whatever speed, you know back in the late 2000s. It was still mostly spinning media That was all except if not all spinning me drive. Yeah. Yeah, so, you know, they got beat up pretty heavily and then You know, yeah, sure a few years down the line 2012 1314 suddenly hybrid storage came a thing and it got a little easier to your 20 Yeah, 2015 16 17 was when all flash started to become a thing and that was when storage admins finally dug out of that hole But yeah, you got your anyways. That's my yeah, that's me Completely diverting away for a moment So, yeah, it really comes down to what works best for you What works best for your organization? Most importantly, what works best for your applications? Because that's what is oftentimes, you know, the public face in your company, whatever's making money or creating widgets or you know Whatever you're you happen to be responsible for Okay, we've we've died tribes enough on that one You want to answer a question real quick before we go into this. Yeah, is it possible to have two clusters? with Admins for each cluster being unique. I mean, yes, it is in theory But I probably should have asked for more clarification to this one now that I'm reading it Yeah, so I see that Cy Krishna Apologies if I'm mispronouncing your name. Is it possible to have two cluster admins for one cluster? So I'm going to interpret that as like could Chris and I both be cluster admins for a single open-shift cluster? Like the answer to that is absolutely. Yes, and I'll actually show that Okay, cool. You're here in a few minutes. Yeah So, yeah, we'll take care of that and then what will you ask to see your question about key cloak? Yeah, I do want to talk about that a little bit. I don't have a lot of details Simply because I'm not a key cloak expert But we'll track any of those down. We'll have that conversation. We'll track them down and get those answers for you So the next thing I wanted to talk about this one comes up on a semi-regular basis Let me share my screen I want Are you who I want you are who I want Yes, there you go The docs so let's go to where am I going here? So Search for graceful Because I know that there is a docs page called shutting down a cluster gracefully So we talked about this a little bit last week. I think it was last week It's just kind of talking about certificates and how bringing back the cluster can sometimes create certificate issues So you can absolutely shut down a cluster. I do this all the time inside of my lab I know customers that do it regularly, right? They shut down for their life environments. Yeah, so and there is no magic to it It's literally Shut down the guest OS. We don't don't hard power it off. Let it shut down, you know normal Naturally, yeah, but naturally. Yeah, there's no order to it turn them all off at the same time right turn off it turn off Worker nodes and then control plane nodes, right? It doesn't really matter. I always turn everything off at the same time Yeah, through them go. Yeah, so no magic to it shut it down. It'll be fine When it comes back It is possible that you could end up in a scenario where it doesn't come back because of things like certificates So when that happens you effectively need to just connect and re-approve CSR's I Have shut down and left shut down Hundreds of clusters at this point. They've been down anywhere from a few hours to a few months And I've never not been able to get back a cluster. Really And it's it's a simple. Yes. Wow. And it's every time it's been as simple as I Need to use the kube admin or the The kube config file that was created with the cluster that has the system admin right credentials Which is the certificate base straight? Connect to the API and basically do an OC get CSR and then approve whatever pending CSR's are there and You can do like an OC get node What you'll see is all the nodes in a not ready status And of course if you do like an OC gets CO If it responds because remember CO is going to be a part of the open shift API server Which may or may not come up depending on what certificates have expired But yeah, it's more or less go through and approve all those CSR's and then give it a few minutes and everything comes back Yep So it should be I've never done it for months though. I think I've ever had a cluster live that long Yeah, here's the restarting with cluster gracefully Nice, so yeah, there is some KCS is be careful with the KCS is around Like restarting things because this yeah, because this process changed So back in the 4.3 or 4.4 time frame Was when we introduced the automatic sort of certificate renewal process So prior to I think it was 4.4 so 4.3 and earlier You'll find KCS is where it's you know This you got to connect into the control plane and execute these commands and refresh these certificates and replace these files And you know do all these things none of that's necessary anymore. So just be careful with those You can always open a support ticket, right? Right. Yeah, that's literally what what supports there for so if you have a problem If you have an issue when it with it coming back up if it's been shut down for a while Call up the support guys, right? So they'll help you with all that stuff And I know that they do it You know, that's that's their thing Now how do I move? Oh, there we go. I Zoom windows or zoom bars. Yeah. No, sorry The last thing that I've got and I know we're taking a little bit longer than normal today So the last thing I've got is Coro s versions and OpenShift versions so if I go to mirror OpenShift.com hub Right, this is where we go and download all of the resources So for reference, this is actually the site that like if you go to cloud.redhat.com OpenShift and you go and download the install tools. This is where it takes you to Right. So this is or if you click the link that says download Coro s This is the site that it takes you to I just skipped that process So I don't have to log in again as well So if I come in here and I look at dependencies and I go to Coro s and we'll go to four dot seven You'll notice that there is Three releases in here latest just points to four seven thirteen There's zero seven and thirteen the current stable version of four dot seven is Seventeen I think sixteen seventeen seven so you notice that these don't align if I go to four dot six You'll notice that there's dot one and dot eight and I think the current stable version of four dot six is like 35 or something. Yeah, it's pretty high. Yeah, so If I'm installing a new cluster Do these need to match right? I want to install four dot six dot thirty five, but there's only a Coro s four dot six dot eight No Right effectively what happens is it will yes, it will install a four dot six dot eight The node will pull its machine config right and among other things It will use our PMOS tree and update that operating system as a part of it So new clusters you basically want to use the newest version Inside of your why release so why release is the second number so four dot six would be the why So use whatever the newest one is is generally your best bet Where this gets complicated is existing clusters especially clusters that have been around for Since four dot five right and adding new nodes with UPI in particular to a older cluster So four dot five and earlier used ignition version two four dot six we move to ignition version three dot oh and Then recently actually with four dot seven dots 13 14 something like that. Thanks. Yeah, we move to ignition version three dot two So in particular the version two to version three move Broke that right. I now have to update my for example worker dot IGN If I am, you know, my cluster was upgraded from four dot five to four dot six to four dot seven That's worker dot IGN is still For is still up words words are hard ignition two dot two based So I need to update that well there happens to be a handy dandy KCS on exactly that So we'll paste that in here. It's probably going to ask me to authenticate which I'm going to skip So I'll paste that there. Thank you. Yeah, this This KCS has a down at the bottom after you authenticate Has just a little short couple of scriptlets that you can run that will pull that ignition file and then update it to the new spec And then you just use that to boot your worker nodes So just be aware if it's an existing cluster, right? So you updated from, you know, four dots five to six to seven You're going to install a new node and you use the four dot seven coro s to install it. You'll need to update your ignition That being said, in theory, you can continue to use the original coro s version. Right. So this is why So think about it. This is why with IPI it generally just works. Right. So let's say I did a VMware IPI deployment to vSphere Right. The first thing that it's going to do is it's going to upload the OVA and create what effectively becomes the template virtual machine for all the notes. So when I created the cluster, it was 4.5 based rate the OVA is 4.5 based If I life cycle that cluster or if I upgrade that cluster rather Right. So I go into the OpenShift console and say upgrade. I go to 4.6 and I say upgrade and I go to 4.7 That template was never changed. It doesn't reach out to VMware and say, Hey, here's a new OVA Right. The cluster doesn't do that. Only the installer does that. Right. So when you go with IPI and say scale this machine set It's going to at that point clone that even though it's a four dot seven cluster that OVA was 4.5 based it will clone that machine and that is what gets introduced And machine config operator does the same thing. It updates that operating system to whatever the latest whatever the version it needs is So this is why Sometimes it's an issue. Sometimes it's not an issue. There's no harm, for example, if you go in and replace that 4.5 template virtual machine with one that is 4.7 based if you create a new machine set and use a 4.7 OVA You just need to update the the ignition file that it's using inside of it and the The KCS will walk through all of that stuff. So anyways Just be aware of that. There is KCS posted in the chat here. We'll include that in the blog post So on and so forth. Yes, if you're not familiar with the show, Andrew takes all the notes and every little tidbit of information we talk about and does a very quick turnaround of a blog post that gets posted every Friday So yeah, Andrew does an amazing job with this blog post I don't know that it's quick. It takes me like 24 hours. Alex handy. Who's the one who manages the blog? The one that makes it. Yeah. Yeah. That's a good point. I sent it to him and five minutes later. He's like, yep, it's ready to be posted. There you go. So all right. Um, today's topic authentication authorization. Yeah, so let's look at a couple of things I usually like to as most people know I like to start with the docs And my reason for that is The docs should be our source of truth. They should have most of the things that we need in order to learn about a particular subject. And if they don't, we want to fix that Yes, and we can do that ready. You can open an issue up here. You can reach out Yeah, you reach out to us. We have friends on the docs team. It's amazing. It's awesome. Yep. So and we're happy to Uh, I'm going to use the word harass. We are happy to harass the docs team on behalf of of you all our audience are customers Um, so inside of this page and let me link this in the chat here. We have this understanding Uh, authentication underneath the authentication and authorization sub topic So what I really want to focus on here is these top two sections users and groups So with inside of open shift, there are Roughly three Types of users right three broad categories There's a regular user, right? This is just somebody who is Connecting in they're meant to be using it, you know administering the cluster deploying applications, right? It's often a human that is behind the scenes here. You're out of focus again chris. Oh my gosh There is system users, right? And you can see that these are going to be prefaced by system inside of the inside of the name there These are users that are created by the system to do certain tasks for certain things, right? So so we have this open shift registry system user or this node account, right type of things inside of there And then last but not least we have service accounts Service accounts are exactly as they sound they're designed to be machine operated or machine used Or application used not a human backing up so a classic example of a service account would be When a pod gets instantiated, right? It's using a service accounts to run underneath that particular context So we need to give permissions to the service account or create a service account and assign it to that pod In order for it to do the things that it needs to do The other thing that we want to be aware about is groups and Exactly as the name implies just as you would expect from, you know, any sort of authentication authorization experience with the groups it is simply a collection of users And users can be assigned to one or more group So by default there are a few what the documentation calls virtual groups That we need to be aware of and then we'll look at some kind of user or in our case We'll be syncing them with LDAP. So some other groups that we that can be created for whatever purpose we see fit So probably the most important one here that i'll talk about is this system authenticated oauth Effectively, this is a group that is assigned to any user who is authenticated with the oauth server inside of open shift, right? So that's the default authentication mechanism this this will be configuring in order to use In our case htbd as well as LDAP authentication The system authenticated is as you can see associated with all unauthenticated users You could if you wanted to write assign some permissions to an unauthenticated user, right? Maybe I want anybody to be able to come in and see Metrics on some service or I don't know something like that It's technically possible. I struggle to come up with any use cases for something like that So the authenticated I believe that this would be used if you are using a third party or a non Oauth authentication mechanism inside of your cluster So the user would be assigned to that virtual group and then you can assign default permissions defaults things from there All right, so let's take a look at a cluster So for my camera problems in advance I think my shirt is throwing it off to be honest with you a little reflection coming off the logo Yeah, maybe And three or keeps autofocusing again. That is strange. I don't like this Uh, so I am running an open shift 4.8 RC inside of this particular cluster. Uh, you happen to notice that this one is actually a single node open shift deployment So if you aren't familiar, um, you can go to Cloud dot redhat.com. I don't know why I forgot the url So cloud dot redhat.com slash open shift. You can go to install Select the tab for on-prem and up at the top. There's a thing that says assisted installer And with the assisted installer you can deploy a single node instance So this is running it right now actually yeah Yeah, and it it works great in the background as I as we're streaming here So if we look at the nodes here, you can see it's just just one single node inside of here Mine happens to be I think the minimum requirements are four vcpus and 32 gigs of ram Which is what i'm using here and it works flawlessly. I haven't had any issues so When we think about and if I browse down here to users, right, I have no Users inside of here, right? I have no identity identity provider configured I can create users kind of arbitrarily if I so choose And I can associate permissions with a user id before that user id exists So for example, if I were to go in and say, you know, give the user see shorts, you know system admin privileges Even though your name wouldn't show up underneath this until the user account is created Or until you log in essentially the first time through whatever the authentication process is That permission would be there and associated with you So how do users get created? Generally speaking there are they are created automatically the first time that you authenticate with the system Right, so let's look at what that actually looks like So I need to I'm gonna switch what i'm sharing here I'm gonna see if I can make this work So let's stop that share and I'm going to See if I can share a region of the screen so that way I don't have to constantly I know I don't have to constantly very big Flip back and forth All right. Hopefully that's better. Yes. Thank you All right, so inside of here Is I have things staged to hopefully work for us Uh, so oc get node we can see I'm connected to my single node open shift and if I do an oc get user User I have no resources right no users inside of here So let's cd into our httpd here So I have an ht password file, right? This is created in the standard way. So ht password dash c dash blah blah blah If I look at that I have two users and solov and andrew right, so What I want to do is in order to configure httpd authentication I need to make sure that it has access to that ht password file And then I need to tell oos. Hey go look at that ht password file whenever somebody tries to authenticate So I'm going to search for oops I should have found this command earlier and I did not Here we go Yeah, so we have this command I'll just put that one command on the screen here oc create secret generic. I'm going to use the name ht pass secrets I'm going to use the file I'm going to give it the name ht password inside of the secrets I'm going to use the source file in the current directory also named ht password And we're going to create this secret in the open shift config namespace So if I do an oc get secrets ht pass secrets Open shift config we can see we have our one secret inside of here right rocket sites Yeah, so now let's look at our oos file here So extremely straightforward all we're doing is configuring oos with an identity provider named ht password of type ht password And saying hey this secrets the one that we just created has all of your data So we got oc apply dash f it'll complain because applied didn't originally create the resource. Thank you very much I configured it Yeah, so now we want to come back over here and we want to look at our Cluster operators we should see among other things The authentication operator will update. Sometimes it takes a few seconds for it to do its thing But it'll update after a second and it'll be it'll become available to us Now I just did all of that from the command line You can do this directly through the GUI. Yep. So if you go to global configuration, we go to We search for oos And down here you see I have this oos this ht password provider configured I can come in here and I can say create an ht password Authentication provider and it will all I do is copy and paste the contents of that ht password file that I just showed Copy and paste them into here and click add and we're done Or you can browse and if you have a big file selected and off you go. Yeah, so I Either way works equally well, and I didn't I hadn't realized that they'd added this into the GUI I think this is a 4.6 or 4.7 edition. I think it's a little bit newer. Yeah, so Anyways, one of the simple ways that you can configure that now you notice nothing has changed I'm still logged in as kubadmin right, I have no um If I look at the user management here if I refresh that right, there's still no users inside of here But if I log out Instead of just being presented with the login screen because there's only one login Instead, I will see How which authentication provider do you want to log in with kubadmin or ht password? So at this point I can you also have sorry If you install code ready workspaces, there would be a login Field for that as well because it can use a different provider if you want it. Yeah So I'm going to log in with one of my accounts here. I actually will log in And It allows me to log in and it gives me the default set of permissions Nice, which is right limited Well, yes ish So you'll notice that I didn't create a user account. We just looked we saw that there was no user accounts If I switch back over here if I now do an oc get user All of a sudden that user I just logged in with exists Hey, he's there. Yeah, you have no full name though. Sorry. Yeah, I know so I am given effectively the default set of permissions and from inside of here I have access to various things by default Users an authenticated user is given the ability to self provision their own projects All right, I can go in here and create a project named test one And I can hit create and now I can come in here and I can deploy things as I see fit It will follow the default project creation template. So if for example, you have You know a quota put in place or a limit ring or whatever the project default project template happens to be If you want to change that We need to effectively remove the authenticated oauth right group from the self-provisioners So Let's paste this documentation link in here And we'll paste that thank you. So we have this disabling project self-provisioning So let's switch over to I'm going to copy this guy Let's switch over to our terminal again And we'll make it easier to read So you can see here. I have a default group called self-provisioners Which as the name implies means that you can provision your own projects and use them, right? And The group system authenticated oauth belongs to that role Right or is a a member of that role So if I want to disable if I want to prevent any authenticated user from just coming in and being able to create their own projects All I need to do is remove this particular config. I'm not going to do it in this one But just know that it is in the docs here and the docs walk through precisely how to do that, right? We do a patch on that self-provisioners. We remove That group from inside of there And then we update right remove cluster role from group We remove system authenticated oauth from the self-provisioner cluster cluster level So relatively straightforward, yeah So what if I want to assign permissions to a user when they authenticate so remember my OC get user so remember there's only one user right now and right but what if I wanted to What if you wanted to cluster admins? um And where is it? See this is what happens when I don't stage all of my commands. Oh I know here we go so OC adm because we're doing a cluster level permission change policy add cluster role to actually want to add this to a user User cluster admin Pay in solo if remember that's the other user that was defined in our ht password file So yes, it's going to complain user a and solo was not found If I do an OC get user you can see it's still not there But I can do right. I can see if I were to look at that cluster role. So OC get cluster role Binding.r back Uh cluster admin And I do an oh yaml We can see subjects Oh ref. Did I It might not be listed in here as a result of the user not existing yet, but Anyways, it will associate that with my particular user. So let's Clear that let's switch back over to our gooey here So I'm going to log out as Andrew and this time I'm going to log in as in solo So remember that user doesn't exist yet until now And you'll notice that the moment I logged in I am now a full cluster administrator inside of here And I can see and change and modify and do all the things that I do as an administrator And I can add users as needed into that particular configuration So let's say that I want to add chris as a cluster administrator. All I need to do Is come back over here And I would update I would add your user to the ht password file I would re upload that secret And then you would be able to log in Now because that is per user authentication Right That's kind of a pain to manage. We don't necessarily want to do that. Yeah And you know, essentially every time a new admin admin joins the team, right? I now have to go into every cluster and give them permissions inside of that cluster Every time a net and admin leaves the team I now have to go into every cluster and remove them from that cluster and so on and so forth Not an ideal scenario You can see hopefully pretty easily where I'm going with this which is Using some sort of central authentication authorization mechanism is definitely ideal So Let's look at active directory So first I'm going to start by and I know that this is probably going to be a little small. Um, I don't know how to make The windows It's rdp. I don't know how to make it bigger Because it's just difficult. Oh, yeah I know I mean you can make it bigger within the thing, but I don't know if you can with file explorer So my apologies. I know that this is small I I'll I'll use words to try and paint a picture as much as possible So this is my my ad domain Really really simple because this is just kind of a demo. Um, if you look down here in the corner This is actually an eval. So I'm not really attached to it So I've got an oh you in here for domain users inside of there I just created a few different sub oh use right so one for administrators where we have this user Andrew Sullivan We can see that my username here is going to be a in-cell of And then I have a couple of groups open shift users open shift admins I have a couple of regular users user one and user two You can see I'm ultra ultra creative. Oh, yeah, super creative And then I have a service account creatively named open shift dash essay so straightforward The only other thing to be aware of here. So I have of these two groups open shift admins Only my account Is listed as an open shift admin Open shift users Has user two as a member Right not user one not open shift essay So that's really all we need to care about for the moment inside of active directory So let's go to Our LDAP directory here And there's a couple of things that we need to be aware of So first I'm going to switch back to our documentation So we do have a question real quick and yeah, please syntax you're using so frequent viewer, uh, I don't know how to say your name dmi 3 mis Why not use oc create cluster role binding instead of the old syntax Um, just user preference or it's just a habit of mine. Yeah, okay Yeah, all right easy enough. So yeah, I'm slow to learn Among other things like you're busy. Yeah, among other things. So yeah, there's no no real reason for me to use one way or the other Um, I think I was actually just copying the docs in that instance without really thinking about it. So yeah, bro Do you like our docs? Um, yeah, uh, so I forgot to mention with the ht password We do have this in the docs here. You can walk through and it shows you step by step Here's the ht password command to follow Right here's Where is it? You know on windows here's creating the secrets, right? This should look incredibly familiar It was literally the command that I just used right creating the ht pass secrets um, and then adding it into The cluster using a uh the oauth yaml file So let's look at this for ldap And ldap is going to be A bit more complex for a couple of different reasons The biggest one of those is We don't necessarily want everyone in an entire organization to be able to log into our open shift clusters We would prefer that you didn't configure it that way. Yeah Yeah, you know sure in my lab. I've got you know one user me who is you know Several additional virtual users I was recently, you know, there was a customer that we were working with for a bug Which I'll talk about in a moment. They have almost a million users and their active directory instance right so You don't necessarily want everybody to be able to log in especially if those who you do want to allow to log in You want them to be able to create and use their own projects, right of Having a you know a test cluster or a lab cluster where hey, you want to go test out open shift You want to start containerizing kubernetes is sizing. I'm gonna make up that word Your apps, you know, yeah, go right ahead. Here's how you log in So ldap authentication the big thing that we need to be aware of here Is this ldap url This is how we are telling the system how to connect to our in our instance active directory domain And then importantly we want to pay attention to base dn. So basically Where is the root right? Where am I looking for user accounts? And the filter so this is if I want to limit Those who are allowed to authenticate What filter do I need to use to do that? It defaults to object class equals splat. So be careful Yeah, so base. Yeah, so basically it'll anybody who is found inside of this base dn would be allowed to log in Yeah, so let's move back over here And let's look at our ad oauth file here I'm going to actually open it in vim. So it's a little easier to to see with the syntax highlighting So I'm going to ignore these things. These are pretty straightforward with active directory. You just want to make sure to use the uh Sam account name for both the id and the preferred username So preferred username you can use mail if you want Basically up that affects what the display name is up in the upper corner so if we look at Where's my cluster here? So up here where it says a and sell of yep If I were to use mail for example, it would show my email address instead of or I could use my principal account name Which would show my full name, right that type stuff. Cool So bind dn This is the service account that will be used Only to authenticate against the LDAP server again active directory in this case to then validate the users This account doesn't need any permissions inside of open shift It doesn't need yeah, it doesn't need to log in it doesn't need to have Nothing all this is used for is to authenticate against our LDAP server to then validate the user accounts So definitely Encouraged great suggested to use a service account that has effectively no permissions anywhere inside of your active directory organization Right to be able to just do that that bind checking You can see that the password there is just stored in a simple secret inside of here So mine you see this insecure true. Uh, I set this up in like 20 minutes and my She didn't manage to provision certs and all that fun. So no, I did not create a CA. I did not, you know No, none of that stuff. Okay, fair enough. We'll let you slide. Yeah So insecure is true. Um, if if insecure is set to false It doesn't matter whether you use LDAP or LDAP s in the url here It will always go to LDAP s to do cert checking and all that other stuff If insecure is set to true and use LDAP here It will use standard unencrypted LDAP on port three eight nine So the next part of our LDAP url here. This is the name of the LDAP server you want to connect to Now you'll note here that I used the name of my domain and Andrew's opinion and we don't say this in the docs. I think the docs just say what is the LDAP server that you're connecting to right host and port and many of the examples show Connecting to a specific, you know server instance My personal preference is to use the name of the domain The reason for that is because active directory will automatically Resolve that to whatever domain controller is at the site that you're on So let's say that you are creating a generic deployments right here Maybe using get ups and you're deploying to a dozen sites across the globe Right inside of active directory each one of those sites is going to be defined And each one of those sites is going to have a set of subnets and a set of domain controllers associated with it So by pointing it at the domain name It will automatically resolve that to an active directory controller That is local to that site or close to that site If you were to choose so for example I've got you know one one data center in here in north Carolina and another one that's in Australia And I'm not thinking about it and I deploy my australia, you know instance And it's connecting all the way back here to north carolina To do that authentication. It's a lot of extra latency right potentially a lot of extra. I've actually seen that problem in organizations Exactly. Yeah So yeah, my personal preference is to use the with active directory is to use the domain name here So that way active directory and dns which are Very closely tied together. It can do what they do best and manage themselves So after that we've got Where to find all of my users so you can see this is very simply the domain users all users in the organization are inside of there But I've got this filter So this filter and this is a standard LDAP query is going to say so I've got ampersand which means both of or and so an object class of user And then what's inside of this set of parentheses? And inside of this nested sets. I have an or condition that says you must be a member of The group open shift admins or the group open shift users Now you may be wondering what is this one two eight forty blah blah blah blah string This is basically so that it does tree traversal so you can use nested groups So it is important particularly with active directory where we do a lot of group nesting rate. Oh you nesting Inside of there. So it is important to use that with active directory if you happen to be using that So, okay, let's go ahead and implement this We'll try using the the GUI this time. I think And I want to copy this guy out And I know I'm coming up a little short on time here. So we'll get through this as quickly as possible So you've got your eight minute warning make it seven So we'll come down here. We'll go to add we'll go to LDAP so we'll name it active directory so our URL here is going to be just like that So remember that was the thing that I just explained our bind domain. Oh dn, which is the user to do our bind checking is our Uh service account Give it that password Remember I said we want to use the sma account Name here So the sam account name So the name cn is fine Mail right and I thought there was one other that we wanted to change in here ID here we go We're not using ldap here now I say that I'll use this. Um, this is probably going to fail or it's going to cause drama So I might not use this. Um, there is no option to do to set the, uh Ugh, uh, uh insecure equals true. Yeah so What's uh LDAP find secret is there. Okay. So So what's gonna happen here? No not LDAP sync uh, a do off All right, so what's going to happen here is Remember I just applied this to that. Uh enough config so what I'm going to expect is is stir-o-yaml, it replaced my HT password provider. So now it's gonna go through, it's gonna reconfigure everything, if it knows you get CO, we can see that it is going through and it's now updating its status. So we'll give that just a moment. In the meantime, one thing that's interesting is, you know what, I'm still logged in as this user and it's not gonna boot me out. So even though I changed authentication providers, I still have an authenticated session and it's not going to re-validate that until the session expires. So just be aware of that if you're going through and changing these things. We'll check this guy again just to see, there we go. So now let's log out. And this time you see that I have my lab.lan option. This is my LDAP configuration. And if I log in with my user A and Solove, I'm going to get a authentication error occurred. All right, we'll try something else then, which that could be a result of a, it's still progressing over here. Yeah, I was about to say. All right, we'll let that progress. So at this point, notice that this is just authentication. Can the user access the cluster or not? Not assigning permissions. What if we want to create groups to manage those permissions? So I'm going to very quickly go through this. I have, I actually have all this typed up in a, and I'll publish it as a gist. And then I'll link that from the blog post that'll come out on Friday. So you'll be able to see all of these things that I'm doing inside of there. So let's look at, so we need to do an LDAP sync in order to synchronize groups and group membership from our Active Directory server in this case. So same as before, right? We're going to synchronize against our cluster. Yes, it's not a secure passwords. I know it's just a lab, it's okay. So I am, because I'm using the augmented Active Directory and because I want to avoid spaces in my group names, I'm going to map a specific CN to an internal OpenShift name, right? Just easier to use something other than space. And then down below that, I'm just providing these kind of, hey, here's where to find users, here's where to find groups, right? And go through and basically synchronize all of those things. And then the other thing that I need, again, because I'm using the augmented is a group allow list. So again, just saying, hey, I want to use these specific instances. So I have a command here. So OCADM group sync. I'm going to use my group allow list and I'm going to provide it my sync config. And then if I don't provide the confirm, it'll just do a dry run and it'll tell me what it's going to do. With the dash, dash confirm, it basically says, okay, we've created these groups. And now you see I have my two groups and the users who are a member of those groups. I don't know why that's still progressing. I'll have to look into that at some other point. Anyways, at this point, it's now relatively simple to go through and assign permissions, just like you would expect for those users. So now I can do my open shift admins and I add cluster role to group. And just like that, anybody who is in the open shift admins group is now a cluster administrator. Note however that these groups are only updated when you go and update them, unless you create either a cron job or you can use, so the COP has very helpfully created a sync operator. Nice. So if we come here, the group sync operator, I'll post a link into the chat here. So very simply does exactly what it says it does, synchronizes those groups and their user membership based on a timeframe. So I know we're out of time today. Thank you everybody for watching. I hope that this has been beneficial. I hope it's been informative. I haven't been paying attention to chat. My apologies. No, that's clear. You're good. Okay, I will address any questions in the blog post. I will also link to the gist that will have all of this information inside of it, inside of that blog post that will come out Friday morning. If you have any questions, if we didn't answer anything here on the stream, please don't hesitate to reach out. You can reach me at Andrew.Sullivan at redhead.com or on Twitter at practical Andrew or Chris is Chris short on Twitter and short at redhead.com. Yes, exactly. Yeah. Last thing, there will be no stream next week. I will be enjoying the week off and relaxing before my kids go back to school. So I will see you all into weeks. All right, take it easy everybody.