 Good morning. Good afternoon. Good evening everybody. We hope you are doing well out there in twitch land This is my third stream of the day. I'm very very happy to be here with Andrew and Christian my fellow teammates, these are my direct co-workers on the technical marketing team that I'm on here at Red Hat a Couple things real quick So someone mentioned this morning that by the way these videos live forever So you might want to mention to some folks that are listening or watching that Yes, these videos do live forever including potentially the chat if we're talking about it. So yes, there's that so be nice to each other Don't do anything your mom wouldn't like or else I will tell her also. We have a follower goal So by the end of the month of June on twitch I want a thousand followers, right like we want a follower like Storm to happen the month of June. So tell your friends. Tell your family We're at we already got like 30 followers today like we can do this That's a team everybody here together. We can totally do this Like the I like to call the arms nice. Yeah, I mean, let's get it. Let's let's do this, right? Let's do a big so today Andrew What's your name Christian? Sorry, it's Tuesday Introduce yourselves It hasn't been a long day for me yet, but we are getting there. So go ahead introduce yourself Yeah, we will fix that for you. We will make sure that it's a long day. Awesome. That's you Yeah, so Andrew Sullivan technical marketing manager. So I cover among other things virtualization open shifts duties as a signed Yeah, so Christian Hernandez part of the same team technical marketing manager Focused on the open shift core platform other cool things I get ups and Things as assigned right Andrew as as Andrew said seems to be the norm nowadays things as assigned so Yeah, so it's good times good time. So what do we wait so we are talking we're talking about our back today Rollbase access control, right? So everything you want to know but we're too afraid to ask we're too afraid to ask I think that was a great title that I can Yeah, so like I've been give myself a pat on the back I've been pushing it out there on Twitter. And yeah, it's a great title I've been pushing out there on Twitter and LinkedIn, right? Like I hope some people show up and like actually poke at us with some questions check here So Nibble is there has a big question on our back one. Oh, yeah, how do I debug explore it? That's a good question That's a great question. All right, over that when I get handed an RBAC config from an install doc like datadog How can I read their RBAC config and validate whether it's using least privilege? That's that's like that's like great questions. Thank you so much. That's a that's a that's a that's a great question, right? So, um So I want to I want to touch on that right so Just a little little bit of background right before before we start before we start hacking away Red Hat has been a big big our have our had our back for opponents for a long time, right? So As many of you know kubernetes Didn't originally have our back right originally Had yeah, well had um the The idea of like the admission controller right and all that but they didn't have a Really really a concept right? I'm talking about like early early early on because I remember I remember using Opuship 3 right when the first one that first I came from the Opuship 2 world So when we kind of we had to do that paradigm shift all of us right to the Opuship 3 Because it was we went from you know our own thing to kubernetes and I remember You know reading about kubernetes, and I'm like well like kubernetes doesn't have like a lot of the stuff That like enterprises needs right? I mean it was the you know The initial release that one release right and we were we're gonna help build it right? So that was the whole thing if anyone wants to listen to a cool podcast there's a kubernetes podcast by by Google and Kate and Coleman was he was he was on that podcast and he kind of described the history of Red Hat and Google and trying to build kubernetes Which everyone you should go check it out if it's it's on there Yeah, so our back was first released in kubernetes 1.6 Yes, those mini moons ago, so that would have been almost exactly three years actually three years ago. Yeah, yeah Wow. Wow. Yeah 1.6. Yeah So that was that means that because I remember using our back in Opuship 3.0 Which was 1.0 right for kubernetes, right? So that's six releases After and a lot of the work Wow a lot of a lot of essentially we just donated what the work we did, right? So a lot of the work that went into our back Was basically used in the kubernetes upstream so So it's it's It's it's one of the cool things I was like to think about like the history and how we got here that red hat did Amongst other things right so like there there's always this thing about like Opuship a you know Like why do you have routes instead of like using ingress points like well? Back in the day. There was no such thing as even an ingress point. You're lucky to even have that object now because You know like I'll be like one of those old guys get it off my lawn right like back at my day We didn't have ingress so we had to write our own sort of Sort of thing is so similar idea same thing with with our back so um So yeah, so that's a little of the history of our back and I think Think I'm just gonna start where is share my screen share. Here we go. Yeah Desktop I think so I always I always struggle to find which one is it this one. Okay, no way. Here we go Yeah, hopefully guys you go don't don't see anything You should see my you should see a CLI. Let me see if I do see a CLI. Yeah, you're good. Yeah Browser window, yep browser window cool cool So let's go through To go through something simple right so let me get my my console copy link Um This is brand new cluster because I never logged into this Let's see here brand spanking there. That's not dangerous at all. It's not dangerous at all Yeah, this is where this is like live live. I finished building right before I Believe we get users, right? See if that works. Oh, it does work. Okay, cool Yeah, I'm just logged in as user one, right? It's never good when you're surprised when it works. Yeah, right And we are not even 10 minutes into the stream. So yeah, here we go Yeah Well success so far, right? So let's let's do It's create a project right um project. I'll call this So we had we need a naming theme we need a naming theme, right? I always like doing naming things we did Star Wars planets last time done Star Wars. Yeah Yeah, we've done Star Wars Mario characters Mario brothers characters. There we go Mario characters. So I'll just Yoshi Yoshi. Perfect. Yeah Okay, this project Yoshi To create this like that sounds like something real like yeah, it sounds like yeah It sounds like it's a upstream project or something right like you guys use Yoshi. It's the the YAML Office skater shell interaction. I don't know. I'm trying to figure YAML obsolete or service Hi So let's deploy Right, let's get to a the developer view right because we all love the developer view here. Um Let's go. Let's I have have an image. Let's go to container image Hopefully quay's not down today. So we'll see yeah, that'd be nice And um Get hope was I think down the other day get up was like I feel like Was last Friday like everything went down towards like the three to five p.m. Time frame on the east coast Yeah, yeah, it was literally like the world suggesting everybody go ahead and just stop working Yeah, just just start your weekend now because right up and quay was like one of the biggest registries out there Yeah, I would say like the second one or second or third I don't know biggest registries out there and then get hub Which is probably like the the place where everyone works from the lynch pin. Yeah if slack would have gone down It I just would have thrown my laptop in the corner. Yeah, I would have been like, you know I'm just out So we'll call this app this is just a dumb app here We'll just do a deployment and create a route, right? So we'll We got this it's deploying So dooth coupon or as mentioned in chath hope to search what goes with no Issues under the hood. Hey, just to let everybody know we don't like we provision everything with self-sign search, right? Like we can provision our clusters with lots and crypts We typically don't just cuz it's an extra checkbox for us and Once we're in we're in and my waste of precious calories, right, right? And you know like let's encrypt as a great service, but we don't necessarily want to burn them down creating You know clusters left and right like we do so using self-sign search something when we do just kind of take it easy on the internet but yeah, well and I think that's a That's a good thing like why if it's just a test sort of cluster. Why would I want to let's equip, right? Right, exactly. Why do I need right? Like yeah, like if it's just for me But I mean, you know hey, whatever Well, I mean you did a pretty cool stream using let's encrypt right with OBS ninja. That was really cool I was watching that. Yeah, actually need to break that out into a separate thing and like share it with the guy I actually put it on the OBS ninja Reddit page and they were like, oh my gosh, that's so cool. So yeah, I was kind of fun So there we go. Cool. My app is up Where's this guy here? All right. Yeah, it was up another question in chat How do you get past hsts with self-sign search on ocp for we don't have hsts? Configured for the domain that we're using so that's how That's how we get around it. It's not there So Yeah, so now that I have this app this cool. This is actually new for the for the developer view in version 4.4, right? Yes is a project access, right? There's an actual tab here. This is a project access. So this is For For open shift, right, there are There are built-in roles, right that you can that you can assign people so you don't have to Triple over yourself and trying to define custom roles like trying to build a Role and a role binding and trying to figure out figure all that out. We have built-in ones So the ones that are built in are admin edit and view Right, so kind of the the bait the basics, right? The biggest difference between edit and admin is that an admin can delete the project other than that It's pretty similar And then view obviously is like read only right so this is like what you get a Christian Let's let's take a step back and sure so prior to or I guess contributing to all of that are kind of two important things right roles and role bindings and Both of those are used so So roles define a set of permissions role bindings associate those permissions with a user and then the user comes from right the The authentic a authentication provider. Yeah, so I don't know if you want to walk through any of those permission sets associated with a role binding or excuse me a role definition or Draw a pretty picture draw Yeah, so there's so there's it's a It depends how deep the rabbit hole you want to go right so it's Coming from the the fact that I'm assuming you've gone through the trouble of doing authentication With with open shift because that's another that's a whole nother whole nother beast trying to tie an authentication to Kubernetes So for those For those of you who aren't familiar Kubernetes itself kind of takes a hands-off approach down a Kate authentication and basically says I trust whatever you tell me to trust So for example the The most basic way just just talking about Kubernetes in general To authenticate is using a certificate based authentication, right? So there's you you create a certificate TLS certificate and then you use that certificate to interact with With the API right and that's just like just bare bones There's a way to plug in Different authentication things like SSO things like LDAP Things like The the most common way of doing is doing I think it's open ID I think it's almost the most common way. I think it's either open ID or The one that gives you a JWT I Think it's open. Yeah, I forget that that that's and that's another common one Essentially what Kubernetes does is that you told me to trust this Give me the token. Yeah. Yeah. Oh, right. Yeah and and Then it'll it'll trust that to write it'll give that token to the back end system that guy says yeah, trust that token That sort of thing right so once you're logged in I assume right that you've done all that all that authentication piece right so now now we're where it's a conversation of authentication and authorization, right? So now we're in like in the like okay like now that I know who you are. What can you do? That sort of thing right so So if I go to I think it'll be If I add a user, I think it'd be it'd be easier to do this here user to and then Kind of do be you right? So if I do view This person has read only so I'm doing this as a regular user Right this project as yourself as a I'm doing this yeah as a non admin cluster admin kind of correct Yeah, so like me as the owner of this project, right? I'm giving people access right so that's I can Me as a regular user so this is part of the The self The still service aspect of OpenShift right is basically oh, hey Me as a developer I can have a project and I can then give that access to other people other groups as well Things like that right so so like here as user one. I'm saying hey give user to a view access Right once I save that And then I go to let's go here log out You can do user two I believe it is this guy yes, okay, and then Yeah, I can't this group but I do have access to Yoshi That's always good. That's always good access to Yoshi, right and I can I do Almost like that was planned almost like this was planned. Yeah So here right so here if I do I think I need to go to the admin view for this Workloads deployments. Yeah, there we go if I go to deployments I Can't do anything with this pod Right and if I go down to let's copy logging command This guy all right your own look away All right, so if I do OC get pods I Get the pause if I OC delete pot Let's get this guy Right, so I'm no longer. I'm not allowed right to GTFO. Yeah GTFO like you're not allowed right and so if I do Andrew what's the command to show OC get I Should probably finish my question. What's the command? What's the command to show the? The roll bind is it OC get cluster all bindings, right? I believe so. I believe so. I need to log in as an admin for this, okay? log in Let's do server user user name, I think it is open TLC You spelled the wrong you said open LTC. Oh, yeah Hmm TLC Okay, can never remember this which which one are you looking for are you looking for the? That one I don't know Yeah, okay, there we go so OC gets cluster roll bindings Well if you want the roll definition, it's just cluster roll. Oh cluster really yeah Cluster roll and cluster roll bindings. Yeah cluster. Well, yeah, we want to see the view the big two. Yeah Right. Um, oops Right, so then um, let's do a edit on this Because I'm a VI guy and I like my Yeah, my VI thing so um So this is the cluster roll, right? So some of the What is exactly like like what Andrew was saying I assign what is exactly a cluster roll, right? So I assign the the name the the built-in one which is called view, right? So this is that this guy here To this user for this namespace Right and so some of the things is it's essentially what you need to do what it's doing here is that You have to list each and every single API that this This user needs access to so for example for the API group operators core s calm the resources cluster service versions catalog sources install plans Yada yada what is allowed things I get list watch, right? Let's do something a little See pods Where it is here API groups This would be where is there should be one that says If only there were a fine command, yeah, well I'm slashing for pods, so Or maybe maybe that's not what I'm looking for. I'm looking for oh, yeah API groups So someone correct me if I'm wrong If a group's image API groups project, which one are you looking for there we go there we go Yeah, yeah, so I'm looking for I'm looking for a Simple example here, but here is this projects in the projects you're of You're allowed to get right essentially or basically just is basically It's like the OC get right or QCTL get That's the verb you're allowed to go against This this API right and you can go and you can go down here and Just look at all these All these are the API out of various namespaces. I'm sorry the the various API groups and What resources you're allowed to do what against right and this these are the the built-in ones Before you would have to like like type all this by hand or Know what you want to do almost like the the AP the AWS Let's see the the I am roles right almost like exactly have to like know what? What what to do here some of these it's kind of nice to have some of these built-in and if I'm if I'm not mistaken Maybe Andrew, you know the name of this so Here in this line 953 so does an empty set mean all I Think it means Because I you either all are none. It's none because it's I thought I had to have a quote star, right? Yeah, I think a star was old astros is all okay. I think could be wrong That's very very deep in the cockles of my mind. I Call in the cobwebs everywhere floating around. Yeah. Yeah Yeah, so let's see So that's that's basically okay, it's a nibble eyes are in chat a good red flag when reviewing the policies You're given is if there are any quote asterisks quote. Yeah, so for a really broad assignments like system Empty so and our back. Yeah. Yep. And yeah, so the easy way to do this is OC describe the cluster role dot our back space view Well, I'm sorry slash view either want to work plus the world are back like that dot dot our back cluster role And then you you yeah, there we go. It'll give you a summary This is actually a better view than the even the array Yeah, I had to look up the command because I couldn't remember it. I knew that there was an easier way of seeing it Well, yeah, there we go see describe cluster role dot our back and Then the name of the role that you want to look at and you can also do a local role so if you don't have cluster level permissions so cluster role cluster local role is the other one Yes, I never do cluster local roles or I never do local role. So OC get cluster local role Dot our back view Just try roll binding No, no cluster because it wouldn't be a yeah. Yeah roll binding Dot yeah, I can I can't I can't view it. You're logged in as user to yeah, try to do a Roll binding dot our back same as before Yeah, I won't let me I'm just user to I'm just lonely. Oh, yeah, okay. You got no right. I got no rights Because I have you right I have the view So let's change that Let's log out Log it out log it in user one. Let me begin Okay, and then Developer So we got here Project access and then I'm gonna give this guy edit right So now as user to be able to Now I should be able to do something here Pods no Where is How do you get deployments from here? I Want to know okay. Well, this is a detail so I can scale up here. Okay There we go into now right now I can edit this here so So one of the things I actually wanted to go over was some of the Some of the common thing I ran into right when I was out in the field was with the fact that The fact that people Customers usually like the edit role. They really like the edit role, but one of the things they don't like about it and let me Let me go here. Let me log Who am I? No, I'm not a user to Token there we go OC get pods and then I can do OC rsh Right now I'm in the container Right So so some of the things that the that some customers would say is like, you know what we really really like the edit role But we don't want people to rsh, right? We don't want people to remote shell into Into the shell is evil. Yeah. Yeah, or like, you know, like we don't we don't want people just logging in right because I guess in in In theory if you're like mounting secrets I can see that file, right if I just rsh into it. Yeah, and you don't necessarily Yeah, yeah, you don't necessarily want developers to be able to see that right you want to be able to deploy the containers But you know, why don't they need to log in right? Well, it's the same principle of not sshing into servers. No, you know, pet versus cattle kind of deal that that whole night That whole nine yards. Yeah. Yeah, and Yeah, and like they may have a policy right like we don't let developers is station to the servers Why should a container be any different, right? So I totally get that So one of the the quickest way to see what what goes on if you do an OC You rsh again. Oops except actually add Log level equals eight you can actually see all the API calls that happen Ding Ding I think I need to add it on this four before. Yeah. Yeah It thinks it wants me to run a command call log And so one of the things that happens here You should be able to see it Maybe you don't authorization bearer tokens mask, thankfully Bear who's bear? Mask yeah, I guess that's or was it nine I think if you go Go higher if you go higher it's unmasked. Let's do higher Does it go to 11 though? Does it go to 11? So there's a little bit more here. You should be able to see what just do like fine for bear Who's bear I wish I can like less this Oh, I guess I can Can you know? Oh, no, I'm stuck now. Yeah, you're screwed I'm a broad Oh, and Oh, wow. Yeah, I'm really I'm really messed up here control. He said Yeah, type reset. Okay. Yeah, I was in a type reset. Um, I did it again This is weird. Yeah, it was really weird. Um, you should be able to see Um, it might not be in there I saw it before I wonder where I was looking You should see pod exec is what I'm looking for So it's at the end of the url that it uses, right? So if you look, um, it just jumped around Welcome php. Oh, there we go. You'll see pods And then the name of the pod and then slash exec exec. All right. So we're looking for pod exec So the the api it calls as pod exec So, um So what we so What I used to recommend to customers is like fine instead of like writing a lot of um, a lot of this by hand Find something that's close to what you want And just modify it, right or like make a copy of it, right? So, um Who am I again? I have to make sure open tlc You know That guy Um, so if I do an oc get a cluster oops cluster role of view Right. Yeah Oh yaml Right. So this gives me the yaml for view. Oops. Sorry not for view We want to do edit right because we want to be able to edit but then Remove the ability rsh, right? So this right this to a file Um called no rsh Like mics in my way, so I have to look That's why I keep that's why I keep clearing the screen. By the way, I did a twitch stream with, um, who was it? Jason right and he he's like, why do you keep clearing the screen? You don't give me time to read anything and it's because my mic's in my way, right? Like I'm trying to I'm trying to work around the mic. So gotta get an arm, man. You gotta Yeah, I've been I've been wanting to get that arm the boom mic to try to Try to be able to do this here. Actually, I'm telling you dude, like I've got all the stuff If you can't find it online, I'll just ship it to you because I've got all the stuff for my old one Wow, you upgraded hand me downs. I don't mind hand me downs actually. Oh, I mean, it's it's a it's a pandemic Man, we gotta do what we gotta do. Yeah, yeah With your microphone, you can put it up above the monitor too So so that doesn't my camera sits right up there. Oh, yeah, you can That that works. Uh, you have a condenser recommended to do like this Because you'll pick up your keyboard sound. Yeah, well, he has a condenser mic not a dynamic anyways, so right That'd be cool And also if I put it in front of me, then I will be able to see my So yeah, mine usually lives down low. Yeah, like right here in front of my face so I can see everything as I type Yeah, so cool. Um Exactly, right? So so essentially I'm looking for a pot exact um So one of the things that you need to do is normalize this here because they got rid of a kubectl export Um, much to my chagrin So we don't want a uuid self link. You could take that out Christian's timestamp. I always leave as blank Uh resource diversion get rid of that guy So, um, here's uh What's that? Yeah, change the name. Um We'll call this no rsh another thing you need to When you're doing this One thing that you need to uh To keep in mind here is this an aggregation rule. So this aggregation rule what what ends up from my high level What ends up happening is that it'll um It'll include It'll include this role Into the edit role meaning that it's it's kind of like when you if I don't get rid of this It'll take this are this cluster role And then it'll um, it'll add The edit role to this role Meaning that if I take away pot exec, it'll read it When when it aggregates um when it aggregates the the edit roll back into my so the the idea is What the aggregation rule is that you can maybe add two or three things and then aggregate Um a bigger role to it, right? So, um Yeah, creating more granular files that correct. Yeah result in one bigger applied So if you're familiar with mco, right mco works on the same principle Have I defined multiple machine configs that are then aggregated into the one that's applied to the node? Yep, I think uh did eric do uh one with mco or he threatened to do all of them mco He threatened to do one and it hadn't done it yet. So okay Yeah, it's that one is you're gonna need like pot of coffee and four hours for it because it gets It gets pretty it gets pretty pretty gnarly how things work the second I have to install an astronaut bathroom is the second the streams are too long. Yeah Yeah, that's right That's that's your that's your uh That's your threshold, right? You know as soon as I have to do that, right? I'm gonna start breaking these up. Yeah So besides removing um some of the things like creation time stamp and uh self-link and things like that I remove the aggregation rule as well because I want this to be a standalone, um, right rule um So I did that remove the aggregation rule Take off all the um all the stuff that kubernetes adds My creation time stamp change the name as andrew pointed out or else I would have done something bad And then Start removing things that you don't need So for that for instance, um, it's like hey, I love it. I love the edderall Don't want you to you don't want uh do our stage, right? So um, so remove the exec part And then there you have it so if you do this is also a good way to mess with your friends if they're If you're yes, exactly Mess with your friends have fun Um, so once I oops object parsing time. Oh, um Let's take out the creation time stamp Yeah, just delete that one. Yeah. Oh, we don't need it We'll work around it We're alive There we go. Ta-da. So I have that created um So let me log out So was eerie uh her wallet is saying that uh, uh pods attach is similar And I'm trying to think Pods attach is for storage though, isn't that I believe so. Yeah um Again, it's it's let's do whatever let's do whatever it says. What do the doc say? Let's google it live. Let's see the doc say So let's see here. Well, the first one that pops up is get into a shell running our container. So that's encouraging. Well, well, I notice it's not blue Yeah, yeah, because I booked for it. Yeah It should be a like a reference like an api reference an api reference. So kubectl a reference docs I got right here Keep couple commands kubectl. I do kubectl. Okay. So this is exact right cuddle. I cuddle my cubes. I'm cuddle my cubes So it's edit. It's a way. Wait. It's Exact attack working with apps x attach auth cp describe There it is. Oh it attaches the pro also like uh, is this is similar to like an ns enter It's like what you're doing a namespace enter Is that what you were talking about? So attach attaches the to the process of the pod. Um, so hit hit attach instead of exec Oh, there it is I don't see it attached process that's already running inside of the existing container Attach So you would essentially pick up the standard out of whatever process was running Like you would foreground the process maybe essentially Yeah, okay. I think that's enough on this the first container in the pod gotcha. Yeah, okay attach Pod running timeout standard in oh, okay. So actually this is this is actually pretty cool. So I removed um Pot exec, but let's see if we can still get into the container even after doing that. So um, that that's So I think exec basically creates a new process inside of inside of the container And whereas attach attaches to an existing process. So if you're Something that is non-interactive. So maybe you're running PHP command or something like that it would Basically you're attaching a php if you're running a bash script Sure, you're going to be attached into into bash itself and probably have some interactivity Yeah, so if I do uh, obviously project, um So, uh, just to kind of expand to what Andrew said, um, if I do an ocrsh Ose get pods and do an ocrsh. I'll remember I'm an admin, right? So this is not the user too. So if I do oc get pods And also do sage. I think it's oc can I? Um, yes, can I there's another one. Um, so if I do ocrsh I can get there ocrsh is basically a shorthand for oc exec Um Oops, I forgot the And that's I oops. Why did I there we go? exec it Um bin batch, right? So, uh, what Andrew was saying Was that since this this this container actually Um, the entry point for this is htp-d foreground right so, um I can't get a shell on this because um to this process because I I can't There's no this bash isn't running So doing an oc exec Basically what Andrew says forks another process And attaches me to that whereas um Whereas attach Here attaches to an existing process already running So I don't I don't I'm not forking another process. I'm not I'm not um Yeah, I guess that's that's that's uh, that's a short of it. Yeah, so try um, so go back to your command line It's oc space off space can dash I And then you can do like pod detach like that Pod space Oh pod and attach to it Yeah, or as the other user or I guess you haven't tried it yet, but the uh exec It says yes, even though it gives me a warning Let's look at the docs OC can I help? idiom Or oc off rather. Oh, yeah off can I Can I help? Oh, okay, so can so it's the other way around So can I exec pods? Yes Well, that's good. Can I um, what was it attach? Yes, okay cool So then um, so let's log in as user two Um You memorize the token yet oc can oops off can so like remembering places of pie Yes Decimals, can you remember? So you haven't applied the new It's a pod. I forgot. No. Yep. Can't be but it's for verb object Oh verb object. Okay. Um, is it help? Uh, yep create so can I Exec pod I say yes No, why not? Let's see who am I because I didn't apply the uh OC project Yoshi make sure I'm on the right project Yoshi I'm already on Yoshi Can I um List pods yes, how come I can't exec? Is this user still view only or are they edit? Oh Maybe Let's go to Project access This is user has edit they're at it So I should be able to unless I overwrote edit That would have been that would be funny. Um No, we changed the name. No, we changed the name. Yeah from something else. Yeah So let's uh Did you or did you not apply it? Well, I applied I applied the role Um Because some role but I I didn't um I didn't assign I assigned this guy edit so I should say yes No Why Shoulda why yeah Um, so I think you can do uh, if you do the oca off can I and do star star? I think you'll have to put it in quote single quotes. It'll list all of your permissions Let the star star. Yeah No I can't do everything. Okay, that's good. Okay Or maybe that's just checking for Administrator access. I thought there was one that will print out your current help Did they hit the the help again for it? That's what yeah Well, it allows me in interesting. Hmm. I don't know why it tells me no wait Do I have to um, I don't know why this is by the name space? No. No Hmm, but why Uh, yeah, so you said h, right? Yeah, so see the oc can I suppose? Yeah Uh That means all So if you can't do everything Yeah, because I'm on edit I'm an edit man. Uh, they're the bottom one oc off can I dash dash list? There we go. Ah, okay List and then um, yoshi Yoshi, okay I'll use the list Don't do Okay, and then Exactly the other four pods exec get this watch create delete delete Interesting. Hmm. There must be a must be a reason why Yeah attach get list I wonder it's just it's just the way the way i'm typing it The way i'm invoking it Because I would think that would say yes because Because you can actually You can actually do it. Yeah Right but Let me see if Yeah, I was about to say try an exec command And see if that fails you Nope Oh Okay, interesting So now we're asking ourselves our back questions. Why isn't this working? Well, I'm just one. Yeah. Well, I mean it is it is um, so I do have so it's working as expected because user two has edit Um, oh, that's right. Sorry. Yeah, and and I can do it. It's just I'm just wondering why Um, the can I command says no says no when I can obviously can right d So in I might be invoking this, uh, can I command wrong? I might Yeah, I'm wondering Is that that's the only reason you need to type in a real pod name? I wouldn't think that would be No, because it's just checking the permission against the api. Yeah Yeah Get list watch delete, etc Sure cuts and groups will be resolved Non resource your is partial your sources slash name of the game that would be pretty big. Okay, so without any help Jones, you would think that would say yes Right So let me um I don't want to do that Are there any nested things that are making it getting rewritten someplace else? I don't think so Yeah, I don't think so because I removed the aggregation rule and and I would think the aggregation rule Um, wouldn't be take place at least at this. Um, I just juncture used your name user one Just go to user one This is this guy can I know interesting But I'm an admin on this one at least on this at least on this name on this name space the Wow So that was that wasn't expected. Um Yeah Let's see if I see anything coming out here Uh, so what's it checking against? Um, so there's a bare token Verge create delete Okay, so these are all the things that It's posting and it's posting against So I wonder if this is Yeah, it's going to the authorization Okay Well that that goes that theory. I was thinking maybe it went to like some open shift specific thing Hmm, I don't think so Or maybe it is going through an open shift specific thing And we're asking the authorization api from kubernetes. Oh I see there might there might be like a different API Yeah, it might be a disconnect there or the different different command. Oh, wait, which commands are you using? You even k or oz. Oh, see Okay Um, so if I do like oh oops Okay, so this is cute No Huh Well interesting Yeah, I'm looking at the same thing on my side Yeah, I want to see I wonder if this is um So I wonder if this is something we go to the the kates office hours for later this week. Yeah Huh interesting. I'm just wondering if Although I would consider that a bug, but I'm just wondering if right All the authorization stuff is like open shift is handling it, but when I do a can I The the kates api is answering um That that doesn't seem like something we do it doesn't yet that wouldn't Like my brain is short circuiting trying to figure out how that would even happen Right, like there would have to be something that says Ignore all this open shift stuff. No really go look over here. Yeah. Yeah Uh, can I explore exit pods? No And then can I Huh But I can get them, right Okay Can I list them? This is like like asking, uh magic eight ball I wanted to say out look not so good. Can I I've got an early magic eight ball right here Oh, there you go. Wait, what does that have? What's that logo the power symbol on it? Oh, no, that's pretty cool. Remember think geek. Yeah. Yeah. Yeah, that's pretty cool So it says win whatever you're doing you're winning you're winning. Yeah win. Okay nice Be root It says that as well Be rude anytime just be rude be rude Well, okay, how I got to ask from a programming perspective how much further did we go down this rep but whole? Um Can I watch? I'm gonna try this thing here Yes, probably because I'm admin You're you. Yeah, I'm rude at this. You're open. Yeah. Yeah, open tlc. So I'm rude Um, was there anything no that it told you before it's all these it told you know before No, no these um the only thing it tells me no are things that it should have said yes to right So but now okay, so no what i'm trying to get at is you were logged in as user Two and it said no, but you were able to do now you're logged in as cluster admin and it says yes And of course you're able to so Yeah, and then that's super weird And I am able to so all right so instead of going further down that rabbit hole we may come back to it We will transition rabbit holes Well, trying to so yeah, which rabbit hole do you want to jump down? Um So, I mean the the did we go down the how far down nebulizer's list do we go down? Nebulizer I think he gave up question. No, he didn't give up How do I debug and explore it? When I get handed an artifact, how can I read and ver validate that it's using least privilege? Yeah So, how do you debug and how would you know that something is the least privilege? Obviously the shorter it is as far as a definition file the fewer religious Yeah, the fewer. Yeah, the fewer rolls it has. Yeah the splats are bad Um, so obviously more specific is always better, right? So if you're if you get if you're getting your r back um files because I think Some people just like blindly load. I know I do. I know I'm guilty of it. I just blindly load So especially when you're doing like operators and stuff like I just want to get for shame for shame So, yeah, don't So don't so don't do that. Don't do what I do is just blindly load just oc create dash f or cube ctl create dash f Um, you know your operator dot yaml, which includes the r back in the binding Mm-hmm. So, yeah, that's obviously some of the things you want to look for the splatting, right? So you don't want to do splat api splat You know verb splat, right like you obviously don't want you you want to go. Um Uh more specific Is always better, right and then um and seeing like some of the things someone mentioned in the chat I think it's very good to um to look for um, if you have things like system unauthentic, uh, um authenticated meaning that um If you're authenticated then you're allowed to do it So you don't you you don't necessarily want, um, just authenticated users. Yeah. Yeah Um, so just basically looking um Looking for those things right and looking and just making sure that it's As as specific as you can get Um, yeah, it shouldn't have some admin level permission that it would never need Uh, just cut this right. Yeah, like Hmm fishy, why would it need to be able to touch storage when it's a stateless app, right? Like you don't correct? Yeah Eric christian, I just used the zoom chat to send you the link to the datadog Uh helm template for our back or agent for the agent I'll drop that in the twitch chat for our friends here So Big gnarly thing It's surprisingly, um condensed actually I mean, it's Yeah, but Because where it says api groups blank api groups blank, right? Like that's that's a good thing, right? Like Yeah, well blank quotes are good because there's no definition So remember each resource has a different set of api groups So if we look at the the thing that I just linked right so starting on line 13 Is the first api group and the first set of resources So essentially it's saying for services endpoints pods nodes component statuses any api underneath those You can do get list and watch. Yeah, so likewise for events. So line 28 You know, it's adding create inside of there. Yeah So and that's kind of the So you can see an example of a specific api groups on line 34 right So if I'm looking at a You know, for example a crd I'll I'll pick on Openshift virtualization right which creates its own api group You can do the same thing with you know, almost any operator is going to have You know the crd said it applies to that's where you would apply to its subset of resources inside of there Yeah, so for for example, I actually have a Operator that I wrote Openshift operator Install Yeah So don't this this is a while back. So I have a cluster role and I actually Had to add I had to add this From operator to work Because I needed to specify the openshift route, right? So the I'm kind of you needed to give it Unique permissions to make go routes or yeah, so my operator creates a route for this application Um, and it needs to be able to create update delete get list watch. I should probably get get rid of Do we need do I need to watch? I don't know. Um, but it like it messes it messes with the routes. Um for for this so like the So as Andrew was saying if you have like an operator that Deploys a crd, right? So if you're writing an operator, you're deploying a crd. You have to just remember to put that um into api groups um our back What's this? Roll binding right and not not the binding the um The other one on that side. I'm drawing a blank Cluster roll cluster roll binding there it is man cluster roll sorry um So, yeah So if you see enough of these it actually starts getting I know it looks it looks like for me at first. I was like, yeah, I never do this. Sorry. I needed to update I think I need to update verbs all just just do everything. I was debugging something. Um, Yeah, so in those api groups you can look at with Um, so the simplest way I know of is oc space api dash resources. Yeah um I have to keep checking these commands because I use them so infrequently that i'm Yeah, like this is all like again dust and cobwebs territory. Yeah. Yeah So you see at the top where ip api group is empty that would be the blank string So anything inside of there is going to apply and that's um, this right here, right? Or yeah, so that's kind of the core set of capabilities associated with kubernetes And then as you get down below it's all of the crd's That are implemented through various things Yeah, so for mutating webhook configurations for this api group, right? This is not namespaced Yeah, I'm trying to remember and I'm my google is Failing me of how to how to get a list of all of the verbs available for each one of those Oh Available verbs. Yeah. Yeah I think it's that easy Is it not Should be easy Like that's it right? I'll see you get See if you get cluster roles So that's uh, well, it's there by the way, uh, two dash o wide dash o wide Ah, there we go on the end of the, uh, oc api Oh on the api. Yep. Oh, so cool There it is See yep, and you can also get a look at the sub resources, but I think it's um, if you do in oc api resources and then dash dash api dash group It's like hold on. Choose one Yeah, I want to choose. Yeah, choose one. So we um Oh, does it get api resource, right? So you don't have a on mine and I have cobert So do like a console.openshift.io. So oc api dash resources space space dash dash api dash group equals Gotcha group and then still kind of do like routes Um, does that work that might work? It doesn't say anything. Um, what was the the the other one you were saying? So do like, um, console.openshift.io It would be that middle column or this So and then there's there's the different things inside of that api group that you can apply different or selectively Choose which yeah the herbs you want to apply to So if I do why does it tell me? Yeah, so verbs Delete delete collection get list patch create update watch Yeah, so in combination with The command that you did earlier from a cli perspective with the dash Is it dash v equals eight? Yeah, so you can see which api endpoint it's talking to and then you can dissect from there You know, that's the specific command But that'll at least help to find which api endpoint you want to talk to and then using these two commands You can figure out one what verbs are there and two what other objects are underneath that api. Yeah cool So what you're saying is I can make a lot of money if I got very good at uh, A kubernetes arbeck and consulting for people. Yes Well, I think yeah, so I think I think I think we're we're at the points in this whole kubernetes thing where it's like you find a subspecialty and just Um, well, yeah consultants in that so like Yeah, I think we've been at that point for a while, but definitely infosec kubernetes. Yeah. Yeah. Yeah. OC get dollars. There we go. Yeah OC gets your dollars I like that OC get dollars So, yeah, there's a lot of Especially an open shift because we put a lot of uh time effort energy into the configuration of it. There's a lot of Uh, it was you know api restrictions in place policy that You know keeps the guardrails on but you can easily easily turn them all off if you're not careful. Yeah Turn them off Yeah, or just lock yourself out of them if you're not, you know being super careful so Tinker with care when it comes to these things Isn't that one of those there used to be a thing It was uh, like a competition of how to recover or how to handle these scenarios and there was one where it was somebody replaced the cp binary with something else or or Like ls or something like that like how like if if you were to have yeah Yeah If you lock yourself out of you know remove your own role for cluster administrator, how do you recover from that? Right? Couldn't do it. Give me a new cluster Give me a new cluster Open ship delete cluster and and open ship create cluster Reapply all configs. Thank you. Yeah. Oh, what do you mean? You didn't have your stuff in version control. Wow But the data the data open ship container storage now hates you. Yeah, that's right Hold on I have um again. So by the way another shout out to Andrew block He's uh He's telling me to send him some stuff. So I'll do that. So I'll do that live So, yeah, uh nebulizer is asking if he has any recommendations That they might want to change about the data dog supplied config. I'm gonna go look at it again Yeah, let's go look at it. I feel like this is a loaded question Yeah, yeah, it might be but We know we'll do it live Yeah, let's see Oh I don't think so. It might be a group. So by the way, um quotas Another thing to take into account for authentication. Uh, sorry for um for our back Is that if you are part of the group some things Will um, we'll overtake other things, right? So if I assign a user So let's take my example here of no rsh, but I'm part of a group that allows rsh Um that'll it'll it'll be overwritten, right? Yeah, so So it's one of those things that the most permission wins Sort of things. So if I'm part of a group, um, so which I don't think I'm part of the group It's an important differentiation to make of most instead of least Right, like it's not least privileged by default. It is if the if the multiplexing of roles adds up to Like user user has higher permission, right when you pile everything together I'm still looking at the data dog thing. It looks pretty clean And they've even got Am I part of a group that might be uh No, I am not Or maybe I am There's no groups Okay, so that's not it cool, um, we're looking at the data dog Yeah, so I I kind of agree with chris of I don't see anything in there that stands out to me Yeah, there's nothing jumping out as being like, oh my god, don't do that I mean, they are creating some stuff that I wouldn't be familiar with in here, but um Or applying configuration that I wouldn't be familiar with in here. I should say So it's also important to take into account the service account or the account in general that's associated with the role, right? Because The the role bindings a cluster role binding is not namespaced. So Whatever namespaces the service account has access to it's going to be able to do those things To any objects inside of there, right? So it is a two-step thing and I think that's an interesting one that often gets overlooked of When you execute a pod it has it is being executed with a service account There's even there's a default one as well And from inside of that pod you can pull out the token and you can then authenticate back into the cluster to make queries So even if your user accounts doesn't have permissions to do things if this service accounts that is being used You can still make things happen either directly or indirectly that may or may not be um Desired by the security or administrator team That's a good point Very good one. I learned that one the hard way by the way. Yeah How did you learn that the hard one? I am also curious of the story Yeah, it was it's come from it was in a positive way. Um, okay good. It was it was many moons ago. I was working for a different company and Was trying to research why we're having deployment issues and it turns out that We weren't executing the pod with the right service account So I kept banging against the api saying but it says I can do this like it can't like it's it's letting me do this Well, why can't you do this and then I had to Basically, oh you were using your service account Yeah, and it was working And the service account so it's like if I created a user on my linux system and the user didn't have something in its path Yeah, basically just because it's not gonna work Yeah, it was one of those just because I created the pod doesn't mean the pod is running as me Me the pod is running as the service account. Even if that is the default service account Yep, right Cool cool So let's uh I'm gonna move this here So just to complete the thought I have to go here. So in uh, so if I do here uh user two By default the developer view has Only the it'll only show you the the built-in ones. You have to actually go to the admin view And then administration, I believe it's administration or is it user management? It's one of these things here Someone showed me the other day No No, what are you doing? There we go. Um Got distracted by a text message from my bed That's interesting Oh, yeah user two headed. I wonder what this is Disappointed that you have a life outside of twitch chris This is right. I know right like it's amazing Um Oh, okay. This is just the role-binding. Okay What are you looking for christian I am um, I'm going to assign. I want to assign the I've done it with the the I want to sign the nor rsh to the user two Oh through the ui. Yeah other ui. I saw before go to Thought I don't think I think it just shows under resource quote. No, it's not resource quotas It arranges. It's not I would think it'd be under there. So let's go to projects. Let's go to this guy here. Did my findings Did my emergency backup cluster spin up yet? No, no, it failed. So let's go to Uh, I know how to do it from the command line, right? So if I do oc oc who am I? Okay, so let's do user. So let me use user one uh user name user name One day, I'll get this right That's not it What happened to my here we go user name is user one And then I want to do oc Policy add role to uh user No rsh To which user user two in the namespace Yoshi And I think that should show up here. Yeah, so that shows up here So we got no rsh here. It shows up there, but not in the developer view from what I understand. Um Let's go to project access Oh, okay. Yeah, I have to remove this There we go Well, I guess we can show that after we can show how like one supersedes the other Um So role bindings there we go no rsh. So user two has no rsh. So now if I log in As user two And I do oc get pods We'll see rsh this guy Um Actually, let me do oc who am I to make sure drum user two which I am Yeah, so now now I'm forbidden right so now I can rsh into the pod Um what I can do though Oops, let me get the log out of here Open shift Um I can still um, I can still edit this right so I can still scale up scale down Right if I can rsh to it no one can I can scale to zero um So I can do everything Needed to do a um To do edit but without doing rsh, right? So now So The big takeaway from from that just to complete that thought was to Take something that's already there And just modify it for what you need. So um, I am curious though. So what happens if I do can I? Going back just getting ready to suggest that Yeah It's probably still gonna say no. Oh, I was gonna say it's gonna say yes That would really Just to piss you off. Yeah Yeah I'm gonna have yes, right, uh So oc get pods Let's see rsh. So I can as user one Uh, can I as user one? It's gonna say no Lock level equals eight sure The only reason I'm doing this is because Andy asked me to do this By the way, Andy had a great, um Summit talk. So if you guys want to go to the summit talk um um I think I think the videos are still up on summit, right? They'll they'll be up for a while for it right now Yeah, they should they should be up for a year Yeah, so he he did one on our back I'm searching It says allowed equals false Do we put them on the main redhead? I think we did No, you have to go like to like the summit page for redhead like redhead.com slash summit. I think yeah It'll rewrite you right. Yeah I'm still on this can I this is this is this is gonna this is gonna You have to register apparently This is gonna I won't be able to sleep at night um If I don't if I don't figure this out Good stuff so, um Okay, do that show you animal. No, so I would think that Well, I saw that in the um in the api call going back I wonder if it's a sub resource. I don't think so pods or pods Off can I list jobs batch bar? No, okay, so we say Um, is it also we'll see there's also uh who can No There was something I wonder if it's because So I want all of this is probably fail. Um Yeah, yeah, see I'm thinking that's because I'm part of a Maybe opuship puts me in part of a group that I can't see Then that that wouldn't explain it That would explain if that says yes, and I couldn't You're just not gonna sleep tonight. Are you I am not gonna sleep tonight? Yeah I know that there is a command that will print out all of your permissions But I cannot remember what it is and I can't I'm struggling to remember what even the sub command is in order to try and Search for it. It's not get it. It's not get it. It's something else view No We'll see get off. No, not off No, um describe That's a namespace Um, this is not what I was expecting This just shows Uh, oh see get cluster roles I'll think it'd be a cluster role. Yeah So the cluster role cluster role bindings Yeah, because there should be um Not all namespaces. I want it maybe on yoshi I want yoshi and that's the same thing. I think it's just a global one Um, and then I want Okay, so it's basic users. I found oc policy who can I think that that's that was the one I was thinking about earlier. So oc policy, can I oc policy who can oc policy sec review Okay, there it is Okay, so uh oc policy who can execute pods The namespace yoshi We got open tsc manager and then these users. Okay um System cluster admin system masters. Okay, so that wasn't So that goes that theory. Okay. Well, yeah, well, because I was thinking that because like when I go to um oc get Uh, yeah cluster role bindings and I was looking like maybe like a basic user or authenticated Like if I'm part of a group That is allowed to Or isn't allowed to or some are you in the right namespace check your namespace Yoshi Said Yoshi Do you know who can oh in you mean like this? yeah Yeah, so, um By the way, thanks chris. This is actually a good command for the flow. So for those of you on on the stream Yeah, sorry I don't want to gloss over this. This is the oc policy commands super super super fun Yeah, so like who can do x right like you can literally say like who can do exec who can do rsh who can do anything list you name it um Would it be cool? Let's see. Let's see. Hold on who cannot We're not so lucky We're not so lucky I'm still looking here oc Yeah, I'm not seeing any so Because I was wondering like if I'm part of a group I it's still just it's it's still making me crazy that how I How it displays? No, but I can it wouldn't make sense to me if it were the opposite way, but Can execute odds in Yoshi? So um And it says system Cluster admin i'm trying to view So these are all the service accounts attached to this It's interesting because it doesn't list user one even though user one Is specifically in there. Yeah, he's the admin of this of this. Um Uh of this namespace this is This is like weird. This is like let me go find that engineer type stuff. Yeah. Yeah Let me go find an engineer Any engineers out there come help Uh Local hosts. Okay. So Yeah unless it puts uh It unless it puts it under the group, right like it just No, because it's the one that would have listed as a user. So that still doesn't make sense system cluster admins But then there's no system cluster like their system admin Right, so it's not like it's under like you see what I'm saying, right? Like it's not like it tossed your user under another user or group or anything. Obviously you can't toss it under another user Um, so yeah, why the hell isn't and and user one isn't it isn't a isn't a system cluster admin Right. Yeah, and it's definitely not a master of the system. Um Yeah, that's weird Yeah, it would be nice if there was a where do I get this permission from? Yeah, exactly. Where do I Oh, there wasn't an inheritance thing I saw the other day. Uh, crap. No Okay, now I got to find my history Like we'll see policy tree would be nice, right? Yeah Service account. No, we don't need a service account These are for current project authorization overview It might be a different api that it's calling. Um Because it may be a project Then I'm grabbed grab grasping at straws here Cube cuddle off can I? Mm-hmm Can I create deployments dash dash namespace? Yes or no? No, so try it. Wait, were you using the off or not? I was doing, um, you're doing just oc can I right? No oc off you're doing off. Okay, never mind. Okay Yeah Can I um exact pods Into yoshi right like you're right like specify your namespace Yeah, and you can also specify a user as dash dash as if you want to like make extra sure you're using the right user Yeah So as um user one, right So it's no Weird God Yes oc off Who can right? That's what we're doing earlier, right? Yeah, oc policy. Yeah, oc policy who can yeah, sorry Yeah Yoshi Because it would be what roles are assigned to that user too Uh dash o yaml Describe maybe yeah oc describe cube cuddle off Reckon Reckon styles everything Oc describe user user two nothing like way digging into the docs here I do a yaml describe with a yaml does that work? Just says i'm a kind user I'm no part not a part of any group Get user user two o yaml Um i'm deep in the docs here Yeah oc get um Yeah, let's let's let's do a eric jacob's um docs. Yeah, that's that's that's why we're here. We're learning together learning together. That's right Feel like i'm close To the api discovery roles Default role binding authorized unauthenticated So I think it's in the gooey, but I don't know the cli command Oh, where is it in the gooey? Oh, I think it's under the user details Oh what they can do specifically Well, what what roles are assigned so under user management? Uh, i'm just gonna do this as admin Yeah as well, so Um, yeah user management users Right user two role bindings I'm just no rsh on the namespace yoshi. That's the only thing i'm doing i'm part of And then if I do user one role bindings I'm an admin Let's look at the admin yaml So the role binding admin Okay in yoshi User one Okay, cool. So there's a question Christian, uh, would have thought configuring the users on the open shift platform would be much more simpler than looking at This thoughts question mark. So it's it is simple. We are actually trying to debug something Which I think we might be getting a little lost going down. Yeah um, so what we're doing is We've configured a user that shouldn't be able to do certain things and yet it is able to do certain things So no the the it's actually the other way around. Yeah the other way around which is which is which is more even more confusing, right? um I'd say it's even more nuanced than that in that They're allowed to do it But the command that queries whether or not they're allowed to do it says no no Yeah, but they actually can do it. But when we do deny that Activity it does behave as expected. Yeah. Yeah, right So I don't I have not seen I don't know if there's any plans for Like a GUI based tool and this is Andrew doesn't dig around in the GUI a whole lot either Um around so creating me a role with a bunch of check boxes, right? Very much to your point earlier of is there a tree view? So that's through the administrator console I could go in and say, you know, create the no rsh rule and check check check check check check check Or copy slash clone this rule into a new one and then edit it which you can do through yaml, of course But that might be a good suggestion for the ux team If we haven't if there isn't something already out there for that And and that's been a request for a while now and Yeah, well it's since since the beginning right so since the beginning Of version three right before kubernetes had our back We had to open shift and that was a request even then like Okay, like this is this is all cool. You get pretty granular um But play it to me make it so make sense of it for me. Yeah. Yeah makes sense of it for me. Yeah, exactly. Yeah Okay, no rsh. Okay, so that doesn't give me anything So I got groups. There's no groups. That makes sense um Yeah, so user one It says I have admin to yoshi, which is yeah, okay. That makes sense user two Now has no rsh, right, which we that's We did that but user one Cube cuddle plugins that let you do this are enumerating every bindings. There's no real way to look up. I'm reading a Chat yeah, there's no way to look up what users or members of a roll cluster roll binding and open shift for kates Need to enumerate all bindings to identify the user The cube cuddle plugins that let you do this are enumerating every binding so we would have to uh Blow out everything right like describe all this stuff and then extrapolate from that what that user could do Which I guess looking at it from a kubernetes perspective kind of makes sense because it's kind of cloud native You know big systems design not necessarily like oh this one user. Are they allowed to do x? No, that user should be part of a group that group should be allowed to do x so forth so on right like It enforces best practices That's an interesting one. I wonder if a cm will solve that for multi cluster To be clear that will a cm is red hats Advanced cluster management new shiny new product. We got yes shiny new product You know namespace the midrange create truncated So that that makes sense to me So it's not like we're we're asking anything weird in the namespace Yoshi. Can we exact? pods It says false even though user wants admin So it's telling me no even though it's yes, so that that's what we're trying to debug. Why is it telling us no and can you Like after this can we sit down and like just spell out exactly everything we did and like Maybe ask somebody like how the hell this happened because I don't think it should be Yeah, I mean I I I agree. I think I think it should tell us what's actually going on Or if it isn't it's not clear to see why It's the way it is right exactly So it's a so I can't say his name to save my life, but i'm pretty sure the person that's writing in chat Is also the person that wrote the the kubernetes operator the book. No the kubernetes patterns book Patterns So said it might be someone in chat say might be an effect on how you look up objects That's what that's what i'm wondering it is. Yeah, what am I invoke me all by means where the user equals yo Like that doesn't exist Yeah Like ldap right like ldap you can get pretty you can get pretty gnarly with the ldap Search Yeah, i'm ml biome. I think is bill jim abrium from kubernetes patterns fame. Oh, okay. I think the names line up. I don't know I could be wrong Identify yourself random user from twitch land Random user from twitch land. That seems very educated. Nope. That's not the person. Okay. Well, you're very smart on kubernetes Thank you for joining So Post this to Reviews Huh, i'm i'm just not seeing it I'm not seeing why oh, it's mark. Uh, oh mark Okay Yeah, so i'm not seeing why it's telling me No, and in fact is yes, so it is yes. I know it's yes So it's it's not like uh the behavior is wrong It's the reporting isn't matching what the behavior That we want so we want this behavior. It's almost like it's almost like complex systems are hard. Yeah It's it's actually in the name complex systems. Yeah, uh, the What do they say what is about rocket science? No, uh bomb text if you see me running follow me, right? Yeah Yeah, if you see me running follow me. Yeah, I think that's the only direction you need right if you see me, yeah This might be one of those problems you see me running away from unless I can file a bug against it was somebody to be somewhere but i don't even know like Yeah, well, so, um mr. Mark Says what's wrong? So what's wrong? is so So to summarize I have I have a user user one that owns a project named yoshi Um, and this user is an admin So that's okay. So that's fine but when I ask oops Can I right so like as user one can I execute? Exact pods in the name space yoshi. It's telling me no although We'll see get pods as expected. I can um oc exec into this pod You know bin bash That's expected because as user one I'm an admin I should be able to but it's just telling me That I can't even though I can The reporting I guess what's wrong? The reporting is the reporting is dead wrong The reporting is wrong. The behavior is what I is is fine is what we want, but The behavior is exactly as we expected, right? It's not that we're specifying something and it's yeah Not letting us it's we're specifying something and it's telling us we shouldn't be able to Oh, see yeah, oh see who am I? Yeah, that's we did it all that We've been there We walked down this this rabbit hole. Yeah, like this is a weird one like in mark like I'm like like the fact that Like we're walking through this together. I think is interesting because that's yeah Super weird out of all things that I didn't expect to work. I just wasn't one of them I expected that that like I would have created a policy like the noris h and like it would break somehow Right like we'd be working on policy all day, but yeah, exactly policy works fine It's the reporting of it that doesn't Exactly. Yeah, the policy is fine. It's just it's just for whatever reason. It's telling me Um, and then mark just put a question mark. That's I think that's Yeah, like yeah, that's pretty much where I'm at Pretty much summarizes Um drug emoji What else support other what other mind-bending things can we throw? Yeah Can you attach pause now is pod equal to pods? Yeah, it's yeah, it's it's it's plural or synchronous or whatever. Yeah, you can use the uh OC api dash resources to see all the shorthand versions Yeah shorthand is po and there is no singular So we we decided I think the can I might be wrong? Yeah, so that that's what I'm thinking like the way the way I'm asking it might be wrong is what I think I just can't see how I can't see that. How else would you ask? Yeah? Yeah I'd just like to point out that pod is so difficult to type that we took out the d Validating web hook configurations. We have no shortening. Nothing Nothing horizontal pod scalers. We put hpa hpa. Yeah. Yeah, pod. We put po not p Just po So funny or cluster roll. Yeah, so off can I Um, he says it's right. That's what I'm thinking that I might be invoking it wrong Can I create can I um? Yeah, that's what I try and um, can I get pods slash exec Ah, that's a good question. I'm just wondering you must specify today. So I need we need to do verb And then resource and then option Right Try um oc oc off. Can I um exist or create pod dot exec And see the server doesn't know pods Okay, basically I'm making that up on the spot, but yeah, so This is a good point though, right like oh, hey, there you go. Okay. There it is. Yeah. Yeah, so um, but but so Yes, we found the answer but to Andrew's point no matter what the default was it's going to tell you know Yeah, so warning the server doesn't have a resource type pods blah blah blah. It's going to tell you know if it doesn't know about it So can I create so the logic so the logic there is remember when we looked at the verbs that were available Exec is not a verb. Yeah, what you're doing is you're creating an exact what it was Against the pod so exec is essentially a sub resource. Wow. Thank you mark Yes, so um, what was that one where you were um, Andrew that the command where it listed all the verbs Dash a wide Okay, okay Yeah, so I have the yeah, so it's um, if I go do pods. Oh god messed up. Okay vi Oops is it pod? Unexpected arguments. Oh wait, now I know what I'm doing. Sorry and then pod So pod you could get create delete delete collection get list pass. Yeah, so exec is not part so, um So the create so this is like the basic crud thing, right? So in order to exit your yeah, okay So we we figured out why So we want to hit when it be able to do the verb create pod exact because the exact is the sub um A sub what what what they call it a sub resource Right. So here it says pod log deployment scale So that's what that was. That's kind of a different convention then I mean because he look like even like deployments extension Like I get it as a sub resource Not not an object. Oh, okay fine. They specify like today I learned Yeah sub resources are Uh annotated differently than objects. Okay, cool So here. Yeah. Yeah So can I create xd. So yes, and then we'll see who am I? um To bring this back full circle i'm user user one so um Now you could you could run that same command with As dash dash dash as user two Not not as user one though. I mean you gotta this let's try that I don't think it let's I don't think it let's stand it. Okay. Okay assume roles of other users. Yeah, you can impersonate That's a good idea You should become user two user two Such a user Such a user. So can I create exact pods? Huh, yes That should be no Now we're getting the opposite problem Yes But you you didn't change anything and yeah, no i'm forbidden We were almost we were almost there I think Okay, so I can't execute uh, can't I create pods? I said This is yes. Okay, so now we have the opposite problems You can tell by looking at the api docs for an object Dot sub resources or listed as additional you are as additional URLs. Okay, cool The api docs for uh for kubernetes, I would imagine For an object sub recent now, we've got the inverse problem where it's telling you that you can but you can't but I can't But the behavior is fine Because the behavior is what we want but now sage christ is it possible that can I it's just one of those things that sucks It could be Or it's it's what it's one of those commands that no one like no one really like has given proper attention to maybe So can I lingering create pods Um, so do uh, let me yes Oh, why do uh Crap now I'm trying to say like do a list Can I create pods exact? Can you read pods as r is rsh? Okay, um, what would do uh So here project uh projects delete, okay so Can I delete It's not namespace slope This is no, okay, which is expected And then user name So user one Can I Delete project yes, okay, so that's expected Because I am the admin of this product user ones the admin of this project So now what's stripping me up? Is the fact that Yeah, so, uh, can I Mother may I Telling me yes, even though it's a no Yeah, it's telling me yes, even though it's a no Yeah, me man Like I'm gonna send this whole thing off to like erin krickenberg and just be like what the hell What's going on? What is this? What is this? I'm trying to find this the list of sub resources That would apply um I just I don't understand why I would say Yes When we've proven it defaults to no and you can't Oh different stocks api equipment is dpi This does not have what I want. Yeah, it's the concept stock, isn't it? I think you may be the first one that I was looking at It's an old one Yeah 1.11 This uh 17 17. Yeah Hmm. You've got like pod v1 core there so exec would be a I guess we just look all at all them. Yeah, we're just on core containers It might the exec might be That would be all containers in a pod I was just wondering if we're not specifying something else like something more specific Because when you do a an exec It assumes the first container If has multiple containers you have to specify the actual containers you want to do the exact on Yeah, that's just a dash c command line off. Yeah option So there is an exec action in v1 core Where's the exec action? It's an action you said There you go. Exec action v1 core Command is the command line to exceed The command is simply executed not run inside a shell. So traditionally shell instruction Unless it won't work to use a shell you need to explicitly call out that shell. Yeah, okay I wonder if is it is exec one of those things where it's like It's short some underlying permission that exec calls upon or something Or is it an aggregation like an aggregate is it shorthand? Yeah, yeah And one is recognizing the shorthand and the other isn't it isn't recognizing. Yeah This is yes, okay Let's do a log level. Let's see if I see anything here. I was a log level 80 finals um So says create status allowed equals true Although it answered false here so Keep kind of exec process creates a process and then Checks to see if it has access to the case api server, right? Yeah so It's user controlled tainers shouldn't matter like I thought exec was like one of the simpler things So it says reason are back allowed by role binding no rsh cluster role no rsh to user So maybe He got something nested in there. That's what I'm thinking um But if it's nested how the hell would it not work? So pot attach Yeah Yeah pods port forward pods proxy attach port forward proxy resources api groups that's for metrics Get list watch. I wonder if it's grabbing something else from over here. Uh metrics case.io Get list watch. There's no create pods Get list watch. There's no create Yeah, that's that's Let's see if I can Just Um, I'll just say can I but I've been I I just I don't trust this so yeah Can I I don't know it'll say yes or no you may or may not though It may be the opposite So there's a question and we've only got a couple of minutes left. Yeah Can you lead to a guide for setting up users and roles? For example, we have two different teams that will need to access to the open shift platform to maintain and deploy their product containers Without seeing one another's namespace So I'm sorry. I'll read Reading can you all link to a guide for setting up users and roles? And open shift like that's it two teams that will need access to ocp paths to maintain and deploy their product containers without seeing another Okay, I think that's the uh, that's the trick That's well, I think that's the default. So yeah, they won't they won't be able to see each other's yeah yeah, they Yeah, they won't even be able to see like Anyone's namespace are not a part of right Yeah, so if I go here correct, you want me to be able just just to see Like it'll just look like it doesn't exist, right? Like there could be some resource they might see Or some tension they might see or usage They might see but they won't know like where it's coming from even if I do an ocp projects I only see the projects I have access to right, um and then the Setting up users and groups, um the documentation Is I'll drop this here in the chat. The documentation is um, this is the slices Silsis there it is um This is a good, uh A good overview of users and groups, but the the default behavior is that You only have access to what you have access to so if you give access to a user or a group to one namespace Um, whatever it is like admin access to a namespace they can admin that one namespace locally yeah Yeah, like it's And this is why like for operators, right like we've realized that Some people might want to Be able to control their own operator and their own namespace Right now we have it set up so you have to be a cluster admin to run operators That's kind of becoming like a problem So we've realized that you need to give more granular access control to operators so that People can run operators in their own namespace because well they can't run them anywhere else unless they're cluster admin right now And that's putting a burden on everyone administering open shift clusters right now. So yeah, uh This is definitely something that is granular as all get outs can add a lot of You know burden if you're not careful, but by default It's pretty safe right like you create a user over here. That's all they're going to see is what's over there If you create a user over here in another namespace, they're only going to see this namespace Uh, if you create a user that sees multiple namespaces, but not all they will only see what they can have access to I think I figured it out So no, I can't rsh okay So that's true Because I am user two if I are rsh I can't so that makes sense and then um user one Can I create pods substance resources to execute? Yes, and that's true because I can rsh. All right For whatever reason it just doesn't like to slash interesting yeah um Because user two if I do uh, can I oops, can I says, uh It says yes, even though I can't Hmm That's expected Wow Yeah Syntax will get you every time Every damn time. Yeah. So so thank you for joining this two hours of us Yeah, this this this is we'll come back to another arbeck ask us anything At some point tell me the future I feel like hopefully um So yeah, syntax like you said syntax is everything syntax is everything right like it's timing and syntax right like those two things If you get those right everything works if you don't that's right. No, you got big problems But you got you got big problems. This is big problems. But yeah, so uh to wrap this up Thank you all for joining us today. Uh, we really appreciate all the help from the audience all the help Uh With you know just figuring out and being patient and working with us here Uh, we have a follower goal in progress. We are trying to hit 1000 Uh twitch followers by the end of june. We are currently at five hundred and 29 I will actually drop the uh stream labs Uh Like counter thingy and chat so you all can watch it update live if necessary or so so so you see fit Um, but please we'll have a big party, right? Yeah, we'll have a big big big party on twitch when we were on twitch a big Virtually distant party. We could have virtually distant We could have a happy hour on twitch I have no problem with that where you come in and ask your questions and you know We do some fun things on on twitch, you know, maybe Uh, once we hit that thousandth follower, we'll get there, but uh cake and hats. We can send you Cake and hats. That's right. I don't know the cake might not make it that that can get a little shifty in transit I feel like that's right. Um, but yes, please tell your friends Please subscribe. Please get the word out there that we are doing this You know every day for open shift commons. I know they're scheduling stuff We art my team specifically we have stuff multiple times a week Uh, we just had a wonderful developer advocate session the first thing this morning So yeah, keep joining us keep after us. We're very happy to have you here tomorrow. We will be on again with uh, i'm gonna do a rerun of mark's show Because I had some audio issues first thing in the morning tomorrow Um, and then at noon we'll be joined by open shift commons for Uh runtime component operator No idea what that is, but it sounds exciting. Yeah, arthur day maga Arthur from ibm will be joining us tomorrow Like how you just gave up That reminds me that reminds me of office space where it's like, um What's his name and he goes naga and not well not gonna work here anymore. So it doesn't matter That was like one of my favorite scenes All right, as always, um, I need to get uh like an faq panel built into our twitch channel here in a few minutes But as always if you're curious what's coming up on the the the calendar here, uh drop it into the channel red Slash stream cal all one word You can subscribe to that google calendar and uh be well aware of what's coming up in the uh live stream We're trying to get things uh scheduled further out in advance so that we can uh get y'all heads up sooner rather than later We know you're busy and it's hard to get blocks of time in your calendars for something like this So thank you. Thank you all so much Any pardoning word gentlemen? No. All right. Thank you for uh, um We're putting up with us and uh, it was a lot of fun. Well, it was fun for me. I learned something new. So Yeah, I learned something new too. Wow. I mean, that's that's intense. Who would have known it was just syntax is that what I had to learn? But hey Well, yes, thank you everyone All right, thanks guys