 Okay, cool. I would say ladies and gentlemen, but I don't know anybody's pronouns Hey, everybody. Thank you for coming at oh my god. It's early here at lovely Boston University the Topic for today's conversation and hopefully we'll be a little bit of a conversation is delegation I'm Adam young close Lucian's architect, which means I help people build their castles in the air. I Have many years as a software developer before deciding I should actually go over and Talk to end users of our software So if the past couple years, that's what I've been doing when we talk about to delegate means to Give to somebody the power to do something a task and trust a task Maybe multiple tasks for somebody and usually you're passing this on down the line when my little visual here is our Delegates as they drafted our Constitution and the idea is that this is how? Organizations scale. This is how you can do something as big as run a country You can't do it all yourself. You can do anything. You can't do everything So if you need everything done, you have to give some of that to somebody else If you have a so here's a org chart doesn't really matter with a little box to say We all know what an org chart looks like and up at the top of the board directors who delegates the CEO and eventually things get down here to Information Technology Department to actually execute things. So what have they delegated down? They've delegated power they've delegated the ability to execute tasks and and this Concept this idea of delegation is kind of Ignored you know when we when we talk about the the personal Devices that we have here Delegation is still important, but it doesn't become the first-class citizen that it really needs to be when we start talking about the cloud So what I'm going to do in this talk is I'm going to take it from The the small scale to the large scale and talk about how you can build better delegation into what you're doing and so when I delegate to To you the ability to do something important like drive my car That's not my car, but it looks a lot like that one I don't give you my whole key chain Right, there's a lot of stuff on here including my other car The key to my house which you don't need if you're driving my car and You certainly don't need the key to unlock my son's bicycle what you need is the targeted delegation for the resource that you're going to be commanding and thus We give just enough to get the job done the further you delegate and I say this as a joke I love Howard's artwork and his stories, but the point is is true that If you want to feel comfortable that your organization is scaling and getting things done You want to feel comfortable delegating on down you want to do this in such a manner that you know that only What is necessary goes on down? Now we don't typically do this in systems we typically Make the admin do everything Until we get to the point we can get to self-service provisioning. So it's kind of all or nothing. So what are we talking about in? the realm of Linux as far as delegation files Really everything comes down to files within Linux is you could say it's system calls and Yeah, there's there are a few other system calls that don't quite fall into the file category this stuff like iOp of that for the vast vast vast majority of things including sockets and File descriptors of all different sites you can read you can write and you can execute And that's the level of granularity that we have and We can delegate this we can assign on a specific resource on a specific file to a user a group or the world The ability to perform one of these things. Okay, this should be a shock to no one This is all old had you guys you like why the hell am I in here? I should have gone to the containers talk Okay, yes, right. I want to give you and only you the ability to Read a file. I wrote How do you do that? How do you make it so that only one specific targeted user? Hello, everybody. I guess I'm gonna have to start all over now. Thank you all for coming on in I realized a really early thing. You've missed the drama. You missed the saxophone although I might be a little bit at the end But there's enough new people in here just to go a quick catch up. We're talking about delegation There was a lot of big drama about the ideas of how you delegate down a hierarchy and give power to people down the bottom Now that we're caught up. We're at the point where we're talking about file systems. Okay, because delegation in Linux is targeted at the file First of all, most that's the primary resource that we have and I'll go back one slide And then I'll you should be close enough to cut up. You can read you can write you can execute That's the level of granularity you have on a given file And you can say a user me the person who owns it the group that owns the file that the file is labeled with or the world and That's the that's the these are the rules of the game. Now. Yes. I understand that in Linux there are capabilities, there are other ways of doing things and You should use those two what I'm going to do is focus on this right over the plate way that most people think about managing access because capabilities are a whole Bigger thing and yes, they exist there. Yes, you can use them. That's not what I'm going to be talking about here I'm going to say how do you make a better system for handling what people expect? Because the bottom line is if people don't know that a A feature exists. It doesn't exist Right. So how many people here are familiar with capabilities in Linux systems? Okay, so you I drew a security focused and security concerned group. So Bit of a selection bias there. Good. Good. Yeah, and I'll treat you to trust you to go out and read about them and be able to use it I'm talking about for now just the basics and As I was saying here is Pointing a random audience member out completely randomly. I want to let him Read a file. I wrote in only him. How do you do that? Short of capabilities With that granularity that we had before The only thing that allows multi-user access and limited multi-user access is the group abstraction Okay, not a super Powerful tool, but that's the limitations that we have to work with it Now I can show it. I can say oh he can he can He can read it now, but at that point he owns it to which point he can say oh you can no longer write to it or He can share it with somebody else. I don't want to give away that much power I can change the group of it, but I have to be a member of that remote group Okay, and in order for him to be able to use it He's got to be a member of that remote group and the permissions have to be such that Owner has right permissions and group has repermissions And we need a two-person group so guess what I have to go to the admin say hey go create a two-person group okay, so Etsy groups is owned by root I have to go to the the system administrator and if it's you know my laptop I am that system administrator, but then again, I'm not gonna let you on my laptop We're talking about multi-user systems here. We're talking about systems where people are going to be sharing resources and So in the Etsy group's file we have a group we call it say that this one is an example of a group called testers No password. That's what the X means there The group ID and then a list of the users that are in there You can see a young's in there. I put route route in there cloud user. So come from the blah blah blah Okay, so this is what I need to be able to control Now if I want a group like For this do thing I have to use the admin to create a group add me to it Add you to it. All right. Now. I want to add her Or her whichever her I want to add somebody else to this. Okay. Now. I have to go back to the admin Okay, you can see for a single file for a small resource. This is way too much overhead If you are dealing with something though that you have a group forming and Dynamically changing you want to be able to add and remove members from this group yourself The admin is going to be the bottleneck and that's the part that we want to get away from here So while this limited two-person group is maybe too small a scale to really give you a sense of the problem That's that's really the abstraction that we're talking about So I want to be able to add a user in a group without having to ask that administrator So here's Here's a hack. I've been thinking about three years to be able to solve this at this all and I promise you This is not the end of this. This is I realize I'm going a very slow pace Let's create one file per group now. Why don't we have this now speed right now? When the operating system has to check as a user member of a group it can look in a single file look it on up But we're not talking about that and group yet. We're talking about how we would administer it Okay, so if we create one file per group where the name of the file is the group You can add a user to the group if you have right permissions on it If you can write the file you now have control of the group But you need a utility to then take what's in this structure in update at C groups. Okay? Again, this is using Simple abstractions, but this abstraction doesn't exist in the old Unix and now Linux a few of the world And we can do the actually we could do the same thing with with Etsy hosts, right? Okay, so before we go on there I'm going to give you a little sense of what this looks like and I have to say I'm feeling particularly pleased with myself Because I'm doing split screen here because I have presenter view here and that there and I couldn't figure out how I could Not have to switch between that a mirror view to do it So I'm using T mocks and it turns out with T mocks if I do Type here. You can see it up there. So this this this is pretty now. Can you see the screen up there? Is it is it too small? Very small. Let me get that one larger I'm gonna go full screen here. So what happens when I echo is not exist. Yeah, my jokes are falling flat. Okay, so What we have here is I don't have to look up there. I can look down here. Let's see That's that's the whole point of doing it this way, isn't it? Okay, so We have this thing called group man and We have an Etsy groups file, you know, that has Let's let's let's do it with less. Okay. Now. This is usually a world readable file as you can see and It the top half are the ones that you get when you do a basic fedora install and then down at the bottom I created At the bottom I create a bunch of my own group I create a group called admins You see and in pre-fee are members of that I have a bunch of users I created the system J Williams is a pain in the neck as he likes to have his you know His username matches email. So and a couple other things. So what do we do? We want to be able to then take a look at a Groups file and I have a group dot defile a directory and underneath there. You can see I have a bunch of directories Let's take the web dev group And that's a bad one. There's nobody in web dev admins Right. So I want to add Adam and remove pre-fee. Okay And if we look at one of the other ones like DBA sim similar kind of thing Nobody there Network. Okay. So I want to add myself to the the network group. Now you can see Actually, okay This is why I never do live demos. It's only a member of it. So you can see I'm I own this right? so I can go into This and so before before I make a change there when I want to show you just what happens when I make the change that I have having these files pre Preset you to do this as route So this utility has to be run as route because it is the thing that has the power to update the file and The utility could either be done as a set UID Permission in which case anybody could run it whenever they make changes, which is what you would want But that does have the potential for abuse The alternative of course is that you put it on a cron job and then there's going to be a lag So you have to figure out what is the right method for your organization to be able to do it, but Hopefully now this works. Let's see CD. I'm gonna have to use a brain so Python 3 Group man I'm one level too low makes you Now you guys are probably going. Why did we not go to the Kafka talk? Okay, so that's gonna blow up because I need to give it some parameters. What am I going to give it? I'm gonna tell it where my Etsy host file is and the reason why that parameters in there Of course like all good things. I want to be able to test it so I want to be able to use it on the non-production workloads and And then I need to say where my data directory is that's the host. Yeah, you can tell what I'm used to I'm glad I didn't go. Wow, that would have been bad. Okay, so what did I just do? You can't really see much of anything, right? Well, I actually kept a previous copy of it So what I'm gonna show you is first of all what it looks like now And you can remember it was nice and pretty before and in all that Oh guess what doesn't alphabetical order so you can see in the middle aside from pre these groups and J Williams and all that kind of stuff There's that web dev group. I created the DBA group Andre Pat Randy are in there now They stayed in there because remember those files were empty. So I only want to be able to make changes I don't want to go in. I don't want that to necessarily have to be the canonical thing I can't I can leave them all in their plus format, but really What I want to say is this is my way of making changes so you can see that for each line there There's a plus and there was a minus, right? So now these groups exist and Arthur William Preathy They're all members of web dev web dev. So if we go into our data directory here That's the web dev And we're gonna say I'm gonna add myself I'm gonna remove pre-thee because pre-thee is no longer doing web dev Okay, and we run That again and it blew up Let's screw something up I have a feeling it has to do with it the data that I just put in a very very new code here This was really done as a approval concept. Let's See you everything and get very good. I was still blowing up. Okay, I ran once we're in first time, right? So you get the general idea. Sorry for wasting your time with a poor demo, but the whole I Was that up there like this the whole time that just happened when I switched on over. Okay, good So you get the idea and Better coders than me and people with more time to put in this could bake into a utility that would actually be useful But you can understand the concept, right? Yes. No Okay, I wanted to do a proof concept actually I originally started writing it as a way to learn rust I have one for doing Etsy hosts in in rust I Decide I was trying to learn the library to do A group management it was too much to do and I actually have a day job It doesn't allow me to spend as much time coding on this as I want to but the whole idea is that the two most important Resources that you have in distributed computing users or the groupings of users and the hosts that they can have access to Are things that you should be able to control now When I look at the Etsy hosts type delegation One of the things I realized is you only want to limit to a sub domain So I want to create a sub domain like you know young logic comm create pre-t dot young logic comm and pre-t can manage that any any Host that she adds underneath that sub domain. This is our on her Okay, so And that gets into the whole question about what do we do for more than one computer? Okay now How many people here have created a user stable for an application that they've written? Okay, so we have one honest person and a bunch of people who either are liars or have not had to write that much code yet It's annoying that we have this concept that the users are owned by the application where when you get out into the Real world that's not the case. You don't own the user database the user database for Whatever reason is owned by active directory It's owned by some directory structure L DAP out there that you are then going to consume and you're gonna get this information through a very variety of sources But the thing to remember is that you don't own the set of users and that's okay Because what you really want to focus on is what groups are those users in? Okay, so L DAP I spent a lot of time though. How many people here know what L DAP is okay, how many people here don't know what L DAP is Okay, L DAP stands for lightweight directory access protocol and the word that you should take out of that is directory Okay, because a lot of times we'll talk about an LDAP server and that's just a server that serves a certain protocol. Well That's kind of necessary. Why is it lightweight because there was a heavier weight one before? Okay, so really and protocols for accessing things so really is directories and what is it directing? Remember that the big graph I showed you at the beginning with the CEO in the middle and they all that that's a directory It is a database a hierarchical database that holds information about an organization And the most important thing it holds is the set of users that are in that up organization. I Worked on a project called free IPA, which was an attempt to take a Directory and make it more usable and easily more easily installed So the example that I'm showing a little bit makes use of that, but what we're talking about here is really Generic anything where you talk about users the users are going to come in from a web app And I'll say you're in an organization that's using Kerberos. How many people here know what Kerberos is Okay, for the two or three people who did not raise their hands Kerberos is a way of authenticating yourself that is actually secure as opposed to handing your password across the Open internet, which is what a lot of us still do because that's what a lot of systems allow So any organization that has tried to lock down security has looked at different mechanisms and Kerberos is one of the ones that a Lot of them use came from right across the river to MIT project project Athena And it uses symmetric cryptography. So I take a secret and I encrypt it and if you have that same That same key that I used to encrypt it you can decrypt it and that's how you prove that you are who you are Key sharing that goes on there. So in an enterprise in a big place using Kerberos You you log on with Kerberos and you identify yourself to some system using this mechanism And then that application is gonna say, okay now that I know, you know pre-tease here Now that I know that Adam's here what groups? Are they in and from that set of groups? I'm gonna use this to apply permissions in this application There's gonna be two to three groups that really apply Maximum for a given application. I really want to know are you an administrator of this? Are you a user of this system or are you just a read-only browser? There may be more there are often more it may be more granular per resources, but Understanding when you have a centralized repository like LDAP, you can't get super granular in what permissions A user can or cannot have for each application Because again, you're pushing off to the centralized admin to be able to control it But what you can do and what we see it a lot of organizations now is this idea of two levels of LDAP server Okay This gives you how many people here are running labs or running groups of computers separate from like the main organization that they're in How many people have like their home lab or? Okay, this or how many people here work in perhaps like a graduate lab or something like that in in grad school or something like that Okay, that's another example You often have this idea of a limited domain where you need To be able to manage permissions for people in there, but you still need to consume from one level up So what you have is what's called an AD trust or a Kerberos trust between the two same idea where you're gonna manage the list of users centrally people who are students of the university people who are faculty at the university people who are Somewhere in between But what you want to do is be able to manage this set of servers these hundred machines And once we start talking virtual machines, perhaps thousands of machines with a centralized to your lab LDAP server, but still consume it from higher up once you do this then you can say okay There are more interesting groups that I can put people into But you understand they put people into I'm back to talking centralized I'm back to talking about what I can do as an administrator, and that's not what we want to do What do we want to do? We want to be able to delegate So we want this idea of group managers And so the major shift that you have to make is thinking the group itself is a resource and just like we were doing files At the Linux level. I want to make the groups within the directory Manageable by people other than the administrator, okay, and What's neat about LDAP as a as a directory protocol unlike a sequel database I don't just limit on the Table level what permissions I have I label on the specific object who can do what in fact I can go down to even just a field. I can say on this field here Preathy can modify it, but anybody else can just read and Then the whole overall object well that has to be an administrator to create so to create the group You have to be an admin, but to add or remove users from the group. I want to modify in this case The member field of a group object is a very standard structure in LDAP that you have These things and so I'm using the IPA command because actually modifying permissions within LDAP is one of things that Various from LDAP server to LDAP server and this is the one that I know Once you do something like this now I created a group called the Beowulf Manage Group and I created a target group called Beowulf and a permission Beowulf Manage Group a permission which allows people to in that group to write The member field so people who are members of the Beowulf group can manage the Beowulf And you can tell what I was doing what I was thinking about when I did this stuff not Old English fantasy But Beowulf clusters Now what I have is the ability to say okay for any resource I have out there. I'm going to create a Group in LDAP. I'm going to create a Permission for that group and I'm going to sign the people who I want to manage that group that mission And then they can figure out who's coming in there because it might be hundreds of thousands of users From that centralized LDAP perhaps we're talking about a graduate lab where you have you know 500 people Working in there with a hundred of them changing each year This is the kind of thing that you want to be able to have the lab manager handle not the centralized admin So that's the way we're thinking about things, okay? I Want to talk a little bit about the domain model of role-based access control before we go on and I Can't remember where I first saw this it's called the party pattern. I want to say it was out of a Martin Fowler book I know Martin talks about it. I don't know if that's the first place. I saw it But it's a great way to think about the relationship between people groups of people and permissions, so You notice at the top. I have a the role assignment In bold this is the and it has a start date and end date This is the idealized case your systems may not have all this way of doing things but it may and Once you have a role assignment that means you have a role and for a given role In a given organization that allows you perform an action on resources owned by that organization Okay Let me put this, you know a little bit more Concretely if I'm a member of the lab Organization and I have the manager role So I have the role assignment of manager on the lab Then I can put the computers in the lab. The action is boot the resources computers now The majority of what we deal with now is the web we're dealing with resources that are exposed by a web API's and How many people here know what rest stands for? Okay, how many people here don't know what rest stands for but know what rest is Okay Yeah, it's something state representation representinal something Representational something state transform. It's basically using the web protocol the way the web protocol was designed to be used And in this you have resources and actions just like we had in our domain model and the resources a URL a resource is the name The URL is the name of the resource because guess what this stands for a universal resource locator it's not it's not random and Here's a handful of the actions that you can do Against one of these resources you can post you can get you can put you can delete now I put down here about XML or JSON RPC is not rest But you can use the map to it there are a bunch of other ways of using hypertext Protocol to access and to control and to manage resources and a lot of them Don't use this gentleman, but you can think about them that way if you look at free IPA again It was JSON RPC and there is an action For verb in in the payload So you can't do it at the URL level, but you can do it at the payload level There's a way of being able to map it But hopefully when you're designing API's now when you're designing systems now you're thinking rest and you're thinking in these terms because this Is the least surprise Okay, so when you start thinking of permissions you want to think who can perform the post action on the specific URL that I have there And we go back to LDAP What we're going to do is we get enough information to figure out that decision I want to talk a bit about open-stack keystone Which is the system that I worked on for a long long time where we made use of this How many people here have heard of keystone? One okay, how many people here have heard of open-stack a little bit more, okay How many people have not heard of open-stack? Okay, open-stack was a Effort to Provide a standardized method of doing what EC2 was doing at Amazon creating virtual machines in somebody else's cloud Amazon Went from being a bookseller to the world's largest data center management company And EC2 this this this Elastic cloud protocol or set of protocols was one of the ways that they were able to do it It is a way of telling Amazon's data center to turn on turn off create destroy virtual machines for you It does a lot of other stuff too, but that's the heart of it Keystone was a piece of Of open-stack that kind of fragmented off to do Access control management separate from all the other things that you can do in the data center It was when you create a virtual machine you have three big things to work with you have Compute the memory you have storage the disk that it stores things to or reads things from and you have networking And those all became separate I'm not I don't want to call micro services They're not really macro services or models. They're just kind of services. They're just Blobs there that do things and some of them do more some of them do less and they're complicated And what we want is we wanted the networking folks to focus on networking and we wanted the storage folks to focus on storage And we wanted the access control people to focus on access control between all those and keystone was that piece And so with keystone we had to deal with it rest API or a set of rest API's that are Provided for managing the data center in the large and again They wrote a user table That's that's what I joined the project. I'm like You don't own the user table users are in LDAP and the project leader looked at me and said LDAP who uses LDAP anymore? He came from Google they don't you know They don't do things the way that the rest of the world does and maybe they're right, but we're not there yet But when I said look the users should exist somewhere else. He's like, yeah, I Didn't actually want this thing here. I was forced to build it So we were kind of in agreement and that's how I got involved in keystone by doing LDAP integration into Keystone way back when and As I said, it's web-based. It's it's a rest API. It became more of a rest API as we went over time That's why it's version 3 2 was a little bit less So and it can consume LDAP or federated identities and what do I mean by federated identities? Well Remember that whole Kerberos thing I talked about that nobody here had actually heard of or a lot of people had heard of There are other things like that. How many people here have heard of open ID connect? Okay, how many people here have used Twitter? Okay, so we have at least a couple liars out there. Everybody's used Twitter Some of us are willing to admit it more than others. No, I'm kidding. Oh, I Need to look up absolute disgust there We've at least been put on web pages where somebody has embedded a tweet from somewhere else on there And what's interesting about Twitter is it had this need to be able to Just like we were talking about it opens up different services talk to each other have a bunch of Microservices or maybe just services and centralized authentication They need a way to be able to share it the same kind of delegation that we're talking about here So there's a standardized protocol for sharing delegation only we usually think of it as an authentication protocol It's called oh off and oh off one and then one a and then two is This thing that grew out of Twitter's need to be able to Mind his micro services together. So you've probably used open ID connect You may not have been aware that you're using it if you go into launch pad, which is the Like the bug tracker and stuff like that through Ubuntu in canonical and all that kind of stuff When you go to comment on a bug, it's going to kick you off to a page where you log in and then kick you back to the bug page That's open ID connect going on there. There's another protocol like this called SAML Security assertion markup language. It's very similar to open ID connect except it's an XML But they all do the same thing you go over here and authenticate improve your identity and it's going to give something Cryptographically signed that you can hand over here and say hey look I just prove my identity over there Okay, these are called federated protocols and groups can be managed Excuse me and I so the the the systems that we're talking about are going to consume these Identities from these federated protocols when you log in But the neat thing again to remember is that the groups can be managed separately from the groups that are in these Assertions we use the term assertion. Why don't you go and authenticate against open ID connect? It doesn't just give you The identity the name of who you are it says a set of groups that you're in or attributes about you But groups can also be managed at many many levels anywhere in between here and that's where your power comes from Now when we get into Keystone We're we're back in an RBAC format our backs is for role-based access control group apologize for using the acronym before defining it a Role gets assigned to a user based on the groups so you can have groups that you consume from LDAP And use that to do access girl eventually this gets down to the point where he's saying okay Can't he do a put on the URL that allows you to create a virtual machine? Now I put this number in here because this is a case of where people mess things up And there's a bug in Keystone that makes this kind of safe delegation that I'm talking about really hard to do It's 968696 I actually figured out what the tones were wrote a song about this is a long-standing bug This is how to not do things right This is how to very much not do things right and this is also goes to show The proof of the old adage that programming is like sex Which is if you make one mistake you might be supporting it for the rest of your life And this is a case of a bug that was built in very early on and we've been battling ever since the idea If you're if you don't have at proper access control and force all the way down at the right level You can't do proper delegation and it messes up the system But assuming you can get around that kind of bug then you could do and we will have here Highlighting in black is some of the code that you would actually implement That is actually implementing keystone that does allow you to say if you're a manager of a given project if you have the manager or the admin role in this case on The project or you're some sort of global admin because you need that for break the glass capabilities Then you have what's called a creative grant and this this again will map to the the URL So you have the ability to add users to your projects. This is the way you should be thinking about things when you're building your systems Okay, I want to be able to do role-based access control I want to be able to create some grouping mechanism and I want to be able to give somebody ownership of that grouping mechanism so they can control it and then I can go off and Whatever it is you do to waste time. I don't know You can go off and not spend your days adding and removing users of the groups and in doing so They can then get their job done without bothering you and you won't screw up They're getting their job done and everybody's happy and this is our organization scale Kubernetes everybody heard of Kubernetes. Okay in Kubernetes in Red Hat Kubernetes is spelled like this It's a system built around Kubernetes with Kubernetes the core and one of the interesting things about the Red Hat interaction with Kubernetes is that we needed role-based access control and it didn't exist So we worked on it at midstream still open source, but not enough Because we wanted to make sure we got it right and we had iterations with people on this and once we had an open source our back system for Kubernetes we contributed it back to the the core and It is now part of core Kubernetes So the same way of managing things. They just talked about for Keystone for open sack that exists in Kubernetes And guess what? There's no bugs 9686 96 So it actually is and it has the advantage of always going through a single API server whenever you make a change in Kubernetes You go into a single point and thus you don't have this whole problem. I have to make a change in Nova and Neutron In Clantz and all the different top projects to make up open stack So by having a single point of authentication, and this is the this is the value of how many people here have been in any like the service mesh Talks and stuff like that and talk about Kubernetes when you have a single place where you administer your policy a centralized policy management scheme, then you have the ability to fix things much much more quickly Okay, including big bugs like this. Okay, so with Oh admin is the OpenShift admin API and there's a couple different ways to do this There's a more per pure cube control way to do But this is the one that I had from my example that I know works you add a role to a group for a group name So you can just a lot of people will use the add role to user and manage users directly But if you're consuming groups from LDAP, then you can do it the same way there And then if you're using LDAP as your mechanism to manage your Membership you can centralize it and that way you could actually share that group It might not just be for Kubernetes. It might also be used on the storage side It might also be used for your software to find networking So you can consume it here, but you don't necessarily have to manage it here Very very powerful concept give power To the people who have to get things done As I said before users and groups almost always come from federated identity from external open ID connect in when you set up Kubernetes, there is a concept of identity providers OIDC open ID connect is that is Kind of the expected way that you can do it. It's a first-class citizen. It's actually written in go But you can actually wrap the Kubernetes web server with like a patchy and use what's called request header and then you can do Kerberos or Sammel or whatever other protocols you come in there that are handled at that level, but the bottom line is Some or and you can even do direct LDAP integration But the identity the users come from from outside, you know, you have to deal with is consuming them How much how much are we over time? three minutes, okay So this is my takeaway slide my last slide so I'm gonna leave this up here You don't own the list of users, but you really want to make it possible for people to manage their own groups Okay, I talked through a couple mechanisms here On the last page I'll leave a slide to my proof of concept code so we can figure out what I did wrong though during the demo But to be able to manage the set of users in which group be able to push that on down. This is how organization scale There are absolutely no time for questions