 Hello, everyone. Today's webinar's topic is about securing access to your Kubernetes applications. We're going to talk about a project called DEX that you can use for authentication. And we'll also cover role-based access control for authorization. We will be doing two demos, one for authentication and one for authorization today. Before we get into more details, to introduce ourselves, my name is Omkai Bhatt. I'm an engineering manager here at Casting by Veeam. I've been here for a little more than a year now. Deepika, would you like to introduce yourself? Hi, everyone. I'm Deepika Dixit. I'm a software engineer here at Casting by Veeam. I've been here for about three years and most of my work has been about Kubernetes and how we can extend it and also all the Kubernetes primitives. So we'll be talking more about that in today's session as well. Yeah, we've both spent some time solving problems in the areas of authentication and authorization and we'd like to use this opportunity to share some of the things we have learned with the community today. So when it comes to securing access to your Kubernetes applications, I talk about a couple of types of users that might exist out there and that we have interacted with while working with some of the users. You might be an IT administrator who's interested in deploying an application in a Kubernetes cluster or you might be a developer who's interested in deploying your app in Kubernetes. And while those are some types of users, you can also roughly categorize them into two categories. One where users are still learning about Kubernetes and they're early on in the cycle of, hey, is this the right approach for me for my apps? And then you might be in another category of users where you know you want to deploy production applications in Kubernetes and maybe you're already doing that. But irrespective of the type or category of user here, these users are usually thinking about how do I avoid reinventing the wheel when it comes to the authentication workflow. Their users are accustomed to using their existing credentials for accessing apps. And so the question that's usually there in the user's mind is, can you use your existing authentication providers to let your users access applications in Kubernetes? And the answer to that is yes, you can. And we're going to talk about DEX and show a demo about DEX, about how you can use that to achieve that. And in the second half of the talk today, we'll talk about authorization. So now imagine you're that administrator or developer who's figured out the authentication piece of the puzzle. But now you're thinking about how do I grant varying levels of access to different types of users. And when I refer to types of users, this might be an administrator who has access to the entire cluster. Maybe you're an app administrator who wants to take care of an app in a specific namespace or maybe just a few namespaces. Or maybe you're a read-only user who's only allowed to view data and not modify it. And Kubernetes has native constructs that will let you do this and take care of authorization by using role-based access control. And Deepika will be covering authorization later along with the demo to show how we can do this. So to start off with the topic about DEX. DEX is an identity service that uses the OpenID Connect standard to drive authentication for apps. OpenID Connect is an identity layer on top of the OAuth 2.0 protocol. OAuth 2.0 is an industry standard when it comes to authorization. And so DEX acts as a portal to one or more identity providers through the concept of connectors. In this diagram, you have these entities in the authentication flow where an end user is trying to access a client application. The client application is configured to redirect to DEX when an end user is not authenticated. DEX then checks if the client app is registered with itself. See if the client app is registered, DEX continues to move on to the upstream identity provider. Once you are able to successfully authenticate with the upstream provider that might be Octa or Google or GitHub or Active Directory, the user gets redirected back to DEX and then DEX redirects the user back to the client application. And so the end user then is able to access the dashboard and interact with it because the client app is now validated the identity of the end user. And so this describes the authentication flow. And this client application, you can consider this as an application deployed in Kubernetes. DEX is also deployed in Kubernetes, but the upstream IDP might be something outside of Kubernetes. And very often we have seen Active Directory show up as an example where it's an existing authentication provider that users want to leverage for their Kubernetes apps. And so then the next part of the discussion is what's a connector? A connector is an entity within DEX. You can configure one or more connectors at the same time. Each connector implements the logic for authenticating against a specific identity provider. I've listed only a few of them here in this slide, but DEX supports a large list of identity providers. In this list here I have the LDAP connector. And I'll be doing the demo using the LDAP connector today. You might be a user who's deploying applications in an OpenShift cluster. And you'd like to leverage the OAuth server's authentication capabilities. And so DEX has a solution for that. You can use an OpenShift connector. And I've listed GitHub and Google as two more examples here. And they represent an example of a connector that does an open ID connect style of authentication. But there are many more providers that you can look at in DEX's website, along with features that each type of connector provides. DEX has published a Helm chart that you can use for installing DEX in a Kubernetes cluster. Over here we have two commands. The first one is to add the DEX repository using the Helm repo add command. And the second one is the command for installing it in a namespace in Kubernetes. And notice that there's a dash fdexvalues.yaml argument here. And I'm going to spend some time focusing on the values file. The values file contains configuration for DEX and we dig deeper into that in some more slides. So here's an example of a configuration on the left side. And I've mapped each piece of this config to one of the boxes in the authentication flow. Remember that the entire config is provided as input to DEX. But each piece represents one of the entities in the authentication flow. On the top section here, you have a section that represents DEX as an OIDC provider. In the middle you have the connector config that represents the upstream provider. And the last section represents the client application. So to dig a little deeper into the first section, you will see that there's an issuer config that points to HTTP colon slash slash the local host IP and the port 8080. This is going to be the URL where DEX is going to be available as a service for client applications. DEX stores persistent state for various tasks, for example, in order to store refresh tokens. And it supports multiple types of storage options such as HCV, SQLite, MySQL, Postgres. But for the purpose of this demo, we're just keeping it simple. We're not setting up any database. We're using the type memory storage option. The next config that I'm going to talk about is the client's config. Here you will see that there's a client named example app that's registered with DEX. And there's a secret and redirect URL. So the way these two fields work is that whenever a client application sends a request to DEX, it's going to include a secret and redirect URL in the request. DEX is then going to compare those fields in the request with the fields that have been registered here in the static client's configuration. And only if they match, will DEX continue on with the authentication flow to the upstream provider. And the last part of the config represents the LDAP connector. On the right side, there's a diagram that represents an AWS simple Active Directory service. I have a reference to a blog later in the slides that shows how you can deploy this. For the purposes of this demo, I've modified it a little so that the Active Directory service is reachable from outside the VPC. And you'll see here on the left now that in the connectors config, I'm referring to this Active Directory service under the host config at ldap.aws.casten.io. There are some fields here that let you enable or disable SSL. There's the bind DN and bind password that represent the admin credentials that DEX will use to connect to the Active Directory service. And then there are two sections here, one for the user's search and one for the group search. These two pieces of config help DEX understand how to look up a user and the groups that the user belongs to from the Active Directory service. So now that we've gone through all of the config that goes into deploying DEX, the next step now is after having installed it in a namespace, we're going to enable port forwarding so that DEX is going to be listening on the local host IP and on port AD80. For the demo today, we'll also install and run the client application. And we're going to clone DEX's GitHub repository and build the app and then start that. The client application is going to connect to DEX. And then in the demo, we'll show you how you can log into the client application with the help of DEX and the Active Directory service. So moving on to the demo, I have three windows open here. The one in the middle is the one where I have access to a Kubernetes cluster in digital ocean. I'm able to list namespaces here just to confirm that I'm able to access the cluster. I'm going to run the Helm install command to install DEX in the default namespace. I've used the values file that I showed earlier with this installation command. Now that the install is complete, I'll be able to show all the resources that have been deployed for DEX in this namespace. You'll notice that there's a service for DEX. There's a pod deployment and replica set associated with DEX. I'm going to use the instructions in the nodes to set up the environment variables called pod name and container port. And we'll be using this to run port forwarding. Okay, so we have DEX set up. It's listening on port 8080 on localhost. The next step is to install the client application. I have the instructions noted down here for pulling in the DEX's GitHub repository. We'll then CD to the example app. We'll build it and then use this command to start the example application. And notice that I have specified the issuer field as the URL for the DEX service. The example app has successfully connected to DEX at this stage and it's ready to accept requests from the user. This is a URL that I can now take to the browser and then access the client application. So I'm now at the login dashboard for this client app. I'm not going to fill in any of these fields for the moment. I'll go ahead and click, notice that the port number is 5555 here, which means I'm right now in the context of the client application. When I click login, notice that the port changes to 8080, which means that the client app has redirected me to DEX for the purposes of logging in. Because I've configured an LDAP connector, DEX is presenting a button to log in using LDAP. Once I provide my credentials here, and when I click login, DEX is going to interact with the Active Directory service. And once I get authenticated, the flow returns back to DEX at this stage. Keep in mind that we're still at port 8080 here, so we're in the context of DEX. And once I grant access here to the example application, I'm redirected back to the client application now. Notice the port is 5555 and the client application has the logic to parse the token. It's a JSON web token that's returned by DEX. And the claims section here contains the output that the client app generates after parsing that JSON web token. So you'll notice that my username is present here, which means this is the token that was generated for me as the user who logged in using DEX. There are a couple more login floors that I'll show. Notice that there is no group-related information right now. So let's visit the client application again. In the extra scopes section, I'm going to request for groups scope this time. I've logged in again. I'm successfully authenticated with Active Directory. We're just repeating the auth flow at this stage. But notice the difference this time is that we've obtained the group's information for my username from Active Directory. And DEX has nicely wrapped that inside the token that's returned to the client app. Now the last login flow that I'd like to show here now is with a different username, come back to the one that I used for my name. And I have an administrator username configured for this service. I've been successfully authenticated as the admin user and I've now been redirected back to the app. And notice this time the token shows a different username with a different set of groups. And to quickly summarize what happened, we installed the client application, which represents the box on the left side. We installed DEX in the Kubernetes cluster. The upstream IDP provider was the Active Directory service in AWS, which was already pre-created before this demo. And I was able to show login using different types of users with different groups information in their tokens. And this is just one example of the token that we saw during the demo. And make note of the username and the group name that we've seen here. For the rest of the presentation, Deepika will use that information for the purposes of authorization. And so we'll now switch over to the authorization part of the talk. Thank you, Oankar. So now that we've looked at how you can authenticate users and groups across your Kubernetes cluster with the help of DEX. Now what happens about authorization? How can you maintain and control access within your Kubernetes clusters? So Kubernetes has its own native RBAC and it defines it as a method for regulating access to your computer or network resources based on the roles of individual users within your organization. So Kubernetes allows you to use your existing roles and existing users and your existing groups and assign them roles according to Kubernetes. So RBAC is a very flexible and powerful method where you define the rules once and then you can use it multiple times, not just within the cluster, but maybe across even multiple clusters. It allows access control not just with our resources within the cluster, but also components within an application itself. It clearly defines who has access to what. So that allows for a dynamic calculation across the application across the cluster as the application changes and grows. So now you might be thinking, why do I need this RBAC? I already have all my users and groups set up. What happens next? So we've seen especially with clusters and applications maturing after you've had your initial hurdles of infrastructure set up, after you've migrated all your applications to RBAC to Kubernetes especially, multi-tenancy becomes a really important concern. Administrators often have a hard time deciding how can I restrict users to just their applications or even components within those applications. Users tend to not want to be aware of what's happening apart from the things that they are concerned about in the cluster. They just want to have access to their own setup and not worry about all these other users in the system. Administrators often are worried about separation and security between users as well as applications. So Kubernetes RBAC gives a lot of primitives for us to help identify the problems, define the roles and make sure you know who has access to what. So let's look at some Kubernetes primitives or Kubernetes objects that are already there to help you with that. First up are roles and cluster roles. So this is an example from the Kubernetes documentation itself, which we have linked towards the end. So on the left-hand side, you can see that we have a Kubernetes role. It's part of the RBAC authorization API group. And as you can see, the role has a name. It also has a namespace and then you have a set of rules defined. The rules are what you want the user to give access to. So for example here, I want this role to define access of get, watch and list over pods and secrets. I want this to be in the default namespace. Now as compared to on the right-hand side, it is a cluster role, which is again part of the same RBAC authorization group. But as you can see the namespace is omitted. This is because cluster roles are not namespaced. That means that the rules are on a cluster level. So the difference between the two roles is that on the left-hand side, the role is specific to that namespace, whereas the cluster role on the right-hand side gives access to create, get, watch, list of secrets across the entire cluster. So that way you can define between what is specific to your namespace if your applications are namespace versus what is defined at a cluster level if you have applications that span across the cluster. Now that we have the rules elaborated here, let's dig deep into them. The rules are specific list of permissions that are additive in nature. So for example, a user by default has no access unless it's bound to a role. So these rules are not a deny list but an access list. So you can keep on adding rules if you want to give more access within a role. The rules can be as broad as a star, which means all. So on the right-hand side, I have a role that is all verbs to all resources present in all API groups. Versus on the left-hand side, I have a more granular role which defines that I have get, watch, and list access to just a specific pod called mypod within the API group mygroup.example.com. So you can go as broad as I want to give access to literally everything that is present in the cluster or everything that's in a specific API group or all the resources. Or you can go specific to I just want to give access to this one pod in this one specific API group. So that gives administrators the ease of use that they can define users that have multiple access or they can define users that have a very restrictive access. Now, who are these users or who are these subjects that we have in the system? So subjects can be of three types. They can be users, they can be groups, or they can be service accounts. Now service accounts is a Kubernetes specific object. So they can be used when you have access within the cluster and you want to manage objects and roles within the cluster. But we did discuss users and groups that we already have existing before the Kubernetes cluster even existed. So if you have existing users and groups, Kubernetes allows you to bind these existing users and groups to these defined roles that we have over here. So from the example that Onkar mentioned, a user could be a user that you already have set up in your AD somewhere. It could be a user Onkar, it could be a user Alice, it could be anybody. It could also be a set of groups that you have predefined. For example, you might have front end admins, you might have domain admins, you might have just cluster admins, for example, and Kubernetes will honour that as long as you have the correct binding in there. So how do we bind these subjects with these roles that we had defined? So Kubernetes allows you to do two types of things. The first one is a role binding. Now the role binding is specific to a namespace. So as you can see here, we have a role binding called read pods and it's in the namespace default. Now the role reference in there is the role that we had looked at a while ago, which is it's the pod reader specific role. And now I'm binding this to a specific subject that is the user Jane. Now, as you can see the subject here is just one user, but it could easily be a set of users, or even a set of different kinds of users, like I could have a user Jane, I could have a group admins, and I could also have a service account in that. This is a role binding specific to a namespace. But what if I want to bind across the entire cluster? I can use a cluster role binding, which again does not have a namespace since it's at a cluster level. It references a cluster role, and it has a list of subjects. So here I have the group manager, and I have a service account that is bound to this rule. So that means I want to give the role reference secret reader access across the entire cluster to these subjects that I have listed right here. So I'm going to switch over to the demo to show this in action so it's more clear and we can go from there. So let's just look at a role that I have in the system. So as you can see, I have a role goal get pods, which gives you get and list access to pods, and it's specific to a namespace are back. So now I'm going to apply this role to my cluster, and Kubernetes will create a role for me in my specific namespace. So if I do cook, I'll get roles in my namespace are back, I should see that it has the get pods role. Now I want to bind this role to a specific user. So I can do a role binding to bind this to a specific user into a specific namespace. So here I'm using the same get pods role, and I'm using the same namespace, and I'm binding user on card that we saw from before that that does not exist in this Kubernetes cluster but with the help of authentication index, we can use this user in here because we already verified the role and the access for this user. So now I'm going to apply this. And it will create a role binding for us in this cluster. Now let's look at a cluster role, I want to give some cluster role access to different user. So I now have a cluster role that talks about get watch and list on secrets, and there is no namespace and it's a cluster role so this applies throughout the cluster. So I can apply the cluster role, and then we can apply a corresponding cluster role binding as well. So let's look at the cluster role binding. So now here again I'm using the same cluster role get secrets that we just applied, but this time I'm applying it to a set to a group called domain admins. So this means I want to give access whatever is in this role at a cluster level to the subject that is this group domain admins. So I can apply the cluster role binding. And that's it the cluster is set up correctly, and we have the role bindings and cluster role binding setup. But now that we've done this how do I as an administrator or I as a user know what access do I have what's happening. It is very difficult to understand what to do when. So let's take a step back and look at what what our needs are and what to do when. So the first thing was a role versus a cluster role. Now you can use a role when you know you want to give limited access and you want to give that in a certain namespace. So you know what your namespace is up front and you know what your rules are, you know what resources you need, and you and you know exactly what you want to do. That's when you use a role because you know you can set up all your boundaries at the beginning when the role is defined itself. Whereas a cluster role can be given when you have set of rules that are across multiple namespaces, or you have resources that you don't know exist in a certain namespace at a time or not. And if or if you even don't know what namespaces you're going to have when your application is running. So it's easier to set these rules up in the cluster role first, and then you can decide if you want to specify it for the namespace later on. So that brings us to binding. If you notice when we looked at the binding, the binding subject can be a role or a cluster role irrespective of the type of binding. So for example, I could have a role binding that uses a cluster role, but I'm still bounded to a particular namespace, or I can have a cluster role binding that uses a cluster role and gives access to all the subjects across the cluster. So the binding decides what namespace that the role is going to take effect. So user role binding when you know you want to act limit your subjects to a particular namespace and use a cluster role binding when you give you want to give a cluster wide access to your subjects. Now what about the subjects, when do I use users when do I use groups or when do I even use service accounts. If you're already using an authentication system and have existing users and groups, you don't need to worry about service accounts. And you know what user you want to give access to simple user user. But if you want to make sure that you know as new users get into your system, you don't want to keep adding new roles or you don't want to keep rehashing your RBAC use groups, you can easily add users to groups and the group level binding will take care of it. RBAC, whenever a user logs in or whenever a user does any operation, the RBAC controller constantly verifies that the user has access to what they're trying to do. So it's not set up once and use that it will constantly dynamically keep checking for what access the user has. And to make it simpler Kubernetes has a tool called kubectl auth can I, which allows you to check your user access. It allows you to check your roles bindings across the cluster to verify what access you have, as well as it also allows you to create it as a specific user or a group to see what access you would have if you were that user or that group. So let's look at the kubectl auth can I tool and then we can see what else it can do. So if you remember we had set up examples for on car user to be able to list boards. So if I do kubectl auth, can I get boards, and this will be in the default namespace so if I do it as on car. The answer is no because I cannot do this in the default namespace. But if I try to do this in the namespace that we have set up in the role binding which is the RBAC namespace. So the answer is yes because yes obviously I can list pods in the RBAC namespace set up as my role binding. Now let's try to see if I can list secrets in this namespace. So I have set up as, can I get secrets and the answer is no because my role binding only allows the user on car to get pods in the RBAC namespace. But what if I can try it as the group that we set up. So if I try as group and if you remember the group was domain admins. And the answer is yes, because even though the user is on car I'm still now addressing is an impersonating as domain admins which is set up in the group. So I could easily be a different user, but still impersonating as domain admins. And I would still get the answer yes, because my binding has been done at the group level. So it does not matter what user I log in as as long as my group is authenticated correctly. So as you can see, these users were set up in our authentication system, and we can use them to use in the Kubernetes authorization system. So even if the authentication system changes we can still have this binding set up correctly one time and keep using them multiple times. And these are the list of references that we've spoken about throughout the demo. Feel free to check it out. And we took the examples from this differences as well. So you can try them out yourself. I think you highlighted a really interesting point that they pick up which is be having the ability to even change your authentication system. We have the same RBAC rules defined with the help of Kubernetes RBAC objects. And that's really useful for users if they decide if they want to switch from their Active Directory authentication to maybe Google based OIDC authentication, the RBAC rules would stay the same. And on multiple occasions, the Auth can I tool has been invaluable for us in helping our users figure out what level of access they have within their clusters. And hopefully this talk today was useful would be useful for people who viewed this webinar to understand how they can set up authentication and authorization for applications in Kubernetes.