 Hello, my name is Allison Hi, my name is Dylan We've been researching GCP for the last year and a half and we've come with some effective ways that compromise most GCP organizations All the ones we've looked at anyway. These techniques involve moving laterally with service accounts and cloud APIs We have a lot of ground to cover. So let's get to it. I think everyone has a slightly different reaction when they first start learning about GCP I am What the fuck? One common thing I've observed is people often compare it to AWS I am but there are some pretty key differences between GCP and AWS I am So let's talk about some of those differences first in AWS We have user identities and those user identities can be exported in the form of credentials Which looks something like this and GCP the equivalent is a service account and service accounts Can also be exported in credentials that look a little bit differently. They look like this And in both cases we have owners of these identities But where things get a little bit different is when you start to ask the question what can my identity access In the world of AWS you basically have a policy that determines all the different things that the identity can access And the person who can create, read, update, delete that policy is the same person who owns the identity So here's an example of what that looks like In GCP though this is where things start getting really different Instead of having a centralized policy for a given identity we have policies for resources So in this case we've got a bucket and a bucket owner and that bucket has a policy The bucket owner controls who can create, read, update or delete that policy And that policy applies to whatever identity the bucket owner wants it to So here's an example of what that looks like for storage What's interesting here is that the owner of the service account actually doesn't have any insight into this policy They can't view it, they can't update it, they don't even know that their service account was given access to this bucket Instead the bucket owner controls this and if the owner of the service account doesn't have IAM list on the bucket Then they won't be able to even see that that policy was granted And I think this is one of the harder concepts it was for me personally to fret my head around in GCP Is if you have an identity in GCP like a user or a service account As the owner of that identity you can't actually know what that identity has access to In fact no one can answer the question what does a service account have access to Instead resource owners control what the policy is for a given resource So you can answer questions like who has access to this bucket But you can't answer questions like how many buckets does the service account have access to Or what buckets does the service account have access to Those questions are unanswerable by design in GCP Likewise we can have other resources like BigQuery for example And we have an owner for the BigQuery and a policy for the BigQuery And the owner of the BigQuery can again grant access to any identity they want However the owner of the service account has no control over this and can't see this One thing you might be wondering is before when I mentioned that the owner of the service account can't fully answer the question What does the service account have access to You might be thinking well if you own an organization can't you just get the IEM policies of all the resources in the organization Well someone outside your organization can still grant your service account access to their resources So when I said before you can't actually answer the question what does your service account have access to It's true you cannot get the answer to the question because people outside your organization can also grant access to their resources We'll come back to that later and explain what an organization is and how it's structured But for now I figured I just mentioned that in case you were wondering As you might imagine not having control or visibility into what your service account can or can't access comes with some challenges And one particular place I think those challenges manifest themselves is in a multi-tenant Kubernetes environment Google has a managed Kubernetes engine that's often shorthanded GKE If you're familiar with Kubernetes you have a bunch of nodes and within those nodes you have workloads In GKE those nodes are powered by VMs and those VMs have service accounts attached to them If we take a look at console we can see a Kubernetes cluster here that's powered by four underlying nodes Then if we hop over to Compute Engine and click on VM instances we'll see the same four instances And if we click into one of them we'll see that we have a service account that's attached to it We talked a little bit about how workloads interact with the service account in our B-side San Francisco talk this year So feel free to check that out But in short anything running on the node can access this identity So it's a good idea to de-scope it and not give it many privileges But let's think about what that actually means We have a Kubernetes admin that runs the nodes and the service account powering the nodes And then we have a bunch of developers that run the workloads on the Kubernetes engine Those developers may be on separate teams and may be in control of separate resources from each other and from the Kubernetes admin So you can imagine in this case we have dev1 and dev2 that have both deployed workloads, workload1 and workload2 Likewise they both control resources In this case we'll say that they both own a bucket And they want their workloads to be able to access that bucket One really easy way to do that is to just give the service account on the node access to the bucket But the downside to doing that is instead of just giving their workload access to the bucket They'll give every workload access to the bucket The developers in this case control this story They control what the service account has access to because they're the resource owners The Kubernetes admin does not control this and cannot see that this even happened So you can imagine in a situation where you have a small team of Kubernetes admins that are trying to maintain a cluster That want to open it up to give many developers the ability to deploy workloads to this cluster It can be difficult to secure because they can't control what the service account attached to the nodes has access to And some developers may want to just give that service account access to resources Rather than using something like Kubernetes Secrets or Vault to pass credentials to those workloads They may choose to just give the service account attached to the underlying node access to their resources But when they do that, they give all workloads access to their resources Not just one workload And so you end up in a situation where a Kubernetes admin who wants to onboard developers into their cluster Ininvertently gives every new workload access to a bunch of resources Even though they can't even see that or have any control over it Okay, now before we go any further, it's important to point out that up until this point I've kind of implied that resources are like these standalone entities that receive single role bindings directly to them And while resources definitely can operate that way, most of the time they're actually bundled together into groupings And there's a role binding that goes to the entire group of stuff So instead of getting a role directly to a bucket, you instead get a role to a collection of buckets So fundamentally, there are four nodes, resource nodes within GCP There's an organization node, which is the parent node This is generally associated with your domain or cloud identity service There are folders and projects All of these different top level nodes are mostly used for IAM And something that's important to note within the resource hierarchy is the concept of IAM inheritance Where wherever you have an IAM binding to one of these top level nodes Your IAM binding is also going to be inherited to all subsequent or child resources within this hierarchy So if I have a role binding at the organization, I have access to all subsequent resources that are created within the environment Same at the folder and project level Generally users mostly interact with resources at the project level That's where services like buckets, compute and general services that you may interact with on a day to day basis are Now let's dig into a few different IAM workflows and try to understand why project level IAM bindings are so prevalent in GCP If we navigate to IAM and we go to grant a member access to our resources What we're doing here is we're actually creating a project level role binding Which will give this member access to all resources within our project If we wanted to create a resource level IAM binding We would actually need to interact with the resource specific API and set the IAM binding there Another IAM workflow where a user is prompted to create a project level role binding is when you create a service account When we create a service account we'll actually be prompted to set a project level IAM binding that will grant this service account access again to all resources within our project So we can see that when you go through general IAM workflows you are going to always be setting a project level IAM binding Your IAM binding is also going to be inherited to all subsequent or child resources within this project Here we can also see something called conditionals This is an advanced feature within the IAM service that does enable you to set more granular IAM policies But also at the project level within your environment So I think that's really interesting that the main IAM button in the UI actually takes you to the project level bindings A lot of people don't even realize that you can do resource level bindings Because you have to actually click through into the resource to see that there are IAM settings there And a lot of people just assume that if they go to the IAM page they get the whole IAM story So when we backtrack a little bit to when we were talking about resource owners and service account owners Realistically what would actually happen is you would have project owners And within those projects you would have resources and service accounts So before when we were talking about role bindings we mentioned this idea of Kubernetes nodes Having service accounts attached to them And developers deploying workloads to the Kubernetes cluster and granting them access to resources But more realistically what that would look like is they would grant that node access to their entire project Because again IAM bindings are typically done at the project level So instead of granting the node access to a single bucket they would grant it access to all buckets within a given project Now this isn't always the case sometimes developers do resource level role binding And it should also be noted that the UI isn't the only way to apply these role bindings You can also apply them via Terraform or via CLI It's pretty common for developers to do it at the project level Because most of their first exposure actually comes through the UI Okay so if we recap to the earlier section you might remember when we talked about service account owners We mentioned that the service account owner can't actually know what access their service accounts have Well it's not really the whole story really what we have as a project owner And then that project owner has a bunch of service accounts in their project And then those service accounts have access to stuff And the project owner doesn't really have the whole story of what their service accounts have access to And so you might imagine how that can start to become dangerous is when I hand over access of my project to someone else I don't really know how much access I'm handing over because again I don't even know how much access my service accounts have in my project So if I hand that over to someone else I'm handing over an unknown amount of access So what is an example of this look like? Well we have a project owner that knows the email address of a different project service account They can then give that service account access to their project through an IM binding And then that service account has access to their project's resources Now one question you might be asking yourself is why you would even need a cross project role binding But when you think about it when you have two projects they're not particularly useful unless they have some way to talk to each other And one common way to do that is through Google Cloud services like PubZub or Storage And so it makes sense that we would want to share some resources between projects But again where things sort of start to become more dangerous is if we share service account access between projects Because again project owners don't really know what access their service accounts have So we're missing out on some important existential questions like how interconnected our projects actually are Because again we can answer questions like who has access to a resource But we really can't answer questions like what does a service account have access to So we came up with a solution that let us sort of answer this question within our own organization We can't actually get the complete answer to the question again because of these things called cross organizational bindings But at least within our org we can get the answer for cross project bindings By just introspecting every individual project that we own and pulling all the IM policies We then took all that information and we made a graph out of it We actually went over to GitHub and we crawled a bunch of IM policies that people either intentionally or accidentally committed And we took all those policies and we kind of compiled this pseudo org generator So let's take a look at one of the graphs we ended up with As you can see there are two different colored nodes There are red nodes and there are blue nodes What the red nodes represent are projects and what the blue nodes represent are service accounts Those are the only two things we're graphing here Then there are two different types of edges There are contains edges and there are bindings edges A contains edge is basically when a service account lives in a given project And a binding edge is when that service account has a role binding to that project Most of the time a service account has a role binding to the project that it's contained in But every once in a while we'll end up with a service account that has a role binding to a different project A cross project binding So in this case we have a service account that's contained in project number 67 And it has a role binding into project number 57 If we zoom out we get the view of the entire organization We have all the different projects here in a map And we have all the relationships of the service accounts within the org that are contained within these projects Now we're not looking at organization level bindings in this graph Or resource level bindings We're only looking at the project level bindings And we're mapping these relationships So because we base this off of real IAM policies that we're able to pull off GitHub We can say with some level of confidence that this is an accurate representation Of what a sufficiently large company might look like We can also say with a little bit of personal experience Based on the large companies that we've looked at This is an accurate representation of what a sufficiently large company looks like Now again most of the time a developer is just going to be the owner of a single project And they won't really see the whole picture All they'll see is sort of tunnel vision for the one project that they have control over And so they'll see the service accounts that belong to their project And they'll see what has role bindings into their project But they won't know where their service accounts have role bindings out of their project And so when they hand off access to their project to another developer They won't actually know how many additional projects they're handing off And they themselves may not have a good understanding of how many projects That they have access to through their service accounts One other interesting thing that we found here Is that there are certain projects that have seemingly disproportionate Large numbers of role bindings and resources in them We found this was a pretty consistent story At multiple companies that we looked at And so in the center here you can see there are a bunch of projects That have really tightly bound role bindings And then at the bottom we have smaller projects That only have one or two service accounts That may not be connected to the broader web And then we have a bunch of projects that have no service accounts What we found is that most of the sensitive data Or sensitive workloads are in the interconnected projects Where most of the development is happening And so it makes sense that it would have most of the IEM role bindings Connecting it with other projects So when we first started building these graphs We were thinking to ourselves Oh gosh, these projects are supposed to be isolated boundaries Are they really this interconnected? And the more we dug into it And the more we thought about how project owners Can't actually see the role bindings Their service accounts have been granted We realized this is a problem that's really remained invisible And it really takes this org level introspection to even see it Now there's one final piece here that we haven't talked about yet And that's when a developer actually does hand over Access to their project to another developer Pieces of that might be okay It might be okay to share storage for example But things start to become really dangerous to share access to service accounts Because again, that's where the problem is hidden Where the owner of the project doesn't actually know What access their service accounts have And so if we share that unknown access That's where things can start to go astray So we'll come back to what roles and permissions Are actually required to hand over that level of access And we'll talk about how common that is And so far we've talked a lot about roles And mentioned that some are more dangerous than others So now let's dig into that a little bit more Within GCP there are three distinct types of roles that you can use Primitive, which have been around since before Cloud IM that we know today Predefined roles, which are curated by GCP And service specific And custom roles that enable developers to specify Any permission that they would like That is supported to fit their services needs Let's focus on primitive roles Within primitive roles, there's an owner role Which provides all of the different permissions That are available within the platform An editor role, which closely resembles the owner role But excludes a few different things for administrative capabilities And that just might not be supported yet One thing in particular that's excluded Is the ability to set project level role bindings And the viewer role, which is exactly what it sounds like It allows you to view resources But not perform any state changing actions How are these primitive roles used? Well, earlier we saw that when you go to Cloud IM through the UI These primitive roles are what are first suggested to users There's another way that these primitive roles Manifest themselves within your GCP project And that's through default IM role bindings Whatever identity creates a project Will automatically be granted a project level role binding Using the owner role So we know that the owner role Contains all of the different permissions that are available But what about the editor role? Well, the editor role contains at the time of this talk 2,576 distinct permissions That's a lot This means that with the editor role You can, for example, access all buckets All databases, all VMs And much more within your project Out of these permissions that we talked about earlier Which of these roles actually enable a user to manage service accounts? Well, both the owner and the editor role do While the editor role does not enable you to create new service accounts Or apply project level role bindings for service accounts It does allow you to manage service accounts in two different ways And that's by creating keys or credentials For a given service account that already exists And by associating a service account with a resource So if a user can create a resource Specify a service account GCP will go and check If this user have the IAM service accounts act as permission On the service account that they're attempting to associate with the resource If so, it gets attached And then that user can interact with GCP services and APIs As that service account identity and its associated role So how common are these IAM roles in GCP projects? There's another way that GCP creates automatic role bindings for identities And this way is a little bit different When you enable a specific service such as Google Compute Engine It will go and create a default service account in your project And then grant that service account a project level role binding Using the primitive editor role that we chatted about earlier This also happens with another service When a user enables the App Engine service A service account called the AppSpot service account Is created in your project and also granted a project level editor role binding Now as we discussed earlier That editor role contains a permission that enables identities To associate service accounts with resources So that means that both of these default service accounts The Compute Engine and the AppSpot service account Are able to associate any service account within its GCP project to a resource So where are these service accounts used? The default compute service account is associated with GCE VMs by default So whenever you create a VM The default compute service account is automatically associated with the VM There's one thing to note about service accounts And the way that they are authorized to interact with services from a VM There is the role that they have as well as something called scopes And scopes define from within the VM What services and APIs can that identity interact with Regardless of the role binding that they have By default for VMs scopes are restricted to only a few Particular services like storage From what we have observed Developers will commonly open these scopes up So that they can interact with many different services And how are the AppSpot service accounts used? Cloud Functions are Google serverless offerings Which are similar to AWS lambdas They're meant to just be lightweight ephemeral small chunks of code That run fast and exit once they're done If the App Engine service is enabled within your project When a developer goes to create a cloud function That default AppSpot service account with an editor role binding Will be associated with cloud functions by default When you create a cloud function You actually do not have the ability to specify to set it without an identity You always have to associate an identity with the cloud function So if you don't know to create a new service account And associate with your cloud function You'll always be using the default AppSpot service account That has that editor role binding Cloud Functions do not have scopes And get all the permissions of the editor role And coming back to the VMs A lot of managed services leverage VMs as their underlying infrastructure Things like GKE, Dataproc, Dataflow They also get this default compute engine service account associated with them In AWS when you spin up something like a VM or a lambda It defaults to having no access There are no roles attached But in GCP, things default to having thousands of permissions Earlier we talked about cross project role bindings The ability to leverage service accounts in a project To get access to other service accounts And potentially move laterally across GCP projects We talked about developers potentially not knowing what access their service accounts have We mentioned the idea of handing off access And what this realistically looks like Is the developer applies a role binding with the act as permission Like the editor role Let's look at what that could look like Now it's demo time To help understand how to exploit the act as permission To start out, we need a base identity There are a number of different ways to get a base identity One way for example is you could Oauthfish a developer that has project level editor access Another way is maybe a developer accidentally uploaded a service account key To github or payspin And that service account has a editor level role binding Because the primitive roles are the first thing that are suggested to you on the IAM page Or maybe the most likely Maybe the base identity comes from exploiting a service And because most services run with project editor by default Maybe that's how you get your first foothold In any case, in this case, we've simulated this By just starting with a service account credential We can see this credential here And we can see that it starts in the project BBS2 Next, we'll run the list operation on the G-Sploit tool We can see that there's no output The reason there's no output is The list operation lists all the service accounts that you've managed to take control of And it doesn't count the base identity We'll run the gcloud command projects list on the base identity To see what project we have access to It looks like we have access to BBS2 Next, we'll try and act as exploit on this project And we'll target every service account in the project Let's unpack what's happening now Keep in mind the editor role has two ways to take control of all the service accounts in a project The first is through creating tokens And the second is through provisioning resources There's another role that we haven't talked about yet And that's the service account user role This role is also pretty common And is often given to users in service accounts that are meant to provision resources Because some things like cloud functions require service accounts to be attached to them This is the role that's usually granted to do that So because the service account user role and the editor role both have the act as permission We'll use that to take control of the service accounts Because it casts a little bit wider of a net than the token creator permission Which is only in the editor role So what G-Sploit is actually doing under the hood here Is first it's running the service account list operation To get a list of all the service accounts in the project And then it's spinning up a cloud function for each one of those service accounts Then we keep a database of all the cloud functions we've harvested Each which has its own underlying service account attached to it This gives us access to all the service accounts in the project Through the cloud functions that we spun up We control the code that lands on the cloud function So we'll just deploy code that allows us to run whatever gcloud commands we want It takes about two minutes to spin up each cloud function So we'll just speed through this really quick And now when we run the list command again We can see all the new service accounts that we now have access to One of them says owner That service account actually has an owner level role binding to the project that we started in That means we now have full control of the project through this service account If we wanted to we could use this service account to add ourselves as owners to the project So just to pause and think about that for a second What that means is if you ever have a service account that has editor level access And another service account that has owner level access The editor level service account can always privilege escalate itself to owner That's the same with developers If you grant developer editor level access to a project But you happen to have a service account in that project that has owner level access That developer can privilege escalate themselves to owner through that service account Okay, let's take another look at those service accounts that we have There's one here that's kind of interesting that says do nothing It might not sound interesting, but keep in mind that the developer that created that service account in this project Although they didn't intend to give it any role bindings Again, the owner of the service accounts can't control the role bindings And so it's possible that a different project gave it a role binding So let's run the project list command from this service account that we've compromised Something interesting here that the base project, BBS2, doesn't actually show up when we run this command This is because the service account has no role bindings there The developer didn't want it to have any role bindings But BBS3, this new project we haven't looked at shows up What happened here is the developer in another project granted this service account And editor role binding into their project So let's run the act as exploit one more time This time from this do nothing service account into BBS3 And see how many service accounts we can harvest in BBS3 Again, this process takes a little while, so we're gonna fast forward through this And now we can see a whole bunch of new service accounts in the BBS3 project Service accounts that we didn't have access to before That we were able to get strictly through lateral movement There's a couple of service accounts in here that look interesting There's one that's labeled networking that maybe controls networking for the organization But probably most interesting is this one that's labeled organizational admin If while you're moving laterally through projects You happen to land on a service account that has an org level binding Through inheritance, you get access to the entire org that way So to recap, we started with a base identity in BBS2 that had act as on the project We then used that permission to gain access to every service account in this project Via cloud functions One of the service accounts in BBS2 had an owner role Which allowed us to elevate privileges in the BBS2 project A different service account had a role binding into BBS3 This allowed us to use act as to take control of all the service accounts in BBS3 One of the service accounts we found in BBS3 had an organizational role binding And this allowed us to take control of the entire org So we talked about when you provision resources When you attach service accounts to those resources GCP will check to make sure that you have the act as permission first Before allowing you to provision the resource Because some APIs like cloud functions require that you have identities to power them This leads to a lot of people getting the act as permission Well, it turns out that some APIs will actually allow you to provision resources With a service account attached to them without the need for the act as permission These include the data proc, data flow, and composer APIs Let's talk about the data proc API The data proc API is Google's data processing API Among other things, it includes Managed Apache Hadoop and Managed Apache Spark When you spin up a data proc cluster, like a lot of things in GCP It will default to using the default compute engine service account This means by default your spark jobs will run as project editor You only need the permission data proc cluster create to spin up this cluster You don't actually need the act as permission on the default service account Because of this, anyone who has the data proc cluster create permission Can actually make use of the default editor service account Put it other way, service accounts that are only intended to be able to spin up Data proc clusters can privilege escalate themselves to project editor And then they can make use of that act as permission To take control of the rest of the service accounts in the project We built support for this into G-Sploit So let's take a look and see what it looks like If we take another look at the list of service accounts we've compromised There's one here that says data proc Let's run the gcloud projects list on this service account And see what projects it has access to Looks like it has access to a bbs4 Now let's run the data proc exploit on bbs4 Through the data proc service account We have to spin up a whole data proc cluster to do this And that takes a little bit of time And then we delete it afterwards so that we're not built too long So we'll fast forward to this part But at the end of it when we run gcloud list again We can see a new service account has been added And that's the default service account in the bbs4 project We were able to get this service account by spinning up a data proc cluster in bbs4 And then running a spark job on that cluster Which enabled us to grab the credential off the instance Because our data proc service account credential Had the ability to spin the cluster up It also controlled the scopes for that cluster And so we just defined the scopes for this cluster to be wide open Next since we have a default editor service account credential at this point We can run the act as explained on this project We'll again fast forward to this process And finally if we run the gcloud list one more time We can see a whole bunch of new service accounts that we now have access to in the bbs4 project So just to recap Some APIs in GCP do not require the act as permission To spin up resources and attach the default service accounts to them The identities that can spin those resources up Can also control the scopes So effectively in cases like data proc and data flow And Composer Any identity that's granted permissions to those APIs Can privilege escalate themselves to project editor One last thing to note here Is because this data proc role binding was actually cross project This allowed us to again compromise the new project bbs4 And because the default service accounts are in every project This allows us to take control of all the service accounts in the new project Google has a new feature in their asset inventory API Called the IEM Analyzer We worked with Google on this feature And one of the user stories actually came from us We came up with a list of dangerous roles and permissions That can be used to take control of all the service accounts in the project These can be seen here We won't demo data flow and Composer But you can think of them as working the same way as data proc Using the IEM Analyzer and the dangerous permissions that we were able to find Were able to run a bfs search on a base identity Without the need to spin up expensive resources exploiting services As you can see from the output here We get the same list of service accounts from the same base identity as we did exploiting things Except this was much faster We didn't have to spin anything up You need an org binding to use this So this is a tool for defenders to be able to get quick answers To how an attacker might be able to move laterally through the organization If we cut back to our mock graph Let's move into a seemingly innocent base identity This service account, guadvcl Was meant to only run spark jobs in its base project 14 It only has the data proc editor role Now we'll run a bfs search from the starting identity And highlight the path in blue As you can see this allowed us to compromise all the service accounts in the starting project Through the default editor role on our spark job Most of these service accounts have benign roles Like storage or pub sub And most of them are not cross project That said, some, like this one Have dangerous role bindings that are cross project In this case, the Dataflow developer role These allow us to take full control of service accounts in different projects As we start to zoom out, we can begin to see the whole picture We've managed to compromise most of the service accounts in the org With a few notable project exceptions That just don't have any dangerous role bindings to them We're only looking at project level bindings in this graph So if any of these service accounts have org level bindings It's not reflected here So we just saw some really interesting techniques Around privilege escalation and lateral movement But we also wanted to provide a way to detect this So we fingerprinted this tooling And we'll be releasing monitoring and alerting To detect the behavior of this tool in your environment I think one of the nice things about doing security work in the cloud Is how much help you get from your upstream provider If we were doing work on a web application or a binary We'd kind of be left to our own vices to fix the problems we found But because we had a close partnership with Google We were able to work with them to roll out some new features That help with the security story of some of these problems One of the people that helped us do that Is a security engineer named Bakunov And we gave him an invitation to do a brief cameo And talk about some of the new features that we helped Google roll out Hello, Black Hat I had the privilege of working with Allison and Dylan As a result of our collaboration We at Google Cloud launched several features That helped customers address concerns On the prevention side of things We now have an org policy constraint That prevents the editor grant To the default service accounts for Compute Engine and App Engine And we advise customers to turn around We recommend that customers create Custom service accounts With the necessary set of permissions On the target resources As opposed to the broad project level editor grants Allison and Dylan Briefly touched upon the investigative IAM Analyzer tool As you saw, we built IAM Analyzer To be able to address the use case Of finding resources Which are granted access to any specific identity The other investigative feature available Is the IAM Recommender It uses machine learning to evaluate The permissions granted to an identity Against actual activities Performed by the identity It's a great way to find overly generous permission grants And to identify the actual permissions That are needed for the workload I spent many years working with all Major cloud providers And this statement is generally true for all of them Starter cloud UIs Those flows are optimized for ease of onboarding And so the fundamental challenge For large organizations Is to make available Samely secured defaults As well as preventative guardrails For their developers Those are usually in the shape of Cloud foundation code bases Such as infrastructure code samples That are built with security in mind Policy as code evaluations As part of the build and release pipelines And preventative controls exposed by the cloud providers Such as org policy constraints On GCP And the accuracy of those Starter UI flows on the cloud providers Or Stack Overflow code snippets We're gonna end up in a bad place If you're a Google cloud customer And are looking for guidance in this area You should try to engage our account team So that we can help out Alison and Dylan, on behalf of Everyone at Google cloud We are very grateful for the collaboration On improving our customers' experience On the platform and for the opportunity To have a cameo appearance in this talk 2021 happens in person And we can share stories in the face-to-face setting Until then, please wear face masks And don't forget to turn on two-factor authentication Wherever possible Thank you Thank you, Bach. We also wanted to provide some context And a little bit more insight And let you know a few of the limitations Or constraints to these tools So one of the main limitations to the organization policy Services you actually have to have An organization resource Either through cloud identity or G Suite To use the organization policy service If you only have projects You won't actually be able to leverage any of the constraints That are available Another limitation within the organization policy service Is that constraints are not retroactive So when the policy is applied All resources will be affected by the policy Or evaluated by the policy After it is enabled But existing resources will not be modified To match or meet the requirements Of a given constraint Another thing to note is that for the new And grants for default service accounts That constraint will only Enforce that when the APIs are initially enabled For new projects that the default Service accounts are not granted At the time of the APIs being enabled But there is no enforcement mechanism To ensure that the default service accounts Are not granted a Project level binding or other binding There's also no way to restrict The cross project Bindings that we talked about previously Where if a user knows the name of your service account They could still give The service account access to their project Or project's resources So the IAM recommender is very powerful Some of the limitations on the tool Is that you have to have a 90 day period Where the policies or behavior Of a given identity are evaluated So that you can have a recommendation be made And there may be more IAM bindings That you may not be aware of Like the cross project bindings The IAM analyzer can provide a lot of context For the IAM bindings or associations For a given identity And in order to use this Auditing tool you have to have Administrative capabilities in an organization To be able to see the policies That are being analyzed So in order to use this tool to the fullest extent You need administrative capabilities In your organization as well as Within G Suite if you want to Expand group membership to know Who has access to a given service account That is in a group How Kubernetes workloads have access To the underlying service account That's attached to the node And the suggested way to harden Your clusters is to use a service called Workload Identity And Workload Identity allows you to Associate a Kubernetes service account With a cloud identity And this is how you can harden your cluster As well as have a one to one mapping Of Kubernetes service accounts To cloud identity service accounts We brought up earlier to ensure Project level role bindings or other Role bindings after the Disable automatic IM grants for default service accounts Constraint is enabled That you require additional tooling And to demonstrate a way that you can Enforce these configurations We will be releasing a Terraform Enterprise Sentinel policy set that can Demonstrate enforcing that default Service accounts are not granted project level Role bindings or that roles That enable service account Takeover or management of service accounts Cannot be applied at the project level Sentinel policies are Terraform Enterprise Policy configuration language That enable you to Either restrict or alert On specific configurations in your runs So just to recap Some of the key takeaways IM and GCP is very resource centric Meaning that if you are a service account owner You may not be aware of the configurations That are associated with that service account Or how it may be used across a GCP organization This can lead to projects being interconnected In many different ways that you may not be able to see Role bindings are often done at the project level And can grant more access Than a user intends to A lot of managed services and general resource Provisioning operations Can hand off access to Your project's resources and IM That you may not be aware of This kind of access that's granted through resource Creation can lead to Privileged escalation and lateral movements Within your environments Another thing we covered is that default service accounts Are granted administrative roles Within your project automatically And that's with the editor role Unless you're using the disable automatic IM grants For default service accounts organization constraint Reduce usage of default service accounts Where possible by creating Service specific service accounts With IM roles that are scoped to your services needs There are a few new cool tools In the platform that have been released To assist you in understanding Who has access to your service accounts Such as the IM analyzer And also be really helpful for understanding IM dependencies and access Within your environments