 Hello, DEF CON. I speak to you today from my own version of the Batcave, the ultimate in social distancing. This is Hacking the Hybrid Cloud and I am Sean Metcalf. I'm the founder, trimark, a professional services company that helps organizations better secure their Microsoft platform, including the Microsoft Cloud and VMware infrastructure. I'm a Microsoft Certified Master in Active Directory and a Microsoft MVP. I've spoken at a number of conferences and I'm excited to be back speaking at DEF CON, even if I'm not physically at DEF CON. Last year, I keynote at the DEF CON Cloud Village and three years ago gave a talk on Hacking the Cloud at DEF CON. This talk will take a journey through the hybrid cloud, starting with the comparison between cloud and virtualization, discussing how on-prem Active Directory domain controllers can be compromised. I will also walk through the cloud-hosted, managed Active Directory deployments from AWS, Azure, and GCP. Since this is hybrid talk, I will walk through some hybrid components and see how they can be attacked. I'll touch on cloud administration and identity access management, where I am concepts, and finally walk through escalation scenarios for compromising cloud-hosted on-prem domain controllers AWS and Azure. First, we need to talk about what hybrid cloud is, at least in the context of this talk. There are likely lots of definitions of hybrid cloud. Due to this, I'm defining hybrid cloud as something on-prem and something in the cloud. This typically means infrastructure on-prem and in the cloud, more IaaS than strict SaaS. Now that we've defined hybrid cloud, let's look at three common scenarios. The first is the on-prem Active Directory environment with Office 365 SaaS services. SaaS stands for software as a service. Many organizations are leveraging the cloud for infrastructure as a service, or IaaS, as their cloud data center. The third scenario here is an on-prem Active Directory with a cloud-managed AD as an additional or resource force. Since cloud can be used in many ways, it's likely that most organizations use a combination of these, among others. This section covers virtualization concepts and controls as foundational information to better set up and describe potential attacks. Conceptually, the cloud is virtualization, certainly from an IaaS perspective. Amazon has used Zen and a switch to Nitro. Azure uses a custom version of Hyper-V Core, and Google Cloud Platform uses KVM. Now, there is a cloud fabric that ties the virtualization elements together with orchestration, which is what makes the cloud the cloud. So this diagram shows Azure IaaS with a VM and related components. This is similar to on-prem virtualization. Amazon's Nitro is pretty cool. The Nitro Hypervisor can take advantage of Nitro-optimized hardware, including a Nitro security chip. Any instance created prior to Nitro is still running Zen. AWS also has this concept of connectivity between the AWS-managed Active Directory that they host themselves, and EC2, which is their IaaS environment within the AWS cloud. And then, of course, since you can host Windows servers and Windows systems and other types of systems, you can also integrate and connect to other cloud environments. This demonstrates how cloud can get complex pretty fast. And you can run VMware on-prem and in AWS and Azure and GCP. Domain controllers control the keys to the kingdom and require special security protections that are often ignored on virtual platforms. Compromising physical domain controllers boils down to either gaining physical access or compromising an out-of-band management system like ILO. ILO hosts a web server on port 2381, and we can scan for this using PowerShell. So if ILO is hosted on the same network as production systems and clients, such as workstations, we can scan and identify ILO. ILO has had a number of security issues with firmware patches available, but the issue is that pretty much no one updates firmware on physical servers. Airbus security has identified many of these security issues with ILO from accessing memory to an RCE to get read-write memory access to the ability to execute arbitrary commands via DMA to gain host access through ILO even when ILO is disabled. And I have a link at the bottom that includes a list even beyond what I have up here and a Go scanner to discover vulnerable servers running ILO. When VMware is hosting virtual DCs, which is probably around 95% of the organizations are better, certainly the larger ones, there's different ways that you could compromise VMware to get access to a virtual domain controller. This could involve compromising an account with rights to VMware. Often, this is a regular user account that is a member of an 80 group called VMware Admins, or we just need to compromise an account that has VMware access to or at least one virtual domain controller, potentially more. If we can compromise a system which runs vCenter, we can ultimately compromise VMware. Also, every ESXi server has a root account, and most of the time every ESXi server has the same root password, which may be the organization default password, which could be guest or found in a password file. If we can compromise the vCenter system, we can compromise the vSphere environment. And the VMware VIX API has functionality that allows for direct access to guest OSs, which is used by solutions such as VMware Site Recovery Manager. This requires VMware tools to be installed on the guest OS or VM. The VIX API allows guest operations such as starting, stopping the application, file directory manipulation, uploading, downloading files, all within the guest OS without requiring any network connectivity to the virtual machine. This functionality may also be used by vSphere accounts with limited privileges to access a guest OS without the need to authenticate. There is a scenario where an account with vCenter privileges can use the vSphere API to interact with the local OS installation of VMware tools to allow a pass-through session for an unprivileged account to connect to that virtual machine and execute local commands. Since this applies to virtual domain controllers, Active Directory could be compromised in this manner by allowing execution of commands locally on the virtual DC, with accounts not normally allowed local logon access to the DCs. For HyperV, the attacks are similar to VMware. You can compromise an account with rights or find a way to compromise a server, either through a server admin account, discover the local admin password, or modify group policy to gain access to the server. So I decided I would take a look at the so-called Big 3 cloud hosting providers managed Active Directory environments and then focus on what this means to pentesters and red teamers. The managed Active Directory environment does not provide AD admin rights to the AD environment, so the customer will never have domain admin rights or even domain controller access. The cloud provider controls AD infrastructure management and delegates rights to OUs and AD components to the customer. Let's start with Amazon AWS Directory Services for Microsoft Active Directory. That's quite the mouthful. The AWS managed AD has an AWS delegated groups OU that contains the delegation groups which the customers provided membership in and access to. The AWS reserved OU is controlled exclusively by the AWS system. There is a customer OU created in the managed AD domain that is typically named after the AD subdomain or the first part of the fully qualified domain name or the domains net bios name. This customer OU contains OUs for computers and users. AWS managed AD spins up two domain controllers which are in different network segments but in the same Active Directory site. The default domain admin account retains the default name administrator, though this is moved to the AWS reserved OU. When a customer creates an AWS managed AD, their first account in the AD environment is called admin and is granted full delegated rights to the AD environment for that customer, not to the domain controllers or AD admin rights to the full AD. The domain password is default, still seven characters, but there are five AWS created fine grained password policies which are delegated to the customer to modify. Based on the group policy linked to the domain controllers OU, I was able to identify a pretty decent audit policy that was missing Kerberos auditing, which means Kerberos thing activity would be missed. The first customer account admin is created with the description that notes the customer OU distinguished name. So it's very nice that they provide this description, so that way you can quickly and easily identify where this OU is. There are several delegation groups in the AWS managed AD environment. All these are named with prefix AWS delegated and are contained in the AWS delegated groups OU. The AWS delegated administrators group is the primary customer admin group. There are also delegation groups for our managed service accounts, adding workstations to the main Kerberos delegation, DNS and server administration. Let's take a look at Azure Active Directory domain services. While Azure Active Directory is not exactly Active Directory, since it's a cloud directory with no Kerberos or NTLM authentication, Group Policy or LDAP, Azure AD domain services is Active Directory, just like you would have on a corporate network in his Microsoft's managed Active Directory. So this environment, Azure ADDS, has several top-level OUs that are created. These are prefixed with AADD and the AADDC computers and AADDC users OUs are the primary customer OUs. The Azure ADDS domain controllers are in the same network, and at least in this instance that I created had sequential IPs, .4 and .5. The default domain admin account is called DCAAS admin and remains in the default user's OU. There is one fine-grain password policy that's pre-created. I'm not going to try to read the name. As it is, it's a bit tough with all the AA and DDs, but the customer is delegated to modify rights to this fine-grain password policy object. I can't see any auditing configuration set in GPO, so I have to assume that they are set via local policy, which I can't see without domain controller access. Azure ADDS has a few delegation groups which are prefixed with AADDC and are in the AADDC users OU. The primary customer group is AADDC administrators, which is delegated full control to many OUs and AD components. I'm pretty sure when Microsoft came up with the names for these groups, they were not thinking about having to say this out loud in a row. So thank you, Microsoft, for that. Moving on to the third and final managed AD environment that I reviewed recently. This is the Google Cloud Platform, Managed Act Directory. It's interesting to note that the GCP managed AD seems to cost about twice what the Amazon AWS and Microsoft Azure managed AD environments cost, because they're about $100 each. The GCP managed AD environment has a single root OU for the customer called Cloud with a single child OU called Computers. The Cloud Service Objects OU contains delegation groups. GCP managed AD is different from the others that I just talked about in that domain controllers are running Windows Server 2019, though the Active Directory force functional level is still set to 2012 R2. Also the Active Directory Recycle Bin, which enables rapid undelete capability is not enabled, which is interesting. The default domain administrator account keeps the default name administrator and it's disabled. So GCP spins up a second domain admin account called Cloud SVC admin, and this account is enabled. The domain password policy is default and the customer can create fine-grained password policies. And with this the DC event auditing is not configured through group policy, so again I couldn't identify how it may be configured. GCP managed AD as several delegation groups. They start with Cloud Service, and these are stored in the Cloud Service Objects OU. The Cloud Service all administrators and the Cloud Service administrators are the primary customer admin groups. GCP delegation groups include computer admins, managed service account admins and others. This slide captures some of the common themes of the three managed AD environments I reviewed. I used a version of the Trimark Active Directory assessment tool that I wrote to perform the review of the different AD environments which enabled me to capture data for all three and identify the key security features and delegation in about a day to put this talk together, or at least this part of it. The bottom of the slide provides a link to an Active Directory Security Review PowerShell script that I wrote and published in June. The link points to the article where I highlight key AD security items to check and what to check for, and the PowerShell script simplifies the data collection part so analysis is much simpler. So it's good to check that out. It would be helpful if you're looking at an environment such as these. So since the customer does not actually have AD admin rights to a managed AD environment, there's not much point in attempting to escalate to DA. Better to identify which managed AD environment it is and enumerate the managed AD privilege groups to determine which accounts are highly privileged. Interesting note about Azure Active Directory domain services, Microsoft's managed AD, it can be managed by Azure AD accounts and these are synchronized into Azure AD DS or even by on-prem AD accounts that are synced in from the on-prem AD environment. And these are synced using the on-prem Azure AD Connect system through Azure AD to Azure AD domain services. And if password hash sync is enabled, which it usually is, then the on-prem AD account password hash is included. Normally, password hash sync does a hash of a hash, but when syncing through to Azure AD domain services, that original AD password hash is retained and synchronized, so that way the password can actually be used in Azure Active Directory domain services. So there are several cloud components that connect on-prem to cloud. This connectivity has some very interesting implications. Amazon's AD connector is effectively an on-prem AD authentication proxy. AD connector is designed to provide an easy way to establish a trusted relationship between your Active Directory and AWS. So when AD connector is configured, it enables the ability to sign into AWS applications with on-prem AD credentials, join AWS Windows instances to the on-prem AD domain, and provides federated sign-in to the AWS management console by mapping these on-prem identities to AWS IAM roles. And AD connector can't be used with custom applications since it's just for AWS. Basically, a user opens the secure custom sign-in page, supplies their Active Directory username and password, this authentication request sent over SSL to the AD connector, and then the AD connector performs LDAP authentication to Active Directory. The interesting part happens at this point. Once the user has been authenticated, the AD connector calls the STS assume role method to get temporary security credentials for that user. Using those temporary security credentials, AD connector constructs a sign-in URL that users use to access the console. And if a user is mapped to multiple roles, the user will be presented with a choice at the sign-in time as to which role they want to assume. User session is valid for an hour, which is pretty standard for cloud apps. Since the AD connector environment is hosted in AWS, this would need to be targeted in order to potentially impersonate the users and be able to leverage that STS assume role method. Microsoft's pass-through authentication is similar because with Microsoft pass-through authentication, or PTA, it provides users the ability to authenticate to Azure Active Directory, which is passed through to the on-prem AD and validate it in Azure AD by getting the status back from the PTA agent in the on-prem environment. PTA is enabled via Azure AD Connect. The users use the same passwords assigned into both on-prem and cloud-based applications. The communication between an agent and Azure AD is secured using certificate-based authentication, and these certificates are automatically renewed every few months by Azure AD. I couldn't figure out what happens if you can get one of these certificates, but that could be pretty interesting. But ultimately, this supports user sign-in into all web browser-based applications and into Microsoft Office client applications that use modern authentication. Adam Chester, aka XPN, has done a great blog post on how to leverage a compromised Azure AD Connect server with PTA enabled in order to gather user names and clear text passwords that were used to authenticate via PTA. Since pass-through authentication is managed by Azure AD Connect, you can compromise that server, and then during this authentication process, Azure AD sends the clear text password, not hashed, to authenticate the user. So it's sent to Azure AD Connect. Azure AD Connect PTA component is going to check and validate the username and password with Active Directory, then send back the status to Azure AD for that. So Adam identified that you could inject a DLL to compromise the credentials that were used for PTA and then have those credentials for anyone that was using PTA. Azure Active Directory's seamless single sign-on, or Azure AD's seamless SSO, automatically signs users in when they are on their corporate devices connected to the corporate network. This graphic is interesting and nice, but I prefer this one because it has a lot more of the flow. But basically what ends up happening is that users don't need to type in their passwords to sign into Azure AD or even really type in their user names. It provides the users easy access to their cloud-based applications without any additional on-prem components like Federation. This is done by converting the on-prem authentication to cloud authentication through a computer account in Active Directory. So Michael Grafner from DS Internals posted this article a few years ago, or a couple years ago, where he was talking about how the Azure AD has a publicly available endpoint that accepts these Kerberos tickets and translates them into SAML and JWT tokens that can be leveraged for cloud apps. Effectively, if you can compromise the Azure AD seamless single sign-on computer account, the password has associated with it. The computer account is called Azure AD SSO ACC and can be identified as in the computer's container. If you're able to compromise this and get this password hash, you can generate a silver ticket for any user you want to impersonate and the service that you use in the silver ticket is going to be aadg.windows.net.nsatc.net. You inject this ticket into the local Kerberos cache and Azure AD seamless single sign-on, the computer account password doesn't actually change. Microsoft does recommend that it gets rolled and changed at least every 30 days. What's interesting is because it's a computer account, there is no actual computer that is behind the scenes that is using that computer account other than the Azure AD Connect client. Normally, an active directory would update the computer account password that's associated with that computer. Since there isn't really a computer here, that password isn't getting changed. Maybe in the future, Microsoft will code into Azure AD Connect to automatically update the computer account password for this. Three years ago at DEF CON 25 in Las Vegas, I co-presented a talk called Hacking the Cloud. I talked about how Azure AD Connect's password hash sync permissions provided Mimicat's DCSync rights. The author of Mimicat's DCSync, Vincent Leto, was inspired by password hash sync to create DCSync because PHS requests password hashes for synced users on a regular basis. We can scan active directory domain permissions looking for potential Azure AD Connect service counts. Or if the Azure AD Connect install would use the express install option, then there's a helpful description on the account noting on which server Azure AD Connect is installed. We can then check in active directory to figure out where that computer is stored. Very often, this is in the server's OU or somewhere similar. Once we know the server name, we can check for, and its location, the actual distinguished name of the OU that it's in, we can check AD for any group policies that add AD groups to the local administrator's group. Compromising any account that is a member of the local administrator's group on the Azure AD Connect server would result in compromise of the Azure AD Connect server and ultimately active directory assuming that it has password hash synchronization rights configured in the environment. Also, if we can modify a group policy that links to the Azure AD Connect server OU, then we can compromise the server that way as well. When talking about cloud environments, it's important to mention how administration is performed. This is commonly referred to as identity access management or IAM. We've gone from groups in our on-prem environment with active directory to roles which is effectively these rights that are packaged together. Azure and AWS and GCP have their own methods, but a lot of the concepts are effectively the same. Azure IAM uses the concept of an owner, a contributor, and a reader. The owner has full access. The contributor has the ability to create and manage the resources but can't grant access to others and then a reader can just view things. Azure has the tenant which is the top level container of the Azure environment, and this contains subscriptions. The tenant admins are effectively the owner role on the tenant and subscription admin would be effectively the owner role on the subscription. What's interesting is that Microsoft notes that Azure roles and Azure AD roles are separate and different, mostly, and I'll cover that in a little bit. AWS has a root account often called or referred to as a payer account which is the primary account, though it really shouldn't be used regularly. Then there are account admins. In AWS, the account is similar to the Azure tenant. It's the top level container for the environment for that customer. The account admins have full control over the account. If an account is a root account and an admin account, then that account has full organizational control. It's important to note here that any overscoped roles provide escalation capability. Rhino Security Labs has an extensive article covering 21 AWS IAM privilege escalation methods which allow an attacker to escalate from a compromised low privilege account to full admin privileges. I've listed a bunch of them here, I apologize for the small text, but really what you want to do is you want to read the blog article that they wrote which is linked at the bottom and in the slides. They also released the scanning tool which is on GitHub to identify these vulnerabilities in a AWS user account. If you have an account with IAM read access for all users, the script can be run against every user in the account to detect these vulnerabilities account-wide. If you're asked to pen test AWS or you're providing security consulting services around AWS or even you're working in an organization and you're moving to AWS and you have some of the IAM roles configured, you definitely want to scan them for potential escalation methods. API keys, I'm just going to cover this really quickly. They provide permanent access to an environment often with extensive privileges. These keys can often bypass the typical username and password used to authenticate as an account if it's configured as such. Well, I've covered a number of scenarios on how to attack on-prem and hybrid cloud components. Let's focus on on-prem Active Directory domain controllers hosted in cloud infrastructure as a service. First up is Amazon AWS EC2. Many organizations leverage Amazon AWS EC2 or IaaS as a cloud data center. Some of these host on-prem AD domain controllers in AWS. We start with Acme Corporation which has the on-prem AD environment. They've stood up an Amazon AWS account to use EC2 as a cloud data center. Then Acme configures an IAM role for the server admins and they stand up a couple of on-prem Active Directory domain controllers in AWS. Acme then configures Federation so that users in the on-prem AD can authenticate to AWS using their AD credentials. To simplify access for server admins, the on-prem AD group AWS EC2 admins is created and then that's associated with AWS IAM role. The server team accounts are then added to the on-prem AD group called AWS EC2 admins to enable the team to easily administer the AWS environment. But if an attacker is able to compromise one of the user accounts that's a member of the AD group AWS admins, then they are able to connect to the AWS console as an AWS admin. And then now the attacker can configure the virtual domain controller instance in AWS and run commands on it. Once the command runs to modify the domain controllers copy of AD, the changes replicate back to the on-prem domain controllers and then we end up with the on-prem Active Directory being compromised. So effectively in the scenario, the attacker can compromise an account, a regular user account that doesn't have any Active Directory rights. It's just a member of a group in Active Directory. This group is associated with the AWS IAM role through Federation. And so once the attacker compromises this account, which has these rights granted in the group in Active Directory, but then are relayed and related to this AWS IAM role, then they can leverage that in order to compromise any of the systems that are in there that they have rights to. So I've got a summary slide here that covers this very briefly. But keep in mind that Amazon SSM, which is effectively the Amazon agent, is installed by default on most Amazon provided instances or templates. You do need the role to execute, but once you have the rights over that environment as a server admin would, you can normally go ahead and get that. Hopefully you're logging this and looking at the logs. Amazon CloudTrail can pull these logs and the logs need to be configured to not be deleted. So very often an attacker is just going to delete the logs so you can't see what they did. For most of 2019, I was digging in Office 365 and Azure Active Directory looking at features. As I went through each of them, I found one that was very interesting. Remember earlier how I noted that Azure roles and Azure AD roles are separate? Well, not exactly. Here it describes a way that global administrators could actually elevate their access and have some control over Azure. Surely the Global Administrator role description notes this, right? Well, no, it didn't, at least not as of May. The Global Administrator role provides full admin rights to Azure AD and ultimately all the Office 365 services. The Microsoft Online document provides key information about this and there's nothing here showing Azure capability. I took the screenshot in May. After contacting Microsoft and providing information about this escalation path, the document's been updated in the past month or so, stating that yes, there is a way for Global Administrator to have Azure rights, which is a bit different than what it said before. Once we have access to the Azure AD portal, which is typically all Azure AD users by default, we can view several different configuration settings for Azure Active Directory, which controls many aspects of Office 365. This page shows the directory properties, which now includes the new managed security defaults. Towards the bottom is access management for Azure resources toggle, so we can see what that is. According to Microsoft documentation, toggling this option from no to yes adds the Global Administrator account to the user access administrator role in Azure RBAC at the root scope. This has control over all subscriptions in the tenant and this option is only available to accounts that are members of the Global Administrator role. While this option is configured in the directory properties section, this is actually a per account configuration option. So this magic button provides the ability to manage Azure roles, but no direct Azure rights as into VMs. Further in the documentation, there's additional context on what Elevate Access is and how it works. When developing Azure Active Directory in Azure, Microsoft had to decide if the Global Admin should have Azure rights initially and opted for no, they shouldn't. But they provided this capability to gain Azure access. This is that backdoor of sorts, which has been somewhat highlighted now that the Managed Security default option is right below it. This graphic shows what happens in the connection point between Azure AD and Azure. My biggest concern here is that for many organizations, the group that manages Azure Active Directory and Office 365 are often a different group from those that manage Azure. This means that someone could Elevate Access, I think a rogue admin, and no one probably would notice. This is potentially a serious threat from an insider threat perspective, especially around the detection part of this issue, which I'll cover in a bit. This note in the Microsoft document highlights an interesting phenomena with this option. Only while the account is a member of the global administrator role, can the account toggle this option to yes or no. This means that if an account is no longer in the global administrator role and the option has been set to yes, they cannot change it back to no and no one else can change it either. I also found an API that seems related, which means that an attacker would not need to visit the Azure AD portal to perform this action. You just have to run this API call. And Ryan added this REST API Elevation Method to PowerJer, which you should check out. I have a link at the bottom of the slide. Now, what I'm going to do is walk through a scenario where we have Acme and Acme has stood up. They have a new Office 365 environment. They've had Azure for a very long time. They're using it Azure as their cloud data center. They've got domain controllers there. They've got a bunch of different servers in Azure, but they're just now spinning up Office 365. They're going to use it for email. They're going to use it for a number of different things. So they have members of the active directory team, exchange, SharePoint, a bunch of different people, probably 30 to 50 people are members of the global administrator role. Not everyone has had their account configured for MFA, which is realistic since many had men's. This is from last year's Black Hat talk that I gave with Mark Morrow. And it was only about 8%. Hopefully it's much higher than that, but it's still probably less than 15% if I had to guess. And so this means that most global admin accounts don't have MFA configured. And so if an attacker is able to password spray and guess the account password, then they're able to take over this account. So by default, this option is set to no. So if an attacker compromises a global admin account, they can go in and actually toggle this from no to yes, which gives them Azure User Access Administrator role rights at the root of Azure, that Azure tenant that's configured on the Azure side. So once that toggle option is set to yes, we can see a new account appears in the user access administrator Azure RBAC role. I call this attack account Hacker, so I don't forget which one I'm using. But you'll notice that I also added a couple others that look like they belong. There's really nothing that's supposed to be in this role. Monitoring for this can be a challenge since the primary method is running an Azure CLI command at the root scope. I can't find root level Azure RBAC roles listed in the portal, so I can't go there to see what the membership is. This is something that could be monitored through an API, or if you're pulling logs into your SIM, you could alert on RBAC role or role membership modification. But when I run that command, I get this output, which means I would have to parse it. And so this is the message that appears when attempting to remove root level role assignments from a subscription. Removing an account from a root level Azure RBAC role requires running an Azure PowerShell command. When an account toggles elevate access from yes to no, it is automatically removed from the user access administrator. Once the attacker has user access administrator rights at the root level, this account can modify roles for any subscription in Azure. Owner is obvious and probably monitored, maybe. What about something more innocuous? So I found virtual machine contributor, and this is pretty interesting, because the documentation states it lets you manage virtual machines, but not access to them, and not the virtual network or storage account that they're connected to. And it includes this ability, which is Microsoft.comput virtual machine slash run command. This provides the ability to run commands on the VM as system, such as PowerShell. And so we can click on run PowerShell script and we can do that. The other thing that's interesting here is that it provides the ability to re-enable the administrator account, which on a domain controller in Azure would be the RID 500 account for the domain. So we have seen some environments when we look at them from an act directory assessment perspective, they have had the default domain administrator account disabled, but very often it has an old password, 10 years old or more, and it may actually have a Kerberos service principal name or SPIN associated with it, which means that we can Kerberos it. So if an attacker could just go ahead and Kerberos it, then re-enable it through here, they could potentially leverage that default admin account, which was originally disabled. So we're going to add the attacker controlled account to the virtual machine contributor, and now that our attacker has the ability to run commands, we can update the administrator's group. This will happen on the Azure-based domain controller, then replicate to the domain controllers that are on-prem. Note that being able to run commands on an Azure VM is not specific to customer on-prem DCs hosted on Azure, but also other systems hosted there as well. So we take a look to see what happened, we can go switch to a computer on-prem or run another command on the DC to see what the result is, and we have now compromised the on-prem act directory from Azure. If PowerShell logging is configured appropriately and sent to a sim and monitored, this event should identify unusual activity showing up in the PowerShell log. Another option an attacker has with these writes is running invokeMimicats from the internet or even a compromised system on the corporate network or within the Azure tenant. So here I'm running Mimicats just to get the Kerbs DD password hash, because with that I can create golden tickets, which means I can persist forever in the on-prem act directory, even though I've never touched that or had access to the corporate network. So that means once I get a foothold, I can have domain admin possibly forever. But what if instead of running invokeMimicats directly from GitHub, we loaded it into Azure Blob Storage. So effectively we're going to use a PowerShell script, we're going to upload, I'm not going to call it invokeMimicats, I'll call it immmk.txt, and Azure Blob Storage is very nice because it gives me a URL in which I can interact and connect to this file. So that would work as well. Let's take a moment to recap. From global admin to Azure user access administrator to Azure admin or virtual machine contributor. So if an attacker compromises an organization's global administrator account, either because they are just starting with Office 365 or didn't realize the risks around protecting global admins, either way the global admin accounts weren't locked down with PIM conditional access or MFA. Or a global admin session token was stolen because this admin was using their regular web browser on the regular user workstation which was compromised. This would mean that the attacker was about an hour to perform these actions, compromise the account, elevate access to Azure, get Azure rights through Azure roll membership, remove elevate access rights, perform malicious actions on any or all Azure VMs and all subscriptions, then remove roll membership in Azure or not. The total time required only a few minutes, perhaps as much as 15 minutes total. In many organizations there are two different groups that manage Azure AD Office 365 and Azure. As I mentioned earlier, they have no expectations that the other team has the ability to affect their environment. So why is this issue important? Well, customers don't usually expect that the global administrator role that says it has Office 365 admin rights can also control Azure subscriptions. We need to be very careful what accounts are members of roles in Azure at the root level, as well as on the subscriptions since this impacts the security posture. It's not obvious that removing the account from global administrator role does not remove the account from the Azure RBAC role user access administrator when the access management for Azure resources toggle option is still set to yes. And the key points around detection is there's no way to detect this configuration in Azure Active Directory at all. There's no property to query on accounts. You have to run an Azure CLI command to check the role group membership but that's on the Azure side. When I walked through the Azure AD to Azure access elevation, I attempted to identify a clear event that I could alert on and was unable to. The core directory management set company information log notes that something has changed but not what. The capability for an Azure AD global admin is quote-unquote by design. However, not expected and not well documented by Microsoft when reading about what Azure AD global admins can do. Certainly now the documentation has been updated to mention this, but I feel like there should be better logging and alerting around this sort of configuration. I'm concerned that in this is a scenario where the Microsoft cloud customer can't detect, can't remediate and ultimately can't prevent since there's no real gate to lock if an Azure AD global admin account is exposed and then they can jump over on the Azure side. So the only real good answer I can find for this is to ensure that global administrator groups are managed or that roles are managed with Azure AD PIM, but even this doesn't stop a rogue admin from doing whatever they want. The best mitigation is to prevent compromise of global admin accounts using PIM reduces this risk tremendously. Outside of that monitor global admin role membership in Azure AD and the user access administrator group at the Azure root level. Regarding sensitive systems severely restrict who has access to them placing them in a separate subscription will protect against common over permissioning but not against a sort of attack. So leveraging a separate Azure 10 is the best method for isolating and securing sensitive systems including domain controllers. I reported this issue to Microsoft security in September of 2019 continued corresponding with them providing additional details through the rest of the year. Again my biggest concern with this is that on the Azure AD side there's no way to detect, remediate or prevent a global admin from flipping the switch and gaining full Azure rights. Instead of an external attacker what if there was a malicious insider someone who did not benefit from the companies spared no expense mantra and we know what happened in that scenario. But how bad could it really get? Let's play what if what if an attacker takes control of Azure resources what if the attacker then removes all the accounts from all the roles and what if they decide to ransom the Azure environment. Azure ransomware? Azureware? Yeah that could be pretty bad. So let's take the cloud architecture concepts to the next level pretty much where organizations are headed. Let's say you start with the typical Microsoft server architecture with on-prem actor directory and then we're going to add Azure to it and then we're going to add AWS to the mix and then we're going to add Google Cloud platform and so I'm just going to provide a simplified view of this because really the architecture diagrams get pretty messy we have AWS we have Azure and we have Google Cloud platform and I simplify this like I said showing the on-prem data center and federation to the three different cloud environments but ultimately this federation just means that we have this sort of configuration where everything can talk to everything else and that they're interconnected and this means a compromise in one will likely lead to a compromise in all cloud environments since the IAM story is quite different in each and difficult to track roles and privileges across them it's likely they counter over permission in the scenario and the use of federation provides the ability to jump across between these different environments so compromise of all three cloud environments will likely result in on-prem compromise since there's likely on-prem 80 domain controllers and one or more of the cloud environments federation or any type of authentication flow really between these environments enables a new type of lateral movement the attacker could leverage overprivileged Azure and AWS roles and the rights provided with them and showing which ones can provide system access to VMs despite how they're described is one way that we can identify and lock down those controls or those roles but the attacker could jump across the federation to gain privileged access to cloud hosted services and systems once they compromise a single system or even an account because ultimately gaining privilege access to the IAS service provider so Azure Amazon AWS GCP means that the attacker could do anything with the VMs hosted in the environment including accessing and modifying data on the systems so it's very important that administrative roles need to be limited and gated I recently heard a customer tell me that an executive was concerned about putting all their eggs in one basket so they sign up for Azure and AWS and Google Cloud Platform so now their eggs are in all the baskets and the security capabilities and controls these environments is quite different and it's up to the operation and security teams to figure that part out the cloud provides many benefits and power though cloud integration often complicates security organizations will have to do better to map security between on-prem and cloud systems and authentication it's important as a security professional or security hobbyist to learn about these things so cloud is the wild west at this point is the whole brave new world if you want to go far in your career focus on cloud security because it is wide open there's a lot of areas that haven't been explored and especially in those areas across on-prem and the hybrid environment so there's a lot of work that needs to be done there and there's a lot that we need to do moving forward to improve the security in this environment that's been my time, thank you very much for yours