 Hi DevCon, that's better. Welcome to I'm In Your Cloud. And um, I'm Dirk Jan, I live in the Netherlands, work for Fox IT. Um, I mostly focus my research on Azure AD and on Active Directory. Um, if you do stuff with Active Directory you've probably heard on some of the tools I wrote, like Mitem6 or um, a Python version of Bloodhound. And I also have a blog where I write about stuff. I also wrote there about Perfect Change. And of course I have a Twitter account where I share my research. So today we'll be talking about cloud. And in the cloud as you all know, um, everything is magic and secure. You don't have to patch. And everything is automatically arranged for you. You just spend some money and everything is good. Um, our results fully the reality. At Fox we also have like internet response department. And most things they see is, um, the cloud being compromised because people actually didn't secure it properly. And on the conferences I don't see a lot of talks yet about cloud stuff. There are some focusing on Azure and, um, on Azure Virtual Machines and stuff and AWS. But I didn't really see anything about Azure AD. So that's what we'll be talking about today. And first I'll be giving an introduction into Azure AD. What it is, how we can talk to it. Um, how it's architecture is kind of built. And about the roles, application, service principles. Then I'll take a short step and have to look, have a look at some fun with multi-factor education. Then we'll talk about linking up the cloud and on premise. Then talking about Azure Resource Manager and how it's connected to Azure AD. And last but not least, I'll talk about integrations and as an example I'll take Azure DevOps. I have to apologize in advance. Uh, if you're still tired or hung over, this is a talk which is pretty fast paced and I've got a lot of content. So try to keep up. But first, what is Azure AD? According to Wikipedia, Azure Active Directory is Microsoft cloud based identity and access management service. Um, that's a whole mouthful. But basically it does the authentication for obviously 365, uh, the Azure Resource Manager and anything you want to integrate with it. Um, I always like to compare it kind of with, uh, login with Facebook, uh, because you can integrate that on a third-party website and then you can just log in with your Facebook account and you register it. Only then, um, it has a Microsoft logo and it's called Azure AD. And it's actually on the, apart from that, it's actually pretty similar. And it's called Azure Active Directory. But if you compare it to the Windows Server Active Directory or the on-premise Active Directory that we all know, there's actually nothing that's really similar to it. Um, all the protocols are different, uh, the structure is different, there are different, um, terminology in different setup. So for this talk, try to forget almost everything you knew about Active Directory and we're really gonna focus on cloud. So if you want to interact with Azure AD, there are a couple of ways. First there's the portal, which if you did use Azure AD you've probably used. There are some PowerShell modules, um, there's also command line interface and lastly there are some APIs. And that's already quite a bit of ways and there's a few difference between those. Um, for example the portal, it's very, um, nice and shiny. It's, it works nice. You can click around and configure stuff. But if you try to understand how Azure AD works and you're looking at the portal, um, it's not gonna work out. Cause I started with the portal and then I moved to trying to understand how things work with the API and such. And I found out that the portal just made it more unclear for me. Cause, um, Microsoft tries to make everything user friendly and translate, uh, things from the complex terminology that's actually used in the backend. And so if you want to understand Azure AD, um, don't use the portal. There's also PowerShell. Actually there are a couple of PowerShell modules and I think the oldest one is the MS online PowerShell module. This one really focuses on obviously 365 and as such it has a few features that you won't find in the other modules. Um, this is a bit old and Microsoft is deprecating this one. So you shouldn't be using this, but it still has some specific features that are not available in the other modules. So sometimes you still have to use it. There's also the Azure AD PowerShell module, which really focuses on Azure AD, but it has a different feature set than MS online PowerShell module. And lastly there's the, uh, command line interface and yet another PowerShell module. But this one is focusing more on Azure resource manager. So you can use it to, uh, manage virtual machines and VLANs and that kind of stuff. And the most interesting ones from my perspective are the APIs. Yet again, there are some different ways here. So you have the Azure AD graph, which is, as it said, is really focusing on Azure AD, which also have the Microsoft graph. And Microsoft was like, okay, so we have this graph for Azure AD, but why not put all of our products in one single API? And I'm like, hmm, that doesn't really make it much easier to understand. You have like an API definition that would take a year reading through. And I can't really find anything. So I prefer the Azure AD graph, um, but that one is already, um, considered legacy. So I, in the future you probably have to use the Microsoft graph. Lastly there's the exchange provisioning service, which uses XML. So I really try to avoid that one. But once again, that's the back end of the MS online PowerShell module. So there are some features in there, which are actually not in the other APIs. Now which one to use? Um, all of them have different features and some of them have unique features, but they are, um, deprecated or considered legacy. As you can see in the, in the corner below, they support different authentication methods. So some of the PowerShell modules support difficult authentication, uh, and some don't. But if you want to actually use all of the features, you're stuck with using different APIs and different PowerShell modules all over the place. And to make it even more confusing, there's also a different terminology. Like here I'm using, um, at the top I'm using the Azure AD PowerShell module to list the directory roles. And you see I have five directory roles. But if I use the MS online PowerShell module, I certainly have a lot of more roles. You also see that object ID for the role is somehow different between the two modules. It has a reason, but it doesn't really make it, um, much clearer. Also here I'm looking at the company administrator role. And if you look at the portal, that one is conveniently called global administrator. So once again, a different terminology. So to conclude on this, there's not really one uniform way to talk to Azure ID, I wish there was. And you're also kind of limited to what Microsoft considers important, like in on-premise Active Directory you have LDAP and everything in Active Directory is kind of stored there. But in the, in Azure AD you have different APIs and not all features can actually be called via APIs. And most of this research comes from using both documented and undocumented APIs in Azure AD. So let's have a look at the architecture. In Azure AD there are, um, three major kinds of principles. First you have the users, which often are the physical users that are in your company. And these users can of course have devices. Uh, a device can be like a Windows 10 laptop that can be joined to Azure AD, but it can also be an iPhone or Android that's joined to Azure AD via, um, mobile device management. And lastly you have applications, which we'll talk about later. And your users, um, can have specific roles. And this is also, um, this picture is kind of important. So usually if you hear about Azure roles, they're talking about role-based access control. Azure RBAC roles on the left. That is what is used for the virtual machines and stuff. And it's not actually used for Office 365. So first we'll, uh, focus on the roles that Office 365 uses, which are defined in Azure AD. And there are some different admin roles. The first one, or the most, um, important one is the global administrator or company administrator if you use the APIs. And this administrator can do anything. Now of course you don't want all your administrators to be able to do anything. Um, so there are also limited administrator accounts. You have the application administrator, authentication administrator, exchange administrator, etc. And they all have their, uh, limited part, which they can manage. Now at a time that I made these slides, um, the roles were fixed. You couldn't change that. Actually in the past week, um, Microsoft started to change that. So, um, you see it's cloud, it's changing all the time. And they're actually looking now into, uh, a way that you can create your own roles. But I haven't played with that yet. So let's assume that, um, these fixed roles are all there is for the sake of this presentation. And now we're talking about applications. And I think this is the most confusing part of Azure AD. It took me a long time before I finally understood how these work. And that's also because the documentation is not really clear on this. And once again there's a, a big terminology difference between the documentation, the APIs that are available, and what you can see in the Azure portal. And these applications have a very complex permission system. So what is an application in Azure? Well, about everything. I mean the Microsoft Graph, which is an API, is an application in everyone's Azure AD. The Azure portal is also a registered application in the Azure AD that you're using. In fact, there are about 200 applications and associated service principles in, um, every Office 365 installation. So it's a lot of stuff that is in there by default and you can do quite some interesting things with us. So when I say application, there are actually two parts. And if you're looking at the simplest case, which is an application which is defined in your own Azure AD, then there's the application definition and a service principle. And the application definition basically defines what the application is about, you give it a name, et cetera. And the service principle is the actual principle in your directory, which gets the rights and has the rights to perform actions in your directory. So there are two parts of an application. And in this talk I'll, if I say application, I usually mean the definition. And if I say service principle, I mean the kind of account that's present in the Azure AD. So because cloud is multi-tenant, you can have applications from another directory in your directory. So this is the simplest case when you have both the application definition and a service principle in your own directory. But with third-party applications, then the definition of the application is actually another directory. And once you start using that application, a service principle is created in your own directory. And that is the presence of that application in your directory. For Office 365, Microsoft has some internal tenants in which all these applications are defined. And in your Office 365 directory, all the service principles are created. So once again, it's a split between the definition of the application and the service principles. Now applications can have privileges and are basically two kinds of privileges. In the portal there are called delegated permissions and application permissions. And delegated permissions are only used when a user is actively signed in in the application. However, application permission is something the application has statically and it can use it at any time. So these privileges actually assigned to the service principle portion of the application. Now the permission model is as follows. Every application defines its own permissions which can then be granted to other applications. And like commonly used are the Microsoft Graph and the Azure AD Graph. And they define permissions such as directory.read which gives an application or a user permission to read attributes in a directory. And these can then be granted to other service principles. So let's look at how this looks in the portal first. So here we have the application definition and we see there are a couple of permissions defined for this application. So first is the directory access as users, access as user which is a delegated permission. So once a user is signed in into this application, this application has the right to access the directory as if it was the signed in user. There's also an application permission here which is directory read write all which gives the application permissions to read and write to the directory. If we look at the service principle, we see these privileges again. So the application has permissions to read and write directory data in this case. And also an admin sometimes has to ground these privileges because by default any user can create new applications but of course not every user can give these application permissions. And sometimes the high value or high impact permissions have to be approved by an application before they can be used. Now I created a small table to see how permissions actually work versus how the Microsoft portal or the Azure portal tells you how they work. So on the right is the portal terminology and on the left is how they are used in the APIs which for me is canonical. So every application defines OR2 permissions and application roles. And these get translated to delegated permissions and application permissions in the portal. And an application definition then requires access to certain permissions such as OAuth permissions or application roles. And when an administrator approves those permissions these are granted to the service principle. So you would imagine that we have the Microsoft graph application which defines permissions. We have created our own application which needs to access the Microsoft graph. So we set some definitions of which API calls or which API permission it requires. And these are then granted to the service principle associated with our application. Oh this is clear a bit. And as I said I should previously an admin has to approve these applications. So that's how it works in the portal. But it's not necessarily how it's required to work. So normally you add permissions in the application definition and an admin then approves the permissions. However if you use the APIs you can directly assign a role to an application on the Microsoft graph or the Azure AD graph. There are some API calls for that. And then when an admin looks in the application's overview in the portal it says oh there are no permissions required for this application. Which looks great. If you now move to the service principle certainly there are some permissions that the service principle has. So even though the application definition doesn't say it has these permissions the service principle still has them. And also you can see that these privileges were granted by an admin. But it doesn't say which admin it just says granted by an administrator. So there's not really a way to see hey who was this administrator that assigned these permissions without actually putting them in the definition. It gets even more confusing when you look at Microsoft applications. Because all the Office 365 applications are considered first party applications in OAuth terms. And as you see in the portal they don't actually have any permissions that you can see. So we are looking at the Microsoft Teams web client here. And it says oh there are no permissions for this service principle. Okay that's great so it doesn't have any privileges. You'd think. But when I sign in to Microsoft Teams I get a JSON web token which if you decode it contains the privileges that were grounded to this application. And you certainly see that this application does in fact have permissions. So in the scope variable of the JSON web token you see this application can actually send emails. It can read my information and it can write to my calendar. And it has these privileges on the Outlook API. And as you see this is actually the same application that we were just looking at the Microsoft Teams web client. So why does this all matter? There are some admin roles which allow an admin to manage all the applications in the directory. So it's just a global administrator. Of course they can do that because they can manage anything. Or the application administrator or cloud application administrator. So these administrators can assign credentials to applications or rather to the service principle of the applications. And then they can log in as that service principle using those credentials. So if we compromise an admin account in Azure AD we can assign credentials to an application. And then we can log in as that application. And of course since applications are not humans there are no multi-factor options for service principles. Also if there is an application that actually has more rights than the application administrator is possible to escalate privileges this way. Actually previously there used to be some default obviously 365 applications which had roles assigned that had more permissions than the application administrator had. So the application administrator could just assign credentials to the application and then perform actions that he previously couldn't do. This has been fixed by Microsoft by the way. So just to give an example how this works. You can use PowerShell to create a new certificate and then assign certificate credentials to a service principle. Then you can log in as a service principle. In this case I'm using the Azure AD PowerShell module and you see I'm signed in with the certificate on the directory. Now if I perform any action such as adding a user to a group then the logging will show that this was performed by the application rather than the admin that assigned the credentials to the to the application. So if I assign credentials to an application and then I wait X amount of time that the logging kind of rolls over and there are no more logs that I actually modified the application. There's no way to see which user assigned these credentials and this actually performed these actions. Furthermore a bit of a twist. So application admins by default can't assign roles to the application that allow it to interact with the Microsoft Graph or the Azure AD Graph. If they could they could just assign the permissions to the application which they don't have themselves which would be a privilege escalation again. But this is not possible. But what they can do is assign OAuth permissions or delegated permissions if you use the portal. But these are only valid when a user is using the application. Well of course you can exploit this. If you are the user impersonation permissions to the application and then fish a global administrator to actually sign into its application you can perform any actions that the global administrator could do. And here's a small demo. So it works. So we sent an email to the global admin here. And here's link. He clicks on it. And then we captured his access token. So this is an access token for the Microsoft Graph. And with this access token I could perform all kinds of actions. And you can see here this token has privileges to access directory as if it was that user. So with this token I can perform all kinds of actions and I can modify my own role. I can give myself permissions. I can ask users to group and all that kind of stuff. And as you saw this didn't require any consent from the admin. That's because the application admin already approved the privileges for the application. You can make this even more fun. So because there are like 200 built-in Office 365 applications that the application admin can also modify. And if you assign a whitelisted URL that this application can use to sign users in and you then assign your own URL you can use the built-in permissions for the application to actually fish the admin again. And if you look at the logs this actually doesn't show anything suspicious. Because if an admin is using Office 365 these kind of sign-ins appear all the time. And the logging doesn't specify which URL the admin was redirected to. So you can just add your own redirect URL, get a token and in the logging it won't actually say anything. Apart of course that you modified the application but the sign-in log is kind of useless here. So this was a bit about Azure applications. I'm going to take a little sidestep and have a look at Node Effector authentication. So of course Node Effector authentication is good and everyone should do it. And there are multiple options for this. So the Authenticator app is one of the options. It's kind of similar to Google Authenticator when you have like a code that you display and you use that as the second factor. You can also have a notification that pops up on your phone that you can prove the sign-in. And there's of course the good old text message that I think has been reiterated a million times already that these can be intercepted and are insecure and stuff. So I didn't look at that. But there's also the option to authenticate with the voice call. And this works as follows. You have your phone number registered in Azure AD and then Microsoft calls you and in order to approve the sign-in you have to press the hash sign or the pound sign. Now I thought, okay, what about I break into someone's voicemail and I change his welcome message to the tone of a pound sign because these signs are just specific sound frequencies. So if you can record it in your voicemail message then potentially Azure would call you and hear that same tone. Then you make sure that the phone is occupied. So when you're performing your sign-in you call the victim and make sure their phone can't answer it and the talk will go to voicemail. You sign in using the password that you fished previously or you found it in a leak somewhere. Azure AD will get redirected to voicemail and it will authenticate you. And if you think can I actually break into someone's voicemail and how hard is that? You should see this talk from last year. This guy made like a Python script to brute force all the pin codes on voicemails which is really cool. So here's a small demo. So I changed my voicemail message. I hope you can hear this. This is the sound of the pound sign. Now I put my phone in airplane mode. And I enter the password for the user that I somehow obtained. And it will say okay I sent you a text message but you can also choose to call the phone. And now Azure is calling the phone and hopefully getting redirected to voicemail. And we have to wait for a little bit. And there it goes. So if you do compromise someone's voicemail and you change the message you can potentially bypass his multi-factor authentication. I reported this to Microsoft and they're saying okay yeah we see the issue here but you still need to compromise someone's voicemail first and it's kind of a post-exploitation technique. So we will be fixing this at some point but not right now. So I haven't tested this recently but I assume this still works. So if you do allow voice messages for authentication in your tenant you may think, may want to think about disabling that. Okay up next um linking up the cloud and on premise cause um of course you don't have all your infrastructure in the cloud yet. You also have a part on prem and you would like to combine the two. And previously we talked about the application administrator and how he can backdoor stuff, escalate privileges and stuff. But let's assume that this account is protected with MFA and we can't hack into his voicemail. So what about the on premise infrastructure? If you want to synchronize Azure AD and on premise AD um that's usually done through a utility called Azure AD Connect and you install that on premise and use that synchronizes Active Directory data to Azure AD. And depending on which method uh you use to authenticate your on premise use in Azure AD you can for example use password hash synchronization so that they can sign in with the same passwords on premise in Azure AD or you can use um federation with ADFS both require this Azure AD Connect tool. And the account for this tool in Azure AD has quite some privileges which is all very nicely documented by the way. So if you look at the documentation you see the direction synchronization accounts. And we can see that it has permissions to update service principles and to assign credentials to service principles. And these were just the applications that we abused with the application administrator account. So um previous some interesting things about the sync account. If you use password hash synchronization then the sync account will have access to all the password hashes on premise. So that basically makes it domain admin because you can just synchronize the or synchronize the password hash of the KBTT account and then you can compromise on premise. And of course in the cloud it also has high privileges as we just saw from documentation. And the assets in the cloud may not just be limited to the single AD domain that you're synchronizing. So there's some potential for previous escalation here. Um I wrote a tool to extract the Azure AD Connect password from the on premise server. Uh there are some several protections for this password. Uh about which I presented at Troopers earlier this year. So if you're interested in the technical stuff uh check out the link below. But basically I wrote three different tools that depending on your scenario can be used to dump the credentials of this account. And some work over the network while some others uh can be used in example for cobalt strike through um assembly loading. And these tools get your get the both the on premise password and the cloud password from the server where AD connectors installed. So once we have these credentials there's some fun stuff we can do. We can dump all the on premise password hashes if the organization is using password hash synchronization. We can also use this account to log in into the Azure portal. Don't ask me why but this works. It's a user account so you can just log in. Um we can bypass conditional access policies since there are some conditional access policies to require multi-factor authentication for all admin accounts. But uh the sync account is of course excluded from this since it's not a human account and this can't do multi-factor. Just like the application administrator we can add credentials to service principles um backdoor stuff and do all kinds of fun things. And we can also modify service principle properties. So we can change the authentication URL of a Microsoft application, vision admin and escalate to global admin privileges. And there are some more things where service principles are used. And that's where the connection between Azure Resource Manager and Azure AD comes in. So once again returning to this picture um Azure AD is the authentication and identification access management for both Office 365 and for the Azure role-based access controls in Azure Resource Manager. So that means that if you're a global administrator you can um by design also access all the um virtual machines and network infrastructure that is stored in Azure Resource Manager. And you can also assign privileges to manage Azure resources to service principles. Which can be managed by application administrators or by the on-premise sync account. And if you have an application such as Terraform which is used to automatically provision cloud resources then it obviously needs high privileges in order to create and modify in the lead resources. And this is this can be done with service principles. So if you have a control over the on-premise sync account you can assign credentials to service principles which have rights in Azure Resource Manager. And you can also control all the cloud resources. An example which actually uses this is Azure DevOps. And what is Azure DevOps? Azure DevOps is DevOps tooling. Um it's in some ways kind of like GitHub. You can use source code management. You can collaborate on projects. Um you can have automatic build pipelines and automatic deployment. It's the whole DevOps um toolkit. And I had a look at Azure DevOps pipelines. Um this is kind of cool because Microsoft makes this feature available for free to for example open source projects. And you can automatically build your code through for example GitHub integration. And when you push your a new version it will be automatically built on hosted resources in Azure. An example of this is my own Azure AD Connect tool. I've connected that to Azure pipelines. So as soon as I create a new put a new comment in there um the net binaries will get automatically built on Azure and you can download those. So there are multiple ways to change the definition of a build pipeline. Previously um when I looked at this only the uh a GUI could be used to change the the build definition. But um I think earlier this year or at the end of last year a new feature was added which allowed pipelines as codes in YAML files. So then the pipeline definition is part of the repository where the code is in. And it kind of looks like this. Since this is a YAML file with the um build definition it says okay when I push a new uh comment to master um you build it using the hosted visual studio 2017 profile. You can provide you can set up all build steps in order to um do stuff with this. So let's talk about a hypothetical scenario. Um we have a DevOps team and a team member of the team wants to automatically publish the build artifacts. So um for example executables uh to Azure using blob storage. Um blob storage is kind of like S3 in AWS. Can store all kind of stuff in there. So he links up Azure resource manager with Azure DevOps. And there's this nice button there which uh you can select your subscription and then you click alt rise. And then suddenly the two are connected. Now there is a new user or a new team member at the company. And this team member needs minimal privileges to uh contribute his codes to the repository. Because he is a junior member there we don't give him any special privileges to for example alter the build definitions. Um now the new user isn't exactly uh isn't exactly a nice person because the first thing he does is actually editing that build definition in the code. And then pushing that change to uh to the repository. So even though he didn't have privileges to edit the pipelines. Because the pipeline is part of the code this user can still edit it. And meanwhile is completely unrelated Azure VM which is not related to the source code at all. The following happens. The video plays. Ooh that's a notepad. How did notepad get there? I thought we were editing source code. So what happens here? Um remember that I said that we added um a copy to Azure blob storage for the artifacts? Well there's uh a task for that which is called Azure file copy. And I've edited this build pipeline a little. So what it does it's it gets the Azure file copy script. And it backdoors that to dump some of the authentication data that is used to link Azure resource manager to Azure DevOps. And it will then put the backdoor code back in the original PowerShell script that's used to copy the file. So when the file gets copied uh instead my code gets run and we can gain access to some of the credentials. So if you look at the logs um we see it dumped the tenant ID, service principle ID but they're all um they're all masked in asterisks because Azure DevOps doesn't let you log sensitive stuff of course to the logs. But if you base 64 encode that um then it's all fine. So at the bottom you see that I dumped the authentication data uh in the password for the account using base 64. And we can decode that. And then we see you get the tenant ID that uh is used for Azure subscription. The service ID of the service principle that was created. And you get the service principle key which is the password for that service principle. Now I was a bit confused when all this happened because I just remembered clicking the authorized button and I didn't really get a warning that the service principle would basically get contributory rights on my whole Azure subscription. So when I clicked that little button a new service principle was created and he got full privileges on the whole Azure Resource Manager subscription. So it means that you can edit any virtual machines, disks, access files and all that kind of stuff. And with these credentials that we just obtained we can log in using the Azure uh CLI module. And that password you saw before even though it looks uh like basically for encoded is just a very long string. So here I used the credentials that I dumped from Azure DevOps to actually log in on Azure Resource Manager. Now how about that notepad? Um I added a little extra so it's just the inline script which adds a new custom extension to a virtual machine in Azure. And this runs a PowerShell script which then uh spawns Notepad. So that's just to prove that you can actually access those virtual machines as the highest user there is. So can anyone edit pipelines? Um normally no. Um the pipeline definition uh there's a specific role required to edit the build steps in the pipeline. But since the pipelines move to be part of the code anyone who could edit the code could actually edit the pipeline. I reported that to Microsoft and that is fixed in the latest version of Azure DevOps. So I didn't retest this yet but uh this shouldn't be possible anymore and hopefully now the edit pipeline role is actually enforced also on the source code in the repository. A few conclusions about this. Um be careful with integrations. Have a look at which privileges they actually get on your subscription and um how they are used. Because anyone that can edit your build pipelines can access the secrets that are exposed to that build pipeline. And if you enable secrets for public repositories and you allow um allow builds on pull requests then anyone who creates a pull request can actually extract your secrets of any servers that you integrate with Azure DevOps. Um though this is disabled by default and this is documented uh pretty well I think. At the link at the bottom it actually says warning anyone who can edit your build pipeline can access your secrets if you enable this. You don't want this. Some general conclusions. Um I think cloud can be beautiful. It definitely allows us to do a lot of things that we couldn't do before. But all your stuff is on the internet and you will have to uh secure it yourself because not everything is secure by design. And especially enable MFA for all your sensitive users because uh 99% of the cloud compromises are users that didn't have MFA enabled. And if you have software as a service um it does take away your need to patch manually. You always have the latest patches. You always have the latest features. But you also have the latest vulnerabilities that got introduced by those fancy new features. Such as in the Azure DevOps case if you really lock down your permissions and then suddenly there's a new feature which uh changes that then you have to be uh aware and redo your um risk calculation. And sometimes there may be a vulnerability that makes your security some uh certainly a bit more insecure. And of course there is uh a full trust implied in the vendor from who you are renting your cloud services. So um you do have to trust that they are actually doing everything correctly and following best practices. So this was I'm in your clouds and thank you for attending.