 Tony enjoys skateboarding, competitive grappling, Brazilian juzit tube, and spending time with his wife and kids in Dallas, Texas. Awesome, thank you. Alright, we'll start the timer. So this is my first time at Tourcon, and it might be my last if I mess this up, so thank you guys for being here. So we're going to talk about a topic I really love. I was trying to find a picture of me as a child. I had the rat tail as like a full shaved head, and then the long tail. I didn't have a mullet, but we'll get into some other famous mullets in just a moment here. So let's talk about APIs. Let's talk about APIs. Google Trends has been tracking the decline in things like soap and wisdom, kind of depreciated models for APIs. And looking at more increase in things like JSON, arrestful services, and microservices. This all started, if you guys have the phone that's got the name of a fruit, and it's not a blackberry, it all started in 2007 with the invention of the iPhone. People still go back and watch us on YouTube, the hits keep going up. But this really started something really interesting, which is driving API utilization more and more. It's the app economy, the mobile app economy. And because mobile applications need a lightweight protocol, APIs are increasing massively. So if you look at the difference between a soap call and then XML RPC call versus a JSON API call, you see that there's a lot of efficiency in a JSON API call. The reason is, is because we need a way to have some kind of uniform communication methodology that's lightweight and fast, and it branches out kind of like a mullet to all the different applications in the back end, right? So this is why developers use them. This guy is my hero. So the company I work for sees a whole bunch of web traffic, and as of our last count, 83% of the hits, this is not per volume, but 83% of the hits on the Akamai platform are API calls, which I think is totally insane. But again, if you think about the majority of mobile traffic, that might explain a little bit of that. They're also being used heavily because they are how organizations are delivering and enriching user content and meeting kind of people where they want to be met, which makes them great to attack. They're also a front door and then obviously a back door into your data, which also makes them great to attack. So here are some examples. Somebody thought it would be cool to take control via an API of these Xiaomi pet feeders. Now somebody's cats got really fat. That's not a huge hack, right? But if you look at some other interesting cases, now I will tell you that this was actually not an API call, but this is an IoT device, and they're totally vulnerable. I don't have to go into that. But a Japanese hotel chain had robots kind of watching the beds of people, and these robots were really excited when they got hacked. So a researcher found a way to remotely access these cameras and microphones and record. So I already went over this. I'm going kind of fast. I was originally supposed to do lightning talk, and I made my presentation was super long, and now I'm making this even longer. So there's a lot of different expectations and challenges around APIs, right? So one is that for business innovation, people are using them. Most organizations aren't securing them properly. The ones that I talked to are having huge problems just creating rule sets around API calls and understanding that you have to inspect each call before it hits your web services. And there's also legitimate problems, which is authorization and authentication, maybe offloading that if you're a real big shop, and then quota enforcement and unpredictable load. APIs are extremely, extremely vulnerable to overutilization. So we'll go into some of the top threats that we've seen. So denial of service, this is like 2005, everyone's favorite topic. But denial of service attacks are happening a ton to API services in a couple different ways. So one, you can specially craft a request to create hash collisions, which basically makes the processing time on that web server go up extremely high, and the deserializer process is going to be overwhelmed and then fail. Similarly, you can craft a request that has a really deep nested structure, and this again, it's like kind of decrypting something and then there's something else encrypted and you have to decrypt, so it's kind of this multiple payload problem. So this is what a hash collision request looks like. It's basically again trying to overwhelm the deserializer, and this is what happens on the decode time when you look at those adding up over time. Now if you were to have a way to perform a validation of what a request is supposed to look like, you might see these hash collisions coming to an API endpoint that for all intents and purposes should never be seeing something of that format. So if you could block it that way, that would be ideal. This is still things that are in development in some cases. And then looking at maximum nesting depth, this is like saying, if I get a letter that's got like 15 letters inside of it, you know, is this like, you know, some kind of like a ricin attack or something? Do not open that many letters. It's a waste of time. So from a red team perspective, attackers are looking at your typical application security vulnerabilities, the same types of attacks that were against HTML web pages, SQL injection, you know, basically anything remote inclusion, it all works through an API service. And the funny thing is, most security teams, when they see API traffic, they say, oh, that's machine to machine traffic, or that's secured via the API key that we have assigned to our mobile app, and they kind of ignore it, leaving most problems to be identified by the impact they have on the systems that they're communicating with. Input validation, this is, I mean, we could have a whole talk on that. Broken and weak authentication, again, I'll talk about that a little bit with API keys and Java web tokens, and then deserialization process. So this is, this was kind of a snapshot of like the top attacks we saw against APIs. SQL injection, again, I think it turned like 30 years old this past quarter, which is kind of interesting. So it's obviously still a favorite. A really quick example here, this is a customer who's in AWS, and this call is basically looking at a credit card balance sheet. And then we saw an attacker do the good old SQL injection, Tom Fullery, to get access to more than one person's account balance. So Java web tokens, the place that we see these being misused or improperly used is that people think that a Java web token and whatever documentation you make inside of there is secure most of the time, it's just like encoded. So when you do a decode, you see the person's name, you see an account number, and then a base 64 hash of their password, which is probably not. Sometimes you see like username colon password, which is no bueno, so don't do that. The other thing is that the deserializer portion of an API call happens before the application logic kicks in. So if a request is sent, and it's hitting a vulnerable deserializer, and I'll go into a Jakarta example in just a second. The malicious code that runs is happening before that request is passed back to the application. So if you think about from a security perspective, you're like, I'm going to secure the application even more and like send my developers to security training. Some of this happens before that application logic even kicks in. Here's an example of a remote code execution we saw against a vulnerable deserializer. You'll see here this is someone's calling the FTW.sh, you know, for the win, I assume that that is what that means. This problem has become so pervasive that OASP has added insecure deserialization to their OASP top ten, which means that most executives are going to say, can we protect against the OASP top ten? That's like all they ever ask for some reason. But if you're able to provide a positive security model that says this is what a request is supposed to look like, and maybe even a negative security model that says if we see any kind of like, you know, breakouts here calling a different string, a URL via a different string, that might be suspicious. And if you could block that, that would be ideal. There was a Jakarta multipart parser vulnerability in Apache struts that we saw a lot of activity on. And this is another example here of someone using a deserialization process to make a curl request to this sh file. And when we looked at this sh file, we were like, oh, what is this, this K worker process? And then we got, you know, some other things have been redacted. We found out that it was a Bitcoin mining script. So basically it ran. And because of vulnerable deserializer, it was, you know, active code on the machine. And this helped golem and his buddies mine Bitcoin for fun. So the other big problem is process and logic flaws. This is one of my favorite logic flaws. I tell my wife that, you know, this is why my shirts keep getting tighter. I'm eating salad, so it couldn't be that. But we saw a particular case. We saw a particular case where we had a customer come to us and they're like, yeah, so someone's ordering a product. And then they don't come pick it up. And it's a, it's a, it's not a perishable. It's like a soda or something. And we thought that was weird. And we see that several times per day at a whole bunch of our different stores. And we're starting to, we're trying to figure out, are they validating credit cards? Maybe they're validating a credit card payment. And so they're buying something small that makes total sense. But then we started to look and we saw this thing here. Order number equals 14586. And we're like, okay, let's start tracking these a little bit. And then we saw the next order was order number 14587. And we're like, hmm, this is quite interesting. So effectively what someone was doing, either a competitor or maybe someone trying to, you know, build a better burger. They were hitting each of the branch locations in different markets and they could see how many orders were placed throughout the day and get a total order count of like what store was the most popular. This was not either of these stores. But have you tried the impossible burger at Burger King? It's awesome. Okay. So maybe people are trading stocks finding out that, hey, wow, all of these locations, their orders have dropped significantly because, you know, there's a new burger in town, right? That could be something. So we're trying to help them with that problem. Some of it is, you know, some of them are not just technology problems. Some of them are people problems. That's obviously a big issue. Credential abuse. Raise your hand if your company has seen credential abuse in the past six months. Okay. The rest of you are liars. You all have. Okay. So we took like a four-month sample. 34 million unique credentials were tried against 98 different customers. 67% of the total logins during that time were fraudulent, which I think is pretty scary. The other thing that's scary is if you're in financial services, attackers are targeting authentication methodologies that are built for, like OFX is built for the financial services industry, ultimately. And this is a little graph here of 21 million login attempts in just a few day period. You'll see the time frame there. 19 million of them were against OFX V1 endpoints and then 2.2 million were against OFX V2 endpoints. Now, what I think is really interesting, 37% of the requests to V1 endpoints were failures versus 0.2% against V2 were failures. So what that means is there might be something wrong with V2 that attackers are exploiting. The other thing we're seeing is something called cypher stunting where attackers are randomizing TLS fingerprints over time. And they do that, one, to avoid detection, two, to confuse any kind of automated rule sets, things that are kicking in. You'll see that there's 95% of the time during this time frame, which I think was the same time frame of that 10 days or so. We're seeing this increase massively, which I think is a big problem. So where is this all coming from? Credential abuse is really a problem because there are other username and password databases that get breached. People are like storing these in the clear, which is obviously an issue. If you go to any kind of forum, you can do a search for combo lists, which is like a huge text file that says username colon password or username comma password. Super fun to see what some people's passwords are. And where this happens in the, I was going to like hashtag credential abuse life cycle, but this is kind of the anatomy of credential abuse. They find stolen credentials, they import them, and they create a botnet that is going to make these requests from a bunch of different IP addresses. And these are full, this is not like a DDoS attack. These are full like layer seven connections where they're looking like a real user. They're copying the header structure of whatever browser type they're pretending to be. And then the next step would be if they get a successful login, they then resell that list for more money. And then the end goal is to do something with that login, right? So Sniper is one of the attack tools. This is, well, I'm doing pretty good on time. This is awesome. Sniper is one of the attack tools. It's an automated tool that really you don't have to know what you're doing to use it. The config files are pointing to login endpoints, and most of the time they're API login endpoints. And basically you load up the file, the config is specific for each customer you're attacking or each account you're attacking, and then you press go and then all of a sudden you get this list spit out that says these have a green check mark, they're successful. You can't get much easier than that. So this is why all of you that were lying about not getting attacked, why you're getting attacked. The other thing that's interesting is you say, well, just block the IPs, man, let's go old school. You know, I got my fat piper, my big IP, and I'm just going to, you know, nothing against those companies. I don't even know if they're still around, but single use IP addresses. So IP addresses that are only used once in a 24-hour time frame, that's increasing massively. That's typically because, I'm not going to read through all this, but it's typically because attackers are taking control of vulnerable IoT devices that live within your home IP range. And now they have like a plethora of like thermostats and pet feeders and, you know, toasters and stuff that they use. And they basically create a proxy, they're all running, you know, embedded Linux, so they create a SOX proxy. And then those same attack tools, they're tunneled through each of these IoT devices. And again, this is not like a network-based DDoS. They're making full HTTPS requests against, so they're taking things like Century MBA, and they're automating those requests to look like they're coming from a bunch of different random home IP addresses. So of the campaign analysis, we saw that API entry points are attacked four times more often. That's because a lot of people are saying, well, you know, it's the mobile app. You know, God forbid I, you know, make a resume writing opportunity and like implement a WAF rule that blocks all of our mobile traffic, probably, you know, get kicked to the curb pretty quick. The other thing is that because APIs are built for automation, attackers are using automation to their advantage. So one of the examples is AIO bots. So these are all in one bots. And effectively, these are kind of like a mix between account checking and validation tools, inventory sniping and cart checkout tools, payments and validation tools, and then like a full shipping profile and everything. So one of them I thought was interesting is called Cyber AIO. And the tools actually called Cyber Sol. This is hitting like sneaker brands, high end apparel brands, etc. And the way it works, you can buy it for really cheap also. But the way it works is it has a configuration and it says, okay, this is what we're going to do. We're going to put in payment details, delivery details, you know, separate billing configurations so that, you know, if you're doing a batch and one of the payment cards gets declined, you can switch to another payment profile and you can even start to customize who it's going to be shipped to, depending on which card got paid, etc. It's quite, I think it's cool, but never say that in a client meeting. I'm like, this is so awesome how they're doing this. I mean, it'd be awesome if it wasn't happening to you. But when you look at the configs of these and you open them up, they're all like this dropdown for like what size and all this stuff. It's literally hitting the API resource of this client and it's giving you feedback. So like you choose your size, you choose what product, color, etc. So this problem is so pervasive that they've created many different tools. This is just for attacking customers on the Shopify platform. And again, you know, you guys know how it is in the hacking community. Like anyone who's already using this to hurt someone already knows about it. So I'm trying to tell people about it because I don't want you to be hurt. I care. But if you look at how these bot tasks are configured, they're all API based. This is from the CyberSoul manual. So for what it get looks like, for what a post looks like, they even have like customer service resources that say, you know, I'll get to that one in a second. They even have, you know, resources that say they've been tuning their rate controls a lot lower. So make sure that you dial that down even more so. This next one is not going to build right. I can already tell. But effectively within Shopify, they have a CAPTCHA queue. So when you get a CAPTCHA response, what happens in this would have been really funny. You know when you get a CAPTCHA response and it says click the, you know, click the boxes that have a street sign. And then there's like the little edge of one that has a little bit of a street sign. And you're like sweating because you don't know if you should click it or not. That's what this was supposed to be. I'm glad you left. So one of the options for Google ReCAPTCHA is to open up YouTube with an account. You log in again, lots of jokes here in the back. So when you open up YouTube and you start watching an actual stream, what happens is you get kind of like user liveliness tokens so that when you get a ReCAPTCHA, instead of getting the street sign, which is, you know, real problematic for automation, you get the I'm not a robot, you know, and then you can have that hot dog spinning around. Is that what it was? Or that was, that was something else. But you can basically automate that I'm not a robot push button super easily. So these are, you know, I am really impressed with the time here. These are just a couple of resources. It wouldn't be fair to talk about this without going for a blue team overview. Obviously one of the big issues that are really common is that API endpoints, if you have just say like a picture frame that logs in automatically and it pulls images versus an endpoint that has a login and it's a transactional web page, those are two different use cases and lots of times organizations, they build out things so fast and they've got so much kind of technical debt that those endpoints exist on the same URL effectively. So now it's very hard to separate and say, okay, we're going to do API key management. We're going to do some cores rule sets to make sure that you're not, you know, co-mingling these requests. It's very hard to separate those out. So in any case that you can, try to be sure that you're separating mobile logins from, you know, automated machine to machine logins for, you know, maybe social login capabilities. You want to separate those out from your normal logins. And if you guys want to learn more about API security, Pixie project from OWASP is really cool. And that is it. I am open to any questions if you have any. Yes. Thanks. So I do a lot of API work but I think I do it more granularly, like per client. And the thing I see the most often is like access control problems. I can't count the number of times I've looked at an API and the only access control checks is whether or not the user is logged in so you can access anybody else's resources. Yeah. Do you get to see that pretty often? Yeah, yeah, I do. So, you know, access control is a really big issue. It's funny because these same problems are the same. If you guys have ever worked with like WebGoat or any kind of like intentionally vulnerable websites, it's so common that you can log in as one user and then you're changing the URI and then you get access to another user's profile, et cetera. That's super common. There's a whole slew of things that have to be done to correct those problems. But if you're working in this space, you see how simply these things can be bypassed. The attackers are doing it all day long. And there's little to no intel feedback from the operator side, from the blue team side that it's happening. Unless there's some egregious outcome on a web server or with an account, you wouldn't know what's happening. It's so subtle. So, yes, the problem exists. All right. Thank you, Tony. All right. Thank you, everybody. We're going to be moving the Q&A out to the back patio. That was amazing. I can't believe you turned the lightning talk into a full 25-minute talk. Imagine if I would have had to stay within the... And you can follow me to head out to the back patio or follow Tony.