 Tommy here from Orange Systems. I'm joined by my bearded friends for an uber big incident Yeah, I saw it. We absolutely do not Are not laughing because we know Because we know a lot of these people have some really hard times ahead of them and things like that But we want to try and keep things at least somewhat light-hearted. We we've been there We know what it's like to do some incident response We've been there and things that suck when we have to deal with it. We feel bad for them So we're not here to make fun. We're here to do some lessons learned, but I want to start with a few caveats Context and caveats is we don't really know if this truly is an external threat actor pretending to be an outsider because you know Cue the ubiquity incident. We don't want another thing like that. So we're speculating here based on what we know We do know uber has confirmed they had an incident So they've addressed it and there's some external changes that were made to accounts like at their heck or one So yeah, someone in there was goofing around whether that person's external internal We're assuming it's external for now, but we're not making any hard statements there And also quick reminder, we don't know how much of this may or may not be made up This was an interesting side note to the lapses incident because when you have a bunch of kids doing it for the lulls and They breach something small to make it seem bigger. They may you know, doctor some photos But we have a few things that we think wow they I don't think these are doctored But nonetheless, we'll just throw that out There's a caveat but our goal here is ultimately talk about what we know based on all the external knowledge and boy Is it just pouring out over Twitter right now? We know a lot of people are waking up to a really bad day right now over there Yeah, and we want to talk about some lesson learned because if any of these things are true Well, I'm not sure or not. Maybe you have a password in a PowerShell thing We're hoping to remind you you shouldn't have a password. Yeah, we'll talk about like some lessons learned of things You could do better based on if any of this is true All right, I was want to share that screenshot because Immediately I didn't see this when I first noticed it, but I think Jason knows it right away It's gonna be kind of small to read on the screen here, but there's basically the downloading of log IDs the downloading of a few things they were trying to upload downloading putty like you Know those were downloads of what would probably be malware You know something like a LSAS dump capabilities or Kerberos in capability something off the shelf and you can see failed download Failed download because a defender and then plopping into an S1 console. We're in the top right It says IR team which if you think about that and put that in context Insider thread or compromised asset you have this massive, you know live compromised problem When a member of your IR team gets popped right like that is that could be very Implicitous so yeah, but yeah, a lot of this started with them taking over their slack That was among the first things because there's a lot more screenshots that I'll leave links to vx Underground Just dumping these all on Twitter right now So be linked down and below but they also trolled some of the employees employees thought it was a joke So these yeah, they're going to force those reactors popping in channels. Yeah at everyone and making comments So they played along with it and they're like log out and they're like, but this is someone fun We can talk to you. Yeah. Yeah. Yeah, this is I mean a lesson to be learned there is right like you really so many people I think are gonna try to run their eye hour in their slack soon. It's soon as compromised It's been a signal group, right? Out of band it looks like based on some of the other screenshots I saw it Uber did at least have a way to notify employees to stay out of slack Yeah, but I mean yeah, it's like telling if you tell people not to press the button They're gonna press the buttons I'm sure there are more employees. It's like that ever but that's a good call-up man Like we had an out of band comms methodology that was established from some of our systems But not our core systems and they were separable from the identity plane Right and so it was kind of have you have you know if I ask this one question of any usp or anybody watching You know, do you have a incident response plan that includes out of band contact for all your customers and out of band contact Methodologies for your structured internal teams to communicate on in a major event Yeah, and this is why I like signal even you know how actually we started our conversation and for sure And if I see any of your numbers change, I'm gonna really verbally talk to you and figure out Why did your security information change on signal? It's also the way is I had an employee I'm someone spearfished my new guy and said hey, it's Tom I got a new number and he was like did you get a number Tom? I was like standing. I'm like no But I mean the person wasn't even on signal. So he's like, and that's how we message So, you know having those solid things that aren't easily compromised such as an ad or a slack or something like that Where you can start impersonating because you think if they have a mass notification system The threat actor could have also used that to send a notification like please interact with this person You know do the opposite or whatever. So it's hard to know what's true in the middle of an incident Right and the other thing the nice thing about signal that people don't think about is the disappearing Messages feature is pretty key for things. You don't necessarily want to be discoverable. Yes Well, I kind of just do that by default. I find it just a good practice I don't need to read something from two weeks ago Yep, and if anything ever sensitive was sent it would have at least some degree of a impermanence, right? But but I wanted to jump back to kind of what we think has happened to maybe talk through some lessons learned Yes on that and so, you know We think there was an SE or social engineering attack of some sort right is at least less what the kid claimed If it is the kid right this maybe maybe maybe not threat actor to get then VPN access and then essentially Was able to just enumerate SMB shares as that user which now may potentially have been that IR lead or team member And then this is the other piece which Tom I know you had some takeaways for which is they found a script that had a password. Yeah for for what? Everything is a biotics server. I believe it was allowed them to get into yeah, yeah, which is a password Access management solution a PAM solution. There's some sort of automation. They're trying to do against it Or are they reusing service creds or like, you know, I have so many questions around Oh, if there's an ad back in did they curb roasting or they owning that ad domain that now has Extensibility into the different cloud identity platforms are they consuming cloud identity first or are they the source of truth for that cloud? I'd like It's it's it's there's so many questions in that space. Yeah, but yeah, and that's where I'm gonna throw this up here They've been interacting. Well, they allegedly have been interacting different security people and reporters because They started chatting with them and this was actually, you know, they're very responsive like this is how I did it which almost makes me You think it's an individual they just pivoted their way through all this Based on what we know and they basically I found this network share One of the power show scripts has a username and password for the admin of the Automatic PAM solution using this I was able to extract secrets for all services duo one log in ADSG suite, etc, etc An uber IP range WS yep, and this chat feels legit, right? Because the the shorthand used the s e dash dash dash there wasn't a lot of Elaboration it was very poignant to the point of exactly what it meant. I mean it was to me felt like somebody really just claiming their work I'm wrong, but it's not embellished enough to feel like a lie, right? No, right. It's just simple Here's what I did s e VPN pivot SMB shares and then, you know, voila, right? Not the next light of the situation but but just magic and that's one of the things you have to think about those and here's Here's what I don't know for sure one You know password rotation could have helped fix this because maybe this power show script was from a long time ago That's some other person I don't know how to solve the problem, but I know how I'll solve it now And there's nothing more permanent than a temporary solution of our show script works Here's what I'd say though Yeah I believe that if you think about any of this type of break glass type structured accounts that should be used once and then literally burned right one Time pad is said for a reason right? It's meant to be literally once and then we're done And so when we had our break glass policy I had a physical drive that held key pass databases that each of those passwords were distributed to a third person in the Password for the Bitlocker encrypted drive and so you think of that and when we had it We burned a password there was a four-person process to ensure that they actually burned that password that any use of it Then had extensibility auditing before you check in check out process like I'm just saying If I was an MSP at just under X revenue I want to say right now like broadcast that but you know that had a three-person security team It could do those things Just feel like there's some hygienic things that could be better for this not to exist or be valuable But don't underestimate the fact that uber was built is like a startup using the DevOps you start up culture, right? Which yeah, no, no, I see I'm not security first at all Right, it's yeah minimal viable product will pivot from there right this could be part of some MVP that is time pointed out You know these temporary solutions the problem last for a few years Yeah, they're moving on to the next problem. So I mean they're not While they have grown I would likely guess that their internal infrastructure looks a lot more like a startup than an enterprise Yeah, they're still on that pitch fast break fast move fast in there from 2009 I mean they're your massive company. So they should have kind of But you know, that's where you put the priorities So now that we know everything that did happen next thing is what could have been better? What could some lesson learns on I Might have some again, you know the Cisco owner talking about sounds like it might have some MFA fatigue there I mean somebody just yeah until you hit approve Yeah, we know they're using duo so probably they're using duo for that So it actually makes me think it's exactly like what happened in a Cisco If you're using duo, there's your MFA push and enough push notifications come through you're like shut this thing up But I'll take it back to an AAD centrism or octa-centrism with tongue-in-cheek But this this conversation around phyto tokens as an enforcement through CA is coming where we can do that now in a more granular fashion But just using things like strong authentication Even if you are doing authentication through push make sure to ask me to type in a number that I cannot know unless being also Additionally socially engineered and so that's my first takeaway and the other thing I'll run through a quick I know we're you know don't want to go too long on this I think I like to map things like this back to control so as we go through some of these I'd love to try to take a poke at yes what controls on CIS this kind of maps to so We already talked about the brake glass handling control five and the improvements inside of control five really Specify how you should increase your knowledge around handling these things now back to Jason's point I'm not gonna beat him up for this because I think having a healthy brake glass account takes some internal discipline and leadership from Met you know people inside so anything else we could do gents Don't store passwords and PowerShell accounts Yeah Yep, I mean just yeah, that's also control five so we can call that call that in the same. It's the game kitten. I suppose Yeah whole base access control and keeping that really locked down and restricted. That's just This is something I had a conversation We were me and Jason were just at Datto con and you know I'm talking to one of the vendors that we have no in respect and they were talking about how inconvenient it is But it's just life like there's not any thought to change it We were talking about updating a billing system He goes I do all the dev but I don't I'm not even access to it only when it's needed And that's why it took us this long to solve this billing problem. He goes and we I'm already reduced out of it We just kind of we're just talking security Yeah internally because obviously the lazy thing is leaving this particular person who does a lot of their back-end devops With it, but he goes I don't need access to any of that He goes I looked at it in separate my room I don't have a reason to if you look here both the center one screenshot and the AWS screenshot seem to be this Philly account Yeah, right. So this guy's an IR team member that seems to have I mean it makes sense He has access to the center one course maybe but I'd be might just I would argue that maybe just needs access to logs If he needs further access, he could get a granted later Yeah, he also seems to have pretty high levels of access inside of the AWS account too and I Seems to me like they're not locking down this guy's account to the level that he could be 100% that brings in and one of these challenges. I think is that CIS controls are meant to be iterative, right? It's one of my favorite set of controls But if you think about it shows some maturity when you do combos just like playing pool I can shoot the corner pocket and wobble in a little bit if I need two angles bounced It's gonna be a little more challenging right like that's kind of the point And so when you think about this this takes a control combos six For secure network design or six for authorization management 12 secure network design right and 13 for network monitoring defense So you have to combine all three of those to be able to do exactly what Tom and was talking about as the initial intro This of let's lock down network access lateralization What should we have access to is talk about our back Let's talk about implementing these things that go to even network lateralization like should I be able to do this? Should I be able to do these things now? The challenge I think you find is if it is that our team member the answer to a lot of those questions probably is yes But maybe not as his daily driver Maybe as an admin account that he only uses for those privileged access And so I think one of the things I'd heart come back to as you see very plainly This is the actual name of this human. So it's not quite likely that it's his admin account I think that's a big takeaway is separating admin roles and function accounts for admin access 100% for sure all the time Yeah, those are just those little things you don't think about is making sure everyone has it and Not password reusing those out of convenience Do we want to talk about they kind of commit here right because it seems like there's some caveats with that particular piece of software Yes, yes Jason has something to say on this a lot of things say about a lot of things means Looking through the reddit post, which I'm sure you'll link below It talks about how they basically were able to enumerate all the passwords inside a thiccotic Right and it looks to me based on looking at the product page for thiccotic the ability to do analytics around password issues or bulk password Stuff analytics is a paid add-on. Yeah, security's an upsell with them Yeah, that's unfortunate a lot of vendors. They don't they would try to lure you in with the low price Then they have all these add-ons. Oh you want the product to be secure? Oh, well, that's extra. Yeah Yeah, I very much feel very strongly that if you're selling a product that you store secrets in the ability to determine people are Misusing those secrets is a core feature or not a paid add-on, right? You know, we don't know if Uber didn't buy the paid add-on, but we just think it shouldn't be a paid add-on insert name of any Yeah, so we're misconstrued on that so I think I think if you look at that in my opinion that really hammers on some of the Control-6 right on improvements on authorization and kind of conversations, right? But the other piece that I think I don't want to miss on this either is really just that Training piece. I mean we've brought up five or six or seven tips that we all think might be improvements as a result of what we thought Has been thought through but like that's part of training people, right? That should be 14.2 you train your workforce members to recognize SC attacks, right? Like we're talking about here 14.3 train the workforce members on authentication best practices like staying up to date on modern protocols and using Fido by policy using Fido as an extensibility for more secure things having an admin key and a non admin key and those two have separate identity structures and like This kind of thought process takes training, right? I know it because I learn it and I try to learn it and I'm always still wrong because it moves too quick But yeah, I feel like a lot of people could do better with their training and not just think of it But I did the automated know-before thing or I did the whatever it may be it's teach them Yeah teach people I think duo if they were using duo I think duo is smart enough that when you deny it enough it like locks essentially you can set a lockout timing Yeah, sure a clipping level we should teach people to not like don't hit accept So it goes away just keep hitting deny if you hit deny five times Then it's gonna lock the account out and somebody's gonna have to fix stage advice. Yeah. Yeah I'll also kind of final wrapping things like when they change stuff the intent of threat actor I think it's just to make a lot of noise because why would you you're not ransomware you're not posting anything about money There's not a single Bitcoin mentioned in here. Sure. They changed your you know hacker one uber has been hacked Even hacker and this hacker one account has been also ordered the poster is fully No, that's funny, but not all that I'm true potentially I won't throw that shade But I will say, you know, this does feel like me 20 years ago, right? If I had access to those capabilities and the ability to flex my my chest muscles about it And I was 18. I promise you 18 year old Matt Lee would have done that and so I feel and maybe I'll be wrong I feel like this is at that kid just yeah, genuinely genuinely And you know what might play out In fact, I'll take a speculation on this the Department of Justice what three four months ago basically said they decline to To prosecute people that are good trust hackers And you might find this could be an argument and their attorney uses if you do I want 10% Yeah Well, the question is if he doesn't really release anything other than these screenshots then he's not really other than being a Pain in the ass for them. He's not really Causing that much harm right like you don't see If you don't see all what two petabytes or however many If we don't see that release then, you know, this very easily could be the case where this guy Breached got really loud about it because they clearly weren't paying attention to him And may just be oh, I don't know. We'll see But That's kind of all we know now if more goes to follow it on Twitter follow that red post is good So we'll I'll leave a lot of links. This is something that you know We spent some time reading this morning wanted to talk about a few lessons learned and speculate Yeah, we feel bad for the instant responders because even if they're not dumping data and hurting people They still have to clean it all up. So nobody's getting a day off for quite a while now I really hope my ask uber RCA give us a good document. I want to actually happen here. Yeah Aller from it. Well, you trigger me on this Jason because when you know, what were you asking earlier before the show Tom? And it will wrap up after that is you said listen What what could people do about like the transparency of this like well? If we say that this didn't really happen people say we're lying and my answer is you just need to be humble and transparent Yeah, the more and more humility and transparency around this and the more we become accepting of at least trying to learn from those Mistakes and correct and be better then I feel like we'll have better chances of succeeding in the future And if we keep stigmatizing a lot of this and in those things so yeah Hopefully it's taken in the intent to learn and to push forward as a community most of my learning is by doing something dumb and realizing I shouldn't do that again. So If I can learn from other people's dumps then man, that's way better than doing it myself Absolutely, that's definitely the way to learn Maybe this is an era we're coming back to because I keep like you said that the script kiddies in our early 2000s and what we were like when you're like, I mean, I was never malicious if I got into something I was messing with your printer hands down. I was my go-to I think it was a you Jason who's who changed the printer setting a long time ago. It allegedly someplace to say insert coin Yeah Radiation week. Yeah Or or just set the printer because there was a PCL code where you could just set the text on the display It always said PC load letter even The answer coins my favorite but radiation lake and PC load letter good PC load letters has a special special place in my heart for the aforementioned moving forward It's how we learned that's how we get to where we are not being malicious We always want to learn a lesson or occasionally show someone a lesson But lots of lessons to be learned. It'll be an update on this. We'll get together and do it We also are going to do some more about from how I would hack you to how you would defend against us type things So look for us in the future like and subscribe and all that fun stuff. Thanks everyone