 There we go. Thank you gentlemen. All right. Step one accomplished. Got slides to display. So throughout my career I've done a lot of red teaming. I'm a former red teamer, former pentester. I've done everything from your standard fishing, vishing to physical security assessments at one point. A team I was a part of, we recreated the original Trojan horse and we fedexed a person, a fellow pentester inside a facility. He did make it through security, popped out and he did not do as the original Trojan horse folks did and slayed a bunch of people. But he did let us in and he also dropped USBs and plugged in things everywhere he could. And as part of a lot of that work I've also read a lot of reports that have come out of social engineering assessments. And one thing that's really stuck out to me both from actually doing the work and my own report writing to reading a lot of reports has been that there's a huge emphasis on fixing or training the human layer of this revised OSI stack that we're all working with. And by fixing or training we really mean making non-security personnel and even security personnel in some cases think more like security people. Look at the world through a security lens. And one of the things that I've really struggled with now that I've moved away from the consulting assessment side of things and actually owning a security program is that it doesn't actually work in practice how we would like it to. And so this talk is a summary of things I've been experimenting, ideas I've been tinkering with for a while and how I think we can challenge ourselves both in how we test what we look at what we talk about to make social engineering remediation really scale. So really quickly here is the agenda that we're going to cover today. So we're going to step one really quickly focus on or touch on the state of social engineering today. The widening gap that's occurring or that is happening. Start to look at themes. This is where we'll inject the central concept or premise of the talk. And then talk about putting all of those ideas into practice. Both as a tester and as a practitioner owning a program. And so before I go any further, this is me in a nutshell. I'm DC based. I run the trust and security team at a company called NUNA. We're a healthcare analytics company based out of San Francisco. Not really important to all of this just in to say that because we are a healthcare company we see a lot of crazy stuff come at us in the form of social engineering attacks. We've seen the spoofed emails. We've seen compromised emails then being used against us. We've had all the whaling and the financial based attacks coming after all of our funding. Things like that. When I was, I consulted for the beginning part of my career most recently at a company called Sigital if anyone's familiar with it. And there I ran red team operations and red team services. And I am, as Kaz mentioned, a huge Batman fan. This is me. I do own a full Batman suit in addition to tattoos and everything. Only one of these was actually on Halloween. Why not? All right. So this is the typical situation. Company is trying to, you know, bolster its security posture. It engages some kind of third party security firm. They come in. They do some testing. Social engineering testing. Maybe it's directed at social engineering or maybe it's just part of a broader scope. And through that social engineering testing, lo and behold, they find that people are clicking on links. They're giving up credentials. They're giving up sensitive information in some form or fashion. They're giving up a foothold. And when they call them up on the phone, as we saw a little bit during the SE competition here, people give up information when they're sweet talked a little bit. Or if there isn't something stopping them from giving up that information. Very, very basic. And what that ends up leading to is a report that focuses probably pretty squarely on awareness training. So we need to start looking at training those people to not click on links, to not give up their passwords to websites, things like that. And we've gotten really, really creative around how we how we educate people through games, through video content, through little social nudges at NUNA. We run this thing called or this little program we call the Human Firewall Award. And on a monthly basis, we have this really cheesy little Grammy-style trophy that says NUNA Human Firewall on it. We give it to somebody on a monthly basis to kind of celebrate the fact that they were the best fishing report for that month or they caught the craziest thing for that month. So there's a lot of ways that we can bolster awareness. But it's expensive. A lot of it is anyways. It's expensive in terms of people's time and investment relative to the return on investment that you end up getting in terms of everything that they take away from the training and apply on a day and day basis. And then what a lot of these social engineering assessments end up pointing to, or at least some that I've read in my role now, is that people are constantly clicking on things. They're constantly falling for stuff. And because of that, we have to assume that we're all breached already. Everything is already over. And so instead we have to shift to, you know, prevention is dead. We have to focus on detect faster, respond better. I don't actually believe that's true for anyone that does. I'm sorry. We can duke it out after this. Let me have my moment of fame here. So during that tooling arms race, you end up with more and more tools in the form of capabilities being injected into a program. And all of those come with increased costs. And we're kind of shooting blind. We're stumbling through the woods in the dark, bumping into trees. And every time we come up against a tree, we have we introduce a new tool, cut it down, get it out of our way. And thanks to DBIR and their fantastic Verizon's DBIR, that is, and their fantastic graphing skills, making things that are really easy to understand. We have this graph right here, which is a little bit of an eyesore. But what I want people to focus on is this little tiny middle section here, these, the yellow and green lines. And what those, what those lines tell us on this graph is that over the last six years or so, you see social engineering vet vectors and malware based vectors having an increasingly prevalent or attackers are using them more and more and more as a foothold into a data breach of some form. They are the initial cause of, cause of exploitation or cause of breach. So if we're spending more money on all of these things, yet they continue to go up, something is not actually working. We need to rethink the way that we are actually doing our testing or the way that we're approaching remediation. Because we know we can find this stuff. We know we're very good at that. But we clearly are not very good at actually getting our point across at scale. In, in program environments where, you know, we have a lot of metrics. We have very frequent touch points. You can absolutely see, you know, people's susceptibility trending downwards. And that's amazing. But I don't think that that's actually working fast enough or broadly enough in order to really, truly make a difference at scale. It's definitely, it definitely helps and we should continue doing it. But I think we need to do even more. So the status quo. So right now we're investing in training. We're investing in, you know, bolstering that, that individual or that team of individuals awareness about social engineering attacks. And as a, as a CISO or, you know, somebody leading a security program, I have to ask myself a few key questions. Such as what happens when that person who I've invested all of this time in, leaves the company. They take that investment with them. They no longer are contributing to our resilient security posture. Plus, I don't know how, how this applies to any of you, but any training that you've gone through on day one or, you know, your annual security training or any annual training, I don't know how much of that you actually remember. Outside of the security stuff, because me being a security person, I don't really remember much of it. And I know this is, this is true for a lot of folks. And then bigger than that, how do people behave under stress, especially when they're in a role that's geared around helping people that are interacting with them? IT help desk, people on, like a password, like a customer service role, things like that. They're incentivized and there are metrics behind how they are able to assist, not necessarily how secure they are. Which is another, another slight issue. So this graph, a little bit easier to understand, this came out of some research that was done on retention rates and enterprise training. Not necessarily for security specifically, but training programs in general. And what this graph tells us is that there is a huge drop off on the amount of information that's retained and the days after the training that the, the individual retains that information. And really effective awareness programs are attempting to shift that line up into the right. But unless we can get that, that line to just be straight across indefinitely, then there's always going to be a gap. Everything over here is where adversaries are going to take advantage of an opening and continue to leverage those openings to get into our environments. And so you can liken this to trying to hunt grizzly bears with bird shop. You're likely to just make a small impact. It's not going to be complete. You might have to hit them a million times before they bleed out and you can actually win that, that particular interaction. I don't know about you, but I would not want to hunt or come up against a grizzly with nothing but bird shop. And one of the things that, that is interesting about security, not, you know, outside of a lot of enterprise training programs is that our field is evolving so, so quickly. And so there's always more things that we're trying to pile onto the awareness training plate. And if we're always piling more stuff on and they're not retaining the stuff that we, that we're already trying to emphasize, then it's just going to get harder and harder and harder and harder. And then one thing that I've really challenged myself on is why is it that we as security professionals, we are expecting other people to think like us, but we are not stopping to think about how they actually go through a system or interact with a process or a technology, things like that. So we have to, we have to put on their user experience hat and try to think like them and build better things. And so that is ultimately the core of the theme based approach that I'm going to talk through today. So there's a few things that I want people to, that I hope people can accept it not necessarily as absolutes, but as probable things going through this. So the first being people are always going to be vulnerable as long as they're making a decision in some context. As long as there is a discretionary series of actions that occur, people are susceptible to potentially making the wrong one, especially when they're coerced in a certain direction. Coerced towards, you know, maybe they're receiving a phone call from somebody claiming to be an executive director. And that person needs their password reset because they're about to go into a critical meeting. You know, that person is likely to fall for that attack, give up the password or reset the password for the individual. The second is the person is not necessarily to blame. However, the role that they are playing in the organization is to blame. You could liken this to like child product safety rules or product safety features for to protect children from, you know, pulling a thing up, like pulling a pill top off and swallowing a bunch of pills or screwing up with a toy or something and hurting themselves. We're designing roles where people are susceptible and able to actually blow their own foot off and make a mistake. And then third is that training is not going to stick 100% of the time for 100% of the days in which they're operating in that role. There's always going to be a little bit of a gap no matter how good that we are at training. And you know, for anyone who's familiar with the security engineering space or we're trying to train developers up and get them to, you know, know about things like cross-site scripting and SQL injection. We've seen this a million times over where, you know, we're expecting them to keep up on all of the things that we talk about at security conferences like this, but at the same time they have to keep up with all of the things that make them good at their jobs, put all that together and apply it on a day and day basis. And it's a lot to remember. So if we don't, if they don't have guardrails in the form of security controls, it's easier to make mistakes. So I want to, I want to describe or walk through this process in the form of an attack tree. A very simple, easy to understand attack tree. And consider that an attacker wants to start by taking over an account in some capacity. They might brute force credentials, they might choose to go down the fishing ground, probably a little bit easier. If they go down the fishing ground they may have a few options available to them. They might want to steal credentials. They might want to achieve some kind of RCE with that user, whether it's an attachment, an nefarious attachment of sorts. A, maybe it's a link that they browse to that then allows them to, like a beef, beef infected web page. Or maybe they just lead them to a page that is spoofed in some form or fashion and get them to give up their credentials. And then from there, they can do other things. Maybe they can replay those credentials. Maybe they can pivot. Maybe they can dig in like bed bugs into your environment. Whatever it happens to be, they have some foothold at that point. Pretty easy to understand. So you might ask yourself, thinking through this very, very simple attack tree, a few things, and this is where we start to get into the themes, is why in the first place are credentials able to be replayed? Can we find a way to prevent that malicious code from actually running? Is there a way that we can stop that content from hitting the inbox in the first place? Or if they send that to maybe 15 people and it's detected once, can we really, really quickly respond and remove it from the other 14 recipients? Find who has received it, remove it before the impact gets even worse and you have some kind of epidemic on your hands. So those are the kind of questions that we're going to start to tease out as part of this theme-based approach. And what I want to focus here on is systemic defenses. So finding ways that within an attack tree we can inject control or detection and response capabilities into that flow. And by doing that we can find ways to break out training instead of having like one time coming at you training sessions to break up training into smaller social nudges or smaller technology based nudges into that same workflow. That way the user is getting smaller, more efficient and effective bits and bites of training and hopefully allowing them to kind of interrupt their workflow, think about something and then move on. And you could- this is not a perfect example but it's an example nonetheless. You could think about Google Chrome's user experience around insecure websites, potentially malicious websites. If you browse to something that's not using the HSTS header let's say, they don't allow you to go to it. You know you'll get the certificate warning, it'll prevent- present you with some educational information and it doesn't allow you to go further. If that site is a little bit better they're using that but their certs are out of date or something like that, it's self-signed. You'll still get the same interrupt in user experience. They'll try to educate you about something and then let you proceed if you decide to click through. And it gets a little bit less egregious, the better the site is in. The better state that the site is in. So maybe it's just a little, you know, red lock or something or it's missing a lock in the- in the browser bar. Something to that effect. Alright and that's the big thing here. It's finding ways to use control within a workflow to make things more effective. So you can achieve benefits of scale. And so thinking about this, we'll do an attack and defense tree and that same very, very simple attack tree. We're going to do it with the same attack methods here. So if I wanted to lock that brute force credentials path off of the attack tree I might use very simple controls like locking them out or multi-factor authentication. This has all been done great. We've kind of stopped that or made that much more difficult. If I want to start to make fishing a little bit harder I might have a layer in there where I narrow the attack surface a little bit through some specifically placed controls using things like DMARC or DKIM or better spam filtering something like that. And we can keep moving on through. And maybe we have another layer relative to where the attacker is at in that stage. You can use things like application whitelisting or browser plug-ins or something that you roll out across the enterprise. These are things that you could do on an enterprise ready basis. And then moving on through to the end of the attack tree. There's more stuff that you can layer in. And ultimately what we're trying to do here is either make it harder to traverse a particular attack tree or path down an attack tree or find ways that you can remove branches to that tree altogether. And if you can remove branches of that tree then you can narrow the amount of the scope and the focus of that particular training and ultimately the user's focus. They have to remember a little bit less and so your retention rate might be a little bit better. Or maybe you just make it a little bit harder for that particular capability to or that particular path to be executed against. And ultimately if you have confidence as somebody managing these controls at an enterprise scale, if you have confidence in the controls that you're layering out here then you can start to figure out where you might want to put your training. Like for instance if you're taking advantage of that application whitelist like on the left side of this particular attack tree and the right side is a little bit more on the atrocious side then maybe you focus your training over there on where exactly people are giving up their credentials. Something like that. So now switching gears into putting all of this stuff into practice. First things first is you have to be mindful of the user experience. If you create or inject a lot of high friction touch points or gates that your user base has to you know kind of jump over then ultimately they're probably going to find ways to route around you or they're just going to be in a perpetual state of being pissed off at your security team. Or if you're a tester and you're recommending things that are very very high friction or take a lot of time then you're going to be putting your client in a potentially bad position where they're going to be the ones who are whose users are pissed off at them. So be very mindful of the user experience when you're making these kind of recommendations. And the second thing is your particular context is absolutely paramount to consider. So you know we could sit up here and talk around like how you create a secure help desk workflow. How you prevent against, prevent against password reset processes from being abused. Things like that. But every enterprise or every organization is going to have a unique way of doing this. Very likely. And so you have to be willing to break down these particular workflows for your organization and look at how an attacker is going to step through them within your org, within your context, within your process, your technology. Things like that. Alright. So as a tester we'll focus on this from a few different perspectives. First being from that tester consultant perspective. So we mentioned this at the beginning but a lot of times when we find those issues we find that people are clicking on links. People are downloading attachments and running them. People are doing any number of things that are ultimately going to degrade the resiliency or the security posture of that organization. We recommend controls that focus on the human layer. We recommend that they, you know, go through that training, go through that, you know, go through, you know, like recertification or whatever, whatever the process is. And instead I want to challenge folks to break down, think about issues from a root cause analysis perspective. And think about within the context of whatever it is you found, what is the root cause that actually led to that human being able to do what they did. Whether it's give up credentials, you know, whether it's give up sensitive information, you know, open an attachment, give up credentials on a webpage, whatever it happens to be, what is it that allowed them to do that? And make your recommendations center around those things. They're much more tangible, much more, much more appropriate at scale. And so you can, you could consider this from, from the perspective of going through a few sample social engineering engagements. So these are all real. I have lived and breathed these myself, but let's say you're doing a physical security assessment and you walk in, you smooth, you sweet talks in people and you get them to plug something in and you get it to run, you know, a sample benign application. And then you come back and you say, well, I was able to, I was able to launch that application, that benign application. Therefore, I could launch something much more de-farious. Finding. You know, you write it up winning as social engineering. So in this particular context, you might report it as, you know, why in the world was a USB device able to get plugged in in the first place? In this particular context, I was in a bank posing as IT, like IT audit support. And in a bank, you know, I've been in banking environments where sometimes they don't even allow USB drives, especially not, you know, just like why was, why was there not a call up to, you know, to whoever I said I was there working with to validate my identity before I was able to proceed anywhere. There's a few process or technical things that you could do first to make this a little bit harder. So this is another one. This was also with a bank that I worked with. Doing a vishing test of a call center. Get some, some background noise playing like airport or something like that. Say you're, you know, high ranking executive so and so, you're trying to, you're trying to reset your password, get onto the VPN, whatever it is, and you're having, you're running into issues. You've done your research, you, you know about some of the, you know about the individual you're impersonating's background information that is likely to be used for security questions, et cetera, et cetera. You get somebody on the phone because it is a call center. They're there to help you and you start sweet talking. You start using vague responses. Like, well, you know, I, I don't, I don't remember exactly what answer I used in the, what street did you grow up on? You know, I was trying to throw people off, you know, maybe I, maybe I use this one, maybe I use that one, something like that. And if a call, if a call center person, the person working at the call center, if they can see those answers, they're much more likely or that it is possible that they could make a decision in that sequence of actions to let you have it, to take it easy on you that day because you're going through a bad day, they are incentivized to help you versus something where they have to enter the information you provide them. They enter it into a system, it comes back and says it's right or it's wrong, it allows them to proceed, something like that. Almost an invisible introduction in workflow for something that cannot be gained necessarily, cannot be sweet talked or smooth talked. You know, you could, you can inject a lockout on that person's account through that same, through that same process, you could have escalation points in place, things like that. So, when you're writing those reports, again, you have to focus on, or I would challenge folks to focus on the underlying set of circumstances, be it technology issues, process issues, whatever, that allowed that thing to happen. And instead, propose technology process human oriented system components that could prevent or make that a little bit harder to carry out in the future. Yeah, so a lot of, I won't, I won't read through all of those things, but it's basically drilled onto the root cause, recommend around that. So thinking about tooling, so as, as if you're working on the practitioner, the practitioner side, you know, there's a lot of, you'll, and you come to a conference like this, you'll have a lot of people begging you to spend money with them. There's a lot of tools out there if anyone's ever traversed the, the RSA or Black Hat Expo floors. There's a lot of things going on, and those vendors will of course tell you that they have the perfect solution to all of your problems. They have your silver bullet in mind. So if you were to start, if you were to take that same attack tree, attack and defense tree approach where you're layering controls within a particular workflow that might be social engineering relevant, what you can do from a from a tooling perspective is figure out, all right, vendor A is coming to me. They are providing me this particular capability. It fits exactly within my attack tree in this way. And ultimately this will help you reduce some of that vendor fud. It'll help you figure out exactly where it's protecting you, where it's not protecting you. And you can figure out, you can kind of bucket it into these control, detect, response categories. Maybe you don't want a program that's solely or very heavily geared towards detection and response, because detection is always going to get harder, response is always going to get harder, and that almost inherently puts you in a position where you're fighting a widening gap, whereas prevention can help you keep that gap, keep the possibility of the gap narrow to begin with. All right. So focusing on some next steps, because I always like to end my presentations on some practical things that people can do. So first things first, I would say, figure out what your particular context is. Figure out what social engineering scenarios are most relevant to you. Maybe it's, maybe you're very concerned about phishing. Maybe you're very concerned about the types of scenarios that we were stress testing in the competition leading up to this. The phishing, the calls against help desks, calls against employee directories, things to that effect. Figure out whatever it is that's important to you. Build an attack tree against that particular scenario at first. Don't try to do it for every single thing that's in your organization. You'll be overwhelmed. You won't know where to start. And really trying to solve every single problem at once is just, you know, you're going to be fraught with disaster. So Rome wasn't built in a day. Complete security postures are not also built in a day. So pick one, dive into it, analyze it, break it down. And within that attack and defense tree, figure out where you can potentially introduce improvements through that workflow in technology, process, or new systems. And figure out where you can start to layer them. You don't have to, again, solve an entire attack tree at once. You can make small, iterative improvements. If one particular branch through that tree looks like it is, or based on your historical data, is very likely to get taken advantage of, or you know it's been taken advantage of zillions of times in the past, then focus there first, try to make that a little bit better. And then maybe you shift over to something else that is higher priority. But find ways that you can exert the Pareto principle, the 80 percent of the benefit for 20 percent of the cost, or 20 percent of the investment onto that attack and defense tree. So you're making small investments for big gain and keep going through that process. And then ultimately, I would encourage folks to try out some kind of A-B testing. Try out some way to experiment with this. So you're not always going to get it right. Sometimes you might screw up the user experience. And that's okay. You can always revert, no decisions have to be final and permanence, but find ways that you can make small iterative improvements. Test them. If they don't work, change them. But ultimately, the more and more you're striving towards making these little dents in your security posture or little dents in your workflows, you're going to end up coming out on top. Or at least in a much better position. So that is all the content I had. So I wanted to leave a little time for Q and A. So thank you very much. And I will pause and leave open for any questions. More technical science and things or goals. How do you handle companies that are more metric-driven and focus on like those support folks that are there to help? Sure. How do you get them to change the mentality around when your metric isn't going to work? In that type of environment, with that new mentality? The support metric, that is. Yes. So I would say the question was just a repeat for everyone, is how in a more technology-focused approach, how do you get a function, a business function like a customer support representative who is very heavily incentivized towards helping? How do you change the kind of flip the script on them to not impact the way that they're evaluated and still tackle this? So I would say there, you don't necessarily have to focus on a technology approach, but maybe it's a process-focused or maybe you can change the KPI entirely. So if they're incentivized purely on how many people they help, and that's it, there might be a more efficient way to write that KPI and maybe it's the amount of people that you help and let's say like you can let's say you follow a validation process or something like that. Like the amount of times you've followed the exact process or not had to have an escalation or something like that, there might be ways to either change the KPI or there might be slight changes in process, like the way that they validate, the way that they, the things that they check of the user, something like that where you're still letting them do their job and you're incentivizing them to do their job, they just do it slightly differently. Because a lot of these things are just process issues, at least like that help desk issue that I cited. So I was going after and trying to get them to reset a VPN profile for this bank and like literally I just, I provided a few phony answers on security questions, they let me onto the VPN and like from there it was all SSO driven so I could get onto whatever that person had access to and it's like, you know, if a very small changing process would have stopped me from doing that. So, yeah. Cool. Any other questions? Yeah. Like keeping that line moving up and to the right as much as you can. So sending a bunch of phishing emails you mean? Yep. Yep. And I don't know about you guys but, you know, if my organization could basically not trust email, then that's really tough, a lot of phishing. So, yes. So that is true. And so the comment up here was in this particular program we didn't do a lot of retraining for those that fell for the attack. So the way that we ended up approaching this is we do really frequent and we, you know, we of course vary our social engineering tests like phishing tests on a regular basis. And instead of forcing people into a negative situation where they have a negative correlation with the experiences that they have so they fell for it, whatever, like we'll actually reward some people if they fall for the social engineering test and every time that they fall for it, we have a very small like mini touch point in the form of like a retrospective on like how they fell for it, what they fell for, something like that instead of going through a, you know, big bolt-on awareness test and depending on how they respond to that they may end up, they may end up being candidates for the like the human firewall award. And going through it like that where we give them a positive association with interacting with our security team as well as receiving training we found a lot better just retention rates as well as just people feel better about engaging with us. They feel better about reporting because there's no shame in doing so, things like that. So instead of thinking about it as training we think about it you know, more as like these small little touch points and then we'll encourage them to like a lot, I'm very big on trying to find proxies to do because you know, I don't have the amount of resources to just go do everyone's jobs and like make everything secure so if I can find somebody else to go you know, be a champion for me, be a proxy then I'm going to go and do that and that will also kind of boost their profile it boosts my mission, things like that. So yeah. So just to like put a like fine point on the original question in terms of like regular like trying to keep that line as high high up and as far right as possible you know, we we rely really heavily on very small touch points and as like using proxies as much as possible because if people are hearing it not from the security team then you know they just have a better better experience and I found that like when people feel better about security they're they're ultimately just going like they're going to interact with us more and the more they interact with us the better we end up being like so. So the question just to repeat was oftentimes C-suite people can be a little pushy. I don't know if anyone has ever experienced a CEO who demands something immediately and they may look at the security team and say you belong to me in some form or fashion I control your budget so make it so and try to bypass those those processes that you're putting in place. So one thing that we that we did is let's say for access control requests we funnel everything through a slack bot based access control request and response process and the way that the process is set up is that even if we wanted to there is no you know without a huge amount of pain in the ass and we don't advertise it there is no way to just manually bypass it through you know people shouting and crying and screaming and kicking on the floor and you know whatever else whatever other tactics that may be resorted to and so work around and like basically the process is designed such that they have to funnel through it then you know we've seen a lot more luck with that and if we make the process really frictionless then they have a lot less problem going through it and so like it's not it's not perfect of course like in any ways you may still have a you know a potential backdoor around it or something in that case like if it's something really sensitive like a like a CIO or a CTO or something demanding a you know access to active director or you know something to that effect that there be a secondary sign off at like a department head level or something like that where there has to be an interaction and like I've instructed my team you know for things like that that they just cannot proceed without having some kind of you know some kind of documented sign off you know somebody is being a bad actor they're just being kind of pushy and trying to like get people to to fall in line that they still have to go have a conversation around it and like they're not to they're just not to proceed otherwise and the more that you can make that a technical control for instance with like you know engineering guardrails like you can't land code in master impossible to to bypass the better position to be in cool any other questions alright well excellent thank you so much everyone