 Cizzo of Expel and founder of the Shmoo Group. He's going to be speaking to us about pen testing by asking questions, the art of elicitation. Cool. Hey, me. Woo. Super exciting. For those that know me, normally I run around and I make a fool of myself when I give talks, but I have the plague. I even went to urgent care yesterday just down the road. I will tell you that San Diego urgent care specializes in revitalizing blonde women. There is little places that you can sit will do IV rehydration, and the ad is basically like, had too much to drink in the sun on the beach? Come here and we'll plug you up. If you roll in with like the flu, they're kind of like, well, that's really boring. So I don't have the flu. I have some other thing going on. So I've watched every Harry Potter movie though on E, which has been great. So it's been super enjoyable, and I'm going to go back to that as soon as I'm done. So I started all my talks at the same basic premise. Don't believe anything I'm about to tell you. What I'm about to present is based on my experience. It's based on my view of the world. In general, in this industry, if you're over the age of 33 years old, you do not have a degree in cybersecurity. You may have a graduate degree because you went to Purdue and studied under SPAF, or you went to Hopkins and studied under IRA, or something like that. But the reality is most of us came to this because we like it, because we learned on the job. Sometimes we learned the right thing. Sometimes we learned the wrong thing. So if you disagree, that's totally cool. You're welcome to raise your hand, throw stuff at me, whatever, and we can have a conversation about it. So this is literally a pen test, by the way, on the side. This is one of my favorite graphics ever. This is someone testing pens. Pilot, zebra, crayola. Crayola has a pen, apparently. I was just totally unaware of it. Why is pen testing with words so important? So I ran a company for a number of years and we broke into things professionally. And we broke into things a lot of times by writing really complicated code, learning really complicated system architectures and getting way down to the weeds to figure out how it worked. And we broke into a lot of stuff just by asking the developers the right questions and we figured out how to get in. Now, we weren't doing your kind of normal run of the mill fish you, get inside, roll around and attack things. This is a lot more targeted work. But I think what you'll find in the coming years, as more organizations go to the cloud, as more cloud native organizations get bigger, when it comes to assessing our security controls, it's not gonna be, I'm gonna load up Metasploit, I'm gonna fire, shoot, roll around like a pig in poo. It's not gonna be as successful because at some point in the future, my bold prediction is AD is not gonna be a thing like it is now. So my contention is AD is one of the most dangerous things that's inside of most organizations. For those of you that do red teaming and pen testing for a living, you're pretty well aware that once you get inside, you're on an AD connected host, you largely have won the war. I work in an organization, we have no AD. And I work with a lot of other companies that are new, last three, four years, that have no AD either. And at the end of the day, when you attack us, it's trench warfare, right? You have to go host to host to host or you have to go after the application, you have to go after the logic. And that is a very expensive operation compared to standard pen testing. So when you live in an environment that is cloud native, that uses other people's systems where you may not even be able to penetration test actively, you have to get in the mode where you have to be able to ask questions, you have to be able to listen to information from people. Communication is obviously a big part of our job no matter what. We're just gonna blow past this, I got a half an hour or so, we're just gonna push. So the big question is what is elicitation? Has anyone ever used that word in common discussion before in their lifetime? Sweet, that's not too bad. How many of you are software developers or engineers? Yeah, right? Well, I meant the first, okay, fuck it, all right. It's gonna roll like this. It is not solicitation, just in case you're confused. I'm not encouraging you to prostitute yourself in order to do a penetration test, that's not what this is about. Elicitation is also not about yelling at people, trying to get answers, right? It's not taking them into a dark room, shining a bright light on them and trying to pull information. I think the FBI unfortunately has a very good definition of elicitation that you can see here. It is, you know, I think one of the core parts of our job when we're breaking into the system is to get people to assist you without them realizing that you're assisting you, right? Being able to ask the right question at the right time to help you do your job. For me, this all started as an interest when I was going through as a UAF dropout. Where's the other UAF dropout? There's a couple of three UAF, University of Alaska Fairbanks. If your kids want to go somewhere where they can drop out of school, it is the best place to go. Anyone here go to UAF that didn't drop out? Right, that's the number of hands you'll almost always see up in the air. I took a software requirements class and there was several chapters that Davis wrote on software solicitation, or requirement solicitation. I thought it was fascinating. You know, it was all about how to ask questions, how to structure that process in a way that allowed you to get the most bang for your buck by whoever you were interacting with. I wouldn't say a lost art, but an art that I think is well studied in our domain, in the security domain. So what I'm trying to do here is take my knowledge. I've been doing this for 20 odd years. I'm a terrible coder, right? I have been forbidden from writing code in several states. I'm really terrible at it. But I've broken into a lot of systems, right? And the way that I've broken into systems, frankly, is by asking a bunch of questions and eventually getting to the point where I'm like, okay, now I can do the one little bit I need to to get on the system and get the access and go. So that's how I've been successful. There's been other bodies of knowledge that I pulled from this presentation. Unsurprisingly, intelligence and law enforcement have a fair bit of art on this. But also the legal profession trains people on this, right? I don't know if you know, but there's a lot of questions that are asked in law and being able to understand how to ask good questions helps lawyers do their job, whether or not you think that's a good thing or not. That was a joke, by the way. Anyway, whew, a lot of lawyer defenders here. And then business analysts, which is actually a job description, there are a lot of people that do business analytics and they're trained as well on how to ask questions. So why is this important? I've kind of gone over this, but the reality of this is if you wanna do an online assessment of a product, whatever it is, somebody hands you a thing and says, I want you to break into this thing. Okay, sweet. I'm gonna spend eight hours, 20 hours, 40 hours standing up a lab, getting everything in place, starting to figure out how the interfaces work, do a bunch of crap. Even if it's a white box assessment, like here's the code, here's whatever, it's gonna take me hours and hours and hours to get up to speed. Instead, I can probably tell a developer to tell me in 10 minutes where things are super bad, right? If I can get a bill to report and do something productive from a communication perspective, he'll say, oh, this authentication thing right here, like the front door is great, here's a side door administrative interface that we have, like, thanks, fella, like that's the thing that gets you in the door. So, I love, everyone hears that, right? Like, it's not just all the suit of fed talking. I do wanna talk a little bit about expectations. A person can only tell you what they know, right? This is the problem with torture, if you will, like if you torture someone, this is the problem with torture, right? Like, ignoring the ethical side of things, but the product that's created by torture is that people will still, will make shit up just to get out of an uncomfortable situation. And when you're being asked questions by a security person, it is like torture, right? We can, I mean, be socially abrasive, we can look funny, like they just wanna get out of the room with us. So, they will, a few, wow, I totally lost it. Normally I could just roll, but the fever's just killing me today. People will answer the question you ask, right? But I've had people, I have real asshole friends, that when you ask them the question, they will only answer the specific question you ask, even if your intent was broader. Like, you know, are you hungry? Yes. Would you like something to eat? Yes. Would you like a burger? No, okay, we're just gonna enumerate all the food. No, the right question is, what would you like to eat? Right? Like, it's not a confusing question, but I have friends that are total assholes who will just answer the direct question. I also have a friend that if you throw something at him and you don't say catch and he acknowledges it first, he'll let him hit you in the chest and fall to the ground. Right? It could be a Faberge egg, and if you say catch, and he doesn't say yes, it'll bounce off his chest and hit the ground. I have some real asshole friends, so. This is not all the same friend, actually. I have one friend. People bring a lot to the table that you need to be aware of. The first is, they bring their personal experience. I fail at this all the time, right? People tend to believe that their experiences are everyone's experiences. That what you've gone through in life is the same thing that everyone's gone through in life. I had a really insane childhood growing up in upstate New York. A lot of weird things that we did that I assumed every other high school went through, like, we would, the story's just too long to tell you, but the reality is I made some horrible life decision that in hindsight I'm happy to have lived through, and I assume every other high schooler did the same thing that either we ran from the cops, we played with too many explosives, we played with too many guns, and all kinds of other things, and whatever, and none of us died, right? And we're like, oh, that's the way everyone's life goes. Well, it turns out it's not. So you have to be aware of other people's biases, what their backgrounds are. The other thing is personal biases due to their socioeconomic position, their gender, their race, geography, things like that, those personal biases come into play. This industry is phenomenally white male dominated, and we have to be aware of that when you're going down this road, and you're asking questions that people are coming from different positions with different sense of priorities, different sense of the things that they feel are important, you have to be aware of that when you're asking any questions. Finally, their own personal reality, the way the world works, is sometimes different than the way you think the world works. There is no, I will say this, equivocally global conspiracy to scatter chemicals from every commercial aircraft in the world all over the planet for mind control purposes. They're doing it for other reasons. So now we cross the chem trail bridge, things are about to get serious. When I talk to someone, when I go through this process, I think of, I don't bend them, like I don't like put a label on a person, but I start to think about how I'm gonna engage them based on their character traits. Are you an introvert or extrovert? Are you guilty? There are people who have like legitimately built insecure systems and they know it, right? They're like, wow, this thing sucks and the security guy is gonna rip me apart, right? There's also people who have tried really hard, but still think they haven't done good enough, the guilty feeling, right? I have made a thing that I think is really bad. The reality is it's actually probably pretty good because they thought about the security from the beginning and they haven't actually gone through to have it tested yet. The know it all and know it nothing. You know, there's people who think they're the coolest people in the room. There's people that are too busy. Oftentimes, if people are too busy, the people you need to talk to because they're the ones that actually hold all the institutional knowledge that you need to get your hands on. So each of those require their own approach. In reality, all we're trying to do here is exploit human nature. It's social engineering. I'm not gonna bore you here. You all are familiar with social engineering. All you're trying to do is social engineer your way to find weaknesses in the system. So the first thing you wanna do, this is one of the posters from The Sting. I highly encourage you, if you haven't watched The Sting recently because you don't watch ABC on the weekends whenever they show The Sting anymore, please go watch it. It's actually a fantastic movie. But the setup of having a place to interview a person, to talk to them that is comfortable, that is familiar to them, that puts them at ease is very important. Also, if possible, have a note taker in the room that's not you, right? Have video and audio recordings if people are comfortable with that. The note taking thing is interesting. When I take my own notes, I get lost going back and forth between the process. I run, one of the things I do at work is we run instant response role playing games where we sit down to pretend a bad thing has happened and I drive the IR tabletop exercise with a customer. And if I have to take notes while that's going on, I can't pay attention to the game and I have a very hard time driving it. It's the same with these kind of sessions. If you're the one having to take the notes and then ask the question and take the notes, you will lose track of what's going on. Let somebody else keep track of the facts while you work through the questions and work through the process. You have to make a determination pretty early on. Is this gonna be a group or individual interview? I'll often start as group interviews just to build a collective trust with everyone and allows the group to start to click around whatever project we have. Like, hey, we're just gonna sit down and talk about your product. We're gonna sit down and talk about your service as a group. And then like antelope, you're just gonna start to peel them away from the herd and have individual conversations. Not that they're really prey, but that they're prey. And so you'll be able to then have individual conversations where you can build a trust, not eat them out on the savannah and dig into the knowledge one person at a time. So again, these group discussions I think are great in the early days, but as you dig more into the process, you need to be having individual conversations. Before you even get started asking questions, there are techniques around elicitation where you can gather data from written word, from surveys, from questionnaires, from use cases, things like that. This is the business analytics universe, right? If you go read business analytics literature around elicitation, this is their toolkit, right? All of these things, because apparently they're inhuman and they don't wanna actually talk to people, so most of what they do is fill out this form, which feels very British to me, so I don't know if they're all British or not. Sorry to any business analysts that are in the room or any Brits. So, good questions. Before we get into the specifics of other questions, here's a good question. Tell me about how you handle authentication. Why is it good? One, it's open-ended. It allows the person to talk openly about it, it doesn't infer anything, it just answer the question. It doesn't imply any facts about their authentication, doesn't imply any prior knowledge, it's not biased. Everything's very open at this. The problem is, when you ask a question like this, oftentimes you'll get an answer of, we use passwords. Okay, that's some great knowledge of how your entire authentication process works. Like I got three word answer out of this and what I really wanted was a deep understanding of your architecture. So when I ask this open-ended question, clearly I'm not gonna get what I want and we're gonna have to find other ways to get what we need. So the first thing I like to do, find a personal common ground. This is where you, before you even start talking to the person, hey, where are you from? Where'd you go to college? Where'd you study? Where you start to try to build a rapport with a person so that they can trust you, right? When you come in as an outsider, even if you're inside the organization, if you're doing red teaming or whatever inside the organization, you're doing product assessment inside the organization, they're gonna view you as an adversary, right? When you're assessing a product or a service, you're pointing out the bad things that people have done wrong. I'm not necessarily actually saying that that's true, but that's gonna be their feeling about it, right? Even though you're there to try to help make their product better, the reality is you're in front of them, right? You're a danger to their job. So the first thing you need to do is like, hey, we're all in this together, I have the same background, I did stupid stuff in high school too, whatever it was. So the next thing, start with easy questions. That should be easy. Tell me about your job. What do you do here? What would you say you do here, right? Builds confidence, gets the person to open up. It's probably not a useful discussion, right? Because you don't really care. I mean, you'll care a little bit, but you don't care a lot, but you have to go through to continue to build this rapport. Next thing, leading questions. I don't know if I'm triggering that or it's just happening, but I love it. Someone hacked it, yeah. Let's play with a guy who's all whacked out on meds. So what's a leading question? A leading question implies facts, right? When did you stop beating your wife? It implies that you beat your wife at one point and then you stopped, right? Those are actually two separate facts. You may still be abusing your spouse or you may never have abused your spouse in the first place, but the question does not leave room for that interpretation, right? This is like the canonical example of a leading question. Now, the thing with leading questions, if used correctly, they're actually useful because if you have an idea of where the answer lies, you can lead the witness there, but you need to be aware that you're consciously doing it, right? You use lots of unsafe functions in your code, right? Okay, that assumes that they're trying to do something with safety or input sanitization or something and they're doing it incorrectly, right? Tell me about how you handle input sanitization. Much more open-ended question doesn't leave it. Your developer training seems useful to all your developers. Tell me more, okay? Instead, can you explain your secure development training and how you measure its effectiveness, right? You're in the first part of that question, you're implying that it's effective. They're gonna run with that, right? They're going, yes, it is effective. Let me tell you how effective it is. No, no, no. Don't assume it's effective or not. Tell me about the training process. Tell me about how you measure the effectiveness. Then we'll go from there. You have four different versions of your API which is no doubt confusing your customers. Why do you do that, right? Maybe it's not confusing. You're implying a fact there that they've screwed up because they have so many versions of their API. Now in space it's probably pretty stupid but how do you handle multiple concurrent versions of your API? These are just some examples but the idea here is you can ask a question that leads someone to answer in a very specific way and if you do that, you need to do it with purpose. You need to do it in a way that if you can't get information out of someone, if you have someone who says, tell me about your authentication, they say we use passwords, you get to start to lead them, right? If you don't have to do this, I would avoid doing it. I'm guilty of this all the time. I ask leading questions at work all the time. I try to be very conscious of not doing it but it's something that takes, I mean for me it's taken a lot of effort to get away from. Next up, comparisons, another useful kind of question. Does your logging system aggregate all its logs or are they stored in each host? I have an either or question here. This can help provide focus for people because it's leading, right? I'm only giving them two answers but I'm giving them a comparison where the truth can lie somewhere in the middle and then we can get down to that process, right? So by making a comparison between two options, sometimes I can get a person that I'm interviewing out of the woods, right? Because they're a little lost about what we're talking about but sometimes I also have to be aware that I know I've led them and I have to make sure that they're gonna answer me honestly and openly. Next, prior knowledge. So this is the idea where I already understood something, like in this case I understand I had to upgrade to Python 3, you know, whatever. This is something where we have a common experience, I have prior knowledge about the system, I share it, it builds rapport. It also allows me to then go ahead and build on that knowledge with that person, right? So the idea of prior knowledge of the situation allows the person I'm interviewing to say, yeah, okay, this person understands what I'm going through, I'm gonna work with them now because I believe that they know what they're talking about. Another thing that's not really a question but we use all the time is the whiteboard, right? Can you draw a high level description of your system? Sure, right? Now, this is great for visual learners, I've got people that have gone up and just written out like the most amazing, like, you know, chart of all this stuff on the whiteboard. I've also had people that couldn't draw a straight line if their life depended on it, they cannot draw a legible word ever. I had one client who would just on the board write, like he was writing words, but he was just scribbling, literally just scribbling. And I knew that's all he was doing and I was there with a coworker once and they were like, what is he saying? What, I can't read that. I'm like, he literally just scribbles while he thinks. Like, it is not actually a word. He's like, you're kidding me. I'm like, no, watch him all day long. Not a single letter in the English alphabet will come out. Okay, do, do, do, do, do, do. And he just scribbled on the board. So most people can't draw, you have to accept that, but sometimes it'll help the conversation along. Next up, parroting back, super important, right? Some of the most important things I've done as a consultant was parroting back and adding clarity to concepts that customers had, right? I started an assessment of a company once and it had a pretty large portfolio of products. Like a dozen different, a dozen, hello! This mic keeps cutting out, I'm gonna apologize. I'll just yell really loud if I have to. They had like a dozen different products and just in the process of proposing to them, I came up with a composed view of here's how all 12 of your products fit together in a portfolio. And they're like, holy shit, we didn't realize that. Like, this added clarity. Like, I would have paid you for the proposal. Like, I'm like, oh my God, you don't know what you're building, right? But part of our job when we do these things is to be able to add that clarity. And the only way you can do that is if you can parrot back, right? I heard a thing, I'm gonna tell it to you how I think I heard it and you can correct me. It can be time consuming because it's like SINAC kind of thing, right? Like you're going back and forth, back and forth. But the reality is like it's super important for people to be able to basically agree. Yes, that's what I said. Yes, I agree, that's what you said. Another one, play dumb, right? Just feign ignorance, like I've never heard of Sammel. Tell me more. And there are people who will tell you more, right? They'll think like, this is the greatest thing ever. I have a person who I get to educate today. It will, if you really know something like a topic like Sammel, it will ferret out people who really don't know anything because sometimes you run into those. If the problem can be when you play dumb, they might just assume you're dumb, right? So you have to use this technique kind of judiciously and an environment where you're prepared like I may lose all credibility with this person, but sometimes it's the only way to break down the barrier. Kind of as a corollary, say something false, right? Say something that you know not to be true and that will get a response, right? It'll be a remote emotional responsibility, a visceral response which can be good and bad because sometimes again you lose credibility but oftentimes you re-engage people, right? You're 45 minutes into an interview session and you really gotta be able to get this person to wake up. So what are you gonna do? I'm gonna say something that's wrong and get them to come at me, right? This is actually something I think Beatle and I used to do every once in a while when we were on stage is just to say things. We knew we were untrue to wait for someone to call us out because that was like the greatest thing when people actually, you're wrong, like, correct! We are wrong, thank you, we know you're paying attention. Say something provocative like, I love System D. Yeah, right? It can totally derail a conversation, unsurprisingly, but it has the same kind of effect, right? It may not even true or false, but it's provocative. It gets someone engaged. Another thing you can do is volunteer information. This is the idea that quid pro quo which seems to be a common phrase in our lexicon these days. It can build trust. They realize you have something to give me, I have something to give you, we give and take, we go back and forth. Two more things that I'm gonna wrap it up. This is something that took me a while to understand because when I was young doing this, I thought I knew everything and I would talk and I would fill all available space with words and I talk and I talk and I talk, it's what I do. I had to learn to shut the hell up and let people talk and ask the question, let them answer and then stare at them, right? Because then they'll fill the void which is what you want, right? You want them to continue talking. So you end up giving them the opportunity to say more and say more and say more and that's what you're really gunning for. Finally, maybe not as the last thing but then next to last thing, end on an open note, what else should I know? Because there's almost always a thing that they've been waiting to tell you but they didn't wanna say it because they were waiting for you to ask the right question and somehow you never ask the right question. So say, what else should I know? And that's their cue to be like, there's this thing I've been wanting to tell you about the authentication system. Okay, poof, and you jump into that, right? Don't do it as the last thing because if you find something really materially interesting and you only have five or 10 minutes to go in the interview, like you could run out of time and not be able to dig in but do it toward the end and you'll get good results with that. Here's a list of techniques I just went over, yay, list. The Babcock elicitation model is pretty useful. The FBI elicitation brochure is actually pretty interesting. There's Google FBI elicitation brochure and I just checked today, this Confluence page on Cornell is now behind a user password wall for some reason, unfortunately. Boo, yeah, right, anyway. Here's my contact info if I have any questions. I'm gonna go back to watching Harry Potter movies, thanks.