 Are we almost ready? Good to go, yes, now? Okay. Okay, next up we have cyber warfare games, excuse me, cyber war game tabletop exercises with Gurney Halleck. Please give him a warm welcome. Thank you. Yes, as I said, I'm Gurney Halleck, if I know my name right. And I guess I'm doing a talk on cyber war games, tabletop exercises, TTXs. I'm going to use TTX because it's easier. I am not pimping my employer, like all the other speakers. And I'm using my hacker handle and not my real name, like none of the other speakers are doing. Anyways, so who I am, my, as if you care, I've been doing cybersecurity for a while, mostly offensive, aerospace. So I get to play with a lot of equipment and systems that aren't normally what you would think of as IT equipment. So that's kind of cool. I've been doing red team work. I've been the red team lead for about three years for TTXs. I am a CISSP, so bite me. I collect a lot of useless computers. I'm starting to call myself a cybersecurity nihilist. And that securing all the things is a noble pursuit. But in the end, it doesn't matter. We're all fucked. So that's where I'm coming from. And maybe it's just jaded because I've been working in this area for too long. So what is this presentation going to be? So it's my observations and experiences doing tabletop exercises. I'm not going to talk about any specifics from the engagements I've done because I'd like to keep my job. I like Bruce Potter's comment here that he starts out with every presentation that he does is don't believe anything I say. So this might be a good starting point for you. If you're interested in doing TTXs for your organization, then you should probably go out and do some more research. I'm going to use the term TTX just because it's easier. Some of my clients are sensitive about using the term war games, so we usually use TTX. I hate the word cyber. It has no meaning at all. But I'm stuck with it, so I'll be using it. So a little history for war gaming. Probably since people have been able to scratch maps into the sand or dirt, we've been doing war gaming. Back in the 1800s, the Prussians put together a game called Kriegspiel. It was a war game, an actual game, and it provided rules for moving troops and artillery and terrain and so forth. But they actually started using it to train their officers. And it proved to be very effective. HG Wells actually came up with a game called Little Wars. Again, it was a game, but it had extended rules such that it could be a Kriegspiel. It's interesting because HG Wells was a ardent pacifist. We've gone into, of course, modern war gaming. We have war games and movie. We do tabletop exercises, go to high fidelity computer simulations, and all the way to actual troop deployments and live fire exercises. But specifically, obviously, what I'm going to be talking about is the tabletop. So who are doing these tabletop exercises? A lot of them are emergency response organizations, state and federal. Things like FEMA for the hurricane that we just had. They do tabletop exercises where they play through various scenarios, events, situations that they have to deal with deploying or supplying and so forth to manage those sorts of disasters. Government is starting to do cyber-based exercises where they are dealing with attack events, how they're going to work through it, do remediation and how they're going to deal with these sorts of things. Some companies are doing this as a service and they're starting to be done internally in some commercial organizations. You might not hear a lot about it because they're not really that open about it, but they're starting to do it. So what is a TTX, basically? So it's a simulation. It's a rehearsal. It's a defense exercise. For a TTX, as in a tabletop, it's paper-based or it could be electronic-based, but it's not a computer simulation in any sense. You have selected scenarios. You can only do so much in a single exercise. Most of the exercises I do are a single day and I maybe get to do like four scenarios. Four major attack scenarios and that's not full coverage of a system at all. It's a face-to-face meeting and that's really important and the previous speaker mentioned face-to-face. You get all the engineers, all the developers in together in one room working out problems and that's a special situation and you can really see when you're participating in that, you can really see the interaction going on and the problem-solving going on. It should be a non-threatening, non-avisorial. Whenever I have an engagement, I always bring everyone into the room and tell them, we're not here to embarrass you. We're not here to criticize you or really be adversaries. We're all working together to make the system more secure and that really sets a tone to make sure that everyone's playing together, everyone's working together with the same objective. So what is a cyber TTX not do? So it's not a pen test. We're not actually trying to exploit systems. It's not a vulnerability assessment. We're not scanning, looking for open vulnerabilities. Not a risk assessment. We're not going through and determining the risks of the very system, enumerating them and so forth. We're not going through and doing a detailed review of policy and procedure. Now both some of the outputs of the TTX may include identification of risks. It may include identification of lax policies or gaps in policies, but it's not a review of it. And it's not an audit, so it's not going to give you a check box on your FSMA or PCI requirements. So what the fuck is it then? And why do I want to do this? It doesn't sound like it's going to do much for me, right? Okay, so what does it do? It provides you with a tax simulation. And so you can see or discover gaps in your planning for dealing with attacks, both technical and in process procedure. A lot of times when I do a TTX, we come up with some pretty novel attacks that the engineers and the devs have never thought of before. And they're just blown away and it's like I didn't know that could happen. And we never designed for that. So there's a lot of learning that goes in there. Like I said, policy and procedure, again, if there's attack vectors or methods that no one's really thought of or considered previously, we can look at policies and procedures that are maybe not covering those aspects of how they, the remediation of the tax. Look at the development process, insufficient requirements, insufficient requirements on suppliers. People don't think about this. I'm buying software from a third party that's developing it, it's in Mumbai. All right, so they're creating software there. How much would it take to buy off some guy to put a backdoor in that software? We're shipping equipment from China to the US and we're gonna put it in our system. What's the potential for intercept and modification? Maybe not in an IT environment? Well, not that that's never happened, but we won't go into that. But anyways, so those are kind of things that is your supplier management, taking a look at these things, is supplier management putting security requirements on your third party suppliers. And socialization, which is really important, bringing together the right people to experience effects of attacks and potential deficiencies in their systems that maybe they never knew about. And a lot of times we have people meeting each other for the very first time. I was supposed to start that timer. Anyways, okay, so when should you do a TTX? And this is idealized because it never works out this way. But you'd like to do it really early on. Even if you don't have a well-defined system, it'd be great to get a round table and do something informal. We'll even just discuss some of the security aspects of the system or aspects that haven't been thought of before. You should ideally be running them through the development process so that before major milestones, before major releases of the product, you're doing a review of the security, not only in technical terms, but also in the policy procedure aspects of it. And you'd also like to do it over the life cycle. So many times I see deploy and forget type of situations and the dev team or whoever's deploying it out in the field, they never think that the threat landscape is gonna change and it does. So you really should be doing, bringing the product back in, doing a TTX over again and then see how you can improve the product that's already out in the field. Sometimes there's problems with that because they cost too much to upgrade it, but I mean that's a management decision. There's not a lot that you can do. If you don't have clear goals and objectives or support from your management and the technical staff, you probably shouldn't do it. I know it's hard to sell it to your management, but if they're not gonna support it as an engineer, it's just not gonna be worth it in the end. Oh, I like Rick and Morty and Fallout, so you're gonna see some of that crap. Sorry. So who do you want as participants? You want your subject matter expert SMEs, who are your primary engineers, system owners, people who are really smart on the system, your software developers, hardware developers? Look at your integration and test people because they know how they're testing it or not testing it. Are they testing it for security or not? They're probably not. Get some customer reps so that you know how is this product actually being used in the field? Maybe it's not being used as the way you thought it was. Again, your procurement and the supplier management security other than the TTX team itself, if you got IT security that's involved with the product. Your managers, well, we'll talk about managers and other people who might be involved. So as I said previously, you really do want some strong goals and objectives. So you wanna try to specify it as best you can. So we wanna take a look at this system or this portion of the system and we're gonna look for specific gaps in security technology. You maybe even go down to a specific service and what is the response recovery process procedure that we use to handle an event, a security event in this system? You need to talk to your management about this so they're aware of it and they give you a buy-in and you need to know what sort of data you're gonna collect because if you don't know what you're gonna collect then you can't really write the report that your management's gonna want. So we got a number of teams. You would think normally we have a blue team and a red team, a defender and an attacker. The blue team would be the system engineers, the software devs and network engineers that are gonna play their everyday roles, what they would do normally in their day-to-day job. You've got the red team who are hopefully security experts and a lot of times, especially in my case, I'm not an expert in the system we're gonna review or we're gonna test. So I'm at a disadvantage a lot of times and I have to spend a lot of time doing my homework to figure out what the system is and your red team ideally would be independent of any of the development of the product. There's a couple other teams that are very helpful. Facilitators, you really should have some facilitators. I've been lucky to have a really good group of facilitators that help us put the event together, do the scheduling, pulling together the right people to be in the exercise, who know how to collect data and can do it in the middle of the fray of the exercise. They have to mediate conflicts, which happen. And we give them a heads up of the red team plans, swear them to secrecy, but then they know kind of what the flow of the game is gonna be. You may have some observers, again, this is where I'm like sponsoring management, but they need to know that they're not gonna play. They cannot play. And ideally when we have management come in, we might have them come in and say hi and make a little talk before we get started and then we usher them out. Because you don't want your blue team, especially your blue team, to be playing with management breathing down their neck. Because they're gonna end up playing, may or may end up playing politically correct instead of technically correct. And certainly you want the technical aspects to come out. So red team prep. My mantra has always been no magic. So when you develop attacks in the exercise, it's you don't say, oh, the server is magically compromised. Or this wireless device is magically compromised. You wanna have some references, either exploits that have already happened, theoretical exploits, some sort of reference that when you run your attack, you can provide proof or at least indicate probability or possibility that this exploit could happen. Normally when I do mine, I'm forward looking. I'm not looking at what is today's CVE or last week's CVE. I mean you can play those if you know the exact configuration, this Linux is 10 years old or something like that. You could go back and see exactly what it's vulnerable to. But I like to push forward since we wanna see what's coming up, what's gonna be in the next thing so we can prepare as best we can for the next thing. So that's why I look for some proof of concepts or a paper here at Torcon or something like that that I can provide to the blue team and say, this is how this happened. Facilitators, whether they're prep, they get a lot of shit and I have to put a lot of respect on them for what they do or at least for me. They have to wrangle management, do the scheduling, educate the blue team and the red team on how the game is played. They get to manage disgruntled blue team members who can get very excited and pissed off and infighting between members in the blue team. So there's a lot of management that goes on there. And the blue team itself, like I said, they're just playing their normal every day job. So they come in knowing what they need to do in a normal situation while they're working at their desks or whatever and they have access to any resources they would normally have either documents or on the network or whatever. So I'll go through, is someone snickering? So, Injects, when we play the game we have Injects that we give to the blue team. There's a couple type of Injects. We do red team Injects and those describe the effect of the attack but not the attack method. You don't wanna give away how you're actually attacking although at some point you might have to provide that proof that I talked about previously. But you don't wanna tell the blue team how you've actually attacked. You wanna give them the effect of the attack just like someone would discover when they're doing log analysis or service goes down or something like that. They're seeing the effect. They're not necessarily seeing the attack actually occur unless we're providing that. And so what the blue team does is they provide a response, what they've done to mitigate the attack event or the attack effect. They can also send back questions and clarifications to the red team. A lot of times the blue team is they're not security experts so you have to kind of explain to them this is what happened and this is why you saw it. We also do external events. So that may be something, it's not a red team events but news reporter comes in and wants to talk to your CEO, law horse comes in and your chief engineer's pulled away for two days talking to them so he's now out of the picture as a resource. Those sorts of things, some sort of service outage, a customer screaming at you, why is my service down or whatever. And we also do tracking so that if you're using physical paper cards or if you're doing it electronically with messages or so forth, you're getting time stamps on those. So later on after the TTX you can recreate the flow of what happened and use that for your documentation. So, hopefully you can see that. So here's kind of just a flow diagram. You have these external events and your red team injects going into the blue team. Blue team is sending back responses. You get a volley going kind of attack response, attack response and that continues until you meet your exit criteria. And that could be your goals and objectives have been met. Sometimes you've just played out the game, you just don't have any more moves to go. And a lot of times you got time called. You just haven't been able to get the full gameplay in the amount that you have. Like I said, I'm lucky if I can get four scenarios in in a single day. And depending on how quickly the blue team or how much infighting the blue team is doing, you can burn up a lot of time. Yes. Yes. Sure, we do. So in some of our, well in a lot of our scenarios, you may have an event occurs that is pretty public and people are tweeting about it or putting Facebook posts on it. And so the news media starts knocking down your door or your customers find out about it or your management finds out about it. So I mean a lot of exploits that we see now, that's the sort of things that happen. It will hit Twitter and the cause of the event may not be fully known or researched, but it's hit the news already. And you get in a situation where your CEO may not know what to say. We're still researching it, but people want an answer. And that's part of your process too. You may need to have, if you come up and say, well we don't even know how to deal with this in the press when something goes wrong, well maybe you better have a policy about or a procedure about how you're gonna deal with media when something bad happens. So when a blue team doesn't really know how to play through a scenario and sometimes they just say, I don't know how to start or do something. How do I deal with this? It may be a situation where you don't have any process to deal with events. We've taken a look at the NIST cybersecurity framework and use the lower three portions, which is detect, respond and recover. So at high level, that's what we would tell a blue team to do. Detect, figure out what it is. Do your mitigation and then get the system back to normal operation. So work through that process and go through the discovery of what your gaps are or what you're lacking. Common issues. So I always get, there's always one person who's gonna say that can't happen. And it's usually someone who owns a system who's been working the system for 20 years or something like that. And thinks that this system cannot be hacked because I created it. I have a Plumbus 2000 in it that has a WAF and you can't hack into it. Oh, well, yeah, I don't have WAF filters in the WAF, but you still can't hack it because it's got a WAF. I have the buzzword, so it's secure. So you have to have the facility here is to, again, you go back to no magic and you can say, okay, here's the attack and here's the backup, the references, everything to say, this is why it could possibly happen. And then you need to have facilitators to go in and say, we need to play through the scenario, we need to get through it, don't fight it. After the scenario is done, we can talk about why you think it wouldn't happen, but let's just get through it. A lot of times the blue team will go down a rabbit hole. Those external events, which may be press or another business sector gets hacked with some sort of exploit. Sometimes the blue team will latch onto that and just beat on that when there's another attack vector that is the main purpose of the scenario. And sometimes you have to go in, the red team actually has to go in and kind of say, well, you guys gotta come back. You're going down the wrong way and you're wasting our time, so you need to go focus on this. Internal conflicts. I've had people almost yelling at each other. I've had people bullying each other. I've had quiet people who have really important things to add to the conversation, who are being bowled over and don't get a chance to talk. So you get a lot of internal conflicts. You get a lot of pointing, oh, well, that wasn't in my requirements to do this security thing. That wasn't in your requirements. No, it wasn't in my requirements, so. So you get a lot of those things and obviously that brings up some important gaps that you're discovering. Blue team always usually gets pissed off at the red team. We're being mean, we're going too fast. Sometimes I go fast on purpose. Things don't happen on the vacuum. Things don't happen at the pace that you would like them to happen. So I may hit them with three or four events, one right after another, and they're freaking out. And well, yeah, that shit happens in real life. So you're gonna have to learn how to deal with it. Sometimes they're just befuddled and say, what are we supposed to do? And you have to help guide them a little bit. Oh, and sometimes, like I said, the red team doesn't have, they're not experts in the system necessarily. So sometimes the red team makes incorrect assumptions and we just have to comp to it and just say, okay, we made the wrong assumption but let's just finish going through the exercise. Data collection, so it could be quantitative or qualitative. Since you're kind of playing a round table or a game, you may not necessarily get a lot of quantitative results. You may come to realizations that we're lacking in this area. Oh, maybe we're strong in this area but it may not be numbers necessarily. You may find gaps in RACI, if you're familiar with that, who are responsible, accountable, consulted and informed and you see that someone's not getting the information that they're supposed to be getting so that they can make important decisions or someone's getting too much information when they really don't need it. They just need to get a quick status report. They don't need to get a whole data dump on them that's not gonna help them. You may pick up the unconsidered attack scenarios I think that we talked about in the supply chain and supply chain or lack of supply chain requirements. Supply chain is becoming a really big issue especially in where I work in aerospace not only because we source parts and systems from all over the world but we have some people who don't like us and we need to think about that too. Oh, and security lifecycle. As I said before, sometimes it's just deploy and forget. And some developers don't have their software lifecycle. Okay, we understand that. We're software devs and we know how to do that but we have no idea about a security lifecycle that we need to do periodic reviews of system security even if it is deployed in the field. Post game review. So you wanna get everyone together. Everyone who played, get in there. We want everyone to have equal opportunity to speak. As I said, sometimes people get bowled over but we need to manage the after game discussion so we can really get all the information that's been discovered. You wanna find those ah-ha or oh shit moments and I've had a number of oh shit moments and those are good. Sometimes they're the ah-has or we did a really good job in this area or oh shit, we didn't have any idea about that. Get those improvements. Get feedback on the event itself so that when you do another one, you can do it a little bit better. Then yeah, you have to present it to management. You wanna give them, if you've ever had to work with management before, probably most of us have, you need to give them some actionable items. So you can say these are the technical gaps in our defensive systems. Here are technical gaps in our policies and procedures and then give them a schedule and plan of action. We're gonna work on these and get these fixed and you're probably never gonna get everything you want but hopefully you can get a buy-in on some of it. It's always tough to present to management but if you can give them enough things that they can make, you always give them like two or three options. So you give them the one you want and then you give them the one that you don't want and then you give them something terrible and then hopefully that guides them to pick the one that you really want. So wrapping it up, so yeah, TTXs can be, oh yeah, Pickle Rick, Pickle Rick! So TTXs can be really helpful for your organization or for your products. Make sure you have clear goals and objectives. Without that, maybe you're just doing it for fun but you really need to get some good output from the exercise. Get the right people and that's a tough to do and that's why I have some really good facilitator people because you wanna get a small group of people but a small group of very expert people. And that's it. If there's any, is there any questions? That's a guinea pig that likes a carrot. But that's, I haven't met one that doesn't like a carrot. Well, that's my standard answer. So I mean, what can you do? Did I hear, see a question, question here? Yes, so I guess the question was how do your facilitators get experience if they're starting from zero? Well, one good thing is if they can participate in another one, in another TTX, either with someone, another organization you work with or a partner or even with maybe a college or government agency, a lot of colleges that are doing these. So if you can get keyed in to that and they can get experience with that. Doing some research, obviously. Doing something really small, just with a few people doing just a round table, just having a couple people. You don't have the red team off in a closet somewhere but you have blue team and red team sitting across from each other and just start discussing it. And as you build up that experience then you can go on to something of a larger scale. Anything down there? Yes, so the question was what is the typical size of the teams? My red teams are five to eight depending. I usually don't want more than five because then it's getting, I'm sorry? Okay, the blue team, I have worked with like 10 to about 50 which is really hairy. It's hard to manage all of them and in the scenarios maybe not everyone is playing because it's different areas of the system. So sometimes you've got people not doing anything during a scenario which is not great. So you want everyone involved. More questions, any other question? One, two, three, no. All right, hey, thanks.