 A clock on a Saturday night so we can get this thing kick started so that we can be done by eight o'clock as scheduled. This is the panel on cyber adversary characterization and just to introduce the panelists. I'm Matt DeVoe with the Terrorism Research Center, Toby Miller, Independent Security Consultant, Tom Parker, Penn Test Limited in the UK and Mark Sacks from the Department of Homeland Security. We're going to speak to you on cyber adversary characterization, why it's important and discuss some models that Toby and Tom have been developing to help them in the characterization process. Just to give you a little background, we all met up for a two-day workshop on this topic in 2002, continued to do some research via email discussions back and forth and then Tom put together the panels to come here to Black Hat and DEF CON to introduce the concept and obtain some feedback. A lot of the models, a lot of the issues that we're looking at are in early development. We think there's a lot of utility for them but we need to get feedback from the community as well. We plan on doing future workshop in October 2003 and just FYI the slides will be on the conference websites for both Black Hat and DEF CON because I know they're not here. FYI characterize, there's a theoretical side to it to gain an understanding and ability to anticipate an adversary in order to build improved threat models and that's a piece I'm going to talk to is the threat model component and then improved profiling of attackers at post-attack and forensic level to guide response and to give you an idea of exactly who you're dealing with. I was asked to come in and speak about threat assessment, put some of the modeling in context and then talk about some of the issues that we look at in looking at cyber terrorism or profiling of cyber terrorism. I'm going to spend 10 minutes on that. Biggest point I want to make with regards to why I think this topic is important is because your adversary is not a one or a zero. A lot of time when doing analysis of attacks or looking at risk management concepts, you're told about the packet that's the attack and very little attention is giving to the intent, motivations, capabilities of the person behind the attack. So it's an effort to try and move it beyond just the technical to start thinking about who you're dealing with to guide your response and also if you have a perspective with regards to what threats you're facing, what your adversary's capabilities are, it can also guide your security posture to a certain extent. I'm involved in the concept of risk management which includes vulnerability assessment, threat assessment, doing some of this modeling and you can't do risk management, you can't incorporate a risk analysis without doing the threat assessment or adversary characterization. We're out especially with private industry. A lot of times they have these risk management models with a threat component but for cyber security issues there's no threat piece injected into it. So a lot of times funding isn't provided or the models aren't accurate for what they're trying to achieve within the organization. The whole point is without the threat analysis you're not performing real risk management and if you're not doing risk management you can't have concept of acceptable risk which means you're either trying to protect against everything or you're not protecting against much and thanks to Jay Healy for that your adversary is not a one or a zero quote. What things do we look at when trying to do risk exposure, what threats adversaries exist, that's this piece here that you're going to hear about, what tools, capabilities can you use, also some of the research that they're doing, how attractive is the target, this is where we have to look at the goal and tent likelihood of a system being targeted, what level of access can be obtained, this is classic vulnerability assessment process, this is something that the industry is really good at actually is going out and doing the assessment piece and looking at vulnerabilities and fixing vulnerabilities but just don't put them into context very often. What impact would the attack have and then also there needs to be a safeguard component here, need to be doing it for a reason in order to guide the security process. Threat agents of interest, it's list the ones that we use just for your information, you'll see when Mark gives his presentation they have a similar list that they use within DHS. This is kind of how we break it down and then needed to categorize techniques as well. We obviously can't look at all of the different ways that systems can be impacted so we had to come up with a high level categorization and this is the list that we use right now that includes everything from penetration to insider placement, malicious code, a whole slew of things and obviously depending on the system you're looking at not all of them are looked at. Cyber terrorism analysis, the piece that I focus a lot of my attention on, the camps on cyber terrorism are divided into two distinct communities right now. You really have those that think that in camp A that bin Laden lurks in every monitor and is prepared to take down the power on the east coast tomorrow and then in camp B that terrorists out there have no technical capability and pose no threat whatsoever and that's kind of the environment that I see whether I'm working with government or industry that typically they're over paranoid about the threat or not concerned at all and the truth kind of lies somewhere in the middle, that's some of the stuff that I wanted to speak to. Characterizing cyber terrorists, it's tough because we don't have any historical analysis at least with some of the stuff that Tom and Toby are doing, they're looking at actual attackers, they're looking at history from past attacks but with cyber terrorists in my opinion we've never had a true incident of cyber terrorism so you can't use that past analysis to try and drive your profile, need to be a little bit more creative, need to imagine the threat. At a conference that we were at about a year ago there is an interview with Alvin Toffler and he described September 11th as a failure of imagination that stuck with me on the cyber terrorism side because it is one of those threats that you need to imagine a little bit in order to give it some reality. You can red team the threat which is something that gets done, characterize capabilities and intent, it's a little tougher but if you're red teaming it you can assume different levels of capabilities, you can put stuff in models like what Tom has done, assess and fix vulnerabilities and then understanding what's happening right now. One of the big issues and I think that Mark will speak to it, when an attack takes place you either have to assume the worst or assume the best and we'd like to put a little bit more granularity on that during the incident response process. Key issues for cyber terrorism and these are things that we discussed at the workshop or that I briefed that I thought were important to looking at the cyber terrorism threat. The first being that there's kind of a nature of terrorism has changed from international versus single issue groups, those that engaged in kind of calculated violence versus those that are interested in mass casualty, mass consequence. And the whole point we're trying to make in looking at that first bullet was to show that some groups are going to be displaced and might use cyber terrorism as a differentiator. So we kind of needed to open the scope and look at it as being attractive by groups that necessarily wouldn't be attracted to it because they want to differentiate themselves from the alkyd of the world. The model, the thing that we actually looked at for one assessment was hijacking. It's 700 plus hijackings prior to September 11th either used to flee a geographic region, attract media attention to a particular cause or to use the plane and the passengers as a bargaining chip in some sort of political negotiation. Of those groups that were engaging in hijacking for those purposes, how many now would engage in hijacking because it would be perceived as an intent to use the plane as a weapon of mass destruction. So we thought that was important. The other thing we wanted that we thought was important was the fact that terrorists go through a lesson-learn process and to look and see what lessons can we draw from cyberspace and critical infrastructure failures or from attacks that we've seen within other threat environments that terrorists might look at and learn from whether they're successful or even if it's a failed attack. Long-term planning cycle we thought was important. When folks speak of cyber terrorism, they like to think of it as a threat that's there tomorrow and when you have an adversary group that will take five years in the planning, you know, development of the capability and execution of an attack, we need to think in a little bit more of an expanded time window. Targets once identified will be continually attacked until destroyed. Thought that was important, especially when you start trying to do adversary characterization in event of a real incident. And then convergence, issues of convergence. Will terrorists converge with hackers a la hacktivism? Can capability be acquired or recruited from outside? We've looked at issues of convergence between terrorists organized crime, terrorist nation-states, so I think it was important to look at issues of convergence elsewhere. The definition that I use is a sustained attack. In other words, it's not something that just happens quickly and goes away, a sustained attack against a critical infrastructure. With the intent of having some sort of political coercive impact or causing public panic, fear, and disruption. It's kind of a fuzzy, somewhat definition, but I try and isolate it or move it away from just a hack attack or just somebody's simple denial of service attack to move it into a scope of conventional terrorism where you have a sustained event that has something with much greater impact than just an isolated attack or a terrorist organization in South America writing a virus and releasing it on the net. We wanted to try and move it outside that realm. Likely aspects of cyber-terrorism, this comes from a paper that we wrote and posted on our website several months ago. We thought this was important as well because it kind of guides when you're doing the system assessment. Whether the system you're looking at falls into one of these categories and where it falls into the category will kind of guide the attractiveness of the target. We thought and still think that the most likely aspect of cyber-terrorism is going to be in parallel with a physical or WMD attack where they're going to try and augment the impact of something that's been done in the physical domain. Maybe it's go after telecommunications, after doing a bombing, going after hospital systems. There are a whole slew of scenarios but we still think that that's probably the most likely aspect in which it'll be used is not as a replacement for physical attack but actually to augment a physical attack. Go ahead. Everybody run on here and it's kind of what we're trying to see in there right now with a rack. And then we're losing one or two soldiers each day on a global scale. That's not paying to lose soldiers. That's what happens in the whole thing of war. But the thing is that it's costing a lot of money. Part of why we're going to have the Iraq War begin with. If I was looking for, what's the mass destruction and all that? Why? I'm going to make you guys pay for it tonight. It's actually kind of covered in bullet number two. And let's re-raise that issue during the Q&A and we'll probably hold all questions. We've got 20 minutes set aside and I don't want to impact on the presentations for the other guys, but we'll address that during the Q&A. And it is addressed a little bit in number two, which is to decrease confidence in critical infrastructures or engage in psychological operations where you go after infrastructures that had nothing to do with a physical attack or associated with a physical attack, but you're going after things that will decrease confidence and cause the public to fear. Living in Washington, DC, we saw the impact, the psychological impact of two guys with a Bushmaster rifle. There is economic impacts, people change their behavior, they zigzag to their cars. There were lots of outlying effects based upon little isolated attacks. And from a cyber-terrorism perspective, if you launched a campaign against critical infrastructures and caused that loss of public confidence, started having the economic impacts, we think that that's also attractive as well. To cause physical damage or loss of human life, this is the most attractive to a terrorist organization because it lets them blend two different types of attacks. You still get to kill people, you still get the spectacular, but you also get to launch the attack from Pakistan or geographically different. You don't have to put the person on the target to engage in it. That's also, from our perspective, the least probable, just based on experience in assessing infrastructures and the fact that there's still a lot of physical safeguards, humans in the loop, things of that sort. Not impossible, but a higher risk to try and make that happen, where the other two become more attractive. And then the point I wanted to throw in as well is we tend to put the blinders on when looking at issues right now and to not forget the nation state as a sponsor or using as a tool of strategic influence. I think Mark will address that a little bit as well, but we just want to make sure we don't fall into the blame bin Laden game or any attack, whether it's a hacker or a nation state, starts getting characterized as an act of cyber terrorism. With that, I'm going to turn it over to Toby, who's going to talk about his hacker rating model. Do you want the wireless? Hi, my name is Toby Miller. I'm here to speak on the model that I've been working on for the past year, off and on for the past year. It's about the rating your attacker. If you want to get in touch with me or you have any comments about the presentation, my email address is up there. So go ahead and email me your comments. Basically, my model or what I came up with is like a checklist. And it has points associated with different events and different categories as to what the attacker physically did. Why did I do it? One of the reasons that I did it was that because I hadn't seen any tool out there that an analyst, IDS analyst, firewall administrator, security administrator, whatever could use to help rate the skill level, the potential skill level of his or her attacker. I also did it so it would help management, assist management, in determining incident response requirements. And also to help management with threat level management as well. In my system, I have six categories actually. They are passive fingerprinting, the intelligence gathering, the attack, the exploit, back doors and cover up and others. Passive fingerprinting, this covers, when you get a packet in, coming in, or if you have it, I personally think that the operating system could tell a lot about the attacker. Yes, the operating system can be fooled. It can be spoofed just like an IP, but it's still there. You still can use it as part of your analysis. Intelligence gathering, this covers everything from social engineering to sending a single packet to a service to see if you get a response. The attack, it is like the attack was cross site scripting. If it was SQL injection, whatever, I rate that as well. The exploit, it goes along with tools, which Tom will get into a little bit more in depth than I will. And back doors, in my experience, past five, six, seven years, is usually three phases of an attack. You have the intelligence gathering standpoint. Somebody probes your box or whatever. Sometimes they don't. You have the attack and you also have the cover up. Once they get your box, there's usually some kind of root kit, whether it's a single file, whether it's a torn based root kit, something like that, or even a LKM or a Windows based root kit. And that will cover that as well. And then you have my last one, which is others. And that basically, that covers everything that the first five doesn't. What I'm doing right now, originally, I posted this thing to Incidence about this time last year, incidence.org. I posted it to the Incidence mailing list along with some other mailing list. I got some comments back. I made some changes right now on my website. I'm up to revision number two. I'm working currently on revision number three. I've made some changes for revision three. Like for instance, for revision number three, a lot of the comments that I've gotten back in the past couple of months, as far as this goes, is destination, the destination server. As far as the destination, the destination server can be critical in evaluating your enemy as far as to what he's going after. For instance, in my new model, I'm gonna have a workstation. It's gonna go down to a class. If you're a government person, you deal with classified information, that's gonna, you're gonna treat that a little bit more seriously than you would. Somebody's just attacking somebody's personal workstation. It's gonna also include a bank-related server. All these different things are gonna be scored and have different point values. Currently, the point values are between one and five. I'm about to change that and make them one to 10. And one of the other comments I had, and I'm gonna incorporate it, is a source IP. Can it be spoofed or is it real? Basically, is it real or is it Memorex? That's been some of the other comments that I've had as far as this model goes. Unfortunately, I didn't have enough time to actually bring the model up and go into depth in it. But if you want to visit the model, make comments or whatever, it's at raidingthehacker.net. And revision three should be up in about a week, week and a half or whatever. So that being said, here's Tom to go into tool care transition. Okay, so Toby touched briefly on the idea of the point scoring system that we've come up with. I'm gonna go into some parts of that system in some more depth. Firstly, one concept which we've come up with is the hacker pie. One of the reasons we think that what we're doing has so much potential is because the IDS guys for some years now have been taking log data and what have you and trying to analyze it to get some kind of idea of the kind of people we're dealing with out there. Although we think that is obviously of great value to us, it's not the be all and end all of the characterization. We have different sides to this. We have the psychological side. We have obviously the technical side. The more we do the research, the more sides we're finding to it. And obviously the research isn't completed yet. So the other advantage of not relying upon one piece of the pie is that obviously when an incident occurs, you're not always gonna have lots of information available to you. Sometimes you'll have technical information available to you. Sometimes you'll have a clue who's involved in the attack. And the idea is that we can build up this characterization with separate parts of the pie without having to have them all to create the final characterization. Okay. The first type, the two models I'm gonna go over today are technique and tool classification. Technique classification looks at the reconnaissance techniques and attack it may use before an attack, the types of exploits they're using. For example, port scanning techniques. Is it stealthy? Or are they just going through your network, scanning everything, not really caring about who's watching them. The exploitation technique, are they using mass routers? Or are they very carefully deciding what operating system you're using and ensuring that the exploit will work the first time around? Tool classification, obviously the tools used are gonna gel with the techniques being used. For example, if you're randomly scanning through the network the chances are you could be using Nmap. We're characterizing the kinds of people that use Nmap and the other scanning tools. Another example would be exploits obviously which fall under tools. Our recent one will be the DECOM mass routing exploit which is being traded in the underground. The original exploit by HDM exploited six different targets. Windows, well vanilla Windows 2000 through to XP service pack one. The mass router which was written went through A-class networks trying each target on the individual IP addresses. If the guy that wrote this had any clue what he was doing he would have realized, I hope he isn't in the room today, he would have realized that this exploit is a one shot. The first time you get it wrong the service dies and that's that. And obviously that says two things. Firstly about the person that wrote it which is the first characterization we can make. And the second the kind of people that are likely to use this exploit are gonna be your low end adversary that don't really understand what's going on. Okay, in the tool classification model the first point we have here is availability of the application. Obviously certain tools are less available to certain individuals. You have obviously zero day exploits which are traded normally between quite close knit groups of people. We look at the origin of the applications that people are using and the ease of use with obviously doing an M-map scan is fairly trivial, it's a few command line switches but in the case of exploits does an attacker need to find a return address? Does the attacker need to know something about that system beforehand? Are there other mitigating factors? If they're trying to perform a man in the middle attack do they already need to have access to a DNS server to poison the network or are poisoned or what have you? This kind of ties in with what we call the vulnerability disclosure food chain. This, we originally developed a metric which was pretty much a pyramid. The top of the pyramid representing where someone discovers a vulnerability and the bottom representing the public dissemination of that vulnerability. The idea is that the bigger the pyramid gets the more people know about the bug. We developed that into more of a web diagram because the pyramid wasn't very detailed which basically describes the attempts to describe the path of the flow of vulnerability through to exploit writing. Maybe it gets traded on IRC, maybe the guy finding it just is responsible about it and tells the vendor. And the characterization part comes in because we try and place the adversary on that food chain. And what we think is depending on how high up they are is a determination of what kind of adversary they are. This also applies to group characterizations because as well as having individual characterizations you can also characterize groups of people. If someone attacks your network with a zero day bind exploit and we know that that bind exploit exists and it's being written by a hacking group that is known of we can perhaps determine the kind of arsenal that the other members of that group have access to just as in real warfare knowing the weapons an enemy has it's also good to know the weapons a cyber adversary possesses. Knowing the group which is a certain individual is a member of also helps to determine the age group. The social demeanor of the individual you know the kind of social group that they're going to be in. A lot of attacks like social engineering we find require fairly mature intelligence. And the kind of people that are likely to go out I'm using social engineering in a commercial environment or government environment are one standard deviation age group higher than your average script kitty. Okay I'm just gonna leave you with this. We move on to Marcus. This is a cut down version of the disclosure food chain which I put together in fact for one of the workshops we did on this topic. At the top there you have the vulnerability discovery information shared with the exploit community with the community directly around him. You know maybe it gets leaked, gets traded on FNAT. There you go. So that's the model that we use for that metric. Okay Marcus. Good evening I'm Mark Sox, Homeland Security. Thanks for coming out and I appreciate you taking your time when you could be off of a drinking beer although I'm sure some of you are. And having a little bit of pizza so if I get thirsty if somebody wants to throw something up here I'll take it. Looking around the audience I see a lot of feds so we won't do a spot the fed maybe a spot the hacker. Maybe we could do that while we're here. But glad to see a lot of friendly faces members of the media and others that are in here. We'll keep this brief because I know everybody does want to move along. What I want to talk about is how we do threat characterization. It's a little bit different from what you were just seeing and that may fill up the slide a little bit there but the cyber threat to United States is unique in the sense of we have what we call critical infrastructure. Most of you probably heard me talk about that before. You've seen the writings that we've had in homeland over the last several years but our information networks that we have coast to coast we recognize that vulnerability. This is well documented. We've looked at it over the years and we're doing quite a bit to protect it. But what we're seeing is that both secure and insecure networks now are starting to converge together. We've got some networks that are doing the right thing. Their network administrators or security officials are doing exactly what they need to do but unfortunately they have to do commerce with those people who don't quite get it yet. We have to interconnect those networks in order to make e-commerce work so that raises a level of threat to those protecting networks because they have to interconnect with those who aren't quite as well protected. Plus the complexity of the networks continue to grow. The complexity of the tools, the way we architect them make sure we get new technologies. We've gone from copper to fiber to wireless. We've got new protocols coming down to point. We're gonna move from V4 to V6, IP based. And a key bullet here at the bottom is that cascading failures. We're not really sure if the entire infrastructure of the United States could have a cascading failure or whether it will be resilient like the original ARPANET which was designed to if it took a hit in one location the network just routes around it. Now on September 11th when we had physical damage in downtown Lower Manhattan we did in fact see the data networks route very nicely around the physical damage. But we had catastrophic collapse of the steam networks, the water, the gas, and oil, all the other infrastructures that are in Lower Manhattan because they aren't designed to route around physical damage. Could something like that play over on a national effort if we had a failure of the East Coast Rail network could that catastrophe play over and have cascading effects across the United States? We don't know. We hope we never get to that point to find out. But it is something to think about when you start looking at the different types of threats that come at us. Now as was mentioned earlier the way we like to look at cyber threats and we'll step away from the physical world and just talk cyber. The range is quite vast. I mean I have it at the top but you could think of it at the bottom end. In terms of numbers, the top bullet, hacker, script kiddies, hobbyists, millions. Pretty much anybody who has access to the internet has a little bit of curiosity, wants to kind of play around a little bit. Even a home user that's sitting behind a DSL or a cable connection that's not adequately patched or has not taken any type of precautions with a machine falls into that category. They are a threat to our critical infrastructure because they're not taking the right steps to secure their computer. Somebody would like to come in if you wanna let them join the party. If they have beer they're welcome. If not they have to go around. Discretional employees, insiders, hacktivists, we kinda make our way down the list there. We're at the bottom state sponsored. We're talking nations attacking each other in cyberspace. Very few of those. But if we take these types of threats and we arrange them on a picture we can see that the probability of occurrence of like the hacker, script kiddie world is way off on the high side. There's a lot of that out there. There's a lot of the numbers are large but the potential damage they can do in a relative scale is much lower than the damage that could be caused by terrorist groups or well-funded nation states. However the numbers of well-funded nation states are extremely small compared to the numbers of say criminals or script kiddies. I think everybody can see the relationship there. But a defense science board study done a few years ago showed that that curve that you draw actually is moving outwards in time so that a hacker today might have a certain level of skill but four years from now they have the same level of skill as today's terrorist groups. Of course today's terrorist groups four years from now will have much more increased skill as well. So that whole line continues to advance outward. And we have seen this effect over the years as the level of clue continues to grow on the offensive side as well as on the defensive side. One thing though that about all these threats is the way they manifest themselves is the same. It doesn't matter if you're an individual hacker or script kiddie. It doesn't matter if you're a well-sponsored nation state. The way you're gonna break into my computer is the same way. You're gonna come through some type of internet connection. You might use social engineering. Some vulnerability, some whole, some place is how you're going to gain access. Why is that happening? Well I've got a few bullet items up here. The internet as we all know was never built to be secure. It was designed to connect people together. It was designed to take academia, governments, researchers, the military, glue them together in a network where everybody could talk. Security wasn't an issue. Security today is an issue so we're going back and we're re-engineering the networks. We had a lot of obscure software. For example the software that runs the telecommunications world, the five ESS switches and the SS7 switching networks. Largely proprietary but now we're starting to move over to voice over IP and we're using commercial protocols that many people have access to. So where we had security through obscurity by very few people who understood how the software worked that's getting changed and replaced by commercial software and securities are being moved in. Software companies themselves have always focused on making something that's sellable. We want it to be slick. We want it to sell. Security is not an issue. And of course we've always had problems with leadership not understanding the problem, system administrators being under financed and such. So that has left us in a very vulnerable state so these threat actors regardless of what they are can come at a vulnerable system and it doesn't matter if it's a hacker or a script kitty or all the way up to the nation state they can still manifest themselves the same way. So this is why we're really concerned about the bottom end where there are so many. And I say the word hacker and we all understand the connotation hacker, cracker, et cetera. I'm just gonna use the term that everybody likes to use. I label myself as a hacker also. I've been doing this stuff for years but not illegally. I've stayed on the right side of the law but the threat of course is the nation state. I mean we would expect to see China or Russia or some large nation want to come at us nation state against nation state. Clearly the largest threat are infrastructures. But what happens if some kid out there playing around on his parents' computer high speed connection all of a sudden it's like a million monkeys typing on a million keyboards. One of them eventually types some Shakespeare. Well what's the number of hackers that are out there? Are we up to a million yet? You know is it a million monkeys typing on a million keyboards? It's the same phenomenon. One of them will get lucky. And that one individual will probably score a hit much larger than some nation state can score. So we're very concerned about that phenomena, that one individual that will get it right. So we take it serious. It doesn't matter where the attack's coming from, it doesn't matter who's breaking into government systems, critical infrastructures, we assume the worst. Assume it's a nation state and then we start backing off from there until we can get clear attribution. This is a very hard lesson because when you have people who are trying to just kind of probe at us and let's see if we can screw with the monkey a little bit. What they don't realize is that on our side of the coin we don't know where it's coming from. We have to take it serious in terms of protecting the critical infrastructures and we may misfire not recognizing that it's just somebody out there pranking about. Couple of things I just want to leave you with. We've got strategies and Homeland Security. You can pull them off the White House website. It's the best place to get it. Be sure it's WhiteHouse.gov not WhiteHouse.com. None of y'all have been there before so you're just laughing with me, right? Not laughing at me. Go ahead and please if you do, download the cybersecurity strategy as well as the physical strategy. If you look in the cyber one, you'll see there's a cup. There are paragraphs in there where we talk about the threats. We characterize what these threats look like but we also make a case that say even though we want to understand the threats, we want to understand the risk equation, system administrators and leaders can really do something about vulnerabilities. If you can cut down on the vulnerabilities, cut down on the exposures, patch systems, turn off processes that aren't in use, have a better password policy, train your users, those types of vulnerabilities, it really then doesn't matter where the threat comes from. It doesn't matter how it manifests itself because you've protected yourself, you've gotten rid of the low hanging fruit and now the threat can go next door and go after somebody else. So that's a key point that we try and drive home. In the strategy we have five priorities. I've just shown number two here because we also address that what we're doing at Homeland Security out of these five priorities, number two and they're all equally important is to address the concept of threat reduction and vulnerability reduction. And we're gonna be putting some serious dollars into this into R&D as well as public awareness and training and getting the government more secure, which of course will then mean the buying and purchasing of products which brings product prices down. So we're doing quite a bit of work here to move this forward and we absolutely appreciate all the input we're getting from the private sector, from industry and from private citizens. So please keep it coming. Tell us how we can do this stuff better. With that, I think we open up to questions and we've got at least 20 minutes or so unless you're thirsty. I mean, and let me sit down and then we'll just take the questions as they come. Or do you want to? All right. Any questions? Got one right here. He's asking in my second slide, I say some should be blocked and some shouldn't. And I don't know, I'll go to my slide and you can point it out to me. One, two. Is that the slide you're speaking to? Yeah, I'm not speaking specifically to terrorists or cyber terrorists and speaking within the context of risk management and taking kind of a private industry perspective. It's not fair to ask a particular private industry to protect against the nation state threat perhaps. There might be risks there or exposure that are only in the context of the capabilities of a nation state and in their risk assessment model, it's not gonna be likely that they're gonna protect against it whereas a government system where they see themselves as a greater threat or a greater risk to facing that adversary, they might hold them to that higher level. It's all about the concept of acceptable risk and doing risk management. You'll never have full protection but if you do the threat characterization then at least you can start to say in the industry that we're in, we think we need to protect against the following threats that have the following types of capabilities and we might leave ourself open to a high-end adversary that has a higher end capability but because we have limited resources, we're gonna acknowledge the fact that we can't protect against everything. Yeah. What about the striking asymmetry of the physical security where the future security, we have limited resources in both of the worlds. Now, physical security, it's not that 100% of the people are going with weapons on the street and try to protect against all possible threats. We have police and we have a big incentive not to do any crimes on the street since it's the punishing of the severe. Now, I didn't see for any of the discussions particularly I remember from the moment security government that security should be international and this can be done in one country but I think that you lose a significant job by trying to achieve perfect security. We don't have enough resources to do that. No, no chance. Mark, did you... So what's your question? My question, I don't see significant efforts. I see that the moment security is still trying to focus on the physical protection of our individual systems. Instead, I think we should try to build a process in an allowance for policing and for enforcement. Yeah, we didn't talk, I mean, there's a lot more I didn't talk about. You got a table mic. You guys can hear me, right? I think they're recording those. Are we recording? Yeah. All right, just one, two. There's a lot more we didn't talk about obviously. Where'd you go? There you are. You're blending again, you're wearing black. I'm sorry. Crime prevention, for example, is one aspect of security. Vulnerability reduction is another aspect of security. Threat characterization is another aspect of it. Those in the law enforcement world, when we ask them what is security, a lot of times crime prevention is the angle they go after. What can we do? Just like in the physical world, to prevent crimes, we put locks on doors, you know, we close windows, you park your car in a well-lit area. All that's oriented on crime prevention, but if somebody breaks into your car, like you say, there are mechanisms for prosecution, for surveillance, for arresting and such. We are making a lot of strides in doing that in the cyber world as well, but naturally it's well behind the physical world. In terms of spending, that becomes a risk assessment for the various companies or individuals or whoever's having to make that assessment. Do you put your dollars into physical or do you put your dollars into cyber? That's a decision you have to make based on your risk assessments. Where is the most likely attack against your resources going to come from? And where will the largest dollar damage or whatever the currency is of your country if somebody attacks? What would one shot do to you? If somebody took one shot at a server and took it out, you can rebuild it. What if they took a second one or a third one or a fourth one? At what point does your company break? Or if somebody physically went in and raided your warehouse and did a physical theft, how many of those instances can you survive before your company fails? That's a risk assessment that companies have to take on to their own. We can't tell you how to do that. So there's one clear thing missing from your model and it's insurance. We don't have insurance in the original world like we did here and there isn't enough insurance about the crime score. Now we're going to get way off subject because we came here to talk about threat characterization and you're right, insurance is mitigating some risk by transferring the cost to somebody else. But let's talk insurance offline if you want to do that, okay? Thanks for the question. Yeah, go ahead. Let me just repeat the question because I know we promised we'd do that. You want to know where the rating the hacker falls into during the incident management process? Tell me. For instance, by the way, just as a general reference, this whole rating the hacker thing, I plan on putting into a script that way people won't have to actually have a long checklist and it takes an hour to do it. But to get to your point, it would help if you run through the whole thing and it comes up with your adversaries, say a script kitty, you might not want to dedicate your whole incident response team to doing the forensics, doing this, doing that, bringing the box down, whatever's part of your incident response. Whereas if you come up and the whole attack it comes up and says it's a professional, then you can dedicate your resources. It's a tool that management can use to gauge not only to threat level, but what kind of resources they're gonna need in order to respond to any kind of an incident. You could use this even on false positives and intrusion detection, you know? So. Let's go right over here. Yeah. The question is, how do you differentiate based on attribution when someone might do parallel paths of attack, one which is really noisy, one which is stealthy that are linked to the same person, or you might have two attacks that you link to the same person that actually aren't? Correct? Go ahead. To be honest with you, there's no way you can tell. If a guy hacks a box in Canada and then goes out and hacks a box and hungry and attacks you at the same time, there's no way you can tell it's the same person. Unless you start looking at statistics, which right now we don't have, but we're gonna start getting that statistics. But then again, profiling a rating is never 100% accurate anyway. So the data that I'm working on is coming off my own honeypots and my own experiences within my organization. But we do have plans and I am talking to other people about getting more data to create sampling, to give more statistics rather than just a model. You got anything? As I said earlier, the technical analysis is not the be-all and end-all of the overall characterization, it's only a part of it. But obviously in the case of, so you're saying if you had two attacks on a single machine from two separate locations, right? Okay, well, I mean obviously there's a question of timeframe, if you have a very quiet network and you have, or if you have a busy network in a machine which maybe people don't know about in the public domain and you suddenly get an attack from two separate locations within the same hour and you haven't had any attacks on that system within the last six months, and obviously that might give you an inkling that someone is, or a group of people are trying to target that system. Then obviously you look at the method of attack they're using, the reconnaissance methods they're using. Do you think that in the second attack they're using information they gathered during the first attack? Are they using information in the second attack that they can possibly have known without carrying it the first attack? So I think there are ways which you can establish whether it's. I had a question over here and then I see you over on that side. Yeah, go ahead. The question is on capabilities of terrorists and disposability and will you get the same impact from a cyber terrorism attack as you would from a physical attack? With regards to capabilities, I agree. Typically when I talk on cyber terrorism it's a two hour presentation and one of the slides that I put up is kind of the state of where we see things at right now and it says those with the intent lack the capability. So yeah, they might intend to try and take the power down on the East Coast but they don't have the capability right now. The caveat there though is that we know that this is something that's on their radar screen. We know that they're trying to acquire the capability so we need to be planning for that out over the long term. The other aspect is we don't know how far in their planning window or capability acquisition process that they are. So just because it's not something that they currently could do or are assessed as being capable of doesn't mean that it's not a threat that we're gonna face and we categorize that it's more in the emerging threat category. Also have to remember the issues of convergence where you might have a threat entity or an individual or group that is capable of launching the attack that either for monetary reasons or ideological or political reasons would converge with the terrorist organization and engage in attack that would fall under a more of a cyber-terrorism profile. With regards to impact, I agree the physically blowing stuff up is much more sensational, has a much higher impact and we've debated on this issue since like 1995 over whether can you even call cyber-terrorism terrorism because you're talking about different aspects. In the definition that I provided, we think we need to look at it from a terrorism perspective where it's a sustained attack, it's causing either political coercion, public panic, what we call disruption of social integrity where it starts to impact the operation of the nation, maybe your foreign policy. So we wanted to, even though it's not as sensational, we wanted to address it within that terrorism context. Right here in the gray shirt. Can you speak up a little bit please? But going back to our examples, if I'm going to attack you and I have to know that you are trying to set up a profile, basically I'm going to sacrifice a kid for you and then I'm going to use some of my better self techniques to go in and basically believe all the things. There's no way you're going to see them. And my point is that whenever your profile comes up, your drug analysis, all this, are going to come up as, okay, we have to start getting down to the worry about it, because you basically have to lose control of yourself, whereas the real attack, you probably won't see some errors in the real value of the drugs that's going to be described here. There's a question with regards to value and utility. I won't repeat the whole thing, but I'll let Tom, Toby, address it. As I said earlier, I'll say it again, the side you're looking at is firstly completely technical. It wouldn't take into account any kind of psychological profiling of, you may consider yourself a high-end adversary and not a script pity, and maybe that is a technique that high-end adversaries would use to maybe flood the IDS, so an administrator spends days looking through his IDS logs, and by the time he finds those ICMP packets that you're talking about, the system is already being compromised. But there are two sides to this. There are the theoretical threat modeling side and an incident response stage. And in the theoretical side, we've discussed this as a group, and there's every chance that someone may do exactly what you're speaking of, and the threat model would take that into account. On a practical side, well, it really wouldn't matter, because the incident is going to happen whether you like it or not. All you can do is look at why that attack was successful against your network and how we can adapt the threat model to... Okay. So what you're saying is you don't think that we can learn from characterizations made after an incident. Okay, well, obviously, you're going to... Once an incident has happened, you're going to change your threat model based on your characterizations. There's going to be the side of, well, in this attack, how much reconnaissance was done? Did the guy use social engineering attacks prior to making the compromise? These things, as I said earlier about the whole social engineering age deviation thing, that tells us some stuff about the attacker. It's all little pieces of the pie that we, in this case, might use to pass onto the FBI to help characterize, to help find who we're looking for that committed the... Furthermore, say if you're running a server on a bank network, let's say, it may help you characterize the kind of people who are likely, who are going to try and get into that machine, the facility that they have available to them in order to do that, the exploits that they have access to, the people that they talk to, the people that they know, how high are they on that vulnerability disclosure food chain? Do you see where we're coming from? Let me just add a couple of points to that. No model is ever going to be perfect, and that's agreed, and there's probably a lot of folks in here that would be sophisticated adversary, and they're going to be able to circumvent or exploit the model in order to launch their attack. I have to remember, though, there's probably 99% of the folks that are out there aren't going to, and you're going to be able to characterize a lot of those and use that to guide resources. From a planning perspective, or at least from my perspective, when I look at being able to characterize and do scorecards, even if it's historical analysis, I can now start applying that to the future. We talked about this at Black Hat as well. One thing that we're lacking is, when the FBI goes out, it does a psychological profile of serial killers. They go out and interview 100 serial killers, and they start to develop the profile. In our industry, we don't do any of that profiling. We have a planning component and look at adversary levels, and I have to do this on a daily basis. Mark has to do it on a daily basis. We've got those threat agent types up there, but we don't have any real historical background to talk about what the capabilities are. Lessons learned, and as you start to have a historical use of the model, you can start applying it to planning in the future. And the real-time driving of resources for those that might not be sophisticated enough to circumvent that particular model you're using. Add one over here, and then I'll get over there. Yep. Let me just step out on a limb a little bit here. It's a hard question to answer, unless you're looking at it forensically in terms of gaining attribution as to who exactly did what we're looking at. Part of this is not just the pattern of behavior in terms of, did they break in first by scanning then by gaining a foothold, etc., but capturing the actual artifacts, building a library of artifacts, being able to compare those against older signatures, for example. Much like profiling in the physical criminal world, if you've got a serial killer, they will leave behind forensic evidence that's collected and compared, and you can compare it against other person X, person Y, person Z. And you can start to build the profile where you can see characteristics of individuals or groups based on the evidence they leave behind. Now that will eventually, if done correctly, lead you to attribution towards the actual person you're looking for, but it does help characterize when you see a new event come up. If you can compare backwards, you can find those signatures or fingerprints so that you can continue the pattern. It helps to understand from a forensic standpoint. The gentleman's earlier question about how does that help me in the future to prevent intrusions. It really doesn't provide you with much there. The way to prevent that is patch yourself and close the vulnerabilities. What we're looking at is how do you then characterize somebody who's already broken in to gain attribution to bring that case to closure. And yes, you can spoof it. Yes, you can introduce noise. That's all part of the signature. And you would learn those characteristics of your adversary who does things such as making themselves look like a script kitty or making themselves look like something that they're not as a smoke screen. That'll be a characteristic of that individual or of that group. Take two more questions and then we'll have to call it quits, I think, on the room unless I get the nod otherwise, but we'll certainly all be here. Go ahead right here in the back. The question to put it briefly is what role do you see the model playing in doing trend analysis over time to guide response, internal security posture, alerting, things of that sort, Toby? Of course, the data being shared where the community goes, yes. The data will be shared with the community. We plan on doing the trend analysis. We also plan on trying to get more people in here who can, in our little group who can provide more information that we can use in our trend analysis and that can help us, you know, do the profiling that can help us make this a more value-added model than what we currently have. I don't know if I answered the question. We've got one more back here, then we've got to release the room but we'll all be up at the front for questions and answers. You can ask the question if you've got a beer in your hand, Mark will answer. Yeah. The question is does Homeland Security have any plans to incorporate any sort of funding for training folks that operate critical infrastructure? Yes, is the short answer. One of the major things that the department will be doing in terms of outreach training and awareness is exactly that, raising the level of understanding. From a pure cyber perspective, of course, this is a shared responsibility. It's not something that just the government can do. Each of us have to work with it. From a larger, more physical perspective in terms of the critical infrastructures, part of our role in this is determining the interdependencies of those infrastructures. How does the rail network depend on the power grid? Some of it's kind of obvious, but there are other interdependencies that we're trying to uncover through doing simulation modeling, et cetera. But in terms of the education awareness, absolutely. There's a large part of the department that's being focused on that. We're somewhat immature at the moment. We're only about six months old. If you come back here next summer, I can give you a lot better answers to it. I think you'll like what you'll see. Thanks a lot, ladies and gentlemen. I really appreciate your time.