 I'm just going to do a quick introduction, then we'll move into the discussion. Someone asked me when I was coming in, so did you start working on this when Matt Kofi approached you? And the answer was no. I'd been working on it beforehand for the following reason, which is so a lot of times when you write about cybersecurity, if you're not saying electronic Pearl Harbor, then you're saying the cost of this is worse than the Black deaths. And so I sort of do this myself. And so I was thinking, how do I actually know it's as bad as we say it is? And so we went out and looked to see if there were estimates of how bad things were and found a range of estimates, the lowest being $6 billion and the highest being $1 trillion. Generally, from an economist's point of view, you'd prefer a narrower range. So that's kind of what we started out thinking to do is, how can we narrow the range? How can we make this more precise? And working with Stuart Baker, who's stuck in Texas, unfortunately, we have Teddy, who is going to the State Department here to help out and fill in for him. Phyllis Schnett, who many of you know, has been doing this for a while and is one of the experts. And of course, Tom Gann here in Washington. So really grateful to Matt Kofi for supporting the effort. We had a group of people come in a few months ago to go through an initial discussion draft we had. They were very helpful. I won't list them all, but I'd like to thank all the experts who showed up, including people from ITC and the NIC and Scott Borg and just a great group of people. So we've had a lot of help in doing this. This is, as you've probably seen, the interim report. And we will work on making it more precise, answering some of the questions that we raised here as we move forward. So with that, I think, do I turn it over to you, Tom, or what's the, do I keep talking? Yeah, I think that sounds good. I was going to come in and focus on the moderated dialogue. OK, great. So that means me, I guess. I'll start. When you look at a lot of the reports out there, we came up with a set of questions that if you were going to try and do the estimate a little more accurately, what would you actually have to think about? So the first thing we asked ourselves was what should we count, right? And a lot of times people look at IP theft. That's certainly the big one. People look at financial crime. We threw in opportunity costs. There was a good study done out of Cambridge University a year or two ago that talked a lot about opportunity cost. We looked at the additional cost of security. And people have to spend more because the internet is not secure. And we looked at insurance and recovery costs, a whole set of things. And finally, we looked at reputational damage, which there's also been some good work done on out of the University of Maryland. Two kinds of reputational damage. There's the stock damage, which appears to be the stock valuation damage, which used to be relatively quick. People got hacked. It went down. Now it comes up. Seems to be not recovering as quickly. The second part of reputational damage might be to the brand or something. So when you've got a counterfeit good or you've got a competing product. So we thought those categories of things, IP theft, financial crime, opportunity cost, security cost, and reputational damage were a good start as saying, what should you count? Where we came out. And I'll do the punchline up front. And then we'll move to how we came there. And Teddy and I will talk about it. Is it reasonably an upper limit? It might be somewhere between under 1% of GDP. That's a best guess. This is what we're going to refine. But if you do 1% of GDP, depending on where you're looking, it's somewhere between $70 billion, maybe $120 billion. That's a first guess. And of course, as I said, there's credible estimates that give a much lower number. There are credible estimates that give a much higher number. And so we want to try and narrow it down. One of the things we thought about is, OK, so here are the categories that go into the estimating the cost. How do we count it? How do we actually figure out what the numbers are? And it turns out this is really a lot of fun because there is no data. And that's a major impediment in doing this. It's not so much of an impediment for me, because I used to do in graduate school medieval economic history where the data is bad, corrupt, uses different measurements based on anecdote. So this is a fairly normal estimation problem in some fields, either intelligence or in economic history. But it is a problem. The data is either sparse or distorted. A lot of the previous work depended on surveys. And surveys, as you know, are very difficult to do in a secure fashion. You can get all kinds of biases read into the survey. One of the biggest ones is self-selection. So if I send out a survey to all of you people in the room, maybe only certain of you would answer. And that's not necessarily a good way to do survey data. So we are trying to avoid a survey. Previously, we had done a survey and weren't happy with the results. So survey data is a problem. Anecdotes are fun. And one of the questions that we did at the beginning of this exercise was how many anecdotes does it take to form a database? The answer is it doesn't matter. Because again, the anecdotes may not be distributed in a way that accurately reflects what's really going on. Anecdotes are useful in Washington. Of course, you have to have a story for everything. But they're not necessarily a good approach to this. And the final thing we came up with was extrapolation. And this is where some of the previous estimates have gone wrong. You have some number that you feel fairly comfortable with. And you say, well, it actually represents some percentage of the estimate of cybercrime. And then we can multiply it by some factor. There was some testimony last week. I saw one of our colleagues from the GAO gave some good testimony, particularly in oral remarks, on the problems with extrapolating from something you know to get the bigger picture. And this is also a problem for us because at the end of the day, we're probably going to have to do some kind of extrapolation given the limits on data. So how do you exercise care when you do these things? One of the things we started out doing, I actually did in the lead-up to the McAfee thing beforehand, was, well, what are some analogies? I mean, if it is the worst thing since the Black Death, the Black Death was sort of a noticeable event. And why does this not fall into that category? So we looked at car crashes, good data on car crashes, good measurements, slightly different figures. Again, they fall roughly in about the 1% of GDP range. We looked at drugs and narcotics. One of the lines used to hear was that cybercrime cost more than narcotic trafficking. The question we came up with after looking at it was, that's really interesting. How do you know, right? And the answer is we don't know. We don't have a good handle on narcotics trafficking. But the UN has put out some data, a little bit higher than the 1% range, but depending on what you look at. We looked at maritime piracy. We thought that was acute analogy. It's a global commons, and this is malicious behavior. What we found is that piracy is actually just a fraction of a percent of the value of global shipping. That one, if you were going to say narcotics trafficking is a bit higher, piracy is a bit lower. Finally, we looked at pilfering. All companies expect to have pilfering. They call it shrinkage sometimes. They expect shoplifting. They expect people to take things out of the warehouse. They expect stuff to break. Companies factor this in as the cost of doing business. And so we said, how much is pilferage a good analogy for cybercrime? And that's actually one of the questions we've come up with that we'll look at in more detail, which is maybe people accept this as the cost of doing business. They accept cyber losses as the cost of doing business in cyberspace. There's two problems with that. First, the cost to society might be greater than the cost of the individual company. This is a traditional economic problem. There's ways to deal with it. The second problem, and one that's perhaps more important, is one of the things we discovered is very often companies don't know what they have lost. Good data on this, or companies only discover the loss after some time, or companies really don't have a good idea of how much their intellectual property is valued at. So people could be making business decisions, reasonable business decisions, based on their assessment of risk and loss that says, this is an acceptable loss for working in cyberspace. But that reasonable risk decision could be based on faulty information from an underestimate of how much is actually being taken. One of the surprises is that the US, thanks to some guy named Snowden, I think we can now say this, the US has a really roughly a fair idea of what is resident on foreign computer networks in terms of lost intellectual property. And the scope of this is much greater than we thought. It's much greater than perhaps companies are aware of. Now there's a whole set of issues between having the intellectual property on a computer and turning it into a competing product. But we came away with an initial conclusion that perhaps companies are underestimating the risk. And that would be a good reason to get another bound on this. One of the things we want to think about then is how do you measure effect? It's not one for one. If you have $100 of IP, someone steals it and puts it on their computer network, that does not translate into $100 of gain for their economy. So there's some loss here. If I give you the plans for something, you need the ability to build it. You need to have marketing. You need to be able to translate it into whatever language you're looking at. It turns out there's a lot of friction, a lot of hurdles, into taking the stolen intellectual property and turning it into a competing good. And if it's not a competing good, there might be a limited economic effect. This is a strange place where trade and security blend. So you've all seen, thanks to some of the, including the post, some of the reporting on what's been lost in terms of military technology. It's hard to value that. What is the actual cost to U.S. national security? So we're trying to figure out there's an effect on trade. There's an effect on security. How do you come up with a good measurement of those things? And the way we're trying to do that is we're looking at the trade effect, this clearly affects trade and competitiveness. It clearly harms countries that are IP dependent economies more than others. There's an effect on jobs. The good number to the extent is the commerce number that a billion dollars in lost trade translates into roughly 5,000 lost jobs. So if you can say, well, this is how much we lost in cyber IP, there's some way we can come up with a job loss. And other people have looked at this, but it looks to us like the job losses are significant enough to pay attention to. The one that I think is the most interesting and that we will do a lot of work on in the future is the effect on innovation and growth. This distorts investment incentives. Cyber espionage distorts investment incentives. And that will affect growth. It will affect where companies decide to invest in the future. It will affect the benefits of R&D. So an area that could use some more work. Finally, there is the military effect, the loss of military technology, which people are familiar with. It's hard to come up with a number, but it's something to look at. I'm almost done. One of the things we'll try in the future is we'll try and do a bit more of a breakdown by sector, IP sector, IP dependent sectors are going to be more greatly affected than sectors that are less dependent on IP. We're going to try and do a regional breakdown, depending on how developed the economy is, how much it depends on intellectual property to generate value. The effect will be different. You can obviously see a very underdeveloped economy isn't going to be at risk from cyber espionage as much as a developed economy. So working in that kind of global estimate, we've already gotten some good data, thanks to McAfee and some other folks on some South American economies, on European economies, on Asian economies. We're hoping we can extend that to get a better global overview. What's the net effect? I gave you the bottom line up front, but that's our best guess. We're going to refine it, right? Cyber espionage is one factor among many that affects growth and affects competitiveness. It differs, as I said, from sector to sector. The one thing that we have found though is cyber espionage has something that traditional economic espionage does not. It has economies of scale. It's so easy to do this. It's so easy to take information that we think that the effects are probably easy to understate. So with that, why don't I turn over to Teddy and we'll be going through a little discussion, then we'll take your questions. Thanks a lot. So my boss, Stuart Baker, sends his apologies. He got called away on an urgent matter. So you can just think of me as his younger avatar. And he said I can allow it to say with a bit more hair. We actually talked about this yesterday, Stuart. And I said, Stuart, if you send somebody with a lot of hair, it's going to make you and me look bad. But it couldn't be helped. We thought about having a cut out as well. Kinko's was closed. I think for those of us that see the costs of cyber crime and cyber espionage regularly, the costs are visible. But I think a challenge to recognize is how, for those that don't see it, for those that don't work with these issues regularly, for corporate boards that perhaps are concerned, don't necessarily get into the guts of IT issues, how do you present this objectively? And so a part of the intent here is to come up with an objective measure of what this actually costs our economy. And the interesting thing is that a certain amount, the pathologies that make cyber crime and espionage difficult to combat, also in many respects make it difficult, it's costs difficult to measure. People don't know, many, many, many victims of cyber crime and cyber espionage don't know that they're attacked. When they do find out, they don't necessarily know what was taken. Even when they find out what was taken, it's hard to figure out exactly what that's gonna cost them. And then even for those that know that, many don't wanna speak about it publicly. And so these are the kinds of factors that impact, particularly survey-based attempts to come to terms with what this means. All of those things make it difficult to get accurate answers. And then you add to that the problem that the problem of selective responses that the companies that do know the costs, some of them may, for example, have suffered very catastrophic attacks. And it may not be fair to extrapolate that across the entire economy. So this is an interesting study because what it tries to do is apply a bit of an objective measure. And I think you have to acknowledge when you try to quantify the cost of cyber crime and espionage that you probably never come up with a single number. It's always going to be arranged. Some things are easy to quantify, like, for example, the cost of identity theft. But as the report discussed, it's more difficult to quantify what happens when people trust the internet list. And so the attempt here is to at least come up with a range in order to do that. Now, I think another point is this is, a real piece of the study has been trying to consult with experts on this. And so this began with a bit of a blank slate and working with experts on this. And in fact, this report is a part of the conversation about how best to value it. There is going to be a further report. And so a part of today is also taking questions and being able to discuss this. So thank you. Yeah. Good afternoon, and I want to thank CSIS and our audience and jump into the impetus and why we asked, why McAfee asked CSIS to write this report. And first and foremost, we wanted to open the dialogue. And there have been a lot of numbers out there. McAfee and others have used that big $1 trillion number for a while, and that was based on surveys, as was said before. And what we've discovered as we look at this is it's a much, much deeper issue. And we look at the importance in why we need a number like this or why we need figures like this and estimation is purely for cyber resilience. The ability to make sure that our networks run while they're under attack, to ensure that mission stays up. And the way that we do that, we talk a lot about this being a boardroom issue. Now when you think about how you get the people in the boardroom and the people that make decisions in this country and in others and our business and our government, it's where do we invest wisely? And I have it in my mind, there's a part on page 12 of the report that really talks about one day when you'll spend less on cybersecurity. Believe it or not, as a vendor, that is our goal. And we want to make sure that networks get safer. And how do you do that is by understanding the risk. So one of the key things in good risk assessment and understanding your assets is this global picture of what are the effects of cyber espionage and cyber crime. One of the neat and different things about this that we asked CSIS to do is first up, and they do this early in the report and well is define that. And there are a list of things from espionage to pure crime that are included. The other pieces look at following the trail. This is very different than just a survey. Pure surveys are a victim of the fact that many companies don't report, aren't aware. So we're missing, not only learning from those events, we're missing the fact that it happened. In this study, they look at, for example, if intellectual property is transferred from one country or one company to another, what if the destination or target company doesn't use it? And it takes into account that maybe there wasn't necessarily an economic loss per se to the originator, but also includes in the right way that it happens. And really following that and taking it many levels and many orders of magnitude further than a standard survey, because in the end, we don't think we'll come to one exact number, right? And that number would be always changing if we did. What we wanna do is correct, taking a nice big number that people have used in the past and say forward, we need to take a much deeper look at what makes this thing move and how when we go into looking at cyber investment, whether it's building tools or advising on how networks are provided for, taking into account how the world around the bits and bytes works, because cybersecurity really isn't about just networks and IT. It's about everything else that we maintain from our lights to our water, our gas, electric, and the food we eat. And that means understanding how the world around it works. So these impacts, we hope in the future, we went to the best of the best to start looking globally at how do we understand a framework for getting the right way to advise the public to invest companies and government and cybersecurity with a much sharper, more accurate target. Because I think what we've learned is you can't teach cool and you can't survey cybersecurity. All right, well, I think this has been a fascinating discussion. I think my role here is to really moderate the next and final panel where we're getting the benefit of the views of all of our panelists here from CSIS and also Phyllis Schneck, McAfee's Chief Technology Officer. So toward that end, I think a very interesting question is, given the fact that many numbers have been put out in the past by different analysts, different think tanks on the true cost of cyber crime, what type of methodology do you think is the best? Why is the approach that CSIS took to be preferred over others if indeed it is to be so? Jim? I guess I'm the stucky on that one. And I wanted to have some props here today. One of them, of course, was the magic eight ball, which I used to have for cyber crime that, you know, how much does cyber crime cost? Well, it cost X. If you didn't like that number, you shook the eight ball and got a different number. Things have gotten better since then and there have been some good reports done. In particular, the work out of Cambridge, a few other places have jail, have done a pretty good job of beginning to narrow the scope on this. What we wanted to do was not so much critique other reports, but to look at where there were areas of strength and where there were things they might have missed. So let me give you an example. One of the things that is hard to do but not impossible is to think about how you would quantify risk, right? And how you would quantify the cost of risk. And as we move into an environment where more devices will be dependent on cyber technologies or digital technologies, you should be able to see some change in risk. So when your car is being run by an internet enabled device and you are not really the driver anymore, what will the effect be on automobile insurance? So what we wanted to do was take sort of a approach that would say here are areas where we have in the past been able to successfully quantify risk. Here are ways we could perhaps transfer that to cybersecurity and here are places where the data is insufficient for us to do a good job in saying that. And I think one of the things that we hope to do is just be pretty frank. If we couldn't find data, we're just gonna say so. And then we will try and come up with some way to estimate what that would look like had we been able to find the right numbers. But you know, this is gonna be a range. Just a quick story and I'll stop talking. A while ago, we had some, one of the CSIS's more senior board members who's a well known economist come in and say, you know, people telling me you can tell me about this cyber espionage stuff. How much do you think it costs? And, you know, I apologize to them. I said, I'm sorry, you know, this is embarrassing, but the best we can do right now is a range of between 24 billion and say 140 billion. And he just said, you know, we gotta do better than that, right? And so I think what we hope to do is say, can we narrow, it will still be a range. We're not gonna give you the number, but we will be give you a range that's a little more precise based on a little bit more of a broad take on this. You know, the question of the right number is certainly interesting, but in your report also, you've come up with the question of the impact on loss of U.S. jobs. You know, from a CSIS point of view, why is that important? And, you know, likewise, do you feel as living? Well, the looking at lost jobs is actually a pretty unique characteristic of this report. It's, I think, the first report, or one of the first reports to do that. Thank you. And it's based on Commerce Department numbers that are pretty current and basically extrapolating from what the cost of loss might be and applying that then to the Commerce Department figures in terms of job loss for, I think, a billion dollars. One thing to add is a big concern in cybersecurity, both past and forward is our workforce and recruiting the absolute best and top talent and really having them jump into an exciting field and continue innovating. And if there's a perceived or and or actual job loss in that area, you'll see a downward shift in the incentive to go into that area, learn about these sciences, and you'll actually see probably less cybersecurity because fewer of our good minds will focus on that because companies won't have the slots in which to hire those people and you'll probably see innovation have to take a side step to just core job cuts. So all of this is connected. It's a big ecosystem. When it comes to network resilience, you need top minds on your top problems. You need some automated responses in some of the other areas where we can do that. And we also need to focus so much on these economic pieces because that's going to feed it from the bottom up. The report certainly devoted a good deal of attention rightfully so to the question of intellectual property. Why is IP so hard to measure? And why don't businesses and government do a better job of factoring in the loss of IP into their strategies and public policies? I think the challenge with IP is that there are, on the one hand, you can count the inputs, but the inputs don't really tell you what the IP is actually going to be worth both to the company that loses it and to the people that take it. So it is a challenge because one question you have to ask is what was the company gonna make with the IP without it coming out? A second is what happens to the IP once it gets taken? Does it just sit on someone else's server? Does it go to a competitor? Does the competitor have other know-how that they can use to actually then use the IP in order to take something to market? I think an interesting question is what's the net effect on the world economy versus what's the net effect on the individual company that loses the IP? All of those questions I think require a lot of teasing out. And I think the report names a lot of them, but I think that's an important step going forward. The example we used, I think we ended up cutting it out, was the cement wheel. And it relates to valuation in some ways. If you spend, if you're a company and you spend a billion dollars to design a square cement wheel, how much is that IP worth? And some of the problems with earlier estimates is they said, well, it's worth a billion dollars because yet so much you spent. And our sort of take was no, it's worth zero because that's how much the market will pay for a square cement wheel. So you gotta get somewhere between those two. That's where we have said, what is the effect on innovation? This clearly has some distorting effect. I think I feel safe saying that now. We aren't quite sure how we can measure that. And one of the things that we're gonna look for is does cyber espionage shift the production of innovation from more efficient innovators to less efficient innovators? This would be something to look at. Now this is a hypothesis. We have to test it, right? But if you look at how innovation occurs in the world, some countries are really good at it, other countries aren't. And there's a whole set of explanations for that. Some of which are fixable, so countries can become more innovative. But if you have countries that are good at innovation now, they're gonna be targets of cyber espionage. This is gonna reduce the value of that activity. So what we might be seeing is this, as Teddy was talking about this effect on global innovation, where you're shifting the creation of new ideas away from the most efficient creators to less efficient creators. And the overall outcome might be a smaller stock of innovation for the world. But that's to be determined. It's an interesting hypothesis. We're gonna have a lot of fun playing with it. Right now it looks like it's worth pursuing. So I've got one last question for the panel and then having dealt with that, then we'll open up the discussion to our audience and we very much look forward to your questions likewise. So the last question, I think this is really one that is the most appropriate for you, Phyllis. You know, there seems to be a growth in cyber espionage. Is in fact cyber espionage increasing or is just the awareness of it seemingly increasing? I think we're seeing both. The awareness is obviously something that you sense and something that we hear about. The quantitative evidence comes in, for example, things that we've led at McAfee Labs from Citadel investigation to the latest with Operation Troy, respectively that the Citadel looked at pieces of malware that were all over Northern Europe. Places you wouldn't suspect and we were tracking it. And what it showed us is that the presence of malware is far more prevalent than we might know about through normal means. When you investigate it and you track it back to the machines that are sending, for example, trying to send our customers not so great things, you start to see the worldwide footprint of this. Same with Operation Troy, with an even scarier factor in that in that some of the events you saw back in March targeted toward South Korean institutions can be tied all the way back to December of 2010. So when you look at is it increasing? Yes, it's increasing, but we're also increasing our awareness of what's happening forward and tying that back with the chronology of what happened before, both from a scientific discussion, tracking everything from procedures to encryption keys that tie it together, as well as looking at patterns and looking at actual information shifting, as we saw in, for example, Night Dragon three years ago with copies of oil exploration diagrams crossing boundaries overnight. So the problem really is growing, but back to why we're looking at this report, there hasn't been up to now a comprehensive multi-level methodology that really looked at what is actually happening and what are the effects economically? There's a lot of noise and a lot of chaos and a lot of buzzwords, but we really need to get through what does this mean and what does it mean for network resilience because the problem isn't going to go away, but the resilience can certainly make our lives a lot safer and that's our goal here. Well, very fine. I think this was a very insightful and useful dialogue. I think we have a colleague here from another think tank Brookings and I look forward to your perspectives. Brookings, I really appreciated a panel from Marks and I'm going to read the study. I wanted to have Phyllis's comments on the, sorry. I wanted to have Phyllis's comment on the idea of using these estimates to actually drive action and could you start by telling us a little bit about all the variables that you looked at? Which one has the most power? Which one, if we knew more about, we would have the most predictive ability to say this would raise or lower the impact and similarly does that variable also give us some policy liver points to start looking at? And I should say that Alan was one of the people who helped us out when we had one of our working group readings, but he left early so you can't blame him for any of the things that are in there. It looks like it's going to be intellectual property. I'm not 100% confident of that because some of the other things haven't been looked at as deeply, but that is certainly the greatest single source of value. Whether it's the greatest single source of cost, we don't know and so maybe some of the rectification costs, the greater security costs might be up there, but not a surprise to anyone that it looks like it will be intellectual property. So the original thought we had and it used to live on a whiteboard and now I think it's gone. The original thought we had was if we could come up with a good model that had all of the major variables, we could then say here's the model and at some point we'll have the data that we can plug into the model that will let us come up with a more precise estimate. So that's still sort of the goal is let's get the model that will explain the damages from poor cybersecurity from hacking and then as a second order problem, see if we can get numbers to populate that model. And I think that is sort of behind the research strategy when on the days we remember that's what we're doing. The, our colleague in the front row. Jim's never spoken that kindly of me before, colleague, that is. Having not seen the report and particularly not seen your definitions, this question may be off the mark a bit, but it goes to the distinctions between cyber crime and cyber espionage. And I recognize that's not the immediate focus, but when you get to the world of what are we gonna do about it? It seems to me there are two sets of things that deal with what are we gonna do about it? And sort of gross simplification, the role of the private sector and a bit of government is relative to cyber crime and the role of what are we gonna do about cyber espionage pretty much as squarely the role of government in terms of all sorts of things governments can do. In the NCIS study we saw earlier this year they failed miserably I think to try to separate that oh they wanted to at the beginning. How are you gonna be able to? And is it fair? I'm concerned that if we don't make the distinctions be hard for us to figure out what to do in the resolution side. And part of that sort of a subset of that is are you able to track the leakage that comes from companies that's neither cyber espionage nor cyber crime that comes when Lewis and McAfin decide to go into business in China and know we're gonna give up there's a cost of doing business we're gonna have to give up something to get in there. And that makes sense for the Lewis and McAfin enterprise but at large it may have a greater cost in society. That's sort of a second part of the question but it seems to me if we gotta nail down if we can crime versus espionage because they're different players. Yeah I think it is a matter of teasing that out and also to some extent that goes to the nature of some of the costs as well. I mean I think part of it is that there are some things that are clearly cyber crime, identity theft, stealing arguably stealing sensitive information to then trade on it on the stock market. You get to steal theft of IP and I think it starts to bleed into, they start to, the categories start to bleed into each other. I think there are a lot of reports about state sponsored hacking to steal IP but then that gets treated as a form of almost aid to local industries where the IP then gets fed out. So I think it is useful to tease that out and also to understand where they bleed into each other and certainly there's different, I think cost assessments going into different activities which can be linked to crime versus espionage. Is it feasible, do you think to tease that and to ask was unable to do that? Right. I think it's possible to draw a Venn diagram maybe or a spectrum but maybe not, I mean I think some of these things aren't one thing or another. The gentleman in the first row. Thank you. This is a question I think you can answer on technical grounds alone. Everything in the digital age is expanding as we know at an exponential rate and becoming more complicated. What does this mean for the bad guys? Is it easier or more difficult to deal with for them to deal in this new ever more complex era? This is a great question. I think this is open season for the bad guys. We have created an internet that ships traffic at 320 gigs per second on a high end router, sends bad things to good people with a high quality service and we are as a society trying to figure out how to overcome that as we speak. The more technology we have, the more they use, only they have plenty of money and they don't have any lawyers. Nothing bars their execution. So when we look at cyber resilience and how to build networks that are stronger, that mindset has to be built in and the sort of easier concept to grasp. If you look at bring your own device, that concept, a lot of companies embrace this, consumerization of gadget du jour and we're able to bring that to work. What you saw was a large proliferation of malware toward the phones last year, more than anything else. In fact, we released a report at McAfee, a threat report that saw two things go up. Spam and mobile malware. And we think the mobile malware is an adaptation by the adversaries to say, well there is a lack of central policy and lockdown on the mobile devices. We're finding it harder on the regular stuff so that's where malware stayed flat but we've gone to where society is enabling better technology, faster technology and a little more fun. So you will see very much like in a cartoon, you'll see adaptation and that's one of the things that scientifically, just like your body works biologically, we need to get ahead of that and that's part of this resilience and it goes back to, again, understanding how to make those correct investments. You know, we're getting such interest here. I guess that's a wonderful thing. I have the gentleman in the fourth row. So it's easy to grab the microphone. Brooke Stahlsman from PWC. I have a fundamental question about the direction of the report and it seems to me you were approaching this deductively, looking at the overall effect on the economy and finding out what lots of people have that it's real hard to nail that particular jello to the wall. Would it be more effective for the people concerned to focus at the micro-end, find a way for people to be more aware of, be able to evaluate what's happening in their own organizations? I ask that just as a theoretical question because what you're taking on is much, much harder. Thank you. No, that's a good question and one of the things that we kind of orbit around in this is one of the goals would be, and I think we talked about this, is to get how companies calculate risk to be done differently than it's done now and you're seeing that happen irrespective of whatever we're doing but one of the goals would be that, one of the tests I do is when I talk to people who are on a corporate board or support a corporate board, I always ask them does your board have a risk committee and almost everyone says yes, right? In fact, everyone says yes and then I say and does the risk committee consider cyber risk? And the answer there isn't quite as uniform as you might hope. So I think one of the goals might be to say what are ways you as an individual company would want to change how you measure risk in your calculations and I think that would be very much aimed at the risk managers. If we can get there, I mean we're kind of doing it the long way which is sure I guess a mistake but that's what we chose to do. Let's see if we can come up with the factors that determine risk. Let's see if we can come up with a range and then let's use that as guidance for people when you individually sit down. It looks safe to say that companies do underestimate risk now from cyber crime. How much they underestimate is where we're stuck but I think we can make that case pretty well. If you're a board, you have a risk committee, this has to be part of what you do. The gentleman in the third row to the right. Thank you, Rob Maxim with the GW Institute for International Economic Policy. Several countries have responded to the threats of cyber crime in cyber espionage with policies limiting the free and open internet. One of the noticeable examples of this are a lot of countries are now pushing domestic server requirements and other similar policies. I'm wondering have you tried to measure the economic effects of these policies that are really pushing to limit a free and open internet? Well, it's been more sort of a something we've used to make fun of people because it's really, it would be a good outcome if you could say, we're pick a country you don't like, we're going to have an internet that's open for business but closed for politics and what we've found so far is that would be a really good trick, right? But it seems like it's really hard to do. The Iranians are furthest along in thinking about this where they've said we're going to close the country off from the internet. We're going to have basically a national intranet that limits what people can connect to but for certain select agencies, economic agencies largely and security agencies, they will have that direct connection to the global internet. This is their theory. Someone in Iran has a sense of humor by the way because they're devising their own search engine and it's called all hack. You know, you got to give them credit there. It's like hack, got it guys. I don't know if it'll work. So you see a few places that have come to this extreme of we will close ourselves off. It looks so far like it doesn't work though. It looks like you can't close yourself off from politics and yet remain open for business. So initial conclusion but it seems pretty robust. The gentleman in the second row to the far right. Thank you. Tanas from the Embassy of Estonia. I wanted to ask about the correlation between the loss in economic terms and what the thieves, let's say IP thieves have gained. Did you take this also into account? I mean, there are different costs for research and development in different countries and how much actually there's, these have gained in economic terms. Is there a direct correlation with that loss? No, there isn't a direct correlation. It turns out to be, and this seems to be a pattern in cyber collection and general digital collection and people can acquire a lot of information but then it's hard to actually monetize it and it varies from sector to sector. It varies with the level of development of the acquiring economy but it's not a one for one exchange. So if you steal $100 in intellectual property that doesn't mean your economy gets $100 in benefit. One of the things we hope to do is be a little more precise in saying which sectors are most at risk, where are the areas that seem to benefit the most from this and what are the requirements for taking advantage of intellectual property theft? The gentleman in the first row here. Hugh Grindstaff, what do you think about personal encryption devices? Will that help? And I know, does McAfee have them? So at a high level, any bit of technology helps a little bit. It's more about if you're going to encrypt something what are you going to encrypt? How much did you pay for it? When do you use it? Do you change the password? It's really all about taking away the profit model if you will for the adversary or the entity that wants to steal from you. So even though we sell a lot of security all over the world, I would never advocate that one would solve the problem. It's how you use things together and how you make an investment so that what you use today to secure can be built upon in the future really without spending a whole lot more money. The way to get ahead of this is to make our economic and safety advantage and overall enjoyment of the networks better than it is for the adversaries. And right now we're losing on the economics and the safety side. So it's really not about which technology it is although everything helps a little bit. It's really about looking at this big picture and I keep going back to this report but one of the key reasons we did this is to actually dig in and find what is the methodology to understand this because this is what we can do that the adversary cannot. The gentleman to the left here in third row. Hi, Nova Daly. This may be a question just for you, Mr. Lewis. So it's a two-part question. One, how can we, in terms of the United States, that's part A, or and then part B is how ought we to get better data? No, that was a mean question. Thank you, though. That's I think a political question. Companies are reluctant to share data because of the risk to stock price to reputation to brand. And I'm a little sympathetic to them. The fact that I could use better data doesn't mean that we should put big companies at risk. There is an effort underway at the SEC and maybe Teddy wants to talk a little bit more about that to require companies to report. What is it, material? I can't remember. Material events. Material events, right? And we have to be careful with this because in the name of better data collection, we don't wanna actually destroy value. If there was a way to do that without creating that risk, it would be great. So right now I think it's a political question. I lean a little bit more towards, I'd rather have the company retain value than be damaged because I get better data. But Teddy. And there was guidance put out in 2011 obviously about essentially how one, how companies should think about cyber risks and cyber events in their typical reporting cycle. I mean, I think the issue with that is one could use language to express something and not necessarily provide fulsome data on it. So there is talk now, I understand Mary Jo White, now that she's in charge of the SEC is re-looking at this. And so there is a question as to whether there will be more coming out from the SEC in the future. The fellow in the very back to the right. Thank you, Chris Porter with Verizon. Wanted to thank you guys for putting together this type of information. When we've written our breach report, the impact information about data breaches is something that we've always kind of considered a holy grail. This kind of touched on something a second ago where you talked about sharing that type of information. And I wondered what are ways that you think that companies would be able to share this type of data in an anonymous fashion so that there's more data to make better decisions within the market. Thank you. Hi, Wynn. So first of all, hi. This is a tough one because the more information we can put together, like a weather forecast map, the more data points you have, the better understanding you have of the actual threat and how to address it. The other side is, to Jim's points before, it's very difficult in some cases for companies to share data, either it's proprietary, it's something for some reason or another, competitive or whatnot, they don't want to share. It could have been gathered in a way that may have touched customer data. All these things are sometimes hard to delineate at the time that you need to release it within a certain amount of time to have it be effective. Currently, as we've testified, and others have testified on the Hill, there's no liability protection right now for a company that wants to go out and share something in good faith. So there are a lot of challenges to this balance between maintaining your corporate integrity and protecting your customers, no matter what, with the overall global protection and cybersecurity side. And this is, again, what gives the adversaries an advantage. I used to feel more strongly about the need to share information, but I think that was before, there was the degree of awareness and the strong effort by the government to try and overcome the problem of intellectual property theft. So you see activity with China, you see an IP protection strategy. Two or three years ago, it felt like we really needed to get good data to make the case. I don't think we have to make the case anymore. The gentleman here in the second row. Thank you, Destin Vandenberg from the Cybersecurity Policy and Research Institute at GW. A lot of you mentioned that companies don't know that they've been attacked or they don't know the level to which things were stolen. Now I was curious if you think this has more to do with the investment by the companies in technologies to do that, or if it's just the underlying technical ability to find attackers and identify attacks that's more at fault. And we actually used the Verizon report for a lot of this. So you should take a little credit there for it's a very helpful report in showing the lag that it takes for companies to realize something's been taken. The worst case, as you all know, is Nortel, which lost data continuously for a number of years before they kind of realized it. The other part is sometimes the incidents happen like that, literally like that. And so people don't know what was taken. They know there's been a breach, they don't know what was lost. And in some instances, the attacker will encrypt the outflow, encrypt the data that's being exfiltrated. So you may not be able, even though, so you know there's been a breach, you know the size of it, but you don't know what was in it. So some fairly shrewd opponents who've done a good job of making it even more complicated for companies to assess loss. Phyllis, do you have some perspectives on that one? I'll add to that with just saying we do a lot of defending and too little hunting. So a good investment strategy is to set up your network so that it has some built-in resilience. You see a lot of this, the government calls it continuous diagnostics and mitigation where every, and McVeigh would call it security connected, but every component of your network is a producer, consumer of information, so your network's learning. But there are things that you still won't see from that. And that's where if you can automate some of it and make it easier, you can take some very strong minds and focus them to hunt, to look for other symptoms of trouble in the network. A lot of these attacks have several stages. You'll see them put a program somehow, whether it's a USB or a spearfish, into one computer and have it disperse itself over time. That program will actually be the one that activates the next one that comes in that has a mission, whether it's to take out the machine as we saw in Operation Troy with the master boot record, or whether it's to literally look around your directories and find what intellectual property it might want. And then stage three is the actual action. And this can be from six to 10 years, or it can be weeks. So the fact that companies don't necessarily know this is happening, it's not necessarily that they don't have good security. I think sometimes it's the ratio of the minds that are focused on looking for it versus defending against it. And that comes back to the investment because again, the adversaries are getting very, very good. As we move to virtual environments, they move to prevent us from being able to run their malware in a virtual environment to see what it'll do. So everything we do, they try to do a little better. And what we have to do is get ahead of that by actually practicing some of their own more craft. And that is having some folks really look through the network and focus on that. And the only way to do it is to have a comprehensive investment, which I hate to keep saying this, but comes back to understanding how I assess my risk and how many heads to put on what problem. The gentleman in the third row in the center. Hi, thanks. Jamie Strawbridge from Inside US Trade. Just had two questions about the report, the numbers. One, I mean, I guess one reason why we want to know what these numbers are is it will help us to figure out how much priority to put on this issue and what policies we should implement in response. So if you could say a little bit more about that, in particular, it's a big, big problem and that seems to be accepted and these numbers are really high. That's also accepted. Does it make a difference? If it's 70 billion versus 120 billion, for instance, I mean, do you see a difference there in terms of which policies we should pursue? Or it's more like, okay, we know the range is here, but it's not a trillion, so we can calibrate our response that way. You can kind of elaborate on that. And then secondly, do you think this administration is treating the problem like it's a 70 to 120 billion dollar problem right now? Thanks. Sure. And in some ways, I think what we have concluded from the initial work is that we need to focus on more specific sectors. So some sectors are at greater risk than others. We need to focus on the effect on those sectors. We need to focus on the effect on growth, right? And that's one that hasn't gotten as much attention. And finally, we need to look at some of the national security implications. So the exact number isn't gonna be as important and that's why I think we'll ultimately settle for a range of numbers. It'll be somewhere between A and B. And then say, but whether it's the high end or the low end, here's the effect on the IT sector, here's the effect on aerospace, here's the effect on U.S. economic growth, which I think will be relatively substantial. And here's the effect on U.S. national security. So the precise number isn't as important as how that affects the companies that, pardon me, the countries, the companies and the countries that are the victims of cyber espionage. One way to think about this problem is it's relatively new. And so the internet really hasn't been around that long. That's a news flash, that's why I'm gonna think tank. More importantly, high-speed global internet connections haven't been around that long. I'd say, what would you say, 10 years maybe? Maybe 12 years, maybe 10 years. And it took a while for people with bad intent to figure out that suddenly they were milliseconds away from their target. And their targets were largely unprotected, right? So what we've seen is kind of, and this relates to some of the earlier questions, if you look at this in 1998, let's say the level was here, I sometimes wonder how people did cyber espionage in 1998. Remember, they'll download speeds with your dial up. So if you were stealing like the F-35, it would still be downloading, right? You know? But all of a sudden you've gone up to really high speeds and without commensurate improvements in security. So I think we've kind of plateaued. That means that in some ways for me, recognition of this problem in the US didn't occur until say 2007, 2008. And since 2009, there's been a really a strong effort to put in place better defenses, to engage foreign opponents and to think about intellectual property. Victoria Espinel at the White House, you now have a person in charge of thinking about this. So a lot of progress, does that mean we're better off? Unfortunately no, but we're on the path to being better off. Hi, yes, the gentleman in the second row here in the center. Yeah, I would just start with a quick observation that there's a quantitative and qualitative difference between being the subject of an Oceans 11 style heist and walking down the street with a fistful of $100 bills and getting jacked. So part of my question is within the scope of what you're evaluating, are you making a determination of what is just, sorry, for lack of a better term, the result of gross negligence on the part of the person who's been exploited? We're actually wrestling with that one. So we aren't quite sure what to do. And some of it is just how would you measure that, how would you measure that? And so we keep trying to come back and this is Stuart and I, we keep trying to come back to this. So can you measure it, can you quantify it? So we know that's an issue and we aren't quite sure how to deal with it yet. Is it, do you separate into two separate categories? Do you say one has a higher value than the other? That's kind of in the to be determined category, but it is sort of a crucial point. And I mean, in some ways it goes to the cost tolerance idea. And a part of the challenge is just the cost of figuring out what the cost is. I mean, the process of actually figuring out what the cost of a hack has been for a company can itself be very higher or what the potential cost could be. And that I think goes into how companies assess their risk. Yes, the young lady in the fourth row, I believe. Good afternoon. My name is Karina Ibrahim and I'm with the Russia Eurasia Program here at CSIS. And looking forward, my question to the panel is how important would international cooperation will be on this emerging issue? And how achievable this cooperation will be considering that a number of cyber attacks and cyber crime has been emanating from non-state actors. Thank you. Or state actors, excuse me. Jim, that's probably a good one for you. I was gonna give it to Teddy. This is your big chance. Oh, these are my views. Well, I think that understanding the cost is a good first step towards working out how you actually have a conversation. One of the interesting challenges here is that the costs are not born equally across countries. And one of the challenges in the next phase of the research is going to be that you can't necessarily extrapolate what percentage of GDP costs the US is experiencing to the rest of the world. So a part of the conversation will have to be, part of this is understanding who is most affected by this. I think another aspect of it, of course, is that, and this goes to the IP question to some extent, a cost for a particular company or even country may not lead to as high a particular net loss globally if another country is benefiting from it. So I think understanding those flows becomes another piece of the conversation. Data in itself is useful if you can give it meaning and data is valuable as a way of starting a dialogue, but I think it would be a starting point. Yes, the gentleman in the center. Thank you. Mr. Lewis, you mentioned insurance, and I wanted to ask whether there is, in fact, a market in insurance for intellectual property laws, cyber attack laws. Let's let the free economy figure out what the cost of these things are. There is a market, it's growing, and one of the things that's interesting in talking, some of the interviews we did for the report were with people in the insurance sector that the number of people either buying insurance or increasing their coverage has gone up in recent years. The problem, this has been a problem for about a decade, is there's still difficulty in estimating the scale of risk and the correlation between the company's actions and risk reduction. So, your insurance company gives you a break if you're a safe driver. Well, they don't know what a safe driver is in cyberspace, and so one of the things that was a problem in this is you could follow some of the international standards, you could do everything right, and you could still be a victim. So how does the insurance company put a value on that? This is one of these things where over time, as we get more data, I think they'll be able to do the kind of risk projections we need. Our hope is that we'll be able to take advantage of some of the data in the market about what people think they have to insure for, but it's still not as robust as you might hope. Any final questions? Ah, very good. Eric Fisher from CRS. I was just wondering, I wonder if you could sort of talk a little bit more specifically about this, some people say, or you often hear that the incentive structure for cybersecurity is highly distorted with cyber crime being cheap and profitable and so forth, and I'm wondering to what extent you might, you've thought about it, what extent you see the results of this project helping to sort of lead to a correction of that distorted structure. F to huddle. Yeah, I think that gets back to the question about corporate governance, right, in that if you can get a better sense for companies about what the, and it relates back to the insurance question too, if people have a better sense of what the actual risk is, presumably they'll take different actions, right? And in that case, I think the signs are encouraging. If we can come up with a better estimate of the risk to companies and the scale of losses helps us do that, then I think you would see more attention to this. That's already happening in some ways. One of the benefits of the whole spate of news stories over the last couple of years has been that there's growing attention among boards, growing attention among companies, so I expect to see behavior change. If we can help that along, that'll be great. So one of the analogies that's about 15 years old is this one of anti-lock breaks and things that you can do to make your auto safer that lowers your insurance rates, and one of the things that may come out of studies like this as byproducts are ways that people can be safer, incentivize on insurance, and at the same time that might give some value, as I think was said before, what is the free market going to value this loss or not at, you know, the other piece is we have this interesting and pretty amazing opportunity right now as the private sector to help build through the executive order with NIST and DHS, this private sector view of how you secure our systems. The private sector knows them best and they should look at how to secure that, and studies like this, no matter what the number is, both the methodology and the study itself should help put some logic into what's recommended there as well as we go forward. Any other questions? Oh, the gentleman in the first row. I would really classify as sort of technical cyber attacks. So I guess the question is, have you considered in the study what I would describe more as cognitive cyber attacks or social cyber attacks? Let's give you a simple example. So in China, well, so there was big tidal waves, hit Japan, everybody's scared about the reactors melting down and you got a billion Chinese sitting there scared that this resulting radiation cloud is going to come over and kill them all. So that's kind of the setup, right? So somebody did what I would describe as a cognitive vulnerability analysis and they came up with a great idea and they said we will put out rumors in the Chinese social media space that if you eat a lot of iodized salt, you're gonna be protected from this coming radiation cloud. I mean, this was done really, really effectively and it ended up inducing a run on salt. I mean, like no joke type run on salt, there were cities in China where you could not buy salt. And this lasted a couple of weeks. So the result was that this was a serious spike in the Chinese supply chain and any kind of spike that you have in the supply chain like that results is viewed as an instability and anybody knows the Chinese government really dislikes instabilities of any sort and I can imagine doing this on a larger scale as I like. So that's a simple example of the type of attacks I'm talking about and it doesn't require any technical expertise. It was just using social media as is. It didn't even require a hack. Unlike for example, if you take the AP Twitter hack that happened recently, right? Where somebody sent out a tweet that said, you know, the White House has been stormed, the president has been taken or whatever it is in the market dropped like, whatever is how many hundred points within a very, very short amount of time. And then, okay, so the technical part of it was somebody made it look like they were AP but that's relatively minor. It was the contents that was really interesting and you know, the security change commission said, well, you know, within a couple of hours we fixed this problem. But you know, when trades are 15 milliseconds of pop, I, you know, a couple hours maybe it was open like a million years. So I guess the question is those kinds of things are becoming more and more frequent and more and more interesting and have you considered the cost? How do you analyze the cost of things like that? Yeah, actually we did. And by the way, if any of you are interested in gold shares we have not just a joke, just a joke. We did look at that and in particular the effect on financial markets where they're, the AP story is a good example. There's not yet enough data to say whether that was in some ways an intentional manipulation of the market or whether it was an event that people took advantage of to manipulate the market. My sense is that it would have had to be intentional. The question about using cyber crime techniques to get the insider information that would let you do stock market manipulations or the use of social networks. And this we know happens now. You know, people say this stock is going to really go up. They put it on some stock board. The stock does go up and they cash in. So we are trying to take that into account. I'm not sure how big a contributor will be but it is something you have to look at. It's a different set of solutions but we're particularly interested in the financial manipulation part. Yes. You know, that seems to be one of the themes these days with Snowden and all. So did you look at the insider versus outsider aspects of cyber and financial implications? Yeah, you have to be able to. We felt that it would be best if we could distinguish between an insider and an outsider. And one of the questions we had as a sort of core decision-making factor is would this crime have occurred if the internet didn't exist, if there weren't these digital technologies? And so if it would have occurred no matter what, then we're probably not gonna count it. So that's where we're approaching the insider activity. The solved example is a good one which is that someone could have done that by going around the market and saying that and it would have had a much smaller effect. So you do get this amplification effect from the internet but our basic thing has been if it would have occurred, and this comes up a lot, it's been one of the problems in some of the earlier studies, benefits fraud, if it would have occurred without the internet, should you count it? And we're going to try and say no. That's good. Well, if I'm looking out of a risk perspective and I'm looking on the board and I want to outside components of the threat in terms of dealing with the risk, because you deal with insider versus outsider in different ways, not completely different, but there are some differences in how you would approach it from a risk management perspective. Yeah, we probably won't go into solutions in this report. We'll just probably do the cost estimates, but this is, and again, you could build off some of the data from Verizon. This is a place where people need to think about how they can lose data. We want to look more at the effect of that data loss. All right, very nice discussion. I think we've had today. Any closing questions? Well, very fine. I think we've had a very good, broad-based discussion analysis and I think outstanding questions from the audience, and we thank you and we thank CSIS and the team that put this together.