 What we're going to do is we're going to have two keynote speakers followed by two panels and the first panel will look At the threat and the second panel will look at ways to address it, right? Our two keynote speakers are Dave DeWalt, who's the chairman of the board and the CEO of FireEye He's got a long career in the industry and has done some amazing stuff before that. He was at McAfee He was at EMC all the big names. So one of the the real powerhouses in the industry here The second speaker of course is Chris Inglis who is the deputy director at NSA again a long Incredible career including in the Air Force He Chris is unusual in one sense in that he had the job in London before he was deputy director usually It's the other way around But he is also one of the true specialists who's been working in this field for decades and a real leader So we have our two key notes What I'd like to do is ask Dave and Chris to come on up and sit here What we'll do is let you talk you can sit at the podium and then once they're through We will move to the second panel. Thank you Good morning How's everybody this morning? Pretty good. I love when the weather Turns positive here in the Washington area. I remember many a snowstorm here. So it's glad to see summer here already So welcome and thank you for having me. What a great honor It is to be here very humbled and just a huge appreciation for from CSIS for hosting us and having us Have an opportunity to talk today so I want to I want to spend a few moments and maybe talk about the state of cyber security and Maybe a couple of ideas on how we can improve things in the world. I get a pretty unique perspective As Jim said, I was a CEO of McAfee for quite a few years Monde Delta Airlines is bored and do safety and security for the airline Chairman and CEO of FireEye. I'm also chairman of Mandiant as well so kind of a number of different viewpoints in the world and It's been interesting to say the least and I probably use a quote to probably summarize it best and Many of you probably know this quote. It was made famous in 1966 by Robert Kennedy during the Bay of Pig crisis the Cuban Russian Bay of Pig crisis and the words are may you live in interesting times and Some of you may know it and actually have his origins all the way back to China and While sounding like a positive statement It is got an undertone to it too, which is a bit of a curse and may you live in interesting times is kind of what we We see in the world today from a cyber cyber point of view. What a what a world we live in and Watching it over the last decade has been Fascinating to me to see sort of what's been going on and the severity levels that are that are happening and I call it a perfect storm for a number of reasons and If you think about it, we have almost the perfect confluence of conditions happening in the world today That has created an atmosphere for crime and and theft in the world of cyber and cyberspace and probably the first is one of the most positive things hence the hence the words may live in interesting times is Innovation we have some of the greatest cycles of innovation the world has ever seen and that is amazing We literally are reimagining everything. We've been doing the internet has created an atmosphere that allows us to To do so many things different how we read how we learn how we share How we listen just about everything we do has changed for the better probably and differently and this innovation is is creating an unbelievable unbelievable cycle of architectural changes in companies and entities and governments around the world just think about what's happened in the world of mobile IT consumerization How many of you have more than one cell phone? How many of you more than two cell phones? How many have more than one device the world? we live in has changed dramatically and Corporations are having a hard time adopting to those types of changes, but again, we're changing everything we do think about Sass-based architectures and clouds Software as a service the acceleration of innovation in enterprises is dramatic We're seeing software as a service having you know some of the greatest Momentum probably ever and having the biggest architectural changes and ever we have private clouds We have public clouds. We have hybrid clouds We have companies going public almost every day now that our software as a service type architectures Some of you probably saw Tableau and Marketo go public just on Friday Salesforce.com oracles of the world jibes of the world It's amazing to watch how much infrastructure today is a radically new architecture Compound that with social everybody's probably here on a social network or two or three or five and We are living in a whole new world with social networks Basically exposing every friend you have every professional you've been associated with anybody you've ever met in your entire lifetime is there for for the for the watching and for the for the observation and What's that created? It's created an unbelievable scenario of vulnerabilities and that's pretty much what sort of the next variable is is the the greed variable and It's amazing to me to watch this because in all my time I've never seen the state of things be as as aggressive. They are in the areas of greed So innovation has driven the opposite effect or the curse effect Which is everybody seeks that innovation that intellectual property that's being created at unbelievable rates is Creating the theft of that intellectual property at unbelievable rates And you watch both sides of that coin the same way and we're seeing unparalleled absolutely unparalleled theft in the world today and It's amazing to watch happen compound that with privacy and anonymity on the internet and the ability to protect your information the ability to stay Anonymous with who you are has created a perfect storm again to steal that innovation Because I can basically hide behind that privacy hide behind that anonymity and I can create an atmosphere where I can steal almost almost at will Put on top of that a lack of governance on the internet Again, we've seen just an amazing environment where there's literally almost zero governance on the internet today globally And I'll give you a few statistics in just a minute, but they're actually stunning to see and The governance model around the world every domain is is fair game from a country point of view And of course with all the ubiquity of access that we have today You can access any website in the world with a click of a button And when you have a lack of governance model around the world with the internet You have all this privacy and anonymity and you have all this innovation it continues to just create the storm effect that we're seeing today And one of the other variables that's amazing is naivete as I call it or lack of awareness to the problems that are happening How many people read the Washington Post this morning on the Google hack anybody read that? It's like right up there with homicides and hurricanes and twisters unfortunately And what we're seeing is you know an insensitivity to this problem at times And it's amazing to watch almost every day the headlines have some sort of attack that's occurring on some side of corporation or government agency and Most of the Americans most of the Western world most of the Eastern world is not aware of the level of threats that we're visiting today Or having in the world today So pretty interesting problem when you have a lack of knowledge Couple with all the other variables that on top of all that we have a significant deficit in defense models probably the most interesting Place right now is the the dislocation that we're seeing between offense and defense I've never seen the gap be wider than we are seeing it today And what I mean by that is if you think back over the last 25 years or so when we first saw viruses in the wild Some of you might remember Melissa viruses or I love you viruses or code red viruses Hopefully I'm not dating myself too much here But you know they were distributed on floppies and what happened out of that we created a business called antivirus and The business of antivirus was all about looking for patterns Creating a signature looking for attacks like the Melissa virus and scanning for files that could be bad and Over the years we created more and more signatures and more and more files to scan More and more viruses came out But what was interesting over the last 20 years or so the offense and defense were pretty closely correlated Usually measured in you know a few days few weeks But a signature could be generated because there wasn't that many of them and it could be generated pretty quickly So you were just slightly behind the curve if you look at the last two three years The defense and the offense have been the greatest in dislocation probably the history of of internet history of technology Today we're seeing unprecedented amounts of attacks. We're seeing unprecedented amounts of Information being stolen as I mentioned and the defense model today is completely broken in my opinion and I liken it to the analogy some of you probably know The story between World War one and World War two of the Maginot line everybody familiar with the Maginot line So here the defense architecture It was built between World War one and World War two and the Ardenne forest was all around this deep defense in depth architecture and the ability to create hundreds of miles of defense architecture to protect the the French and German borders and Of course, it took billions of dollars especially in those days to build many layers deep chambers built to siphon out gas launch airplanes be able to launch tanks be able to launch artillery and It was probably one of the greatest defense models ever built history of mankind You all know the story what happened in days weeks the Germans were able to leverage air supremacy and blitzkrieg supremacy to get around that in almost instantaneously So when you look at the defense models today in cyber You have almost the same thing defense in depth at every level of the architecture is built with the exact same engine Antivirus that's been built around for 25 years all leveraging blacklisting and signature models And now we have in essence over 60 million signatures in every single antivirus engine And that engine sits on hosts like endpoints It sits on network devices like firewalls and IPS solutions on email on web Sits on just about every area and what's easy for the attackers to do is that they can get past the blacklist If they can get past the file scan, they've gotten past the entire defense architecture just like that So what are we seeing? amazing statistics absolutely stunning and Today, you know, you can see this a 250 billion dollar problem a year in theft of intellectual property And quite frankly in my opinion, that's a very low number because of your reported companies Who have breaches for intellectual property are pretty minute in the grand scale of the theft that's occurring We have more than a hundred and fourteen billion dollars a year of crime of fraud identity of theft and other types of Crime activities and this is also probably fractional of the real problem Because when you actually look at what the attackers are doing in the world today They're actually going after very specifically Information from public companies so they can trade and hedge stock prices using inside information Very hard to track that as a theft area So what do we find? CFOs controllers finance managers as a number one target particularly in Eastern Europe Where we're finding lots of attacks going after inside information and then trading on the capital markets on that information So when you look in totality of the problem, it's probably measured in a trillion plus dollar a year problem a year Pretty amazing situation. We see 60,000 pieces of malware every day 60,000 pieces of malware every day the average company is infected a hundred times a day a Hundred times a day infected actually successful infections on average Every couple of seconds there's an attack Pretty interesting world. We see 9,000 new websites every day created that are malicious Because of the lack of governance model Pretty interesting We have hundreds of thousands of command and control servers set up around the world 94% of the countries in the world today are active with hosting command and control servers So command and control servers are actually the ability to money launder intellectual property or information and and data These servers are set up in nearly every country in the world over 180 of them So our problem has gone global The problem is unbelievable in size and scope today and it seems like it's only getting worse by the moment When you look at the types of attacks that are happening today These types of attacks are very ingenious These aren't attacks where I simply send you an email with a file attachment like it once was These attacks come down in stages I'll send you one piece of information. I'll send you another one a month later maybe even a year later and Over time I'll download capabilities to your computing device that allow me to exploit the information on that computing device what we call multi-stage What else happens they come down in multi vectors? What does that mean? It comes through a number of different protocol points I might send you an email with an executable, but it also has a web link in it If I click click on the web link, I exercise a brand new protocol or a brand new vector So the exploits are coming down in multiple ways today And what happens is the lines of defense that have been architected in the world are lined up very deeply One protocol at a time Very deep defense around email very deep defense around web very deep defense around file But none of them correlate amongst each other. So the attackers use multiple vectors to attack the networks So the architectures are completely flawed that we have today from a security point of view And what's happening is the attacks become very very present We estimate more than 95% of all companies in America in the Western world are compromised as we speak here today That's just the state of things somewhat ominous here in this morning to say all that but that's the truth And that's the reality of what we see and quite frankly 89% of those attacks are coming from China Amazing statistic 89% Chinese lead And we see that almost every day in our business whether it's mandian or fire I or other security companies We have some significant issues around the world in terms of the types of attacks that we're seeing and the types of threats that we're seeing and The victims are everywhere almost in every vertical So at once was that the victims were focused in on particular high intellectual property verticals What's changed pretty dramatically? It's nearly every vertical in every size company today If you have intellectual property, you're a target We see it in hospitals and health care organizations. We obviously see it in banking. We see it in think tank organizations We see it in manufacturing We see it in energy Almost every vertical and almost every country. We're seeing major exploits of information intellectual property and money Around these industries. So it's gone global. It's gone vertical And it's gone from small companies to very large companies So an interesting problem to to solve and a difficult one to do and many of you probably know some of the attacks Many of you might not but when you look back on some of these major attacks They're directly at the heart of The infrastructure stacks that are out there some of you might have heard of a an operation called Aurora this was made famous in 2010 by Google and This was a specific target Using spear fishing that allowed a web exploit to download an mpeg that unpacked and then ultimately created a key logger So you could steal valid credentials You could log back into the network using valid credentials and you could insert an advanced persistent threat an apt That advanced persistent threat could sit there for up to years And it could siphon out information The Aurora attack was specifically after source code assets a source code control system called per force And it reached over 150 companies, especially high tech companies So what do we see today from the threats? They're now zero days everywhere Zero day attacks or attacks that are unknown to the vendor were the software company That actually has the product. So we're seeing zero days in microsoft adobe Java just about every software stack that's out there on a regular basis So interesting world going back to the types of attacks that are occurring And the types of attacks that are happening today And the list goes on night drag and shady rat b bus ghost rat I mean I can name them all almost every time we see major campaigns major attacks on hundreds of companies at a time So what are we going to do about all this? It's a pretty big problem to solve and kind of a scary one at least from my perspective We we have a history where we've got a lot of challenges whenever a new domain is discovered There's conflicts around that Whether it's land or air or seas or space or now cyberspace. We've had conflicts We need to resolve that We need to figure out ways to address this as a global community And I think about these and kind of four letters They're all start with the letter t I call them the four t's But a little tongue in cheek But the idea here is we've got a number of fronts that we've got to improve upon in order to change the situation that we're in in my opinion The first t just would be called teamwork And uh, it's uh, it's often used word, but a very important word for us today And the teamwork is needed across countries public to public Within public sector itself government's been working together with governments We need treaties around governments to create an atmosphere and cyberspace that allows us to behave properly in the world today The teamwork not Doesn't just need to exist between public and public government and government But it needs to exist government to private So we need better interlocks between private sector and companies and security firms With the government and we're making great progress in that area. That's encouraging to have but we need a lot more of that And we need capabilities to share information create safe harbors Protect public companies from liability around sharing that information But the teamwork is critical Absolutely critical in my opinion And we even look on private on private companies working together The security industry itself doesn't work effectively together And we need to drive the security industry to work better together I don't know how many of you've been to an rsa event. They're kind of fun to go to rsa is one of the largest security shows in the world 1372 security companies showed up this year at the musconi center in february And if you ask the 1372 companies, how many of them partner with each other You might find it about zero Are very small, especially when it's interoperability amongst sharing compromised data Sharing of intelligence using formats And we need to create some formats that allow us to share better amongst the security industry And there's some positive momentum that's happening there like open ioc from mandient or sticks That dhs is leading but boy do we have to take it to another level that teamwork and that t is critical for us moving forward The second t is around testing And standards and making sure that we test our architecture is much better today It's amazing to me to see all the vulnerabilities that are brought in In both imports and exports of technology the lack of testing of these types of technologies If you study the supply chain of technology today that's being put into our critical infrastructure And you look at the testing that's done around it You'd be appalled most of the uh the world's development for technology is done globally And of course the exploits can occur almost at any point in that supply chain of technology Yet we have very little standard for testing to implement critical infrastructure types of technologies And we've done this in other areas. We have the ability to do testing We have the ability to test these stacks to create standards around the infrastructure that we use There's so many examples of this everything from lighting that we see here today like underwriters lab and ul To power supplies to just about anything else that you see today for seat belts to texting and driving We have standards and testing that's done in every area that's dangerous in the world Yet we don't have them in the cyber world. We need them The other areas are training and other t education I mentioned the naivete that we see in the world today with consumers and corporations Many of which don't even know they're breached or if they are breached. They don't know what to do about it The training is critical the education is critical to help elevate the problem To help them understand what to do how to interact with law enforcement how to interact with vendors and suppliers To solve these problems easier And of course when you look at it less than 10 percent of the company is actually report that they've been breached And if you look almost 100 percent nearly 100 percent of all companies that have been breached Were notified by a third party that they were breached. They didn't know it themselves So the education and training is critical to helping to solve this problem the awareness level is critical And the last t is technology We have to advance the technology architectures that are out there today There is some great technologies that are out there I spent the last 10 years in security industry and you can see some of the momentum that's being made architecturally But when you start to look at the standards that have been in place Or the lack of standards that are in place We have no impetus to try to put forth new technology that actually can stop some of the advanced threats that are out there today There's some real promise of capabilities that we have in the architectures models out there The use of virtual machines or sandboxes or detonation chambers enable us to Test web pages and applications prior to them being viewed on host computing So instead of scanning and looking for bad files or patterns like signature antiviruses done Virtual machines can be used and leveraged in brand new ways at higher speeds and greater efficacy than we've ever seen today But yet we're at our infancy of these types of technologies And the ability to leverage them into all the egress points that we have in our architectures could advance these technology models infinitely better than we have today White listing capabilities gray listing capabilities Abilities to create interoperability amongst technology vendors could advance the tea of technology in ways that we probably have never seen before The technology exists today We have to use it and leverage it So Hopefully i'll be standing here in another year or two and we'll be talking about how we've solved some of the problems that are out there I think it's paramount that we do I'm sorry if it came across to a little ominous. I don't mean to i'm a very uh, very optimistic person by nature But it's really interesting to watch what's happening out there in the world today And nearly every day at my company as I see major exploits occurring Major countries attacking our infrastructure and watching it happen at the at the record paces is a little appalling to see But uh, again, there's always a silver lining. There's always an optimistic view And I think if we can all move forward as a community Together across public sector and private sector we can solve these problems today. Thank you So um in conversation with jim, i'm chris englis speaking here What we thought we might do is that i'm going to give some brief remarks and then david and I will take questions together Um, it won't surprise you that the remarks that i'll provide are very consistent with what david has already rolled through i'm reminded of that that old story about how when The conspiracist in the world found out that pyramids were built at around the same time and roughly the same shape in both egypt and The kind of latin american regimes they immediately assumed that it must have been an alien race that came down And kind of conveyed the information about how to build pyramids in both those places As opposed to the alternative theory, which is that it just might be that if you pile up rocks any other way they fall down So I think you'll find that david and I have very similar views on this and and i'll provide some insight from my own experience Perhaps using a slightly different lexicon, but ultimately coming to the same conclusions About what the nature of the issue is and perhaps the strategy out Kind of I think I'd start with an anecdote of my own Which is that you can remember the story about that cyber entrepreneur when asked by his chairman How are things going? How's the business going and he replied in a word? Chairman said sure in a word. He was in a word. Good So and the chairman thought there might be a more fulsome answer. He said how about in two words. He says two words. I'm not good right Turns out both of those views about the state of cyberspace what I would call the internet Cyberspace are right, right? It's still a good thing. It's still an enormously powerful engine for commerce for private communications For diplomatic communications For the progress of all those things that we hold near and dear whether it's aligned with economics private life Diplomatic life even command and control for industries and military. It's a huge engine of all those things But in the same way all of those things that constitute a lucrative asset for those who would then take advantage of those So that stride for stride along with that promise goes a threat And I think I agree strongly with david that the internet was built more for the first than the latter The latter is almost an afterthought and so my own framing of this kind of rolls In the following way, which is that first and foremost we need to understand cyberspace for what it is It is an extension of each and every other domain right that it enables But it has unique properties those unique properties must be understood in order for us to then Treat the properties the value that's stored inside of that in an appropriate way Three come to mind The first is that as opposed to simply reflecting through its communications things of value about other domains Cyberspace increasingly is a place where we store wealth and treasure Many ways to describe that but as opposed to when I got started in the business at the national security agency in about 1986 When communications that I might capture for my intelligence mission or defend for my information insurance mission They reflected things of value that were held in other domains Increasingly those things of value are only in cyberspace might be the blueprint to an advanced fighter jet that the department of defense You pick the country Holds as a competitive edge in their competition with other nations on the planet Might be ones and zeros that are associated or perhaps are the only representation of your child's college savings account And a whole host of other things Now the second property of cyberspace that I think makes it unique is that everything is connected to everything You might call that the property of convergence That's a wonderful thing when you google yourself or something else and you find that there are 100 000 entries But you need to at the same time remember that the 100 000 things that have just reported to you about what they know about you Can touch you in ways and places and times that you might not prefer At the same time you can reach out and touch them they can and often do reach out and touch you And the third I think also referred to by david is the royal the rate of change which is the reality of cyberspace It's almost impossible to achieve a static advantage in cyberspace Whether that's a competitive advantage that you prosecute in your business Or whether that's a static advantage that you might know is a security advantage over an adversary Things change every minute of every hour every day and it's not just the technology that changes It's the employment of that technology. It's the operational practices Of either the adversaries or your users who maddeningly make use of things that you provided to them in ways that you never imagined That's a source of great innovation great delight. It is also a source of security disadvantage If your security depends upon a static advantage and static nature of compliance based standards Your heart's going to be broken on a fairly regular basis Now that then leads to a conclusion then about what is it about cyberspace that is then worth protecting What are the core issues for us about what it is We need to hold near and dear and what if we had a strategy to protect something what would be the things you'd protect Well, I mentioned one of those wealth and treasure Intellectual property or the literal wealth and treasure that's stored in cyberspace is worth protecting We need to therefore focus time and attention on how that's stored where that's stored what that is so that we might then protect it Privacy is well worth protecting. It's an inherent Constitutional right for us in the united states, but it's also a common principle across Our allies around the world and that too must be protected I might say that an implicit right that most americans would immediately subscribe to and I would imagine many other societies as well Is the right of traversal the right of access the right to use it? We need to defend cyberspace such that it is available given the dependence that we've achieved on it or through it We need to make sure that it is generally available And then finally we can't miss the fact that there are so many things that while they're not necessarily stored in cyberspace Their resilience depends upon cyberspace if you ask us transportation command That's the command within the u.s military that gets guns butter people from place a to place b from the place of Manufacture to the place of consumption They would say that 90 percent of their flow depends upon the resilience the integrity of cyberspace because much of what they do In command and control from ordering things at a factory and moving them through a rapid logistic system There are no warehouses depends upon an exquisite Choreography that occurs in cyberspace lose confidence in cyberspace. Do you lose confidence in your flow of materiel? You lose great efficiencies that we have no backup for again. There are no warehouses There are very few grease pencils left in the world It's hard to back up to a manual process if you've depended so heavily upon cyberspace As to threats to all of that I think david too did a really good job and I would simply say that If we were to keep score in the american game of soccer Remember most of those scores break your heart at the end of 90 minutes. It's about one to zero And generally it's some bit of serendipity in between that kind of led to that result If this was the game of soccer, we'd be at a score of about 86 to 50 and we'd be 20 minutes into the first period Any offense will do and there are no defenses anywhere in sight Perhaps all the offenses are clustered around the kind of opposing goalie's goal And that's why perhaps no one's defending our own goal, but but when it's all said and done. There's a great asymmetry There's a great disparateness between what the offense does what those threats can do and what the defense does I think that's because the strategy is always lagging right while there are some good strategies And I think david's right. They're point focused and they're fractured almost by nature While there are some good strategies, they're not holistic and they're not dynamic Most of those are based upon point focused solutions And most of those are focused on compliance as opposed to a dynamic defense So the strategy might be as simple as we should make these networks more defensible We should actually defend them That's a surprise to most to find their networks fouled on a monday morning having left on friday night And we should bring to bear all instruments of power right to enable that defense And I say all instruments of power as opposed to national power And we more often than not especially in the government where I come from think about national power But I say all instruments of power because individuals have a responsibility Organizations have a responsibility sectors have a responsibility Governments plural have a responsibility and to david's point. They must all be done in collaboration Some of the most interesting innovative perhaps disastrous attacks we see Increasingly are ones that are not simply staged as david described. I think that's also a very interesting vector But but they are kind of done in a way that the actual attack occurs two three levels upstream Such that when something arrives on your doorstep with perceptively bonafide credentials You do what you're supposed to do which is you check those credentials You ensure that that person has the right to access what it is They're asking for and you provide it to them. That's why we create the networks That's why we create the connectivity But it turns out when you look one or two or three levels upstream those those credentials were stolen or counterfeited or washed And until such time as we can step by step back and we can take a god's eye view to the various artifacts that exist in the system Integrate them achieve collaboration right within the engineering of those networks And then with the within the dynamic nature of how we operate those networks our defenses will always fail As to who I mentioned that when I talked about, you know, what the strategy might be It's individuals. It's organizations. It's sectors. It's governments plural And as to when now would not be too soon Possibly it is too late. I would close with my own version of the four t's being from nsa. I have to encrypt it though It won't be t t t t But I came up with virtually the same words using kind of perhaps a Synonym for each the first word on my list is collaboration I think of convergence as a reality in cyberspace than integration in our engineering and collaboration the human form of integration must be a necessary response Let's put all of our efforts together in a collaborative way in order to Make this a more resilient more defensible set of networks and to then actually defend them My you know, my version of training is education, right? So that's an e for me I think that this venue and I give huge credit to cesis The reason i'm here this morning is if jim lewis asked me to do something i'm Bias to say yes because I think that his organization has been an enormously Powerful force and a thoughtful force in terms of providing an education that then enables action And I think then that this is a fora that we should continue in so many other ways My version of testing is Policy and law and by policy and law I don't just mean government policy or kind of the law that we might then effect up on the hill and have the president sign But organizations need to take some responsibility within their own house to determine What are the standards for building and operating a network? Does the ceo know the name of the cto or the name of the sysso? And if so, do they see that as a a cost center something that has a drag on profit? Or is that something that creates the very foundation that generates the profit that the ceo is charged to effect And then if that ceo feels that way, they'll take it upon themselves to describe declare what the standards are both in compliance And in the operation of those networks followed through by complementary activities on the part of government Which can perhaps establish policies treaties normal behaviors between and amongst nations in this space And then finally with respect to technology I would broaden that a bit to say not just technology, but human behavior action, right? There needs to be some emphasis on what are the active components that then have some properties within our system And people are perhaps the bane and the boon right of any activity within cyberspace The reason that a national security agency continues to be successful this day In the face of ubiquitous encryption and massive security protocols around the world Is that the end of the day the proof for the pudding is in implementation It is humans after all who manage who essentially conduct the affairs that we know as security practices and protocols And the reason our adversaries are so enormously successful against our networks today again Is it's the implementation that is that issue and therefore we need to focus on the behavior of humans Not just the behavior of technology So in closing I would say that I agree wholeheartedly with everything that david said in terms of his framing Though our words might be somewhat different in our experience of course comes to this this common place from different directions I think we see the space at about the same way It just might be that when you pile up rocks any other way they fall down We'd be happy to take any and all of your questions at this point Well, I really appreciate both of you giving those Very insightful remarks that I like the pyramid similarly. So it really I'm going to steal it as soon as I can No, these are two real leaders in the field. So we have a few minutes for a few questions Could I ask when you ask the question stand up and identify yourself? And then we'll add our two speakers take it go ahead, please Organize ourselves around this concept of defense and depth As we move forward have you thought through what new governance models must exist to change that view? Just what you talked about that that concept of human behavior. We don't think enough about how organizational culture Impacts how that implementation works So in in revisiting our governance and revisiting comms, etc How would you change the way we organize to move away from that majeure line? That's a good question. Although I was still trying to think about how to encrypt chris's four t's to c e p a H or something. I was like, oh, it spells peach. So I got it. Yeah policy Anyways, great question and um, you know, it's not an easy answer because we've spend if you look at the security industry today It's nearly a 30 billion dollar a year industry And as I mentioned, you know, 1300 vendors in the security fields, just technology vendors alone We've been we've been conditioned to build defense and depth as an architectural model And that architectural model today is so easily evaded It's unbelievable to be here and watching that happen because we have some of the greatest cyber mines and technology Personnel engaged in these these activities These are the smartest people, but when you kind of look at all that innovation. I was suggesting earlier It's just created such an atmosphere to exploit That you know, it's created these these architectures that are now just flawed in a way we have to think about it So we need some creativity here and we need some new ways to think about solving that problem And you know, certainly when you think about it, we need we need a way As I think chris suggested of creating some some policies and some technology advancements and some behavioral changes to really Address that problem and that's just about the only way we'll do it Yeah, so I would uh associate myself with those remarks and then perhaps um at three things one There needs to be accountability and I don't mean by that negative liability I think that we need to hold CEOs or kind of the appropriate parties accountable for The resilience the security the integrity of those things that generate revenue or generate whatever the business is of that particular organization They therefore in the same way that they might then kind of pay a lot of attention to their finances under sarbanes oxley Spend an equal amount of time to the integrity and resilience of their networks because that's not simply a commodity Who's kind of fate may have an effect on their bottom line? It's a foundation for their business and i'm not supposing that we have a sarbanes oxley But you need to have the same sort of focus The second thing I would say that I love the analogy that david had about the imagino line And I think if you think in terms of warfare and this isn't warfare This is a defensive strategy in a place called cyberspace But if you really think about the imagino line The solution for that in terms of militaries who ultimately were able to counter blitzkrieg Was I think at least composed of two things one maneuver, right? As opposed to static defenses who did not move in time and therefore had strength and depth but no ability to be agile They achieved a degree of maneuver such that they were able to outflank or counter Right in a very dynamic fashion the forces are right against them and second a joint ability such that you could bring Your various forces to bear in a hugely integrated way Today's u.s military Like a lot of other militaries in the world achieves an enormous power from the leverage that comes from the army navy Air force marines even coast guard joining their efforts on a joint battlefield such that the maneuver that they bring to bear Has great cross leverage And I think the collaboration and integration that was suggested in our opening remarks is a component of that I think if we could bring all of those to bear in an integrated way We just might have a chance and I do believe that this realm is defensible I think we would have a very fair chance in order to defend it stand up defense Mainly mr. Englis How relevant is uh the division that we have now between title 10 and title 50 To your ability to do your job and uh do these lines need to be redrawn So i'm gonna assume that what you mean is that um kind of in the realm of computer offense computer network offense Okay So title 10 title 50 come from a time and place for those of you who aren't kind of close to that and and if you aren't um you live happy lives Comes from a time and place when we essentially had very discreet Places where we did the various functions that we might today call computer network defense computer network exploitation computer network attack So in my business for example, um, nsa has always had a breaking codes and a making codes component But in the early days even when I came to nsa 25 years ago, those were two physically different places We kind of broke codes in one place and made codes in another when I came in in 1986 The soviet union was the principal adversary of the united states They made operated their own networks or what constituted networks in those days And we made an operator our own networks. And so there was a very Distinct separation between the authorities you would then bring to bear in those respective Areas I think what you talk about more currently is whether the distinction between title 10 and title 50 We practice title 50 in terms of my intelligence mission The us cyber command practices title 10 when they defend networks or went on order. They would attack networks Are those distinctions so they continue to be important Those distinctions are not helpful if they talk about different places But they are helpful if they talk about the effect that you intend to bring to bear And so I think we need to hold on to the distinction between the effects We need to know at any moment in time. What is our intended effect? What's the intended consequence outcome of a particular action? But we need to make sure that there is increased synergy between and amongst the forces We would bring to bear in the same way that I talked about how do you counter right blitzkrieg You have to do that through a jointness and through a sense of maneuver So I kind of would answer on both sides to say that I want to retain the distinction between what my authorities are So that I know that at any moment in time I'm doing what I'm authorized and no more But I want to have a join between those so that they can actually aid and abet one another in useful and thoughtful ways Dave you're on the end stack and similar bodies. Do you ever run into title 10 title 50 issues or what do you think about it? Well, certainly from the private sector's viewpoint, you know, we have similar challenges as chris was just describing where You know, we sort of have the The elements of constantly trying to break the code obviously because the attackers are trying to look for You know new ways to crack in we have research teams constantly on the defense trying to figure out how to How to figure out, you know, how to solve these problems and what's really interesting is we now have Vendors who are on offense too in the private sector And you know ones that have built new malware kits have designed new kits and you know It's interesting to watch sort of the private sector sort of have that same sort of title 10 title 50 try elements being built today The commercial world now is creating and capitalism is helping to create both offense and defense in the cyber arena So we're seeing similar similar things emerging and you know, I think that's going to continue to be a trend as we move forward globally I'm calling their names and they don't do anything. I'm sorry Hi, shavon Gorman with the wall street journal I was wondering particularly for mr. English, but both of you if it's possible Kind of following on the first question to give a little bit fuller explanation of your assessment of Kind of the current defensive capabilities and sort of what's what the limitations are and what your concerns are in that realm So I think I would associate my remarks with what david had to say Which is that I think that the defense by and large is increasingly aware of its vulnerabilities The threats are right against it, but not yet joined up Kind of integrated or collaborative enough across the multiple parties that must combine their efforts in order to prevail against those threats I think within the private sector. There isn't yet enough information sharing that is agile enough or timely enough in order to Counter the effects that increasingly are quite agile and very resilient Against them and there's increased sharing between the private sector and the u.s. Government But that hasn't yet. I think achieved full effect I think that there are still some inherent limitations between that The policy that was signed out by the president earlier this year Took I think a very useful step in the direction of encouraging information sharing between the private sector and the government and vice versa I think legislation that is considered on the hill would take a further step And certainly provide some incentives in reducing the liability for information sharing We need to at the same time ensure though that we do not incur upon privacy and civil liberties And so it needs to be done exactly right But increasingly what we see is that information sharing must be done in something approaching real time because that's the nature of the threats Areate against us And that the integration between the various activities we would bring to bear private sector Private organizations individual efforts and governmental efforts have to be applied as a collaborative effort again in real time Yeah, but echo what chris says but also add the following. I mean when you think about defense today And I had a lot of these products at McAfee and other companies were very similar Most companies and most entities set up an architecture a defense architecture that Is very deep and it's it's very comprehensive I mean to watch if you studied the architectures of the critical infrastructure security models today, you'd be amazed at what's deployed The problem is that nearly every Point of defense that they've created the exact same engine is sitting in that defense So what I was alluding to earlier is sort of that maginal line was created At the firewall sits an antivirus solution looking for and scanning for files At the intrusion detection or prevention layer exact same thing signatures built to look for Known attacks at the email level the web level the host level the cloud level Almost everything's the same The difference is at one layer it might be a McAfee or a semantic or a trend micro or kaspersky or you know a Sophos, but the engines are identical. They're all signature based solutions They all have a blacklist that block a known bad files If you don't know about the known bad file your entire defense architecture is is easily defeated So today's attacks are coming in in totally new ways. They're coming in through applications through executables They're coming down in stages And the pattern matching blacklisting model is somewhat defenseless against that architecture And that's what's created this sort of perfect storm that I alluded to So we need new ways to look for these exploits We need new ways of technology to study this and that's why I alluded to virtual machines That nation chambers to test the holistic view of an application study it over time But do it at line speeds of the network It's critical that we evolve that model because we'll be sitting here three years from now If we the exact same known blacklist kind of model at every egress point The defense architecture will stay, you know, where it is which is hugely dislocated from the offense I hope we're in the back Hi Brooks Dalsmith from pwc. I have a question that comes from Dave's comment about the growing market in offensive as well as defensive tools Do you all see a prospect of? Corporations hitting back at the people Who are attacking them and what you would and what would you see as the implications of that? I'd also like to offer speculation that maybe if imagine no line had used more pyramid architecture It would have been more successful. Thank you You know, I'm not sure I could comment frankly on on striking back, but you know, certainly at times We think that that's a necessary Unnecessary solution I mean watching what we see today with all the all the theft and all the all the elements of crime that's out there We need an enforcement model that is eradicating the bad behavior somehow some way and it kind of goes back to some policy And some teamwork and some collaboration capabilities But if we don't interlock the law enforcement elements And the defense elements, we're never going to solve this. So we need a way of striking back You know, do we strike back in cyber ways? Maybe maybe not But we need a cooperation. We need a cross-border cooperation to eradicate this bad These bad actors and these bad elements and that might require diplomatic relations That might mean law enforcement interactions across the world But somehow some way we've got to put an end to that and and we need to do that So I think it's an interesting question I think that at the end of the day that the We need to do more than simply kind of take the slings and arrows that come our way Kind of go into a fetal crouch and kind of be better and better about taking arrows Right in stronger and stronger armor plate. That said, we do not want to encourage vigilantism, right? There are many instruments of power private sector and public sector power That I think could equip themselves quite well in this they've not yet been exhausted Whether that's the standard realm of deterrence, which is that you increase resilience You increase the possibility of attribution You then set up the possibility that the consequences you can bring to bear using those standard instruments Might then be more effective and those might be criminal prosecution public shaming Might be financial sanctions if it's a nation state that's doing something to your criminal organization But I think we need to exhaust those traditional instruments that we use to essentially effect consequences upon those who would do damage to our societies or our private sector organizations Before we contemplate the possible the dangerous possibility that we might somehow implicitly or explicitly Kind of encourage vigilantism. I don't think that we'd be well served by I think we have time for one more Good morning, Andrei Ashelal Issa with Reuters. I had a question for you that has to do with this kind of problem of nsa being involved and the question about Whether in some ways the fact that you know, there's concerns about privacy whether that's hindering your ability From the government side and what the perspective is from the private sector side in terms of that You know dual halting with general alexander been both head of nsa and cybercom and whether you know Whether you think that there's sort of an institutional problem and moving forward to Do the kind of creative innovative solutions that you're talking about So the question and I'll repeat that for those who didn't hear it If I kind of take liberties with the question, please shake your head vigorously Was whether there are some inherent issues with the fact that nsa As an intelligence organization is also involved in defensive issues Trying to bring resilience and integrity and defensive properties to networks. Is that a fair characterization? I would say that that Cognitive dissonance has been with nsa forever We've always been an organization that at the same time we broke codes made codes because underpinning each of those are common disciplines And it's very important for us to then contribute the fullness of our experience Right to both aspects of our mission set. There need to be controls in place to make sure that we're doing it exactly right And that there is not You know any Equivocation taken On either of those mission sets But nsa By virtue of experience and by virtue of what it sees on a daily basis Understands a great deal about the nature of threats in the realm of cyberspace And can make contributions in its information insurance role To both the creation of more resilient networks and ultimately to the understanding of operational threats in real time Such that if provided to those who then build operate defend our nation's networks or for that matter coalition networks Would put them in a better place such that they might then have a better prospect for having defensible networks or networks That they actually succeed in defending and so that's the proposition That's what nsa is trying to do mindful that it needs to make sure that it is completely faithful To the authorities that it has exercising nothing more right than those authorities And that's why general alexander as the commander of us cyber command and the national security agency at once Tries to affect the synergy between those two as opposed to considering that that's perhaps a attention There is no tension in terms of the mission outcomes Though there is some kind of distinction between the effects that we might bring to bear David Yeah, I would just further on at least from the private sector. I've been Very encouraged and very pleased with the progress that we've been making between private and public in the interlock And general alexander and chris have led just an amazing effort to reach out to the private sector and collaborate effectively together And it's been uh, it's been a great process Obviously, um, and we've been able to create an environment where we can share amongst each other as best as we can But privacy still is a is a large element of challenge for security companies Especially public security companies that trade on public markets that are governed by the sec and governed by serbanes oxley On hippa and other types of legislations that are out there We don't often have safe harbors around privacy companies that are breach companies that have a large amount of information that Could be useful in forensics investigations around Who the adversary is or what the attack was around Is is a delicate balance for security companies because today the way the attacks go down is they're often embedded in documents or embedded in Applications and a lot of times this has very important private information about a company It could be about their financials could be about their engineering drawings It could be just about anything patents and so we have to receive that as a security company And yet we're under um, strict, you know, nda's and other types of disclosure requirements around that privacy element So we could do a little bit more here. I think it's important The executive order is a step in that direction the president's order I think we've got some capabilities to work together with critical infrastructure to do more I'm encouraged by the progress and I would continue to try to urge us to To collaborate as as chris said and to create some more team working around that area Particularly around privacy related to security and breaches Thank you. Let me push that one just a little bit, dave You've glued a lot of companies together And so it's inappropriate a lot of times to compare corporate experience with government experience But if you were looking at the federal enterprise, not just cyber command and say the federal enterprise for cyber security What kind of recommendations would you give? Wow, you can talk about pyramids Talk about pyramids. Can I go back to the imaginary defense? No, you know, again, it's uh, you know, we're seeing a lot of capabilities that I think the u.s Cyber command is put forth that allows us to unify a lot of the intelligence communities defense networks the architectures that are there Which is exactly what you alluded to in the in the security world where we're unifying our architectures are trying to do that That's critical I mean as we look across how to defend the networks the better we are at being able to have visibility continuous monitoring of those networks Um some of the legislation of fisma and others that allow us to be able to monitor those networks defend those networks and do it Across a complete command is extraordinarily a positive direction And if you go back just a few years, you know, we didn't have those architectures in place. So we've moved You know much in the in the right direction But if you go back to the state levels or you go to I was just visiting the state of california last week in sacramental 160 agencies 160 different networks 160 different c-sos completely Um lack of collaboration would be an understatement there So you can really see the the need for bringing this together and unifying this in order to defend because it's much easier to attack 160 separate networks with 160 different architectures with no availability and no monitoring capability across that So this is needed at every level of government in my opinion to defend their network and uh, it's needed in the private sector too Chris maybe will give you the last word you've been doing this for a long time. You want to What do you want to say about it? I mean 20 years ago people couldn't spell cyber security. So we're in better shape. Where are we now? I think that might be my tagline Which is this is very encouraging to see the diversity of the organizations the people the backgrounds in this room And to see the level of interest in this and so we need to then capitalize on that And create the integrated architectures technology And the collaboration that is essential for us to essentially push back on the threats That constitute a threat to something that is better. It's good, right? It's a generally better thing than the threats that are arrayed against it And so the choice is do we give up on this or do we kind of continue to defend the things the equities that we have in this space? I think it's clearly got to be that Thank you. Uh, please join me in thanking our two speakers What we'll do now is we'll have a Brief swap here of name plates and then our first panel will come up if I can ask the first panelist to Come on up to the table. That would be great That's encouraging. Do you guys know each other? Can somebody swing the door shut for me? We can get the the show off and running. Hey, Clara if you can You saw them both? Okay, well, we're missing two of our panels. Let's see if we can get them Could you go to the speaker's room and ask ellen nakashima and Tell them now's the time Yeah, well, I'll go ahead and get started while we're oh, here we go. Great. You uh, you you uh, did exactly right I'm really looking forward to this panel. I'm looking forward to both panels But this panel I'm looking forward to in particular because I'm on it and I won't actually have to And I won't have to do anything because these people are so great and know so much Um ashara z's is the founder of fire. I uh cto. I was kidding him about I think I met him at sun a long time ago So you have sun micro systems that sort of populated through the it community and a lot of the The joke about the company was that they were really really smart But just didn't figure out a way to maybe monetize that so anyhow, you've solved that problem and Grateful that you can be here Shane McGee general counsel of mandiant Mandiant you've probably never heard of it's a little company But we're very grateful that he's here James Mulvannon probably I'm going to say probably the leading expert on china when it comes to this stuff That puts you on the spot and he'll spend the rest of the panel dodging questions, but um, he is indeed the leading expert and A friend and ellen nakashima Globally famous Now when I was standing out in the the hall I heard people at the coffee thing saying you were the go-to person in washington for this stuff And I thought oh, I can't ever get away from it. Can I so ellen? We're very grateful to hear And finally shawn henry Now at cloud strike, but shawn of course is familiar to all of us as the The guy who was really one of the people who turned fbi into One of the best cyber outfits in the world So when you think about fbi capabilities, maybe 10 years ago or 15 years ago and where they are now A lot of the credit for that really goes to shawn and deeply experienced with a lot of a lot of these issues Both from his cyber side and from some of his other assignments So with that what i'm going to do is ask each panelist maybe to give a few opening remarks We're talking about identifying the threat maybe five minutes or so identifying the threat And then we'll turn it into a discussion both among the panelists and and with you the audience So ashara why don't we start with you and just go right down the row? Yeah, so good morning everybody and thank you jim for that kind introduction I'll just speak about the threat from the perspective that I have Since I started the company because the threat was the problem that I was looking to solve Back in 2004 So as an entrepreneur, I believe my job is to solve difficult problems And I actually was on the hunt for a problem at the outset or the founding of fire eye And I found this problem Interestingly enough in the archives of the do d The do d felt the pain of the potential of evolution of advanced and stealthy malware And the implications of that malicious code Were actually wide and deep the more I studied the problem the more it became obvious that This was perhaps the defining problem of the 21st century So in finding a problem, I think I found a big one And it's a complex and challenging one right so While there's some good news to be had in terms of the technical advances that we have made in countering the threat There's still sobering news that this is still one of the most challenging problems of the 21st century and I think the recognition that cyber security right now imposes One of the deepest threats to the security of this country above and beyond terrorism Is a recognition of the growing problem here So I started the company in 2004 with the belief that malicious code Would acquire a purpose beyond self propagation, which was its entire purpose in 2004 It was kind of like amoeba Flitting around on the internet with no purpose with no obvious goal in mind But if you could give this piece of software a goal What goal would we give it? and The goals that came to me at the time were obvious that if Somebody can plant code on somebody else's computer Then they can steal the information from it and so theft Of both information in financial assets became the obvious evolution of malware As the problem was being studied in 2004 and this should not be a surprise to anybody This is not something that we should be shocked by Theft is endemic to human nature We've had thieves since the dawn of society and we have had spies since the dawn of organized society and cities and nation states so to the extent that information has flowed into Cyber realm and that assets have moved into the cyber realm We will have thieves in cyberspace and we will have spies in cyberspace and there is no Treaty that will abrogate that just as there's been no law that has abrogated crime So there is going to be no law that will ever abrogate crime in cyberspace People will continue to conduct this so as long as there is a cyberspace People will continue to infiltrate cyberspace and they will continue to evolve their techniques At the time I started studying the problem. It was also obvious that the Defensive measures that were in place Unfortunately nine years later are still mostly the same defensive measures Would be hopelessly inadequate when the malicious cord acquired a purpose So if I'm a thief my job is not to announce my arrival If I'm a spy my job is not to announce my arrival And if the agent of the thief or the agent of the spy is a piece of malicious code then the first thing It will acquire is a stealthy Countenance to itself and it will come in unannounced and leave unannounced And it will not be something that will be easy to identify so the ability for code to be Shapeshifting or morphing constantly to evade the pictures of the code that exist in the blacklist signature databases was the obvious evolution That was anticipated So my hope and my goal was to create a new generation of technology That could better combat this threat And I I say that very carefully better combat this threat because the threat is truly insidious and uh, I don't believe that there's ever Going to be anything like a perfect defense But we can have a defense that is a lot lot better than the defensive measures that we have today So that was my goal and hope in starting fire eyes. So Leave it there Thank you Shane Thanks, Jim Yep red button. There you go. All right. There's only one button. That's I should have figured that out So we're talking about the origin and the problem and I'm from Mandiant and We we issued a report relatively recently about China as it relates to apt1 the advanced persistent threat group one How many people read that report out of curiosity? Okay, great So you think when you ask me what what the origin of the problem is I'd say well china China is not really the origin of the problem. China is just successfully if you can call it that taking advantage of the problem I think the real problem here is what we call the security gap The security gap is the difference between our development and In continuation of innovation of technology And how much we're willing to put into recent how much how many resources we're willing to put into security So the gap between those two things is what we call security gap We're always going to innovate faster than we're going to look back and try to plug any holes that come out of that innovation And that's that's that's how it should be We don't want to cripple our own innovation by doing more than that. So the How do we eliminate the security gap? There there is no technology out there that is going to eliminate this gap for us There's no law out there that's going to eliminate this gap for us The best we can do is invest I think in a combination of good technology And it's good people. I think good people being probably the more important of those two things One of the most important consequences of the security gap. I think is that prevention is Impossible 100 prevention is impossible You heard this morning people talk about this Defense architecture the Maginot line all these things are incredibly important We have to put these protections in place that will stop a good deal of the threats out there But there's always going to be this gap these this 10 5 10 of attacks these advanced threats That cannot be prevented. Uh, and how do we solve that problem? Well We need to be able to detect Contain and remediate those types of intrusions in minutes detect them By far the most important part of that make sure you have technology people and an ongoing business process In your company so that you are always looking out for intrusions things that get past the Maginot line You uh, you detect them you contain them make sure they don't spread to different systems And then you remediate you kick those people off your network Again, I think this is very important as part of an ongoing business process People generally think as incident response is something you bring someone in to take care of You know be it mandant or someone else you kick these people off your network And then you're done until and unless something else happens. No, you have to be constantly vigilant No matter how many defenses you put in place There will be people to get by Also want to talk about a couple trends that we've seen at mandiance We respond to a number of events. So I think I could be most helpful in talking about what we see in terms of of the threat One of the strongest Most important trends we've seen recently is this concept of outside in attackers getting through To your company your networks through outsourced service providers vendors Companies that you're acquiring if they can't get to you directly and Depending on the team they probably could but it's probably easier to go through Another smaller company or a weaker partner to get into your network. We're seeing more and more of that And you know, it's easy to see why in 2012 companies spent twice as much on outsourced service providers as they did on security So they're relying on the security of these other companies who are really just struggling to get by and a lot of them Haven't necessarily put in effective security measures themselves Offering the adversary and back to or into your network A second trend we've been seeing sophisticated network reconnaissance So the apt the advanced persistent threat I think like Dave said this morning the vast majority of that is china So it's really just a pseudonym to some extent, but this uh, this The reconnaissance we've seen by these attackers has Certainly grown in sophistication in the sense that they're no longer just trying to get in any way They can fumbling around until they see something interesting or see what they're looking for and they Pull it out of your network more and more They're focusing on system administrator accounts or compliance level accounts that have information about for example your pc i audits Why do the work themselves to figure out where you're vulnerable if you know exactly where you're vulnerable And they can just pull the report from you and exploit that So it's a much more sophisticated approach than what we've been accustomed to seeing Also persistence, you know the p in in apt is persistent threat, but there's a new level of persistence It's not just a persistence mechanism Something that these attackers put on your system after they get in there To make sure that they're ever present to make sure that if you clean one or two systems off That they're going to be able to continue to navigate your network and pull down your treasures Now it's more even if we do successfully kick them off our networks They're just waiting in the wings with another attack They'll wait a couple days a couple months and then they'll come back and they'll attack you again and more and more It's not just the same team coming back and attacking you There's some level of coordination among the apt teams So that they will come back and attack you from a completely different direction with new tools techniques and procedures And they'll gain access to your network again And this goes back to my previous point about being so important that this is an ongoing business process You do not let down your guard. You absolutely have to have people there watching at all times Finally the fourth trend uh more targeted drive-bys. We're all familiar with fishing at this point We know that the apt and other advanced teams use fishing quite frequently They send you and your employees email messages That look like they're from trusted senders. You click on a link. You're compromised. They navigate from your system They go through your network and they compromise your entire network Now as we're getting more technologies in place to detect and stop those types of fishing attacks They're relying more on drive-bys drive-bys are where they compromise another weaker website That they know that your employees frequent And you will go your employees will go to that website as soon as they go there Or as soon as they click on something on that website They'll be compromised that way and it's a much more difficult problem to solve unless you're just going to cut out all web browsing Sometimes they can compromise entire industries or multiple companies within an industry at least by going to for example An industry website or an industry portal if they want particular information from say the automotive industry They can go to a vendor of automotive parts compromise that site so that all the Automotive manufacturers are compromised. So those are just some of the trends We're seeing just to add some color to uh to the threat. Thank you This is always such a gloomy subject, isn't it but and now to bring us cheerfulness and light That's right. Exactly right. Well, you know, let me go first against my irish nature and start with some good news um and You know it since you know since we're at csis in the spirit of kurt cambell. I have three brief remarks We have come a long way Things do look bad, but we have come a long way. I've been working in the chinese intrusion set since 1998 And then it wasn't even known as the intrusion set because the china's cyber threat that was causing us to cower in fear Was that they were defacing whitehouse.gov's website You know the script kitty patriotic hacker types and we thought that the world was coming to an end But the oh for those halcyon days when we were worried about the whitehouse.gov website But up to about five years ago when senior policy makers would ask me about what we were going to do about the china's cyber problem I would say to the ma'am. We have an attribution problem And the attribution problem undermines our ability to come up with the downstream set of policy Objectives and policy implementation that that we could do something about it. The good news is we no longer really have an attribution problem We have scads of attribution. We have deep deep attribution Um now the qual challenge for the last two years in the policymaking realm has been what do we do about it? What are the steps that we can take and people are basically, you know with a copy of tom shelling's You know a strategy of conflict tucked in their back pocket trying to figure out How we actually repair the airplane at 30 000 feet uh and do something about this cyber threat even with Exquisite attribution Now i in in previous four i've talked about a range of things that we've thought about we've thought of you know Well, let's just declare that we have a deterrence policy and we and we know that the problems that are associated with that Or let's just focus on buttoning up the defenses. It's really about the choice of firewall Or it's about you know the edge and all this and i think that the previous speakers have highlighted The extent to which we actually needed to change our entire mindset from a perimeter defense mindset To a defense in depth mindset where you knew there was going to be compromised hardware and software inside your networks You knew that you were going to have advanced persistent threat But you couldn't just curl up in the fetal position on the ground. You still had a mission to carry out You still had things to do and that's led us to things like virtual encrypted enclaves and other unique ideas that have come along That say okay, my network is compromised, but i'm still going to be able to operate inside of it But that wasn't enough. Now, of course, there are always people who say well the best defense is a good offense Let's scare them straight through our own enhanced computer network exploitation campaigns Let's steal all their stuff, you know, then they'll really feel a pain and everything will come to a balance You know and as someone who recently took his two teenage daughters to the fairfax county juvenile detention facility To emphasize to them why they needed to listen to their mother You know, I I mean i'm sympathetic to the scarum straight philosophy But the problem is the chinese in particular already believe we're ubiquitously intruding their networks You're not changing a mindset by doing that to them um What we've what we've happened upon is one of the tools Is we have to get the chinese and the other adversaries off this idea that when they exfiltrate the data out that it's pure You know, they believe this is ultra. This is the most profoundly successful intelligence campaign they've ever had They believe with metaphysical certainty up until the recent times that what they're exfiltrating is actually true But using deception and poisoning the well and doing things like that in terms of the data exfiltration is obviously not new It is it can be technically difficult But we've seen the tried and true methods that we've had in the counter intelligence and counter espionage realms have really helped us We obviously can't affect everything But we do focus on key areas We want to corrupt the inner workings on their end because we know that if we sow that kind of distrust If the intrusion sets tell their leadership that that the carrier strike group is at one lat long and it's at another That will lead to circular firing squads on their end. Who's the leak? Who's the mole? Of course, every ounce of bureaucratic energy they spend on finding that is an ounce that they're not spending intruding our networks And hopefully in my view it will accelerate Centralization trends that we see on the chinese side for them to move decision making about these operations to higher levels To play into their natural control freak tendencies that we associate with the chinese government Um and move them from a system in which they have a bottom up entrepreneurial grassroots oriented intrusion system where people are encouraged on their own initiative to go out and find data And tools and accesses to something much more resembling our system, which is top down Authority centric in other words I would offer that our policy goal should be that it should we wanted to make it as difficult For the chinese military and the mss to get a c&e operation approved as it is in our system To get a c&e operation approved To me that would be a much better world So finally though to support that we need a different type of intelligence than we've had to this point We still need the technical intelligence We still need the mandians and we still need everybody like that who can tell us the specific malware signatures and things like that But we also need an intimate understanding of the adversary And unfortunately, this means largely doing it in native language Um, and we're and we're just beginning to develop new techniques and capabilities in these areas I'm a chinese linguist. I have large teams of chinese and russian and farsi linguists who look at this problem Um, but understanding rather than laying back and doing the forensics of the intrusions after they're already in and remediating it there We also need to be forward. We need to be inside their system We need to be looking at their websites and chat rooms and bulletin boards and blogs and irc and silk and everything else And anticipating and doing indications and warning of intrusion planning that's ongoing Rather than simply trying to remediate things necessarily only remediate things that are happening inside our networks We need to obviously do it both and we need to do it smartly Which means not going there via, you know, you know conus based ip addresses and things like that But utilizing the technologies that we have that allow us to do is securely Because I think then and only then with the combination of that kind of technical intelligence But also having deep adversary intelligence that can inform The deception and offensive counter intelligence operations the things we want to do Only then can we change the cost benefit calculus of that adversary and really impose the kind of costs That our leaders are talking about in terms of rolling back the scope and scale of this intrusion set. Thank you Great. Thank you, Ellen Jim, thank you for placing me right after dr. Mulvennan. It's as impossible to follow but Thanks again to csis for host and fire I for hosting this panel I'm not a technologist or a cyber security expert and Or policy expert unlike the people on this panel who are the experts in who I Um actually gain a lot of my insights from so what do I have to I do not accept that Oh, yeah, well it's accepting Sean who has never been a source of mine, right? No Um, I'd like to just put that on the record on the record So what I have to offer is as a generalist is just some of my Insights and observations from several years of covering this Not because of any special technical expertise, but just from the average person what I think is kind of interesting to note James mentioned Chinese espionage and counter espionage and now of course the Chinese know we are interested in doing deception to them. So Maybe we should also think about what sorts of deception they are doing back to us But it's interesting to note that with all of the attention paid of late to Chinese cyber espionage theft of ip the widespread for assistant campaign It should also be noteworthy that the Chinese are also Into our networks for counter espionage purposes, you know spy versus spy hacking into the servers of technology companies like microsoft and google in order to Find out who the us might have under surveillance through gmail or microsoft mail for instance It's not something we often focus on but that is yet another Direct link between cyber security and national security if you think about how interwoven The u.s. Surveillance system is into the private sector and how dependent then this surveillance system is on the security of private sector networks Just something to think about Another to me Significant development in this advanced threat space Is one that is playing out right before our very eyes and that is the palpable shift in the u.s. Governments Stance in confronting china You know people like james have been saying china publicly in the same sentence as most aggressive Collector of cyber economic espionage for years, but it's only been recently that we've heard senior administration officials most notably national security advisor tom donlan publicly calling out the chinese to to warn them that if they Don't stop their campaign of economic espionage. It could damage the relationship and that's been also going on At senior levels in bilateral meetings more privately But as james pointed out and others have as well if the next question is well, so if they don't If they don't shape up now, what do we do? Right will our words be backed up with actions? you know visa denials trade sanctions Prosecutions as the justice department has been talking about bringing Against nation states for cyber economic espionage where they can prove that perhaps the ip that was stolen actually benefited a company in china I think those are all interesting trends to sort of to note and watch if that actually happens With a state like iran I think the challenge is much trickier because the u.s. Does not have one thing diplomatic relations with iran. It's it's harder to read their intent But while they are not in the top tier of advanced threats yet They are trying and they are likely to have more motive to want to Disrupt critical infrastructure systems in the united states say than the chinese And we have seen evidence of an effort to raise their game with shamun Last year the wiper virus some analysts do not believe that the iranians were behind that Threat, but the u.s. Intelligence community Believes that it was the work of the iranians. So something to watch there and in any case Just a week or two ago the department of homeland security put out a Threat alert first ever I believe to the critical infrastructure community that there was a cyber threat Potentially to disrupt industrial control systems Again this goes beyond the threat of just stealing intellectual property and the alert did not mention any country but There has been renewed concern among government and industry officials of increased activity coming out of the middle east and in particular iran So now that we know the iranians have gone after wall street with didos attacks If they're moving on to industrial control systems, I think that's worrisome Something to watch and finally I wanted to just say while we're talking about advanced capabilities Israel is worth noting their top tier, but because we assume they don't want to attack us You know, we don't mention them as a threat but When people with top tier capabilities don't adequately protect their tools that can sometimes lead to unwanted Discoveries so in at least one case Some suboptimal tradecraft Resulted in the discovery of a sophisticated cyber espionage tool called flame jointly created supposedly by the u.s. And israel to gather intelligence on a wide variety of targets including iran And u.s intelligence community experts believe it would have remained hidden Had israel not launched wiper virus against iranian oil export facility facilities last year that caused minor disruptions, but led the iranians to investigate Through kaspersky labs for instance and thus to discover flame Which had done the reconnaissance work on the system so in the interest of Keeping my remarks brief and getting on to the real interesting part which is shan and q&a. I'm going to stop here Okay, thanks ellen. Thank you jim and fire eye for hosting this. I really appreciate it I had a list of some talking points and after david wall and chris englis and this esteemed panel I kind of checked off each one as if somebody already said it and the very last one i had ellen just said So i've got nothing to say Good night Let me let me just reiterate a couple of things that i think are really important at least at least for me I spent 24 years in the fbi much of my time in the last 10 years focused on the cyber threat Now a crowd strike from the private sector side and kind of looking at this confluence between what we've seen in in the government What i see now in the private sector And and how much has actually changed since i left the fbi over the last year certainly from an awareness perspective To hear the u.s. Government come out publicly and make assertions and to hear the president of the united states during the state of the union address to talk about About this threat and about china very very specifically That to me is a dynamic change I think a lot of that has to do with a lot of people that are here in this room Not just here but also in the audience who have been talking about this to raise awareness to the private sector to really Alert people to how significant the this threat is what the risk is To our national security to our economy to our way of life quite frankly And and that's really for me been been one of the most dynamic changes We hear about china all the time But the reality of it is there are dozens of countries that have aggressive electronic espionage programs in place And it's not just nation states, but there are terrorist groups and many of you may have seen Recent reporting About those that are sympathetic to the jihadi cause who have actually called for electronic jihad Against the west where they've actually called for Young men who are sympathetic to the cause to rise up and to use their capabilities their electronic capabilities to use the tools To target critical infrastructure target the financial services sector that to me is is very very interesting We talk about nation states many of whom have Reasons not to attack in a very very destructive way But terrorist organizations have a very different perspective. They've got a different agenda They've got different motivation and that is is equal or or more of a threat um A char had had mentioned in director muller has has said as well About this exceeding cyber exceeding the the terrorist threat But I would actually ask you to think about it in a different way Because it really is a tool and it's the tool that's used by terrorists and foreign intelligence services and criminals So while it absolutely will enhance the capabilities of terrorists and make their Their threat to us more imposing It also enhances the capabilities of criminal organized criminal groups and of these foreign intelligence services So that all of those threats their capabilities are are raised and they are taking advantage Of the same technology that makes our lives more effective and efficient. They are becoming more effective and efficient themselves um, one of the other pieces Much of what we've seen what you've read in the media through the good work of of many of of the people here to get this message out there still This is just really the tip of the iceberg and much of what has has occurred and is occurring You still have not heard about because that's Below the water line I equate the aggregate of all these cyber threats as an iceberg and what you've heard about You know a million user names and passwords have been stolen and somebody lost a hundred thousand dollars through some fraudulent ACH transfer some denial of service attacks That literally is the tip of the iceberg and what's occurring below that water line Which i've seen because i've been circling it in a submarine for for many years is ominous and Again really important for us to continue to have discussions like like we're having here today I would also Highlight the the point about the supply chain what i call the supply chain And while you may increase your defenses and raise your capabilities and reduce your vulnerabilities The the threat is not just to you but to everybody you are doing business with and the adversaries are becoming increasingly sophisticated in their interest and willingness to target The everybody that you're doing doing business with Let me end here because i know we want to get to questions I have a couple of letters myself I heard uh chris and dave using their acronyms when i was in the bureau I had what i called the four p's and i think that they actually absolutely apply in the private sector as well Being proactive to james point being able to raise the cost to the adversary right now There's no cost the risk is about zero because people have been called on it for years and years and nothing's happening And they continue to do it the value is up here the risk is here And until we invert it and make things more difficult and challenging for them this goes on unabated forever And i think the denial and deception is key Changing the way that we look at these things being proactive on the networks not in an offensive aggressive way I am absolutely not suggesting we're hackback, but on the network that we create Capabilities that make things more difficult for the adversary Being predictive using intelligence to understand due attribution to raise an awareness about who the adversary is Technologies a piece of the solution, but it is not the sole solution There's policy process and strategy that if you employ them on the network You can be much more robust and resilient and that really comes down to using intelligence to become predictive And then preventative where you can prevent the consequences You are not as my colleagues here have said going to prevent the attack But if you identify and detect the attack early enough You can you can prevent many of the significant consequences and the last p is the partnership piece Which really is the government in the private sector intelligent sharing and and making things Much more collaborative in that way, so Jim let me let me turn it over to you and stop there. I know there'll be a lot of questions Yes, great thing. I in fact have seven questions already. And so I know we've got a lot I'm not sure which one to start with so I'm going to start with one that's a little Building off some of the things Sean said that when you look at so one of the things I think we've heard today is it's really not that hard to do this And if you kind of look at some of the tools that are available online You could develop some nice capabilities Why haven't hacktivists? Why haven't the non-state actors been more aggressive? Why do you think when we think about threats? We've talked a lot about nation states and they do appear to be in the lead When should we start looking for what are the signs that the anonymous is or the All sex or somebody like that will be the people we have to worry about Because I don't think it's lack of technical capability. What what is it that's going on? Let me just give you my perspective so so Yes, there are a lot of tools out there and you can buy them if you have money Having said that there is a difference between the kind of tool that allows you to infiltrate into an organization and plant malware there Versus a tool that can cause real destruction, right? We cannot underestimate the difficulty even now with all the tools available to Truly develop a very destructive cyber weapon I would say the cost of that is probably between 10 to 20 million dollars So it's not your average hacker that will do this Why is it that because it goes beyond Having code you need to have physical infrastructure. You need to have the skater controller So stuxnet was developed with incredible qa performed on the physical machines This is not going to be developed in a basement You need to have the physical infrastructure You need to understand and have the reconnaissance to know What systems run where what versions of the code run where so that they can be attacked, right? So Now having said that 10 to 20 million dollars is not that much money, right? And it is certainly a lot lower bar than developing a nuclear weapon, right? So it's not trivial. You can't go use zbot and have zbot crash the grid On the other hand, you don't need to work like the iranians have for almost a decade to go build this if you are focused So I think the the comments made earlier about The nation states not having the motivation To do what they're certainly capable of doing which is destroying the grid Is what has kept the grid up up till now the non-state actors particularly the Jihadis and the other people who have a nihilist agenda, right? So you have to have a nihilist agenda if you want to do something beyond Financial or information gain uh, and so those guys have not had the Either the the ability to conceive of such an attack Uh, but my sense is that that is not very far away That if they can imagine a physical attack a cyber to physical attack Not just imagine it but see it happen to a country that they perhaps care about Uh, that they will be motivated to go construct a cyber to physical attack and Probably the clock is ticking down on that event right now I think excuse me. I think ashar headed that on I also think being state sponsored gives you just so many different advantages in terms of your ability to create this type of malware It's it and it's not just the the resources the financial resources And it's very much so the architecture and the backbone like ashar said, but it's uh, it's also the ability to Coordinate amongst yourselves without any sort of external influence be that external influence your day job And a lot of these activists have day jobs They're not able to to give the same type of time and attention to these issues as state sponsored State sponsored actors, but it's also Just a matter of being able to coordinate with large teams of people the activists are are being investigated They're at risk of being you know arrested in any time. They can't disclose their identities or shouldn't disclose their identities to each other So it's much harder for them to coordinate internally than it is for the state sponsored actors So it's it's the coordination the ability to coordinate the resources The back end architecture everything I think it's very difficult for the hacktivists to Get the resources they Jim I think another feature of it is You know if I was channeling my former rand colleagues bruce hoffman or brian jenkins. I would say that historically Um, particularly the extremist element the the sort of shock value of Explosions and other types of terrorist attacks that have that immediate television Political impact. I mean, I think bruce hoffman in particular has talked about how the real purpose of a terrorist attack Is to get people to see the attack on television and to be scared and everything else Whereas in many ways for historically the plausible deniability of cyber attacks undermine the political impact of a terrorist attack Because it could just as easily be our our shitty critical infrastructure that failed Rather than you know as someone who just spent two and a half hours on the 395 It could just be our critical infrastructure that just folded underneath us rather than something malicious or deliberate But I would say that the reason I would the reason I would caveat that now is moving forward the trend lines that we have Are pushed towards connectivity over security are moved to mobile the way the way social media allows adversaries to enumerate targets Means that basically we have more on the grid. We're more vulnerable We've put you know, we have much more of the critical infrastructure that is now accessible Therefore the impact of these attacks every day could be potentially graver as as residents of northern virginia know When you lose electricity everything else completely collapses around here And so be given that the trend lines over time I think argue that this becomes a much more attractive target for people who want to do devastating damage To the united states and its allies Thank you I don't have much more to add except to say that I would also highlight the difficulty of Doing the reconnaissance work and getting the intelligence on your target That is so crucial to a successful destructive attack on on an electric grid or critical infrastructure It's not just a question of buying an exploit on the black market. You have to do a lot of exquisite intelligence work that is Not often just within the easy grasp of the average Hacker or or hacktivist group And then to James's point about the anonymity of the of the internet I actually think that if um, you know every time there's a metro derailment or some active nature and People think trains crash or people die I I always first thing I think is was this cyber? I mean if it's not active nature It's the trail derailment Is it cyber because imagine if a terrorist group actually put out a a message on a forum to claim responsibility I think the impact of that would be just a lot bigger than something that was caused by a malfunction or malfunctioning system, so What they said Well coming from a guy who's actually arrested some of these folks. I guess we could just take it at that I'm going to do one more and then we'll turn to the audience So in a few weeks, uh, the president will meet with president g of china in, uh California and for a summit that's good And people have asked do you think cyber will be on the presidential agenda? I think that's a fair guess We don't know What would you guys have the president say to the chinese, you know, what would you have him? Would you have him? You know a direct confrontation is not going to be that useful. So what what would you recommend to the president? Sean why don't we start with you and go the other way since you dodged the last one? Ni hao ma Um, I think that this is an issue that has to be on on the table And I think it's actually a priority issue on the table I think that what's been occurring here in the commercial sector Over the last five years is incredibly detrimental to our long-term prosperity And we it's been in the shadows too long You know again, we've had attribution over and over and over again. We've heard the reports We've seen we've heard the government officials and And I see no change in the activity One of the things we did when we we did the comprehensive national cyber security initiative initiative number seven had to do with Really defining what the red lines are and I think that that still is critical Look every country has been involved in espionage for For centuries right going back to the greeks and the romans and probably before that but What is happening is Nation states are using their nation state capabilities to attack the commercial sector And they are empowering their commercial sector to an incredible advantage against us economy The us hasn't done that and I think that those red lines need to be defined and there needs to be a discussion about what the impact is what the Potential retribution might be if you cross those red lines, whether it be diplomatic economic Civil sanctions, etc. But that's got to be a discussion. It's got to be clear and I think that it's got to happen sooner rather than later Exactly right. I think that the laying out the potential list of of sanctions and of Measures that could be taken to hold China accountable for its actions Is what I would want to see put out there Being more explicit about exactly what the stakes are and how companies are really starting to come frustrated with the degree of of Siphoning of their IP that's taking place to the point that some I think are starting to rethink their investments in China Maybe refocus move out or just You know put things on hiatus in in in beijing Which should concern the Chinese if there is still concerned, you know concerned about becoming Retaining economic powerhouse status and I think that's where you know between Hitting them where it hurts with financial Disinvestment and potential for trade sanctions visas and actual prosecutions. You're kind of doing a You know holistic approach to holding them accountable and I think If the if the president can be explicit about that that in a diplomatic way that might be a good step forward Well, let me begin by saying what we shouldn't tell Xi Jinping We shouldn't walk into the room believing that he doesn't know about the intrusion set And our first goal is to educate him because the Chinese military or the mss hasn't told him about it I continue to be gobsmacked by senior policy makers who begin with that question when they talk to me about it And they say well if only if I tell him what's going on, you know, he needs to know this, right? I'm saying no, he doesn't need he already knows The ministry of foreign affairs weenie sitting next to him may not be read into the program But he certainly knows don't worry about it The dilemma we have is we've tried to make this very cute distinction in our discussions with the Chinese in the strategic dialogue level between Traditional espionage in the cyber realm, which we've said we cannot legislate or govern through treaty And commercial espionage, which we've tried to make a separate category And this has been a real clanger with the Chinese because they don't see the distinction because in their system the same people are doing both And therefore and they don't believe our entreaties despite Jim's best efforts at the kicker csis dialogue They don't believe us when we tell them that we are statutorily precluded from doing commercial espionage And then we even give them a very practical reason we say if the united states conducted commercial espionage on behalf of its companies We wouldn't know how to share the proceeds without somebody who didn't get it suing us suing the u.s. Government for anti-trust violations Most of the countries we deal with in the cyber realm have single large state-owned enterprise national champion companies in each sector It's very easy for them to figure out who to share the intelligence proceeds with very complicated in the united states So you make that distinction again, which is not going to be effective, but then you say look It's the commercial espionage that has finally Disproven what political scientists have said in the united states for years Which is that you could never get a whole of government response from the u.s. Government This has unified the u.s. Government including the economic and trade departments of the u.s. Government in ways that I thought from As a political scientist were impossible And you and what has to be emphasized as she is you are now undermining the last remaining pillar of strategic Cooperative sign of u.s relations when the trade and business community Are some of the loudest critics of what's going on on the chinese side who traditionally been the strongest proponents of cooperative Sinister relations you you then say to president she this is imperiling your own economic development Which is imperiling your social stability, which is your number one priority That is the only message that will get through to a general secretary of the chinese communist party Is that economic development and social stability are threatened by the brazen scope and scale of this intrusion so The other thing I say to them is that their technology isn't worth stealing so but that doesn't always go over I now sympathize with shan being towards the end of the line. We have such an esteemed panel up here They're making all the points. I wanted to make I will say that I do agree with shan We have to draw this red line, but I think we also have to tell them they've stepped well over it already I mean well over it there. This is the largest transfer of wealth in history. Was that Keith Alexander? I mean it really is And it's a lot of people have echoed it Ellen, I think you're absolutely right. We have to have a comprehensive approach here We can we being you know all the stuff we talked about today security measures safeguards. We can slow the bleeding But the only way this problem is truly going to be solved is through You know the the the use of diplomatic and economic pressure and we have to take advantage of that We have to tell them that they stepped over the line And we have to distinguish between the economic espionage and the traditional espionage We have to make them understand that That's the only way we're truly going to recover from this issue Right, let me just strike a contrary note here. I don't think anything we say to them will make them change their mind I think every country has its own sense of national security and no leader from another country will come in and change the other leaders mind We could be clever in what we say we could be threatening in what we say The chinese have a notion of national security which is grounded in the nation of economic security The united states is a very different perspective of national security We are worried about attacks from various parties and countries. We're worried about nuclear development in various countries and we by the way Have exercised our right to attack the countries that we feel threatened by Chinese are Exercising their right to steal from who they wish to steal from because they believe it is in their economic interest So while they can have a very polite meeting between the president of one country to the other I'm not optimistic that any action will change Fundamentally because their beliefs about national security and their national agenda are very different and they are going about exactly what they intend to do That's great. Um Do we have questions? Go ahead And please remember to identify yourself even though we all know who you are So my name is harby rishikov I'm with the my name is harby rishikov I'm with the american bar association standing committee in law and national security Um, first of all a great panel put together jim. I love the fact that ellen is there so we can ask ellen questions Uh, maybe you should point out who your sources are ellen will really be helpful for me. Um I guess my question is um sort of falls on the last point, which is that we've been doing this like for 15 or 20 years We've known that the private sector is the real target in a way for what it's where the jewels are But yet we've had not very good public private sharing of information Uh, because the private sector has been very reluctant to make public its range of attacks So the people on the panel represent certain interests that have made it public And that the cost benefit analysis has often been for the private sector They're preferred to do the market is so large that it's hard to offend in a certain way So my question is how based on your experiences How do you change that cost benefit analysis? And the fact that the legislation that we wanted to have with potential immunity for sharing of information has failed Has not gone forward, which is clear what the private sector wants So i'm curious to see what the esteem panel's view is about how do you move forward given that dilemma Thanks for an easy one harvey Um, what I think james you want to go first and then we'll just people can chime in Well, I wanted to start with the perspective of a classified defense contractor that's under the dib sharing program that dod has Um, which many people would say well, that's not really transferable to the private sector because there's all these special security rules And they have all this leverage over us But I would say the following and I think deputy secretary lin would agree with this the real brilliance of the dib program And why it has succeeded is not because the government shared classified signatures with us The brilliance of it was that they stood back and they let all of us collaborate with one another With at least a notional umbrella over us that said we would not be sued for antitrust violations by the department of justice If we shared information with one another Um, and that's the key issue and that was a real sticking point in the congressional legislation Um was you know, somebody's always going to be excluded Someone's always going to feel like they got left out But what we're really seeing now are the rise of these confederations of people who are coming together And and finding creative ways to anonymously share signature data with one another to share threat data with one another um And the government just needs to provide the indemnification and the top cover over the top of it that says We will not penalize you for engaging in collective self-defense Provided as shawn said you don't go too far And we also frankly need to rewrite parts of the computer fraud and abuse act So that we can actually know where the bright lines are of what is actually permissible and what is not Under law rather than waiting for someone to go first and probe the outside edges and find out Exactly where the department of justice's pain threshold is on that issue I am hearing from uh industry including the financial sector in particular that It actually information sharing for instance the latest round of d dos attacks has really Helped spur greater information sharing between and among banks Um in a way that they hadn't done so before so that's sort of the silver lining of that But they would appreciate more um information coming about advanced information about threats coming in from the government And sometimes the impediment to that is a lack of of clearances But um, I think the government is trying is making an effort there to try to get more people in even smaller organizations cleared to get this information Because it takes a while to get classified level threat information Washed through and declassified by the time it comes out to the general public. It's useless um Then the other impediment obviously is to Get information from the private sector back to the government. I think that's one of the biggest hurdles of if not least because of the privacy issues and and um the need for possible legislation to make make To change laws so that sharing of information back to the government doesn't violate wiretap laws or privacy laws in it but those You know We do hear that the government both the obama administration and the hill want to get some information sharing legislation passed Whether or not they'll get it through with everything on the plate remains to be seen I um go ahead the term information sharing really is kind of a Burr under my saddle. I've never used that phrase before that's funny. Um, it it's not about information sharing It's about actionable intelligence right information. The sky is blue is information. That doesn't really do a lot for me I need I want something that I can actually do something about Um, and as it relates to classified signatures There's a the reason things are classified is to protect sources and methods I think that there's an awful lot more that can be shared without compromising sources and methods It's difficult. It's a bit of a challenge. It's very different from what we've done historically But I think that it can be done and it needs to be done As it relates to the private sector the private sector owns the vast majority of the infrastructure They've got all of the indicators all the artifacts of these attacks They could help the government immensely the concern of course is the anonymity and how do we do that? And I think that there are capabilities There are protocols that would enable us the private sector to share information in an anonymous fashion with the government actionable intelligence not information but actionable intelligence that helps towards attribution that will Help towards some of the things that we've talked about here. It's an incredibly complex area There's lots of concerns about privacy and civil liberties. Of course, that's all got to be taken into consideration But I don't think it's a it's a bridge too far. I think it's people sitting down with a different mindset and and really looking at this as a Capability to make the situation much better Yeah, I just want to make a few comments here. So We have built a very large network for Threat intelligence sharing we have lots of organizations that are actually providing us with the threat intelligence and we are able to do it Without impacting their privacy and their personally Identifiable information because you can extract Anonymous metadata about malware and share it generically So people get caught up very frequently on this sort of trap of privacy versus Information sharing and we've been able to actually do both. We've been able to preserve The privacy of the organizations that we have the threat intelligence taken from And and we are able to do that very effectively on a global scale So threat intelligence sharing I think is a good thing, but it is Not a silver bullet You need to have and this is imperative You need to have the ability to generate threat intelligence because you cannot share Things about which you do not know and you do not discover in your network And for the most part the attacks are not being discovered by the legacy technique So if you just walk in with this blanket recommendation that hey, by the way, why don't you share everything you found With the government for the most part I can tell you that the vast majority of organizations are compromised and they don't know that They don't know what to share if they cannot see the attack So it's imperative That they have the ability to see these attacks in real time beyond the legacy security architecture that they've built And then to have an architecture that generates threat intelligence not just consumes it Generates it in real time and then is able to take an anonymous version of that and share that across the globe And and we have actually built a proof of concept system of just that I think a big part of this issue is education and awareness And I think it comes in two parts one two companies In terms of how will you first of all the importance of it? How important is this to your security program? And I don't think that can be overstated It's not a silver bullet. I agree, but I think actionable intelligence is something you have to have to have an effective security program So I think you start with the with the companies and you convince them how important it is and some of this is going to happen itself It's going to happen organically We have some of the industries that are sharing actionable intelligence very effectively right now and others can do it as well I think the other part is educating the public and I'm sorry I get a migraine when I hear from some of the privacy activists on this issue Because I'll sit down and talk to them. I look at my facebook page. I'll have all my friends From the school days, you know saying oh my gosh, you have to stop syspo. You have to stop all this stuff It's they're stealing our privacy, you know, they're taking all our information It's all going to the government and I I engage in these discussions with them I say okay, so what do you think's really happening and they really have no clue They're just following the crowd here So I think there needs to be a lot of education in terms of what exactly we're proposing Is shared to the government and amongst private industry actionable intelligence If done correctly, there's no personal information in that it's IP addresses. It's registry entries It's different things that you're not going to associate with any particular identity So I think that's really important I think once that happens and if we have an educated dialogue amongst the Amongst the public we're going to get some legislation passed And I think it's incredibly important that we do so and just one minor point And I say this to all of the over gratified New millennials who work for me that as long as what they're doing electronically is on a machine That I own they have no privacy All right, I think we have time for a couple Alan and then we have one on the other side. Is that well out? Why don't you go ahead? We got three we'll do those three and that'll be it Alan Friedman from the Birkins institution So this has been a great panel and I've enjoyed hearing about how data has been leaving western companies Could you offer as much insight as you can on what's actually happening with the data once it hits the chinese intelligence organization? You know james, we talked about myths of political science the monolithic state We assume that all of china is the same Maybe some of the different interplays between the different or agencies and how different sectors inside china Maybe using information differently I like to break it into five quick categories because I think each of them have to be dealt with differently in terms of the Policy solutions we look at on the traditional side You know location and movements of u.s. Military assets extremely valuable you get into nipper net you get in the databases You know where things are you can cue other intelligence sensors immediate benefit You break into lockheed you steal information about the j35 Synthetic aperture radar you can use it to fine tune your electronic warfare systems, you know instant benefit You break into the you know the secretariat of some organization You get the state department or the president's talking points three days before the apex meeting Instant benefit your leadership loves it near real-time strategic intel You know all good right on the commercial side you have to break it into two pieces for me One is what I would you know what we traditionally call sensitive business information? So you break into exxon mobile you get into the c-suite You find out what the magic number is of what they're going to bid on that tract in the south china sea You hadn't into your national champion state oil company. They under bid they win the bid Instant benefit the one that's been most troubling to us analytically allen is that last one You steal the source code you steal the intellectual property you take it back to the nest You may or may not give it to the right company They may or may not be able to reverse engineer it Productize it marketize it and then then having the metrics to say and then they demonstrably reduced this us Company's market share in china by this percent and then they then competed with them globally and reduced it by this percent We're only now beginning to get fragmentary elements of data to support that line of analysis And I will say the one thing again going against irish type the one thing that that i'm optimistic about Is that this indigenous innovation state driven research and development and innovation system that chinese have been trying to build since 2006 Is the worst possible mechanism for exploiting advanced western intellectual property And so it may in fact break down at that stage and we they may be able to reverse engineer one generation But the organic knowledge and creativity that undergirded it Will likely prevent them from getting a second and a third generation of innovation out of it So we may see a shallow innovation, but we may not see the deep innovation and that may ultimately be One of the only pieces of good news that we have on the on the commercial espionage side Um, I can just speak anecdotally I appreciate james the the five points there certainly from the military and the government perspective on the commercial side again It's anecdotal, but i've spoken to a number of companies who have reported To me some of the impact that they've seen from the theft of their intellectual property I also agree that this is oftentimes many years down the road because of the time it takes to To monetize that that ip if you will But i spoke to uh head of a biotech or not the head but a senior leader in a biotech organization Back in in february and this person told me that in their business line It typically takes them five years from concept to actual sales to go to market In in their industry and in what they're doing specifically and that their chinese competitors are actually churning out product in 18 months And it's not because they've come up with some new fangled manufacturing process. It's because of all the front loaded Resources to the concept and the engineering are already being done They're being stolen and they're going right to manufacturing and to market And that that's having an impact on their organization and they are just starting to see that now There are a couple of other examples. I won't go into here some of them are somewhat Long long examples, but i've certainly seen it Commercial entities saying it's absolutely hitting our bottom line and we're losing market share in certain areas Concepts from brown university. My question was very similar to ellens But i'll phrase it. Oh, i agree with james's observation about code So is that and there are other examples that can be used as a basis to educate the chinese That is not in their long term interest to Engage in this kind of fact. They said the french do this on a fairly substantial scale I look at their economy and say You know, what what good has it done them? Yeah, uh, brook stallsford from pwc question. We've heard a lot about china a bit about aran and terrorists Nothing about russia who's shown capability and willingness to use Cyber weapons against some of their less fortunate neighbors just any comments about russian capabilities and any comments on russian capabilities and intentions. Thank you While we're waiting for the microphone I'll say I talked to one of our asian allies once and they're complaining about china And I said, well, how about the russians? Do you see the russians on your network? Some of you heard me say this and they said no, we haven't seen the russians on our network And I thought that's a true statement. Just not the way you The michael snell coast guy cyber at what point do we actually start engaging the Overall public and commercial industry from a national level like for example If you look on television, you see today very strong campaigns against smoking against drunk driving Against texting while driving at what point do we actually engage the public? Patch your computers. Make sure you're running Good Operating systems stop using bad software do this at a national level. So we deny The the actors the ability to even leverage our private and commercial sectors as a vector Why don't we go down the row and if you want to hit all three or as many let me just start with that So let me let me echo that. I think that's probably the most important thing we can do Uh, because we have failed to pass legislation to mandate any kind of security control So I think the most important thing we can do is mandate some level of education and awareness, right now It may not be Hey, you got to patch your system because the perfectly patched system is still vulnerable, right? That's the unfortunate reality of today's threat landscape is a zero-day attack will work on any system It doesn't matter how well past it is But the broader point that you're making is a valid one that if we had the critical Infrastructure operators understand the threats that they face They may be motivated to do something about it. And I don't think today There is widespread awareness of the structure of the attacks and the big Dislocation that is now there between offense and defense is an incredible gap between Contemporary offense and traditional defense and I don't think that is widely understood So if there is an education mandate, I think that will be the smartest thing we can do Because I don't think we'll be successful in passing legislation to have any kind of technical mandate out there I want to just address the russian question real briefly. I think the russians Their biggest move in this area is to just ignore what's happening right under their noses I don't think a lot of it is necessarily at least some of the purely economic crime is not necessarily tied to is not Maybe state sponsored, but certainly state ignored and protected to some extent So they certainly are out there in that respect I know if if there are other actors then they're purely more of the traditional state to state Or they're getting in and getting out without anybody seeing them. I think that's a that's a real possibility Education awareness, absolutely in this scenario as well Whether it's mandated however it comes it needs to happen People need to understand that they have an impact on other people's lives that if they leave their system Unprotected then that system is going to be co-opted and used to get around as a hot point to get around other systems And and we see that all the time. So any way we can make that happen any way we can Create a culture of security here in the u.s. I think that would definitely benefit our national security and our economic security Um, the russians are a puzzle We know more about the chinese because their tradecraft generally is so noisy And the sad thing about that is I always have to remind people it's not that they're that good Although there are some very really good intrusion sets that are trying to get into my corporate networks as we speak But they're most of them are quite noisy and it's because we are so bad They they've literally they don't have to leap over the bar. They just have to step over it And so they found exactly the amount of energy they need to expend to get into the networks and nothing more. It's very efficient The russians are much stealthier. They use a lot more crypto It's much harder to do adversary intelligence because they actually Don't regard their language as their first line of national defense. They actually use aggressive levels of encryption They coordinate in silk channels They do all kinds of things that the chinese don't do that make it so easy for us to enumerate them And to identify their building in shanghai and in everywhere else Although I will say that the chinese in my view have always been terrible strategic communicators But they reached a new low recently when their response to the mandiant report was and this is an official spokesman at the ministry of national defense said There is no unit 61398 You know, I mean talk about kafka esk Ignore the man behind the green curtain. I mean we have hundreds of pieces of open source data Identifying that unit. It's a public knowledge, but the chinese response is not We can either confirm or deny those allegations. But as a rule, we don't discuss intelligence operations from this podium They're they're literally the response at the official level is to deny reality just to deny the existence of reality Um, which to me is just a new low from actually on the bill Kind of it's on the paitza on the front of the gate Kind of like nsa, right? It used to be no such agency to I think you were a question about Denying the benefits of all of this stolen intellectual property On a tactical level, some companies are starting to do deception by planting fake but Juicy is looking information in sort of honeypot sites on their websites that they know or they think the The adversary will be interested in stealing but which would then end up being a completely, you know bogus Plan for for something and I think James also mentioned doing this on a more strategic level There are also I guess, uh, you know problems with Making sure that if if you do put some deceptive product information out there You don't have any liability for some, um, you know formula a baby formula that ends up killing thousands of babies but um Anyway, that just came to mind as one area that has not received as much discussion as I think it should or could um, and then we're generally speaking with respect to just educating people about better hygiene and security, um, you know, I I think it's Definitely something that The the administration talks about and we'll see whether or not there can they can turn it into a You know see something say something kind of campaign that actually makes sense Some companies are doing things like Turning this into a service where if they notice that your computer is infected part of a botnet They'll alert you and and and tell you you can go to a certain site for remediation But at least to let you know that there's there, you know, you are potentially part of a larger cybersecurity threat On the far end some countries actually are much more into Creating secure nets where you need strong user identification in order to gain access to Sites that might where you might want to do banking or voting or filling You know prescriptions or getting government services Some some government officials. I know think that's something like that is is a good way to go, but I think there would be a pretty rigorous public debate about about whether that's creating You know too much of a almost a national Identification system that we don't want to go down the road of so That's my thoughts Yeah, not much more to add except um, I think the education piece is critical Unfortunately, I don't think it's going to happen until we see real world physical Implications of an attack the digital equivalent of planes flying into buildings until we actually take this seriously that we see A lot of movement related to Legislation, etc And if you can imagine in august of 2001 if some government official stood up and said, you know, we're going to ask everybody because we're concerned about this Terrorist threat. I know it's hard to go back in time and remember before 9 11, but imagine august of 2001 We've got this terrorist threat. We're going to ask everybody to take their shoes off We're going to take your jackets off take your your shampoo and put it in a plastic bag. That's the one that always gets me And and people people would have been up in arms. This is outrageous. This is an invasion of our privacy We're absolutely not going to do it and then a month later We have this devastating terrorist attack and all of a sudden people are stripping down at the at the At the line at the airport because they recognize what the threat is they recognize the impact And they're absolutely willing to to take extra steps and measures to make sure they're more secure I think it's unfortunate that it may be something certainly not as significant as that although perhaps But but some physical impact where before people it really grabs their attention and they recognize we need to do something differently Well Let me do the following Which is the drill given the amount of expertise we have on this panel on the next panel Is we're going to go directly into the next panel and so we'll switch the name cards If you could keep your seats that would be very helpful But also join me now in thanking this group. They were incredible All right, let's get started everybody if we could I know it's tough and when you have a small opportunity to Take a break Well, as Jim Lewis said, we've we've uh, been very fortunate to have a number of uh excellent uh panelists and of course speakers kicking off the event uh today And um, and I think we'll find that This panel Will duplicate a little bit the previous panel. So they stole a little bit of our thunder, I guess group, right? But i'm sure we'll uh exceed expectations or at least we'll try to in this regard It seems like in uh In my previous life in the government where I worked in the department of defense for 31 years and and it seems like i'm always asked to To answer the more difficult question which is How do you stop something? Versus just describing the threat sort of like when I worked in NSA for For many many years, uh, and I was on the defensive side most of the time and Seemed like our job was a lot harder than the offensive guys. So, uh, so I think I find myself in the same situation But luckily we've got a great panel and this panel will try to take the momentum from the previous one uh to try to I think get to uh some interesting thoughts on uh on stopping This threat that you heard earlier talked about by the by the previous speakers and we as you heard in the previous uh Group, uh, it is certainly a daunting problem that we face uh these days And uh, and so I'm gonna just open it up for the for the panel to to each one of them give a few minutes Of their of their thoughts As as leaders in this field and we're very fortunate to have as I said this group. We've had a few people Uh, who had some unfortunate family emergencies at the last moment who had to drop out But we've got this group together. I think will be as equal if not superior Than our original, uh group And so with that, let me just kind of start and work our way down Left to right to keep it easy if that's okay with you Stuart and Stuart I think many of you may know but Stuart, uh Was at the department of homeland security. He was the first Uh, uh, a senior in charge of policy assistant secretary for policy for DHS and and as we all know when we created dhs after 9 11 That was a very very tough job and he's learned a lot of lessons from that. I'm sure And uh, so let's have steward give us some of his thoughts on stopping the threat, sir Yeah, the lesson is don't do it. Uh You know doing a startup in government, which I've now done a couple of is just deeply painful, but um a So, um, I would like to you can't hear you. Okay. Uh, I'd like to popularize and This with what I call baker's law, uh, which is our security sucks, but so does theirs We can you know the fact is The the real enemy of security is operation on necessity. There's things you have to do You've got to accomplish the mission you take a little bit of a shortcut And that's the end of your cyber security. Yeah I and that you know that that that operational necessity Works on the other side too. They've stolen stuff and they've got to get it Down to the their state owned oil company in time for them to get their bid in as well I and they're going to take Shortcuts and we're going to be able to figure out who's doing this and this is the critical point Uh, I sometimes liken this to uh, pig pen He's got this just ball of dust surrounding him. Uh, this is what we're like in cyberspace. There is just Bits of digital dna flying off us at all times as we take one shortcut or another and find ourselves losing control Of our identifying information. We're doing that happening all the time We all know that and so are the people who are attacking us The the important thing about that is this means that we can attribute these attacks We can actually identify the guys who are doing it. Sometimes I put up that photo of the anonymous attackers Who were busted because they put up a very low cut picture of one of their girlfriends I And to mock law enforcement and didn't realize that the picture had been taken with an iphone Which very helpfully provided the geographical coordinates Of the girlfriend She was they didn't show her head just the rest of her I I've often thought that You know the secret service and the fbi must have arm wrestled for who was going to do the id in that case um So we can we can begin to identify people Who are attacking us? That's the attribution stage. We really can do a much better job than we have In attribution and then we have to bring like i'm a scott's irish kind of guy. Yeah We need to bring the pain We need to show the folks who are attacking us that it's a painful thing to do and they'd be better off Choosing a different career And for that I think we are going to have to get much more creative But I testified last week to judiciary committee and suggested a number of things that we could be doing All you have to do is read the mandient report read the trend micro report Read some of the or the the report that citizen lab did There is there are lots of clues to the identities of the attackers We know where they went to school. In one case, uh, they they went to Sichuan University and the kid Who was engaged in those hacking attacks later went to work for 10 cent, which is the enormous Chinese internet company with a big subsidiary located in the united states Sichuan university needs visas to send their people to the united states. So does 10 cent Why aren't we saying hey, we got an investigation going We'd like you to cooperate if you don't cooperate. No visas you can go home and and and train There's no reason why we shouldn't be doing that today. Yeah Especially designated nationals we have systems for saying These are people who are engaged in trade in conflict diamonds And we want to take the people who are engaged in that Designate them as folks who the us government says no one Uh can do business with those Conflict diamond nationals They do the same thing for bellow russian oligarchs The magnitsky act does this for people who are interfering with human rights in china Well, for god's sake, you know, we have people who are interfering with our human rights right here in the united states We ought to start designating those nationals And causing some pain for people who are engaged in these attacks. We know enough to designate them Let's start doing it. Uh, and then finally and and i'll close with this We need to uh to take the information that we're getting and follow it through not just to the attackers Not just to six one three nine eight, but to the guys they're feeding with our stuff and um We need to find ways to tag that quote information as it goes back to china and then on to a state-owned oil company so that we can say we know where that Uh information went we've tagged it and followed it all the way And now we are going to take every nickel you have in the western world for engaging in economic espionage with Criminal prosecution civil lawsuits and the like we can do all of that if we set our minds to it We're going to have to change some laws, but not very seriously. We just have to take it seriously. Thanks Oh, we got pressed a button. I wondered how that worked. Okay. Thank you Yeah, uh, just just uh, I thought I was going to be the oldest person on the panel till stew walked up So now I feel like I've been vindicated But just just to give you some context when you know and chris english was talking He was talking about the code making and code breaking roles at nsa and I was a code maker For for the early part of my career probably I had been there about 20 years when chris actually arrived on the scene And he was working in something called the computer security center Which was relatively new at that time in the in the 80s because we were we were starting to you know think about Yeah, we've been building encryption. That's what I did. I built encryption boxes. I was not a crypto mathematician per se I never really understood the difference between a fibonacci sequence and a fibonacci coca sequence It was always there was somewhat mysterious, but I could make the boxes work. So that's what that's what I did But we started thinking about okay beyond encryption as you know in the 80s The internet was just starting the early 80s. The internet was just starting to take shape So we started to think about this question called computer security at the time And you know there was the orange book and the red book and the yellow book and I think chris was involved in that He was trying to think of the colors. So I said what next what color is where we're going to publish next? I think that was his job. But anyway, uh, we went on from there And we you know we evolved into information assurance and I think it's still officially called information assurance But it's really focused on cyber security now And I'll say the lot's been covered already. I don't want to repeat what the previous panel has said But you know a couple a couple of observations out of out of the discussion Uh education, uh, particularly, uh, you know, it's a good thing But it's never going to solve the problem. I mean if you're expecting consumers to change their behavior That's really a fool's mission. They're going to behave the way consumers behave and if they get an email So some people will always respond that an email saying you've won the ugandan lottery Just send us your bank information and we'll send the money to you. Okay, we're never going to stop that behavior Uh, you know educating them and you know, I think we're going to have to do more In the consumer space about automating the security processes there You know, we shouldn't have to have the consumer check the box and say I want the automatic updates I mean that's dumb. I mean that ought to be a part. You want to use the system You want to use the the operating system? You want to use the applications? You got to have the updating process in effect so we can go so that the Providers the technology providers can fix the problem for you You know, we we participate in something called the ncsa the national cybersecurity alliance, which is dhs You know industry sponsored thing if you go to the ncsa website It's got 10 10 guidelines for how do you how do you secure yourself? Okay, and this says, you know change your password regularly The one I like is configure your your computer In a secure fashion Okay, so how is the consumer going to do that? I have trouble. I've been in this business a long time and I'm not a computer geek I was studying vacuum tubes when I was back in college So I'm not a computer geek computer science hadn't even invented yet And uh, you know, how do you expect the consumer to quote configure their machine securely? I mean, that's just not going to happen So we have to we have to start to deal with that from uh, if you want to use the technology You gotta you're going to have to accept the fact that your your security has to be managed by the technology providers It's never going to be configure it yourself. You figure that out Okay, the step to the larger question. How do we protect critical infrastructure? Let me say a couple of things. Uh, number one, I think defense in depth has been very successful But it was it's it's the the security model has always been evolving when the internet started There was no security. Everybody trusted each other was a set of uh, You know host machines at academic institutions that were networked together when everybody trusted each other And if somebody got out of line, they were quickly, you know, put back in line by the rest of the of the peers That obviously has changed. I think the firewall was first invented probably late 80s early 90s They still have the bill of incest mcbook on my my desk called falling foiling the wily hacker And it was the the internet firewall was invented And it basically said, you know, we've got to close some of the ports and protocols that you're not using And you'll keep those guys out. Well, that worked for a while But then the hacker said, well, you know, I now I'm going to figure out how to tunnel through that firewall and Tunnel through the protocols you have opened So it's always been and you know defense in depth came along as a evolution of that thinking Clearly we're now at the stage where we've got to move away from, you know, the early discussions about the static defense And go to a much more dynamic adaptive environment And that's going to take some some things that we're going to have to do collaboratively Uh information sharing if I can use that word is is an essential part of that But I I think of it in really in terms of a higher level I mean sharing signatures sharing threat warnings That's all great and it needs to be, you know, we need to do more of that and make it more Operationally focused that's I think somebody already pointed out, you know Sharing for the sake purpose of sharing is a waste of time sharing so I can block a new threat that that's much more useful to me But if I start to think about how do I get ahead of the power curve? I've probably got to do my analytics if I'm looking at my internet portal and I'm analyzing the traffic flows That's good. Okay. But what I really want to be able to do is analyze it at the next larger scale up to You know, in fact ultimately take it to the global network level because that's the way I can actually understand What's really happening in the network and as new threats emerge I can be on top of those and that's going to take some doing to move from where we are today And localized analytics to global base. We do it at AT&T We do it on our global network infrastructure. So we have a pretty good view, but it's only our view I don't share the view with Verizon at the native level We're now doing more collaboration with Verizon and the other tier one carriers As we're dealing with DDoS attacks against the financial institutions, which I think Ellen mentioned earlier We're actually changing our business model in effect to help deal with that But the driver ultimately is be how do I really understand what's happening out in that global infrastructure And be able to deal with threats as they're emerging those zero days start out, you know Somewhere and we want to be able to find them where they're starting as opposed to gee I just got compromised now. I got a clean up the mess So that's kind of I'll stop here because we are kind of you know constrained on time But you know, we got to change our thinking in our approach to security And and a point solution here a point solution there We've got to move to a much more global view of what's happening and that's going to take Global collaboration international collaboration as a part of that We're going to we're starting to try to do that at the internet service provider level And and opening up dialogues with with international partners We peer with you know, a large number of the global carriers And so why aren't we doing more in terms of just developing a common Understanding of what's happening in the network and I can go down to the packet and you know protocol level And and be able to identify those zero days before they become successful That's what you're really trying to do is get ahead of the threat. We also have to work towards Driving the technology base to be naturally more resilient and more secure. We're just starting to understand how to do that But in the commercial world, that's a big challenge simply because they're you know, the technology is always moving And you know, Microsoft's been been trying to deal with security at the native level in their development process But you know, they brought out windows 8. Okay, and windows 8 brings a whole new flavor to things in the operating system realm And so we're going to learn things as we go I see we learn something every day about cyber security because we usually learn several things Just because we're we're there and we're doing it So you never stop and say I I understand the problem In fact, I'll close with my favorite saying in the cyber security business If you think you understand your problem, you're badly diluting yourself. Thank you Thanks, John Okay. Now next is John Gilligan. John Was during the dod days Probably one of the most innovative cio's that we had he was a cio for the air force And now he runs his own company and uh, john, could you give us your thoughts? Thanks, Bob. And thanks to csis and fire for putting on this session You know, i'm thinking back and it's been Almost 40 years since I first got involved in cyber security computer security back then I went to a seminar In a graduate school and ended up getting a graduate at the end of it They said we have graduate assistantships available and that was caught my attention and so I raised my hand and Spent the next couple years designing trying to design secure systems trying to mathematically prove systems I spent most of my subsequent career not doing computer security But designing and building it systems And now more recently, you know and helping manage companies and The the the topic of stopping the threat to me has to be looked at in terms of a business perspective And I come to these sessions and candidly my head hurts And it reminds me of a Of a story that I tell often when I was cio of the air force We spent about seven billion dollars on it a lot of money on Computer security. I had a pretty good background in computer security And what I would find is each year as nsa came in to do their penetration analysis of the services Then they would call us all together said they'd line us up like a panel here army navy air force and they would debrief us on what they found and The first time they did that I was terribly embarrassed because it wasn't Did nsa succeed in breaking in it was how long it took and that how long was in minutes and seconds And and the types of attacks every one of them was successful And I'm thinking my goodness if somebody from the media had been sitting in this audience. I would be Pillared in the in the media for for spending seven billion dollars and not being able to protect even And and so the second time this happened. I was very frustrated. So second year same briefing Very frustrating and I went to nsa and I said I need to know where to start It's not like we're not spending money. I need to know where to start and that ensued a discussion That I'll shorten but at at the end of that discussion nsa came back and said well, we've now analyzed the threat And based on the threat here's where you ought to start That was enormously enlightening And so I want to fast forward that same discussion today So Verizon just produced their latest Verizon data breach investigative report very enlightening. There are a number of other reports that are similar out there But to me what catches my attention is that really things haven't changed dramatically from my air force days The majority of threats In terms of number are unsophisticated And they're attacking very straightforward weaknesses That's really important to second and and some of these statistics were mentioned earlier today in the presentations The the breaches are not discovered until weeks and months after they occur unsophisticated and discovered weeks and months later And most often discovered by people external to the organization And yet we're spending 30 billion was the number that they've dwelt used others have used a lot more We're spending all of this money. What the heck is going on? Well, um, I've spent some time trying to analyze that You know being on the board of several companies. This becomes quite important as gosh if we're going to spend all this money We would hope we would get some return And and what I've discovered through that analysis and and looking at the reports is In fact the most prominent threats are unsophisticated attacks And it turns out they're relatively easy defeated We have now demonstrated through, um, you know, the research and applying Different names but a set of a minimal baseline set of controls That you can be effective in protecting against most of those attacks One set was developed in the united states that's called the critical security controls sans institute nsa a number of other organizations did it The australians have done have come up with their similar top 35 Interestingly enough their research shows that only four of those four four controls Are effective against 85 percent of the threat So the conclusion is what we know how to deal with we know what we need to do We just don't do a very good job of then implementing these threats now I'll tell you a little secret As cio what I learned is It doesn't cost a lot of money to implement these baseline controls. Why? Because most of them are essential to operating and managing the network It's just doing them in a disciplined manner And in fact most organizations are already spending the money in fact many are spending more than they need to because It's not that they don't have the controls is to have multiple sets of overlapping controls Inconsistently applied and so they leave gaps, etc So step one is implement this Baseline of critical security controls now that does not address the sophisticated Threats and I acknowledge that But if you don't do that you're wasting your money Trying to address sophisticated threats. You're kidding yourself So all of the discussions about the sort of the why I recall my kids used to play soccer You know everybody'd huddle around the ball And that's what I see often organizations saying we're going to go after these sophisticated threats. They're shiny We're either exciting But unless you have done that foundation work, you're wasting your energy Now what I have found is organizations that beyond the critical controls what they're doing and what's most economical is not To continue to layer control upon control upon control And I think that's the big flaw in what nist has been Providing in their risk management framework. It's well done But it really just continues to drive cost upon cost upon cost what we're seeing and we heard this well today is that the Very sophisticated advanced persistent threats in the nation state attacks They're agile. They're intelligent. They're dynamic and In order to respond to that you have to be likewise And I think there's been great discussion about that so I won't repeat But you have to implement that time that same type of capability. It cannot be done strictly with tools It has there has to be a human element The sharing of actionable intelligence is critically important the ability to look at patterns of attacks And eventually those that are most sophisticated are actually able to predict What's going to happen? What's the next step of the attack? Why because they're they see the patterns They're they're studying it and so I think all of those Then are the next step and sharing becomes absolutely critical because organizations in general can't afford to do that on their own so Anyway, let me stop there, but I think I think we sort of know some steps It's not to say that other comments. I mean obviously diplomatic and other other avenues. We ought to pursue as well But but I think from a technical perspective There is a better roadmap than perhaps we've been able to implement so I'll stop there And I think john points out An important topic that he's working on with apsia which is focusing on The theme of cyber economics, and I think that's one of the Key elements of cyber where we are today Which is I think if we can Get our resources Proportional to the threat so that we can Take care of the ankle biting problems with the least amount of resources but do it smartly But then focus The other part of our resources on the stuff that will kill you That's what we got to focus on And unfortunately we're spending I think we're finding a disproportionate of our resources on the ankle biting problems Because we're not doing the basic stuff And as a result, we don't have enough resources to focus on that 15 percent That will be the stuff that will give you the the heavy injuries or the fatalities Like like in a case of an automobile analogy So I think that's the situation we're in today Relative to moving from static to dynamic defenses that we talked about This morning so so how do we get at this situation in cyber economics of moving In this new paradigm of security that where we are facing ourselves Right now, and so we've asked her of let's go To talk about an area that he's focused a lot of his attention on He's a csis senior fellow which is called active defense and we've talked a little bit about that So I'd like to you know with that setup or if I'd like you to talk a little bit building upon what john mentioned Okay, thanks bob Right, so I'm going to talk about this thing called active cyber defense and of course the minute I say that We're I guarantee that we're all in this room on a completely different page in terms of what that means because there is no a widely accepted definition So the defense strategy for operations in cyberspace defines it as basically real-time technical protection of the dot mill network But in in the popular parlance and the articles that are showing up in the media. It's often interpreted as meaning hacking back and so There's a lack of understanding of what the term means and often what happens in discussions about active cyber defense is People end up in one of two extreme areas Either looking at this sort of hacking back area, which gets legally very dicey very quickly as any lawyer will tell you Or just saying well, we're just going to look at activities that we can do within our network, which are perfectly legal And have been going on in some cases for decades. So honey nets and gathering threat intelligence in a variety of ways And that's that's perfectly safe legally But there's an interesting gray area that's that's developing and people are starting to pay a lot more attention to this For a couple of reasons The first is that The government simply cannot respond to the magnitude of the threat facing the private sector If you read all these reports, you just see it's a huge problem And a lot of companies and organizations are on their own Now hopefully with some of the some of the initiatives that the government has announced with with information sharing Things might help but unless it's a major breach The fbi just doesn't have the resources to to come and help you in a lot of cases The other thing is the private sector is growing incredibly sophisticated in many ways In terms of its ability to Analyze the threat and potentially even respond to the threat So there's an increase in motivation and capability on the private sector side And so then there's this interesting question that's starting to come up Which is how far can the private sector go to protect its intellectual property and its assets that may be leaving the organization So a lot of folks are starting to look at this and again in particular. There's this interesting sort of gray zone Um where one can start to look at things like Beacons in information that leave your network So can you you can you put a passive watermark on your document? For example and have it leave your network and then you can search for it to see if anyone's stolen Stolen the information. That's one thing with what if you put an active beacon? So it's actually you know signalling home from wherever to so you can track it. Is is that legal? Well, it starts to get more tricky What about information that might leave your network and Self-destruct or something like that that gets really tricky And then there's all kinds of questions one can get into in terms of if someone is accessing Your network and they're connected to your network Do you have any rights at all to Leverage that connection to gather intelligence that you might be able to use Again, you know it gets into really tricky legal questions. So Um There's there's not a lot of clarity right now and in fact what's interesting is There's a lot of debate on the computer fraud and abuse act right now on the hill because some people believe the act is Either too strong or being applied too strongly So the erin schwarz case is one example of that but there's some others where people feel that the It's it's a bit too strong and and that the language is a bit too loose And there's other people who feel that it actually needs to be strengthened So that you can deter this activity more effectively So there's debates there Harvey Rishikoff is here and he's leading a task force At the american bar association that's looking at this issue. There's a lot of interest in this in this issue and one of the things That comes up is this question of First of all roles and responsibilities. So What can the private sector do on its own? What can the government do? How can they work together? And so there's a number of questions that come up there and I hope this tees up Jenny a little bit with things like the ecs program that dhs has developed where there is a partnership between the government and the private sector To share information and provide some protections And one could think about whether that kind of activity should should continue That's also a question of should there be clear lines in the sand in other words Should the laws be very clear about what companies can do to protect themselves? Or is it better to have some legal ambiguity And let either case law sort of work itself out through the system or provide some ambiguity for the attackers So they're not exactly sure what the lines are and they're not sure what steps companies can take to protect themselves So there's debate there So, uh, I'll just uh, stop right there and then uh, happy to discuss it further if there's any questions Thanks, sir Director of us a long title stakeholder engagement and cyber infrastructure resilience Leader, I guess it's here is your title But as herb said, uh, she's got the hard job of we talked a little bit about information sharing and collaboration and jenny has the job of Promoting that at dhs. So if we could ready could you give us some updates and perspectives? Sure. Thanks, bob And um, is this catching my voice on the microphone? Yes Yes, okay. Good. Um, so one of the the things that government can do recognizing that there's a huge scale of critical infrastructure partners that we need to work with And a relatively limited size of government resources. Um, one thing we can do is share information That we have we do have some unique sources of information whether it's from our partners in the intelligence community Whether it's what dhs sees from across the the dot gov What it's our friends at do dc protect at do dc on their networks law enforcement, etc So we have a broad set of information that we can share and and shawn henry is right right that term does get over Overused a lot. It's a big blanket Term so when we work with critical infrastructure sharing information, we need to recognize who we're sharing with what so that they can take action Sometimes that's actionable indicators. It's md5 hash values. Sometimes it's sitting down with ceos or cio So we found to be a very productive group to work with to make sure that they understand the threat What really is the threat landscape? What are those most important things where they want to allocate their scarce resources? Um, what decisions are they making in configuring their networks that may introduce significant risk? Um, and what about some of the new technologies out there? You know, I've had a number of cios come to us and say, you know, should we be implementing application whitelisting? Is it worth the effort questions like that where there are a lot of vendors out there that are proposing different solutions? And they are looking for kind of some objective lessons learned that they can find out from government So how do we work with people that are making strategic investment decisions ceos cios, etc And then how do we work with folks within the critical infrastructure companies that are doing that real hands-on protection? Those are those actionable indicators And actually right now today we're having a quarterly meeting of our advanced threat technical exchange, which is part of our When we have awful names for things I think secret service always has great really cool names for their programs And we always have awful acronyms that don't even spell anything Our cyber security information sharing and collaboration program I know it's awful an awful acronym. I'll take suggestions But that's where we share a sensitive but unclassified information with critical infrastructure companies So that they can protect their own and their customers networks and they provide information back to us about what they're seeing on their networks We do that through a legal agreement called a cooperative research and development agreement that really Lays out how they can use our information and how we can use their information so that everybody is on the same page And it's a program that grew out of lessons learned from the dib csa program A pilot that we did jointly with dod in the financial services sector And now is something where we've learned a lot of lessons and is available to all of the critical infrastructure sectors So far we have 14 sectors not in totality But members of 14 different sectors that participate Both through information sharing and analysis centers or individual companies where companies choose to do that or where there is no Information sharing and analysis center And so what we do through that program is we share those machine readable indicators And we are working toward increased machine readability You know started at very basic csv But we're now working in in a format referred to as sticks and taxi Which some of our partners are actually piloting true machine to machine communications with no human in the loop What we're putting out in those formats is still pulled down off of a secure website But we share information with them they share information with us on a regular basis We've shared almost 20,000 indicators through the program When we started about 18 months ago about 80 of the indicators were coming from government and 20 percent We're coming from the industry partners now it's 60 40 Which I think it really shows that the industry partners are starting to see value and putting more in And we're really seeing unique things about what our threat actors are doing In different sectors that we would not see in government So there may be one threat actor that deals very differently when they're working against a A manufacturing company than they would if they're trying to get into the department of defense So very different TTPs from those groups So we have the the flow of the actionable indicators We have mitigation strategies that go out But then we've also found great value in these analyst to analyst exchanges like we're having today where people come in And talk to their peers and say this is what happened to us and this is how I dealt with it And then people can ask questions and there have been many many examples of where somebody has heard a company from another sector brief And they've been ready for what happens to them in the future Or they've been able to go back and immediately apply something that's already happening to them It's interesting because some people come into the room and will go and throw out their name company and sector And then other people will just say, you know, my name is bob and that's all that they're willing to share We do allow that anonymity for those who want it And we have government partners and industry partners that participate there. So I think that's an important part Of the information exchanges. It's not just all about the ones and zeros going back and forth But it's getting the smart people from industry and government having those technical discussions about what's working and what isn't With the actors So that's our sys b program As are mentioned our enhanced cyber security service is a new effort that dhs has launched It's something that started with the defense industrial base dib opt-in activity And back in the spring dhs took over the relationship with the commercial service providers Who provide those dib opt-in services now with the executive order that came out dhs is able to work with those providers so that the services can be provided to all 16 critical infrastructure sectors So for those of you who are not intimately familiar with what I mean when I say dib opt-in or enhance cyber security services Basically, you know, we talk about tear-lining information and what we share through sys b is unclassified We use a traffic light protocol that governs whether it's proprietary data or whether it's information that came from the government Not everything can be tear-lined. So when we do enhanced cyber security services This is where we provide those classified indicators up to the most sensitive levels of classification To information and communication technology providers So that they can protect their customers networks if their customers choose to buy those services Right now it is Email filtering and dns sync holding those are the two counter measures that are available In addition to increasing the sectors who can buy the services and the pilot that do d did was only isps participating We've expanded the kinds of ict providers who can participate manage security services providers AV companies companies like that have expressed an interest in coming in through the program So it is very new in offering these services to other sectors There are Anytime you do something that's new and different You don't realize all of the little details that you need to sort out until you do it So how do you validate who can buy the services? How do you figure out a process for adding new counter measures? What are all those details that have to be sorted out? How do you get a memorandum of agreement signed and then get the systems accredited at the sci level? But this gives us an opportunity to take that information that can't be quickly Tearline down to an unclassified level get it to those ict providers that a large percentage of our critical infrastructure Community uses and if those critical infrastructure companies are interested allowing them to buy the services Where they receive that protection with the classified information? So it's a very quick overview. Happy to answer more questions and go in different directions Great. Thanks. Good Okay, so we've got about 10 minutes before we have to move on to the the next session and so what i'd like to do is Is either open this up for some questions from the from the group here since we have limited time I've got one right here from this gentleman Yes, uh, Mike. Yeah My name is alex lawson. I'm from inside us china trade Miss menna. Uh, mr. Baker raised the specter of some different sort of enforcement Tactics that the government can take he mentioned the visa issue. He hinted at some financial sanctions This kind of sent seems to center around business strategies for preventing attacks, which is valuable But I was sort of hoping to maybe get some information from you on Tax that the government thinks are worthwhile for enforcement the visa thing the sanctions or are there some other Options that the government can consider there's been a lot of sort of public naming and shaming going on more They were meant, you know china was mentioned in the dod report But I don't know if there's anything with a little more teeth that the government can consider as a next step I can only speak on behalf of dhs and so dhs's role in cyber In our office of cyber security communications is very much focused on Prevent protect building resilience and then responding When there is an incident so I can't speak on behalf of my interagency partners But obviously this is an issue that is discussed as a whole of government discussion Each with our respective roles I think I only understand Another question from the uh from the group Yes, sir Stand up and give us your name, please Here's your mic here so we can hear. Thank you My name is alexander soley. I'm a delta risk and I was considering the idea something That david da wa mentioned earlier about how most of our cyber security is based off of Sorry, most of the antivirus is based off of blacklisting and such and I was wondering if anyone has been considering more sort of alternative to that or any sort of Legality issues involved with creating some sort of autonomous Way of finding critical vulnerabilities and such Is what baby They want to try that or I can I you know, I give you a lawyer's view of Cyber security, which is not necessarily something you should take to the bank or at least I wouldn't code it directly, but Yeah, I think one of our biggest strategic problems I'll tell you a story from the border When I went down to the border when I just started at dhs and I was dealing with the border patrol Routinely they'd say well, we sent out two border agents and they brought back 30 people who were trying to cross the border And I finally said how do how do two agents bring back? 30 people and he said oh well we surround them. I But the real answer was They surrendered because the worst thing that would happen is they would be taken back across the border and let go to try again Um, this is where we are with keeping people out of our networks I After we've spent a boatload of money Stopping up all the rat holes. They spearfishes again and 90 percent of the time somebody's going to open the the email and The reason is it's getting past all of our signature based Solutions and we do need automated mechanisms for dealing with that It seems you know Fire I know has sponsored this but the fact is they've got an interesting Approach to this which is to say let's put this in a virtual machine and just watch what it does And if it doesn't do what pds usually do then we're not delivering it Uh, and you don't actually have to know In advance that this is bad. You don't even have to know what bad things It's trying to do if it's doing something that adobe didn't tell you what was supposed to do You just don't deliver it. It seems to me that that has some real potential to make it much harder for people to get back in And to go back to the border, you know now when they stop people crossing the border There's a distinct chance that there will be a struggle Even gunplay and that is a reflection oddly of how much better border security is because They don't expect to be able to get in if they can't you know break past this if they're coming from Latin america, they're gonna have a long flight. They're not just gonna be let go across the border And I think we will know we're doing a better job when it is harder to get people out of the network than it is today Thanks, jim Any other questions from we got a little bit more time. Yeah Hi, i'm uh dj bailo from robert moore's university talk a little bit louder, please Or get closer the mic. Hi, i'm uh dj bailo and i'm from robert moore's university I know uh hacking back is a big gray area, but how do how is that area How's the us area and viewpoint on that like compared to other countries and how does that affect our defense modeling? Well, let me let me take a quick shot at that that's that's an interesting and complex question First of all, uh, you know, I like to say, you know for every cyber warrior There's a cyber lawyer looking over their shoulder and the at&t is is no different than that And the government, you know my government experience was the same way The problem today is for example the computer fraud and abuse act basically says what you cannot do in Cyber secure in uh, you know in attacking somebody else's system. It says if you do this It's illegal. You can get arrested for doing that. It doesn't define What you can do going back in the other direction. It doesn't define any particular Framework for what you can do in response to a penetration So there's a lot of discussion going on right now including on the hill as we're looking at the cyber legislation Uh, the syspa act, uh, of course has been passed by the house That's gonna, you know, that's the same topic as being taken up in the senate and one of the discussions that's taking place is What counter measures are or defensive measures or active defense measures or whatever you want to call it Are allowable under the law I think irv might have mentioned this that okay if I set up a honeypot and I put dummy data in there That's relatively, you know, accepted today from a legal perspective. There are some liability issues, okay But it's relatively benign. Okay, suppose I set up a honeypot and in that in that dummy file I place malware that's going to destroy the file structure of the machine that it gets downloaded to okay That's probably not legal under today's uh laws And probably never should be uh, and one of the biggest issues there which we wrestle with every day And it's you know part of this whitelist blacklist question is Most of the hosts that are compromised and these are websites and hosting cloud hosting services that are compromised Are compromised not by the owners of the site but by somebody who's a bot master going or controller or coder Whatever you want to call it going around looking for machines to compromise that I can then use to distribute malware So they may be innocent typically the the first thing you see where you think this attack is coming from is an innocent bystander So from a legal perspective, what do you want to do to an innocent bystander? We're having discussions right now with dhs and with fbi about okay, we we see these sites We see a bad site. We can tell right away. It's it's downloading malware to our customers And number one. Can I share that legally with you? Um, you know the identifier information, which is typically ip addresses and urls and and max and things like that that We can see out of the network traffic, but then what can you do with that? Can you go in and knock on the door and of the of the hosting service and say hey guys You got a problem and let's work together to get that fixed That's still a very gray area in in the law right now and people of course and particularly with syspa As soon as the idea of sharing and liability coming along with that Gets the privacy people very upset because their interpretation of the of syspa is You can break the law and get away with it because you got immunity in the process You can do anything you want whatever whatever you violate anybody's privacy And you've been immune immunized by this legislation. You can get a what you get out of jail free So that's kind of where we are in wrestling with that whole topic is and we're hoping that out of this Senate and house legislative process We're going to get some clearer guidance from a legal perspective about what can be shared What purposes for which that can be shared and then you know How do you protect privacy in within that context is sharing an ip address a violation of somebody's privacy? Not clear today what if that is or not so so I I've blogged about this and uh one of the things that I pointed out in a recent blog post is that Uh luxembourg has turned out to have more cajones than the entire united states cyber security establishment They got a guy there who just said he read the mandate report He said well, why do you kind of go from the command and control server back to the unit? Why don't I just go looking for all the poison ivy all the guys running poison ivy and break into their network And he did that uh found all kinds of of interesting stuff The response from the u.s. Government has been to say well, you know, that's could be could be illegal You know, that's a bad idea. Uh, well what they're really saying is we don't know how to protect you But we do know how to prevent you from protecting yourself. Uh, it's absolutely nuts. This is an old Computer crime section view of the world kind of leave it to the professionals You know when you say leave it to the professionals when you're dealing with crime You got to actually be able to do something if you're a professional our professionals are completely unable to protect us In the end people will find ways to protect themselves Currently the the law is written so vaguely that almost everything is illegal many things that we do today As a matter of routine or arguably illegal under the computer crime and abuse act And um until we find a way to embarrass The justice department into starting to say yeah, we didn't mean that you couldn't protect yourself in some way Uh, we are going to have a uniquely Disabled cyber security infrastructure in this country Oh, so just one minute uh on this so so there's a really really interesting legal uh issue Which which has been discussed here, which is absolutely fascinating. There's also a policy question So, you know, let's assume the law gets clarified and there's clarity on what companies can do and and uh not do But then there's a There's going to be an equities Uh discussion here. So for example, let's say that that companies are given more leeway to take certain action That might undermine, uh, us diplomatic efforts to establish certain norms In cyberspace, um, or it might not but or it might be worth Having that or it might not so so there's there's going to be a really interesting policy discussion in terms of What the u.s. Is trying to accomplish internationally in terms of norms of behavior even discussions on the internet governance fora Uh, which you know, jim can tell you all about what's going on there But it might have an impact on that as well So, uh, it's a very interesting issue from policy perspective as well as a legal one So we're at a time, um, I get the hook from jim back here, but I would like to I would like to, um Summarize, you know one thing from my perspective having been in this business a long time Which is that, um We've heard some great discussion today some very difficult legal issues and policy issues that we have to To address but from the standpoint of where we are today versus where we were just looking back five to 10 years ago Um, I think we've accomplished a lot of things now have we accomplished it at the pace we should absolutely not Is the threat having you know a field day? Absolutely But I can remember and and uh back having to go to the hill Uh with shawn and a lot of the other uh folks that were part of the cyber group The hill didn't even understand any of this stuff five to 10 years ago all this discussion we're having right now They didn't have the foggiest notion About this topic and we'd have to bring props into the hill To get them to understand just the basics of what this topic of cyber security was all about At the same time 10 years ago, nsa Thought they could build pretty much anything That was needed in the most critical areas In this space it was all about you know government built or government enhanced Solutions nsa doesn't believe that anymore. They realize the private sector in large measure Has the capability to build the right solutions in this space You look at what ashar was talking about earlier with fire. I fire. I was built By an idea that was germinated By darpa. We need a problem solved in zero days Fire. I was born uh incutel invested in it the cias You know arm for investment and innovation And it was born and that's happening all over the place with silicon valley and things like that So the good news is you know, we have lots of solutions coming into play lots of solutions even in the active defense area You know, so there's a lot of room for optimism But there's no doubt That I think we've heard you know today that there's a tremendous amount of frustration We've got to speed up get the get the car shifted from first gear And get it shifted up to third or fourth gear and what's the best way to go about doing that? We did not get into the issue of you know, how much regulation how much strong policies that we have to put in place to deal with this But the reality is I think every one of us should be a little bit more optimistic But clearly we still have a long way to go in this space. So hopefully The the discussions this morning have been enlightening. It brought up some interesting issues. I know today And I know some of us are going to be around For the next session for further dialogue with each one of you jim. Did you have some closing comments? Okay, and thank you csis for helping to host this