 Well, good afternoon. I hope everyone had a good Thanksgiving. I certainly did and We are lucky today to have really a great set of speakers Thanks to fire I for sponsoring the event. So when you bite on your sandwich, be sure and think of Dave This is really a good panel. It's a good topic We are excited to have this discussion. I'm excited anyhow Let me tell you the format. We're going to have two keynote speakers Dave DeWalt chairman of the board and CEO of fire I long experience in the industry You all know his history Successful IPO unlike Facebook it went up instead of down. So you've got to give him that's a good sign He will open followed by Jane Lute who of course all of you also know President and CEO of the council on cyber security But of course the former deputy secretary at Homeland Security and one of the real drivers in the administration of cyber security policy After our two keynotes who will each speak for about, you know, 20 minutes We will have a panel with a number of folks who have real expertise I'll introduce them when we get up My job is mainly ornamental. I'm going to sit here and smile and introduce people with that Why don't I ask if they don't mind if Dave and Jane could come on up here to the head table? And then we can go ahead and get started Is it good morning or afternoon? I've lost track of time lately after this IPO with fire eyes So I think I was I was in Europe twice the Middle East for eight days back and forth the East Coast 15 cities on the IPO roadshow. I'm so a little You know wondering which which time it is but anyway, thank you Thank you for for coming today and it's wonderful to be back with CS is I was here a couple months ago But different facility and a big improvement. So kind of fun anyways, I want to try to do two things this morning if I can or afternoon and I want to I want to give a little education as best as I possibly can and as Layman way, but also in a technical way just to help you understand the landscape that I that I'm seeing and then secondly maybe a couple of ideas and recommendations of Of some improvements that we could make if we're looking at our critical infrastructure policies and our practices and maybe even our standards and I'll do that at the end here, but I Have some unique views as Jim mentioned Being the chairman and CEO of fire. I I've got some unique opportunities Spent a lot of time as chairman of mandient as well I'm also on the the board of directors of Delta Airlines, and I do safety and security for the airline CEO of McAfee for five years and a variety of other roles so travel around and see lots of security officers and various types of companies and What an amazing world we live in the the advanced threats that are happening particularly the critical infrastructure are almost stunning and I hope you don't get any indigestion after the next 17 minutes because I probably will give it to you It's pretty appalling to watch what's what's happening out there, and I'll give you give you a visual or two But ultimately I'm hoping that we can focus in on how we can strengthen the NIST the framework and Some ideas to do so so real quickly and we've we live in an amazing world as I always like to say and The innovation that we are seeing is Incredible everything we have in our world is is being reimagined just about every day and that's kind of fun I don't know how many cell phones you have Jane, but I have a few now and New applications all the time, but what's happened is all this innovation has created almost a perfect platform for the adversaries And that's that's just the way it is The social networks that we see the mobile computing environments that we see the internet itself a lot of a lot of new computing has just created this very easily Accessible and attacked Infrastructure and of course what is every company having to do try to keep up culturally with adapting all this innovation And if you're a CIO and almost any Fortune 500 or global 2000 company all your employees are asking you can I have the new iPhone? Can I have a new social network? Can I have this or that and of course the pressure to adopt all this technology is creating Literally a perfect platform for the adversaries to leverage and you know, I'm probably speaking to the choir, but when you look at The companies that are breached it's scary and the level of depth of the breach is pretty unbelievable Thousands of critical infrastructure companies as I stand here today are breached. They're substantially breached and I don't mean they're they're losing PCI data or some sort of privacy information. I'm talking they're losing their core intellectual property Their networks are compromised their owned by the adversaries many of these adversaries are nation-state adversaries We my companies have walked into thousands of companies now literally thousands and almost every company 95 plus percent of them have been substantially breached So Our current systems aren't working too well our current controls aren't working very well Our best practices aren't working well and what you're seeing is the offense and the defense have got highly dislocated and The offense and the adversaries are easily able to defeat the defense of architectures that we have out there And you can see here just from you know, what I'm showing Obviously a lot of espionage activities intellectual property loss Hardly measurable, but obviously in hundreds of billions of dollars Very attacks very advanced attacks from China serious cyber sabotage kinds of scenarios unfolding Crime activities unfolding and not to paint too dire for a picture But when you compound the fact that now the adversaries are sharing and you've created a capitalistic community Where you can purchase advanced weaponry and advanced malware kits on the black market You end up having a situation where these advanced attacks in the hands of the wrong individuals Create a potentially catastrophic scenario So it is what it is and again not to not to harp on it too much But you know when you look at the fence in depth not keeping up We've really created almost the perfect platform for our defense to be defeated And after five years at Mac, if we have come to realize this a little bit we preached for years defense in depth defense in depth and We've created that just about every layer of the architecture from firewalls to intrusion detection prevention systems To web gateways email gateways to host security. We put in lots of defense architecture But what's the Achilles heel that's in that entire defense and depth architecture with the exact same Detection engine at every layer of the architecture. I Call it the Maginot line everybody knows the French history lesson there a little bit and We have a situation where the adversaries basically can get around our defense and depth architecture because the exact same Signature-based black listing model for detection is in place at every level of defense and if you can evade That architecture you've defeated the entire model. So what's happened offense and defense highly dislocated? Every couple minutes you're seeing a successful attack 95% of the companies are compromised and it's very simple to Breach the systems an oldie, but goodie right now is spearfishing and all I have to do is send you a little linked in Reminder of our meeting today along with a web link. You click on that link. I can affect your system I can steal your credentials and I can log in as you So we end up having a perfect platform of innovation that now has created a pretty Untenable scenario for our adversaries and for our situation So what's new the threats can't landscape is dramatically changed You all know this the persistency of the actors is different the amount of funding that's going in to the adversaries is Dramatic we're talking tens of thousands of cyber warriors being created in various intelligence agencies around the world The arms race for intelligence is dramatic at this stage of the game And we have a whole new breed of types of attacks that are occurring They're no longer little attachments that I send to you that we could use a model to block and Measure against we now have highly obfuscated Executables that are running inside code and in memory They're talking to sophisticated command and control servers and these architectures are being deployed in nearly every company in our critical Infrastructure today, and if you look at the adversaries many of the adversaries are lined up as account teams as Research teams focused on individual companies that are part of our critical infrastructure It is the facts and that's the reality of what we're seeing So what ends up happening? They leverage a model like I'm showing on the screen All these products that we've deployed and the recommendations and controls that we've put in place Create a defense and depth architecture that has lots of point products And what did the adversary do they come in one vector and out another vector? I send you an email through one protocol I get you to click on a link and it goes out another protocol And exploits the fact that these two products don't communicate to one another the email gateway doesn't talk to the web gateway Doesn't talk to the host security and we end up having fabrics that are under attack that are easily evaded And so if you're a chief security officer today, and you spent hundreds of millions of dollars on security And you're seeing the success that the adversaries are having it's a it's a pretty unique problem and This is the state of which we're in and of course the more Devices we bring in tablets we bring in mobile computing or cloud-based computing environments We bring in it just creates the situation a little bit greater and greater So there is a lot we can do about it and that's what I want to focus on in a second But I'll show you an example or two here I'm not sure if this shows up, but this is a live map Well was a live map yesterday, but the map is really just pulled directly from my my company fire Eye and what you see is a cyber map of all the command and control servers that are live as of a couple days ago And if you just sort of get in a little illustration up in the one quarter I don't have a monitor here I can see but you'll see a couple numbers just during that day over 300,000 attacks that were talking successfully successfully out of American command and control servers and critical infrastructures 100 plus million of these attacks year-to-date That were successfully evading the defense in-depth architectures that are out there just to give you the size If you look in the other corner probably can't see that 90% of these attacks are Leveraging apts or advanced threats So you now have hundreds of thousands these a day on our critical infrastructure 90% of them are advanced attacks and they're successful It gives you live data illustrations of the problem if you look down here you can see well You might be able to see but I'll tell you most of them are Industries that are in the critical infrastructure in America today. So these are you know high-tech transportation Defense industrial base energy and The list goes on you can see the amounts of attacks are measured in tens of thousands on a daily basis So pretty sobering and unfortunately very true and if you sort of look at the campaigns They're very deep and wide crossed a number of industries and verticals and The adversaries are after the innovation that's created here. So you see high tech And some of these are advanced persistent threats names and you can see them Some of you might know ghost rat if you don't know these terms These are remote access tools that are deployed very successfully into our infrastructure and The list goes deep almost in every industry So designed malware for industry attacks that have a very complicated persistent and advanced methodology That are in active use as we as we speak one example of this Many of you might know is what we called operation b-bus operation b-bus was a very successful attack on our defense infrastructure our defense industrial base particularly after UAVs and drones and This attack you spearfishing and just to show you a little how it worked a Weaponized email was sent through a socially engineered lookup on LinkedIn or Facebook They figured out who had access to the systems sent them an email had an attachment to it That person opened up that email that email ultimately talked back to a command and control server and that activity then leveraged a Cryptography that was never seen before highly advanced cryptography and suddenly the malware was erased and Ultimately the credential stolen and ultimately the intellectual property stolen as well And we ended up with an extremely successful attack very organized attack on some of the core defense intelligence that we have in our architecture today and The impact of this was pretty dramatic. Some of you might have seen it we now have a major competitor in the UAV market and Launched just recently in China and now we see an industrial hub copied and counterfeited Almost to the tee in months So this is the world we live in where if I can steal intellectual property from my neighbor I can replicate that technology I can gain an advantage and Suddenly the stakes are much different than today. So this is a pretty public one There's many many hundreds more that we could show you like this where the race for the innovation and the Stilling of that information is at a whole new level So we're a lot we can do to solve this problem Education is critical. Is it going on? How's it going on? What type of attacks are happening? And what are we gonna start to do to fix these things? The very first thing I would tell you is we have to think a little differently about the controls that we're recommending and implementing Antivirus today and this comes from a person who pushed this technology for five years is having Significant trouble keeping up with these attacks if we continue to recommend the same type of control We're gonna get the same type of result that I just showed you We have to augment this with new detection methodologies that are are available Detonation chambers sandboxes Doesn't matter the terminology, but we need a new way to study the behaviors of these attacks And we have to implement that and recommend that as a core part of what we're doing in principles moving forward Second thing that I would tell you has to happen is we have to create ways to put in place multi-factor authentication If any of you read the mandate report the apt one report Nearly 100% in fact, I think 100% of all the attacks came through spear phishing So the idea there just like I showed you on b-bus was if I can send an attachment to you you open it up I can put a key log around I can steal your username and password and then I just log in as you By putting second and third factor authentications in place Which is nothing that has to do with my companies, but an important practice helps for the adversaries to another level and We need to put in better systems, especially interior server systems Those controls have to get in place and most of our critical infrastructure needs to deploy that some of that is in the recommendations, but the wide practice of it is almost never done So we walk into customer after customer valid credentials are stolen interior systems are compromised and There's no level of fabrication other than username and password and many times that is you know Your son or daughter's name and it isn't too hard to figure that one out So a lot of very easy ways to breach obviously alternative mechanisms for putting in Endpoint controls Again are key. There's some great technologies out there to do that and probably some of the biggest recommendations I have to is to do better hygiene and health checks. It's amazing if you're an incident responder just what you see there in terms of APT's have been dormant inside the systems are active in the system for years Some of the accounts have three four or five years of the breach sustaining itself inside the architecture, which means the pen testing models and health check models aren't working So how do we create a better health checking system to really help make? These APT's get discovered inside the networks of our critical infrastructure so very key and probably the last piece I just tell you we need a much better understanding of what the adversaries are doing We need a much better understanding of the risk that the critical infrastructure has Many of the chief security officers that I talked to just don't know much about who it is That's attacking them and why And what you're gonna do about it So a little this is education to helping them understand and map the risk and enterprise risk management framework map to the adversary Understand what classes of employees are going to be targeted what classes of assets are going to be hit and Building a defense architecture around that that doesn't have to mean they have to spend hundreds of millions of dollars more to fix it Basic improvements of the defense architecture aligned to where the risk is the greatest and Aligned to where the people and assets are most deployed can have a huge impact on The success that the adversaries are having All right with that I'll pass it over to Jane and Well, I can't think of anybody to speak more co-gently to the threat environment and projections of where we're gonna be going than Dave So so that was terrific How many of you in the audience are not technologists? Come on. Let me encourage you to raise your hand How many of you are Who are the rest of you just curious So my name is Jane lewd. I'm not a technologist. I feel that's important to to say at the outset I'm the former deputy secretary of Homeland Security and had the opportunity over the course of the past four and a half or five years To be at the center of this government's effort to establish a presence on cyber security and to establish Cyber security as something we all need to think and worry about I see some of my colleagues In that effort in the audience Phil Reitinger and and Mark Wetherford We bear the scars of Trying to figure this out in a dynamic way to try to build this plane while flying it and what I'm doing now is deeply related to that That effort in Homeland Security. I'm running something called the council on cyber security with some of the finest technologists And individuals in this field And what I thought I would do this morning is for this afternoon is talk a little bit about what we're up to what we're doing What we're seeing as we're doing it and what we think is coming I think those three three things might be of interest here first I was very interested in the the question they've tossed over his shoulder. How many devices do I how many phones? Do I carry around well between my husband my youngest daughter and I just the three of us We have five mobile phones two desktop computers three laptops two iPads one mini and six e-readers It's a lot It's a lot This past Thanksgiving, thanks give a cup for some of us in the audience Our family came together and our one of our daughters drove over from Germany gabbing on her iPhone Tracking herself on her iPad while her son was streaming a movie in the car on the way through three countries To get to where we were in Belgium We skyped with our daughter in Jordan and talked on the phone with our other daughter in Manhattan At least we think she was in Manhattan That's what the phone number said but there was really no telling where she was. She's in dance music So we don't ask But the point was We're all of us online all the time certainly all of us in this room And so why was there a need to establish a council on cyber security? Quite frankly because the technologists and the policymakers have not been speaking to each other coherently Up until very very recently and in fact as in part as a result of that We haven't made progress in this country or frankly around the world in where we should To the extent that we should in cyber security at all And so we created this council which is a not-for-profit independent international expert Reference point for what best practice ought to be in cyber security and we're focused on identifying validating promoting and sustaining that best practice over time keeping it up-to-date and applicable Not only for technologists, but for policymakers as well. We're working in three key areas technology manpower and policy in Technology our focus is all about the 20 critical controls The council on cyber security is now the home to the 20 critical controls Tony Sager known to many of you in this room Who's the director of programs at the council has just convened two expert panels one which will be a standing executive panel? To review the controls on an annual basis will report out at RSA every year beginning this coming RSA conference On the update of the controls and the second panel is a panel on threats Looking at what we're seeing in the wild. We're seeing in industry. We're seeing across the board with respect to threats and Asking ourselves whether or not we have we do we have adequate controls for the threats? We're facing some may say we need a 21st control others may say that you know 20 is too big We need to neck that down into four or five that can really do the job and with respect to those four or five This is for me one of the key messages of the immediate period and the coming period We're not in any way. I mean Dave talked about it and talks about it Jim does as well others of you in this room about cyber hygiene From a policy makers perspective part of the problem has been that the dialogue that we're having with the technological experts has been focused on cyber Couture and we need ready to wear We don't understand most of the time when technologists are speaking to us We just want to know what are the things that we can do and the things that we should do first That will have the biggest effect in keeping our systems and our network secure We're not exactly sure with even that means but we do know now that the threat is real That it's getting worse that our vulnerability is increasing and that the status quo is no answer Those were the same factors that drove the US government to get into this space in a very big way beginning in 2009 When we called out for the first time the mission of ensuring the cyber security of the nation's critical infrastructure Person most responsible for that is sitting in this room Phil right and sure came into my office when we were writing the QHSR When we were finalizing that document the quadrennial homeland security review and said we cannot Ensure a safe secure resilient place where the American way of life can thrive unless we call out Cyber security as a core mission and up until that point We had talked about preventing terrorism securing our borders enforcing and administering our immigration laws and building national resilience in the face of disaster and We added because of the the logic of Phil's argument and not to say he wouldn't leave my office until I did We added cyber security as a core mission for homeland security and it has remained so ever since and in part you can trace the increasing awareness and activity on cyber security to over the past four and a half or five years To the fact that Homeland Security began to get in the game in a very big way As the lead agency in the federal government for cyber security and as we were doing that We focused on what our mission would be and we said we needed to protect dot gov We needed to do more in dot gov to prevent bad things from happening and Respond and mitigate rapidly when they did and we also at that time began to focus on fundamentals and hygiene What are the basic things we need to do and what are the things we need to do first to have biggest effect? So So we have been ever since that time and certainly in the council on cyber security now I'm focused like a laser on the top the top 20 critical controls But really on the top four or five The Australians have recently conducted a test where they had 1200 machines in a network and they ran that they ran up 1700 pieces of Unique most common malware against them with no controls one two three four and prevented What percent? What what what success rate do you think they had in preventing attacks? 100% 100% now they don't claim that the top four will prevent a hundred percent of all all attacks that you're facing But the numbers pretty high and from a non-technologist and policymaking point of view the numbers high enough To be persuaded that these are the kinds of things that we should be doing We're persuaded that the threat is real again. We're persuaded that it's growing. We're persuaded that our vulnerabilities are increasing I mean when people ask me, what's the greatest threat that you see out there? Everybody wants me to say the Chinese or the Iranians or you know anonymous and I always say Unpatched existing vulnerabilities that you leave unpatched. That's the greatest threat. We're not doing anything about it Or we're not doing nearly enough So one of the things that we're focused on in the council is the 20 critical controls are the four fundamentals of cyber hygiene I tell non-technical audiences. This is the equivalent of brushing your teeth flossing and visiting the dentist twice a year Now surprisingly Surprisingly, there are a number of people in the technological community that says there is no such thing as adequate hygiene There's nothing you can possibly do to defend yourself unless it's tailor-made to you. That's why I call it cyber couture Well, frankly the policy the policy community thinks that's nonsense It's nonsense. There are fundamentals We may not understand this field deeply to the level of technical expertise that many of you have in this room But we understand complex organizations enough and there are always a few fundamentals a few sound practices That can that can punch way above their weight in achieving the effect And so we're pushing I'm pushing the four the top four of the 20 critical controls We're also working on manpower Because we don't think that the the technology will be in the entirely the answer here. Nobody does How do we understand this field? We're calling cyber security a recent national A report of the National Academies the National Research Council said it's too soon to try and professionalize the field of cyber security We should wait for things to stabilize really really Have you talked to your doctor lately? Ask him how stable the field of medicine is Sure, there are a lot of things that they've learned over the course of thousands of years, but it's changing all the time It's not too early to try and seek professional standards and a focus on what are the mission critical cyber skills for? security that we need There are a lot of important positions in the IT in your IT networks and in your systems and that operate and handle your Information, but not all of them are security experts and when it comes to cyber security expertise How much of that expertise should non cyber security experts such as electrical engineers? For example operating the grid those in oil and gas those with other operating other complex data systems How much cyber security should they have baked into their professional preparation? And be tested on and certified against standards that people recognize acknowledge and respect and that's where we need to go in our view in the area of Manpower on policy the full range of issues is open ranging from everything from governance of the Internet Which is a hot question among governments right now? Down to what individual enterprises can do What's gonna what's going to where's the future going? We all you know, it's it's we're all as good enough to guess at that as anyone else I agree with Dave's characterization of how the threat environment is going to unfold I don't think the Internet will look the way it does even two years from now I think we'll see major structural changes in the way. It's administered and maintained And we're already beginning to see countries move in individual directions in that regard Will we have the same kind of connectivity major applications already don't have that for those of you those of us I should say who try to reconcile iCal and Google Calendar can attest really fix this please fix this So where else are we going? I think we're going to be able to judge the viability of an enterprise The financial viability of an enterprise through looking at one thing. It's cyber security posture It will tell us how well they're protecting their IP It will tell us how robust their systems are and it will tell us how good they are in compliance with the Regulatory and other regimes that they have to comply with only by looking at their cyber security posture We're going to develop that model in the council and we're already having a lot of interest among enterprises in seeing Because they believe that they can use that approach then not only to fix their cyber security But also to improve their business viability Fundamentally what I should say is that we learn this in homeland security in every aspect of what it is We did not just cyber security that the federal government cannot do all that needs doing here And so I differ a little bit from Dave and others in the room as I try not to use language From the national security or from the defense environment in this environment I don't think anyone who I think only those who have teenage boys in their families will accept anything called a detonation chamber in their systems at home But I think we We're all headed in the right direction. No one's trying to get this wrong, but it is going to take all of us working together Will we be able to leave it to the market simply to handle our cyber security? No, we don't leave anything to the market by itself to handle Is this so dangerous and is this so complex that only our government can do all that needs doing? That's also manifestly not true So we need to collaborate together and we will need to find new models of sustained public-private partnership To address business viability address privacy and address the cyber security that we need not in the future But what that we need today? Thanks very much Great presentations by both our speakers. We have time for a couple of questions Dave has a plane at three and even I am getting nervous But so maybe you could start if you have questions for Dave We'll start with them and then have questions for Jane if you have questions for both of them will wing it Any questions go ahead. I'm sorry And could you identify yourself, please? Johnson I've got a question I guess for both Dave and Jane and it's about sort of reconciling what you all Both talked about Dave focused on the very scary threat landscape in particular the advanced threats, but Jane you focused on basic hygiene the 20 controls so the question is Are the 20 controls going to be Adapted, you know neck down expanded to incorporate protections against some of the things that Dave was talking about I'll just say quickly. Absolutely The threat panel that we're convening is designed precisely to look at the things that they laid out I think really cogently You know I was on the phone with Kaspersky labs this morning and they wanted to know if we were going to include Evidence and data that they had had from nation-state Intrusions that they're picking up to see whether or not the controls would work against them So, I mean there's a pretty this is a this is a new landscape that we're in right now one thing I just add on is I Veminately agreed with what Jane was saying earlier. It's amazing. What just good hygiene can do You find so many systems where the vulnerability has been known for a long period of time Yet it was unpatched it just just trying to get basic rigor around just improving what we do know about You know can can really dramatically improve Are we going to be able to protect ourselves against advanced nation-state attacks? Maybe not but there is a lot of Hacktivism that can be stopped just from a lot of controls that basic hygiene can improve upon so others Go ahead, please My name is Frank Barone. I'm a private investor and I'm probably hybrid technologist and hybrid non-technologist So I fit your description a couple things that I heard today, which I was very encouraged about Certainly delighted to see that there are tools that are skilled and capable of identifying what's happening That seems to me like a pretty strong national asset And I think I heard you say that the Australians have some kind of a test environment where they Ran a hundred tests against a hundred test cases So I guess I asked the question is do we have a national policy on test beds for cyber vests? We do that for everything. We do it on airplanes. We do it on tanks. We do it on cars But we would seem to me that you've got the tools somebody the Australians as a case built a test bed of some kind and Rung out the significant amount of information So it sounds to me like it's time to put the policy on the back burner Move the tools to the front and turn up the gain on the test beds Does that make sense? No, no it makes perfect sense to me. I'm not I've been out of government five months I'm not current. There are there are cyber ranges that exist There are a number of tests and tools that are run have we run our own comprehensive tests like the Australians to validate the controls? No, would we like to do it? Yes, that's something we're going to pursue. Is it government policy yet? Not to my knowledge I have to say I kind of like the name detonation chamber, but then I Do have teenage boys and they do play call of duty. So Other questions here got any torture chamber, yeah, I Have one I'm gonna ask it and this will be the last question then which is One of the things that I'm interested in and in particularly if you looked at Dave's remarks the last time when we had Chris English there it looks to me and some of the other work we've seen there's been articles in the Times there's been the Verizon report. What do you think the future is for the AV industry AVG industry? Pardon me. What the technologies that we had developed? Convincingly don't seem to work as well. What's the future industry going to look like when it comes to? Yes, great question. And the first thing I would probably tell you is There there's obviously a place for antivirus. I think for a long time to come. I mean, there's no doubt about it Is it the exclusive place of detection engines? I don't think so and I think that's what we're we're suffering a little bit from today is When you look at the security industry We we are spending Probably over thirty billion dollars globally on security today when you look at the industry and when you look at the Detection engine itself it represents tens of billions of dollars ten to twenty billion of that comes from The antivirus vendors and the antivirus model so you almost have a monopolization that's occurred on the engine itself which is which is interesting because That engine while having a very wonderful place in the world for stopping a lot of the controls and Attacks that are happening can't stop a lot of these advanced attacks. So what do we need to do? Complement it. We can't have just one our defense in-depth model has to evolve from a single detection technology to multiple detection methodologies and I think that's just a little of a learning that I continue to see is we have to involve And recommending controls and proving those controls to say hey any virus is great There is a whole lot of benefit from that that can happen because once the attack is known the controls can work But if you don't know about the attack and the attacks are sophisticated and advanced and they're what they call zero day A type of attacks. Well, what detection methodology do we have for that? And so the complementary nature of the old and the new a little bit or the evolving and the new is Probably what we have to put in place and the defense in-depth has to evolve to have multiple Detection capabilities so at least my view on that and by the way in two years from now three years from now It might be another one, but if we can't evolve the controls to have multiple ones We're gonna miss out on whatever nation-state or hack to this group or whatever it might be who's evolving very quickly too So keep the policy up keep the controls up Keep the methodologies up, and we'll we'll be much closer to where the adversary are than we are right now With that, please join me in thanking our two speakers If I could ask the panelists to come up and if we could get the name plates for them, huh, I don't care One of the things that was funny about the RSVP list is there were so many people who Are experts in the audience that I kept saying to them well if you want to be on the panel Let me know none of them did but I think it will lead to a robust Q&A period I Promised Bob that we'd save him a sandwich. He drove in nobly from BWI and got here on time. Thank you So let me quickly introduce the panelists. We have a great team here Topic for today is really to discuss in light of the keynote remarks What we think the framework is going to do where we think it's going to go what might need to you know Be a path for evolution on it. I'll just introduce them briefly We'll have their bios on our website. We have a website with the white paper Podcasts interviews with people that you'll be able to find I hope at CSIS org Yeah First Adam Sedgwick senior information technology policy advisor at NIST Many of you know Adam from his time on the Hill. He's really the guy I think of as the The Stucky on the framework and has done most of the work in pushing it forward along with all his colleagues, but Really grateful that he could be here Angela McKay Principal security strategist global security strategy and diplomacy at Microsoft Someone we've had speak many times here before at CSIS a real expert in the field Craig Rosen the chief information security officer at fire. I one of the people who's helped build this company and understands the threat in a Def that most of us don't share Paul Kurtz Many of you know a long career the parts that were involved with me You could call checkered but a long career in cyber security dating really back to the Clinton administration So one of the national treasures here now that chief strategy officer at cyber point international And a lot of experience both here and overseas Last but not least Bob Butler who is One of the founding fathers in the field. Wouldn't you say I don't know he may not like that But someone who has commanded respect who comes out of DOD who initiated a lot of the policies that Began our efforts to improve national cyber security And so we're really grateful and who also just flew in what an hour and a half ago Landed 11 so really appreciate the fact that you could be here I'm gonna start with a few questions for the panelists Then we'll open it up to the floor. This is supposed to be interactive in exchange So think about the framework think about what you want to know think about the the threats that are looking up Maybe what I'll do is I'll start with Adam and say if you could give us a little bit of an update on Where the framework is where you think it's gonna go how happy you are with the process Sure. Thanks, Jim I wonder if I should start with that last question, but I'll get to it. Yeah, I Like the title of this of this event because it does give us me a chance to make a pitch which is You know, we are in a period now where We have a preliminary framework as was called for in the executive order And then we're in the period where we have this we're in this four-month period where we're able to take that and Think about how we need to improve it. So The RF the the framework itself has been posted since the end of October we missed our deadline to the shutdown Which causes a lot of angst for some of us But we and we still we commenced a 45-day comment period that it that expires a Week from Friday. So Friday the 13th. It also gives me a good out on any tough questions I have now I can just say I look forward to seeing your submission so we can make that change In terms of what we're doing it while that's going on I mean one thing I would point people to is if you look at the process that we created under the exact executive order When we got this assignment back in February It was really to engage with as broad a community the stakeholders the owners and operators of Critical infrastructure the people providing the services and the policy makers other government agencies and we did it through a series of Open discussions open forums. We had workshops throughout the country And you know basically in every stage we would we would pose these sort of difficult questions starting with the request for information and then and then kind of NIST role was really just to provide that structure and the analysis so that that people can have these conversations and I think one of the things that was unique about it was having folks having that diverse a community in the same room you would have At the workshops you'd have large multinational corporations sitting next to small water utilities Sitting next to foreign governments It was and all with the effort of achieving that same goal and our effort and one of the things we have tried to do throughout the Process is be as open and transparent as possible so people understand the decisions that are getting made And in that context, you know one of the things that we've been doing that I can say you know We posted out just yesterday As we've done throughout the process kind of a summary of the workshops Which are which are true working sessions. They'll be short plenaries and then we'll break into groups So, you know, we look forward I look forward to the discussion look forward to thinking about ways to improve Our time frame is to get the comments and by the 13th We only have a handful now, but that's pretty common. I expect them all to come in Thursday and Friday at 4 58 if history Has anything to say about it? And then, you know, what we will do is, you know We hope that people will be looking over our shoulder and doing their own analysis of the comments and helping helping us determine What where the consensus is what were the things elements we need to pull in? and Then you know one of the one of the other things that we're thinking about doing is you know in February when this is Do in the year under the executive order is up. We also intend to kind of to report out a Roadmap for future action, you know the framework in its simplest way is to look at, you know The existing capabilities to elevate the use of those that we know to be effective But then that third piece of it is vitally important and that's thinking about Where are the next things that we need to work on? How do we? Get these capabilities into the hands of the people that really need them. So that's going to be our focus post-February and I think it's really important and that's part of the document now We have a whole areas for improvement section talking about these Advanced elements, so I really do as well as looking at the document itself I really do invite the people in this room to look at that section and think about how we need to improve and how we should structure that work over the next years Thank you, Adam. Let me turn to the panel now and maybe we could start with a question that builds on the keynotes and just go down the row starting with Bob, but We heard a lot about advanced threats and their success rate, which is incredibly high How do you think the advanced threat is going to evolve in the next few years? What do you look forward if we assume it's a dynamic opponent? They're going to react. They're going to respond. What will that look like? Yeah, so thanks Jim my sense here is You have to take it from the perspective of what foreign intelligence services are doing and what you're seeing today in terms of Convergence so people taking advantage of vulnerabilities as they exist today, whether they're zero days or what have you But also looking at the convergence of insider threat Copper company access with remote network access and how that plays together in a Collusive in a way of collusion So if you think about where we are today, whether you look at the recent, you know hijacking over at Facebook or any of the things that recently happened you're seeing I think Advancements and appropriations of techniques against botnets and using botnets differently You're seeing ways that we can weaponize things a little bit differently, and I think that will continue to happen I think the real dangerous part of this is the collaboration and conclusion of Sophisticated actors working with with others that may not have as much care and are actually out for other motives I think the vectors continue to evolve with regards to to the threat itself I see it in the job that I have at IO as a global data center company You know, we do see you regularly Ways that people are now increasing not only in terms of intensity of attacks with the state Distributed now of service working with financial services But creative ways of stealing credentials looking at Not just classically PII You know stealth and then and then also looking at IP theft but looking at those together and actually building campaigns You know really what we talk about with an advanced persistent threat I also see that you know as time goes on and the world that I live in We are a DHS CIK our asset We have critical infrastructure that's built on industrial control systems And so I don't lose sight of you know what has happened with stuck stent and others and We still have tremendous challenges there and if you know if I'm thinking as a foreign intelligence service or any nasty type of Actor that might want to try to exploit Certainly our industrial control system base or operational technology that drives energy up into the IT stack is is still very vulnerable So I think there's still a lot of vectors out there and as we remediate within the space that we're in today and Again my hats off to Alex and the NIST team for what they have put together at this point I think we have to continue to stay focused on we're living and continuing to live in it and a greater Interconnected world that creates opportunities and threats and the threat continues to advance and so You know I think a document like the framework is a good starting point But it needs to be a threat-driven document It needs to be a document that kind of looks out forward Into the world of interconnectedness and what that means for us as we kind of move forward both in national security and economic competitiveness so Before fire I also spent about six years in critical infrastructure. So there's a long list in my head of Possibilities here to discuss about pick-two I'm starting with kind of the distributed nature of what's happening out there You know, I do remember when a website was a server When you look around now a website's a conglomeration of multiple sites snippets of code It's incredibly complex. So what I think we're gonna see is kind of preying on that complexity. We've seen it already, right? We've seen You know things like watering hole attacks and luring you to one side or infecting a third party of a third Party of a third party who's a snippet of your website who lives in another data center And so it's incredibly complex and it's easy to prey on that complexity So I think that whole distributed model of our assets being out there and then couple that with mobile and it gets Exponentially complex right Jane talked about the six devices alone in her home So, you know, that's one I think that distributed model and the mobile side and the complexity is one and then the second one is you know On the supply chain side of the house And you know part of this I draw my critical infrastructure experience, but it's pretty much everywhere, right? I'm concerned about Things like firmware embedded systems And you know, I think what we're gonna see is we're gonna see these threats kind of go lower into the staff, right? So that they're persistent and they're there all the time So I think that those are the two that I pick out of a long list of possibilities But those are the two on top of mind right now So I guess what I would say here is you know Some things are gonna stay the same and some things are gonna change over time We'll start with the easiest of the stay the same The layer eight in the system in other words all of us the user and the attacks spearfishing attacks on users We have not been able to address that challenge historically And I think that's gonna continue to be a challenge over time and some of the things that Jane was talking about and Workforce education are gonna be important there, but that's always gonna be there Also, I believe that this credential a credential theft and credential harvesting will continue One of the things that we've seen at Microsoft is in some of the more determined and persistent attacks There is a technique called pass the hash that is used quite frequently And what that does is it basically gets a set of credentials and logins and then can use that to either move Laterally through the system or escalate privileges in the system And that's a great way to move around inside of the network in order to find things that may be of more interest And then building on the points actually that were just said by Craig when we think about things that are gonna be new I'm gonna highlight two first. I think is the Effects that we're gonna see in small and mid-sized businesses and so Craig you kind of hit on a supply chain So sometimes people just think about the supply chain side as being you know Malware malicious malware inside of a product or service and that's one angle But another angle I think we will see is the partnership side of the supply chain attacks So moving from small and mid-sized businesses that don't have necessarily the resources or capabilities to secure themselves adequately that our partners are either brought into other organizations through mergers and acquisitions and Then last but certainly not least As we all move to greater encryption because of concerns that we hear about in the news What I also think that we'll see is a move towards greater insider threat Because as we start to make these systems a little bit more secure and use encryption People are still going to want to access the system. So we're gonna have to watch out for that human side inside of the organizations as well Thank you, Angela. That was very diplomatically put about events in the news So Paul Thanks, Jim just really to add on what others have said first point that I would I would note Most of you are many of you maybe where the defense science board report that came out earlier this year has an excellent breakdown classification of the threat actors From those who are relying on existing exploits that exist out there today to those who are able to develop their own tools and finally to those who actually can manipulate the supply chain and I think that was the first time I actually saw a public useful template of breaking down who the bad guys are But problematically just the same it notes that we can have The bad actors as in states or foreign intelligence organizations use those less sophisticated capabilities in order to achieve their ends And that brings me to what I think the the three Developments we might see over the over the coming Years number one Organizationally how are the bad guys operate? I think we're going to see a lot of hackers far higher I'm not talking about anonymous. I'm talking about small agile groups Operating Well below the radar screen that are hired by others whether they be foreign intel organizations Whether they be nation states or that they be terrorist organizations I think we're going to see those kind of cells and units form up and and operate more actively In Bruce wills is what is it live free or die hard which came out in? 2007 which is actually based on an article written in 1997 an article in in wired talked about the you know terrorists if you you using using the networks to cause a Financial crash Around the world and across the country and I unfortunately I think Organizationally the way that the actors operate is is going to change before us And it's going to pose a lot of problems for people the second issue. I noticed the speed of attack And this is if you will good news bad nor bad news story I think there are a lot of companies fire I and others my company cyber point is working on capabilities as well to automate reverse engineering of malware and that would really help the good guys better defend networks We can understand capabilities far more rapidly don't need necessarily all the human talent That we have to throw against a problem now to understand what really may be happening Well, we have to also remember that the bad guys will get a hold of that stuff, too And they will use it to come against us at a much more rapid pace than we have We have even currently see as hard as that may or difficult as that may be to believe in the third evolution that I would highlight is Attacking the cloud. I think this is a enormous problem. I just came from the cloud of security Alliance Congress down in Orlando and when we look at Critical infrastructure, and I know we talk about IT being important It is one of the infrastructure, but the cloud in particular a Small number of players in the cloud will have will be if you will the Critical infrastructure because they will support everyone whether we know it or not and That's where I think where this work and the guides who are putting together. I Is useful, but will also be incredibly challenged because the cloud infrastructure is Incredibly complex that is not easily Defended I think we have a very long way to go and in that case We know that the bad guys are using the cloud now to attack others And The White House has picked that up and through the banking attacks in the now service attacks They've taken place over the past year, but I also think we need to look out and how the cloud itself is going to be attacked I don't have much to add to that I don't think I could do a better job in these four actually in the space a little more closely But you know, I would say that it does echo the things that we've heard about You know throughout our process about sort of the the complexity that develops through the increased flexibility And the new technologies that come in And as well as the the threat evolving and I think you know in terms of how policy makers think about these things It's really important to think about how do we develop the solutions and the policies that are really focused on those outcomes? So that you know, we maintain that level of flexibility to allow for that those change changing landscapes I think it's really critically important for the framework, but also all of our work going Okay, thank you Let me start again. I'll start with Bob and we'll go down the row One of the things that would be interesting to think about in light of what you've all been saying is How does the creation of this kind of? Framework good or bad for cybersecurity. How does it change the landscape for policy and legislation? What is it? We're gonna have to do differently now. I was gonna ask about companies as well But let's do policy and legislation first. We have something now that we can hold things up to what is it? What's different? Yeah, I think this is We have the basis as you said Jim for for now moving forward I remember Phil and I working a few years back on Trying to get at this idea of now. How do you enforce it right? what's what are the carrots and the sticks that make this all work and You know give credit to Phil and the DHS team and a lot of folks that were working in that space at the time with you know finding ways through Through creative means not to impose regulation, but to encourage people to raise the bar You look at champions within sectors that can help in that capacity Especially when you link it with you know the issue that we're talking about with supply chain We have to take advantage of champions in these sectors to help us Strength in the overall sector so in the financial services world I mean the Goldman's the JP Morgan's of the world we need to we need to find ways to help them help smaller institutions credit unions and state banks and things like that I Think regulation is an anathema. It's a it's a hard. It's hard to impose, but I think You know as we saw in DoD We had to impose rule sets to ensure national security whether it was Nuclear C2 was easy of the closed network But missile defense then line force projection. There are certain things that we had to impose within the DFARS Defense Federal Acquisition Rules to make those happen And you know we I think we built an education campaign We we then worked through different approaches which led to information sharing arrangements Which we continue to enhance And I think that's all part of what we're going to have to do in terms of specific legislation. I think the basis For what we need to do is is on the table. I really think we we need to now take that and Say hey, we've got this framework right so this So legislators and policy makers you can understand how this links together I would tell you that I think there is even though the NIST framework Adam refers to Business risk and tolerance processes and other documents I would bring more of that out because I think that's gonna that's gonna have to be a part of the education campaign Incentivizing this you know again working where I work the reporting to boards and working with other boards Just using a framework of you know identify protect detect Needs to be translated into business objectives and risk You know and so if if a company has objectives of international market expansion Or they're being driven into certain other kinds of compliance regimes based on industry This has to dovetail with that and there's got to be a way of characterizing Risk tolerance in that space as we go forward in time So I think those are some of the things that I would lean towards again incentivizing as opposed to regulating finding champions that actually in a sense Highly encourage and actually help folks. I mean work with some companies today that have actually adopted that role Quite well, and I watch in their extended enterprise where they're providing software education training as well as inspections down chain to ensure that These kinds of standards Are continually implemented one last point within the space is And I think Adam brought this up. This is not a static framework again based on a threat driven model Best practices have to become standards. We have to continue that cycle continuously So one of the things that caught me right away, and I can talk a little bit about the risk side of it Is you know looking at this framework? We have the right building blocks I Would like to to see an expansion in the risk management areas as well But I think it's a great start one of the things that's really powerful in this framework Is that it actually blends the concepts of risk management of controls and safeguards and maturity levels together now Just you know it it touches on maturity by describing maturity You know and we may touch on this later, but I'd like to see more of a linkage Between the actual maturity models and the risk so what's going to incentivize people to get to a certain level? And for it to really kind of draw that out a lot more But I really think that for the first time it's really powerful to see these concepts working together With the right building blocks in place And I think we've got you know some some work to do my hat is off the list. I've worked with this before You know on some areas and you know, I know it's a complex process and you can't make everybody happy You know at the end of the day But one of the things that I think we really can do is focus more on this risk management area and look at the different levels of maturity So that an organization can say hey, there's a you know There's a tier four organization What are the things that I can do to put into place that are going to get me to that level? And you know if we kind of take a look at it from that perspective Yeah, I think we'll see more of that behavior and risk management, you know Itself is a process that should be measured right and there are models out there at the electric sector I spent a lot of time in that area, you know, they've got models and I'll never get the acronym right, but it's the E2 CMM C2. Yeah, we've never been able to get that right but those models that are out there that really sort of describe what it looks like to be and so I think that You know having those practices documented and spending time and the framework does reference, right, you know other other Documents you would want to look at to leverage risk management You know, but I've also seen you know, you know playing the role of the sea So the first thing you want to do is grab that one Document and that's what you want to follow and you're gonna flip to where it says tell me what I need to do And you're gonna try to do it And so, you know what we'd like to do is is is be able to highlight those more and more strongly be able to get Companies and organizations to reference those documents or use them so that they can build in those levels of maturity So when I think about how the framework and maybe the EO process is gonna affect the policy landscape I'm gonna think about it at three levels first at the corporate level the domestic level and then at the international level From a corporate perspective. I think that this is driving a conversation and we're hearing it from our customers We haven't had before so while it is not yet into adoption and implementation Which I know we're controversy on terms of the ways that it will happen there's already a conversation occurring in The CSOs and with the board that I think has not been occurring before we're hearing these requests from our customers So that's I think one indication of a positive effect from a domestic policy perspective You know along I've been doing this a really long time and I think forever people have been saying well You know what are the things we should do for hygiene and I try and avoid that word hygiene sometimes But I said, you know, there are a set of things that should be generally done by everyone Across the board and there hadn't yet been anywhere that I had seen internationally a place that was security Outcome focused not control focused but security outcome focus that said these are generally the things that people should be doing To improve security and so I think you know, that is a data point in a touchstone that didn't exist in the ecosystem before And that provides also data when we think about how it gets adopted about what levers will work and not work So here in the United States the approach that has been taken is fairly lightweight I call it a turn the dial approach However, if we don't actually see demonstrable change Resulting from a voluntary adoption I think that then provides a basis for what we see happening in other places around the world Which is a more regulatory approach and so it's going to give some data about How far can the market take us in different places and where there may still be a delta and what those different places are to address that And then lastly I did start to mention the international But I think a lot of people in the world whether they look to emulate or not emulate us policy Are looking to see how this works and they're going to use it to shape their policies Whether it's again follow or not follow and so I think again We have a real opportunity and a real impetus here to work very vigorously Particularly on the incentive side and voluntary adoption to see how far we can go And that's how I think that the framework has affected the policy landscape generally Maybe one last point on domestic which is I do think that there's going to continue to be Various different initiatives from a Hill perspective whether it's around information sharing or the section 10 regulatory review That is called out in the EO those things are going to be moving forward and again Leveraging the data that is provided Hopefully through voluntary implementation Let me roll back there was a question After Jane and Dave spoke about you know testing and how important testing might be To help improve systems and there was a question about well, maybe with that ought to be a policy and I really cringe when I hear Government talk or government potentially take on those ideas and say well we need a policy everybody's got a pen test now We need you know, we need a policy around Having a test grid of some type or or cyber ranges of time some type I say that as a marker as to how I look at the framework I think I think the framework in fact does change the landscape it provides the template The framework that we've all been missing for the past 15 years, but now here's the danger We have to encourage our policy makers to resist Shining in law and regulation Properly, I think the executive order allows for at least some period of time for implementation of the framework and looking at adoption But and that is you know for the private sector to get up to speed on it In adoption, but I would argue looking at the framework especially The five function areas and the tier is the four tier area tier is let the private sector also respond in trying to develop the tools and methodologies against that and To automate it to streamline it to develop those dashboards for senior decision makers for operators on on through the stack up and down so I It's a it's a very narrow path that we need to seek to walk and it's gonna if there I was going to encourage government to do anything in this space. It would be continue to fund That the development and evolution of the framework Don't cut the budget increase the budget allow allow the government to market it to Conduct the outreach at the other end. You've done all the the roundtables and The meetings around the country to take it Well now it would be able to take a back out on the road and talk about it and get people to adopt it and also allow Innovators and entrepreneurs to develop against it And that would be far better than seeing Congress or seeing the executive branch rushed rush to adopt it because I think that Rushing prematurely we could all live to regret the other big portion of peace I would highlight here is the framework as much as I like it is what I would call the in the good enough category It helps 90% or 95% of the enterprises that are out there That allows people to use the current framework and not just wait for the next version to come out And that's something we want to be very open and transparent about and also with the goal to eventually, you know move this actual process out to industry entirely And and to allow NIST to get out of the framework development business and and go back to our role With supporting it through R&D and our work with the standards bodies So those are all parts of the conversation that we come back to With that we've had throughout this process, but we really have a lot more vigorously starting next year So I think that's a great question You know, I think if we're going to beta test it we ought to be very Selective in how we do that and I'm gonna probably sound like a broken record here on the maturity side And you know, but I'd love to pick a few different organizations, you know Some that are in implementation tier one through four and run them through the rigor Of this beta test and I would say that we would also need to have some Solid deliverables out of those beta tests and I guess some of my fears here are that The organization with this framework is gonna have to develop profiles, right? So they're gonna have to actually go through a process to develop those profiles So my fear is that they're gonna grab the framework and stick with the core profile and come out You know, if we just do a beta test that's here you go Does this work in your organization and kind of get the bare minimum? So that's my concern and that's why coming back to this, you know, how do we deal with these? Sophisticated issues and how do we actually measure whether or not it's successful as a result of the beta test? Is it development of one profile to ten five? You know, I've seen some some success in this area on the electric side with There was a grant and an organization called ASAP SG which Advanced security and it was advanced security and acceleration profile for smart grid and they developed these things called security profiles, right? And I thought that was a great model for what needs to be the output of this process And they were very specific and they talked about the systems and that's a lot of work, right? So You know, you know that work was was done. I think that again going back to this beta test We have to be very clear on what are the results and what are we expecting out of that beta test? And what does it look like on the other side so that we can then feed back into the model and say hey This is working or it's not working Just a just a thought building on your on your question. I mean DHS has a process That's enabled through some tech experts out of Carnegie Mellon others called a DHS cyber resiliency review There are many critical infrastructure companies that have voluntarily participated in that An option could be I mean there might be other venues to to link these two ideas together, right? So If you come at the cyber resiliency reviews that take place today really focus on some of the basics that you see in this This continuum that you know, we talk about in the document the framework document Builds into business continuity and resilience as a broader concept, but it's a self-evaluation That can then be used by that company. It's it's confidential, but it's a good way I think based on some sampling of different industries within sectors to kind of get a sense of whether it's you know Whether it's helpful. What's the value add over time? Paul you wanted to Yeah, I Maybe jump into a conclusion with your with your question about beta tests But when you say beta tests it implies that in the end you're going to ultimately Advocate the Formal application of the framework and I think we ought to really resist any sort of discussion That it is a beta test meant toward that end or a series of beta tests I think I would rather see for Advocacy around awareness and help with adoption of the framework and if you will let a Let a thousand flowers bloom so to speak to see what it looks like. I Also think to be the skunk at the garden party or just to call it out Government is not exactly seen as the white knight right now If the willingness of the private sector to cooperate with the government in a sense of an application of a beta test in this space May well be limited And that could be limited for a variety of reasons You know for their current market share for their desire to gain Additional market share overseas all of which have taken a significant hit in the wake of Snowden I'm not seeking to fault government activities here, but there is a dose of reality How much government can hope to do in this space? And so if government can if you will encourage the adoption facilitate the adoption in organizations like Janes Allowing them to succeed in allowing the private sector to develop new capabilities In programs against the framework that will help it is a far more constructive way to go in my mind Then the government starting to push and advocate for beta tests on the part of private sector I'm gonna we have a couple questions on that and we're gonna lay down a marker for a question I hope we can come back to because it's come up a couple times Couple not a year ago. We had a private meeting here. Some of you were here with people to talk about the electronic communications Privacy Act and SISPA and the Privacy guys were saying, you know, we they agree we need to amend these things and we need to do something and somebody Ask them well, how long do you think that will take and and they said well, it'll take two or three years Right to change the legislation That's probably an optimistic estimate And the the person in the who who was asking the question said well You clearly don't share the same sense of urgency that that others of us have regarding this problem and Last week I was talking to a friend who'd been in the White House in the first administration We were laughing about the fact that we had both seen the Patriot Act in 1997 1998 and the folks who had written the Patriot Act or the bits of it Said and you know, we're just waiting for the politically right circumstances to deploy it and then we know and so what I've been thinking about is We're cruising along here like we have a lot of time We've always been wrong when we've said the threat is coming really fast But we all know too that the day after something bad happens and we come really close to it in the last year There will be a rush to judgment and a rush to legislation So maybe when we come back towards the end I can ask the panelists about how do you balance the Go-slow Experiment all this other happy stuff with the fact that we could wake up tomorrow and be whacked over the head right, but While they're thinking I saw a couple hands out there. Go ahead, please We'll get that one. Please identify someone get Andy Thank you Kevin Neumar with the Perry Center at National Defense University Following up on something that the Paul said and really linking with James's comments How do you positively incentivize on the policy government side? adoption of frameworks that will improve the overall cybersecurity of the country In a positive manner, I mean Regulations can be seen as negative and impose costs, but how do you positively incentivize that in ways? It would be acceptable to the private sector sure. Yeah, actually. This is where I think that the framework can really be if you can This is let me see they can spit this out if you can take the framework and the functions to five key functions outlined in the framework and the the four the four levels of maturity if you and you apply a set of controls or a compilation of set of controls along the lines of the 20 critical controls in a continuous monitoring environment Then you ultimately enable The insurance industry to start writing down risk and to start writing insurance associated with the risk that is a very positive Way for us to go which is is not regulation It's not forced adoption. It will help raise raise the overall water Mark on the level of security we have distribute the risk more evenly the What what the framework does is at least give us what appears to be a common template across the private sector Which we which has been absent now I think this is where I advocate for the private sector being able to do more because I think the private sector can Begin to develop the tools that will ultimately allow the insurance industry to get the data They need so the actuarial folks the really green I shade people the computer people can actually do their business of understanding Risk in the space that's it's this that the the the framework should be seen as an enabler Not as an end in itself So I think about how it can the incentives on the market side maybe from both the demand side and from the supply side and when I looked at the List of incentives that came out of the White House I think about it as a little bit more focused on the demand side and some of those things really are Going to matter if they can come to bear And anyone who was at the NIST workshop in one of the voluntary program Meetings I said something about I think we need to to deal with some pragmatism that The incentives that have laid been laid out are going to take time To be manifest and so there's a little bit of time here where there isn't a huge amount of incentives on the table And we have to kind of live with that reality But when I think about prioritizing some of the ones that would help on the demand side of this Obviously government procurement is going to be one of the ones that could have a significant pull and also Depending on how the government procurement is done. It could also have a pull down the supply chain So there are those people who do been directly with the government, but if they require it from their supply chains You also have that market pull One of the other things for large multinationals like Microsoft is going to be actually having the US government and Industry work on harmonization and getting similar things to the framework done in other places around the world So for getting the US government and US industry working along with our colleagues in the EU who are dealing with the NIST directive The more we can get these things more closely harmonized That's a huge incentive for industry because it creates predictability on a much larger basis Obviously the one that I you know don't hold out a lot of hopes for right now is liability protection is very important to a lot of folks in industry And so those are the kind of demand side and then I'd like to hit the things that Paul talked about earlier Which is the supply side as we start to see the conversations move towards how do I? Adopt or implement the framework There is a supply side that will come out of the ICT industry that says, you know We don't have enough people no matter where you're at in the world. There are not enough people We're gonna draw a line and they say this is the security outcomes you need to get to but unfortunately There aren't enough people to help organizations get there and so you're gonna start seeing innovation in the ICT community I think in particular Managed security services based out of the cloud potentially Groupings particularly in the lifeline sectors where they may have challenges where they work together to buy in bulk So that they can start to get scale of opportunity for managed security services So I think about it. I hear most on the demand side, but I'd like to highlight the supply side exist as well Yeah, just when I again from my from an industry perspective, it's you know, we build in an extended enterprise I work, you know across lot the commercial and the government side and looking at what again champions are doing Microsoft's a great example I mean when we write work an arrangement with a company like Microsoft There's an SLA built in there with it and there's a there's a huge security piece to it that actually begins to look a lot like the framework and if we are all incentivized kind of in this this idea of bundled business to Actually promote, you know common prosperity across the an industry or sector, right? It's it's not that you're losing competitiveness actually increasing competitiveness by protecting risk on the on the downside of your supply chain That that would be a way of actually incentivizing it just kind of building on on Angel's comments here Just one comment because I've been sort of chopping into bits to use this analogy here and You know, I used to get a discount on my insurance, you know, just just for seatbelts, right? And you know now I get one for airbags. I get one for being a good driver So my analogy here would be the good driver discount is how well are you doing your risk management practice, right? And your seatbelts and your airbags are these controls that you implement So if we can think about it like that and that's kind of how I frame it in my head Which is hey, here's a great way to incentivize There's a lot of implications for how we roll that out, but when I look at that model Maybe it's a simple way to look at it You know Andy Purdy with the Huawei Technologies first a brief comment and two brief questions for Adam I think the number one incentive for folks to use the analytical risk management framework Which is what the framework is is the ability to sell your products and services and while government is the one That's traditionally been emphasized as Angela just did as Jim did in the CSAS Commission report The number one incentive and insurance someday will be helpful But referring to the Dallas session even the insurance panel said it's going to be years But the incentive for folks that want to sell their products and services but the idea and my first question to Adam is as You think about the work streams coming out of the framework To what extent are you encouraging or do you now encourage the private sector particularly sectors of private companies? who have common interests in the kinds of things they buy and The kinds of things that they need the requirements they have for those that they sell to To what extent does the private sector need to stand up in sectors and try to tell that story try to try to organize Those buying requirements those those requirements for supply chain those requirements You guys call it conformity assessment those requirements for product evaluation if you'd like I can ask the second question now or I can wait Second question is I know it's not perfect And I think it's arguably inconsistent with the traditional NIST model of like 800 dash 53, which is not to prioritize It's to lay out a framework And there are those here that better capable of me than saying this but the top 20 controls the success That's been demonstrated about what they can contribute to organizations now that had been vetted over time Is there a way that the NIST framework can can recommend to folks even though it's a former prioritization that the top 20 controls be part of the mix going forward can tell you that some of us have have mentioned this to them once or twice so Well, so, you know, I'll say to the to the to this I'll start with the second question and I'll say, you know the process that We laid out and you know, I remember Pat Gallagher talking about this at the CSIs event You know the director of NIST is to see what's being used throughout industry and build off of that So the the critical controls are mapped in within the framework core as is 853 which is also used throughout industry, you know I was you know talking to Ron Ross and you know that documents been downloaded over the glass number We saw it was over five million times since April. So who knows what where it is now so both of those things have been proven to be very effective throughout industry and I think the one of the lessons of the Framework process is you know, it's less the the the standard you use and it's and it's Really the capabilities that you're trying to achieve. So so both are in there And I think one of the discussions we had at our last workshop is, you know The the mandate under the executive order was to look at those true cross-sector Standards, but you know, we also had about 300 400 other entries from other Specific industry standards and so, you know part of the work of NIST and it's And its standards role is to really help the market with those and create create something where Those other standards can also be fit in for the organizations that choose to use them because it's proven to be affected to manage their risk On your first question, I mean, I think it is vitally important that we think about The the technological underpinnings and the and the technologies that can be used to support the framework I think that's one of the benefits of the framework getting out there is because it is truly cross-sector And we and we also highlighted the we've highlighted since our first workshop The need for the conformity assessment programs Also to be industry developed and industry run to the fullest extent possible I think that goes to Paul's point as well with true industry leadership So, you know, we're putting out the roadmap also for comments So we want to get feedback in terms of what other people think the priorities will be and that will be another open comment Since we're running a little short on time Maybe we've got a couple questions out there if we could get the two questions and then let the panel respond to them both And I'll make sure they do both questions. So In the back there in the first and then to the three questions and that's it final offer Thank you. This has been a great session Tim Stevens GCS Intel actionable intelligence for the financial sector I'm glad we brought up the issue of insurance. I mean the Situation we're in today is that the market is less than one percent of the Underriding for real estate. I don't think that's where the value of the American economy is in its real estate However, valuable that is I just be interested to hear the panel talk about how we close that gap and protect our intellectual property Ashley piles internet security Alliance and I'll say beta test and incentives there. I said the Larry But my other hat on the Council of Cyber Security I am on that board of review on the 20 critical controls that Jane was mentioning earlier and What I want when fire eyes brought up this question in Raleigh saying where does the advanced threat fit into the framework? And we talked about it doesn't quite fit in there yet And I think Paul articulated that very well We're not the framework doesn't go after that five percent of the threat Which is the advanced threat and should we include it in this version instead of waiting till the next version or the day after To include it Thank you. Wow, this has been a real learning curve for me, but bottom line is this is great I'm really really enthusiastic about everything. I'm hearing here. I'm a private investor in every single solitary day I have oil tankers that go through the Straits of Hermos. I have oil tankers that go through the sedan We've been dealing with risks that are dramatic for a very long period of time Just as you have the Department of Defense as you have an industry What I'm hearing today though from all of you is that there's some things that are ready to be done that are good to go I think this gentleman continuously emphasize that you've got tool manufacturers that I heard earlier You're hearing more about it that have some tools that can identify and monitor and highlight and focus in on some of these things At least one test bed. She indicated earlier Jane Lou did that the Australians had put up Had been able to do some hardcore testing on this with these tools in place So I would like to know as a private investor When are you going to move forward and get on with the job before you finish the policy the rules and regulations? Great, so we had three questions one on insurance and That one is Bedeveled us for a long time. So it'll be interesting to see what the panel has to say One on the prioritization of risk and whether we need to deal with APT now Another way to frame that would be can we afford not to deal with APT now, but it's your choice Or it's a risk decision by people relates back to insurance to some ways And then the third one is when are we going to actually move out and Some of us have been asking that question for a long time Be interesting to see what the panel says did I capture the questions correctly? Okay And we'll start with Bob and go down the road On the on the insurance side, you know We're where I am now we you know, we have cyber risk insurance And the challenge I think we have is is figuring out what the value is of cyber security You know building the actuarial tables so to speak the comprehensive insurance the collision insurance What's the model? You know to frame this out so that is intelligent to not only a board and a CEO but To those that would provide the insurance and I think we're this provides another input into that process what we've basically been doing is working pretty hard on benchmarking Internally and then internally in the data center business you can you can benchmark across a lot of different sectors, right and That has helped us in terms of creating a relative valuation of who does things well Where do we need to put greater emphasis? I think over time extrapolating that translating then to something that might be Used on a broader basis would be would be useful Currently I think you know It's it's the right thing to think about it's the next it's the next big challenge, but at the same time We don't have the data we haven't figured out I think a good model yet to really begin to provide the heuristics to help us with insurance valuation in this space and So we spent we place a lot of effort. I think a lot of energy and more so in the recent Breaches that have been announced on other consequence management type activities So so as we go forward in time going back to Jim's question from about 20 25 minutes ago You know if we if we know a big bang is coming What are we doing now about it other than traditional prevention and protection activities? I think we have to spend more time on consequence management And that's not just testing DR and business continuity plans and looking at PR strategies I mean that's that's soup to nuts kind of using almost like you see in the recovery framework of NIS But taking it I think back to business objectives. I mean it's in the in the business world I mean, it's it's really about you know, how are we going to? ensure That we've got viability with our stockholders And our stakeholders over time and so reputational risk needs to be translated into some some numbers To help you figure that out on the second question prioritization of Risk and as specifically as we go forward with advanced persistent threat There are models out there that that I think are beginning to work You know as we build threat intelligence functions and we think about methodologies like kill chain against APT We think about how we share information I think was actually something that the the NIS framework preliminary framework document called out in terms of information sharing I think those are the right things My feeling and again we do this from where I sit in the data center world We try to build counter APT campaigns, which helps us to prioritize risk as we Look at our customers and and our suppliers and figure out what the prioritization should really be in in terms of the Viability of our ecosystem, right, which is a global ecosystem that goes across different industries And then in terms of moving out. I'm all I'm all On on board with that I mean I think that was one of the challenges that I saw both in government and I see it in the private sector is in the private sector just by the very fact That you have to continue to grow business You're moving out whether you're moving out in the right direction or not I mean that can be debated, but you're moving out in a direction I think the key is to try to sort out The value of the direction you're moving in in the department defense You know it was clear, you know when I came into government the last time There was a desire to move out organizationally and making some changes as well as in in processes and in procedures and policy I think There at least in the national security realm that I was a part of My sense is there was incentive because we saw a real clear and present danger in the world of cyberspace I think as people begin to begin to internalize that whether it's on a corporate board or inside of a Pentagon That's when people move out. You've got an incentive as you move ships and activities around the world Certainly Though that that supply chain that physical supply chain that e-commerce link needs to be protected in order for you to grow business And it continued to to invest and attract other investors So so my sense is it's it really comes down to incentives and internalizing those incentives So that just that you become the champion to move out Thank you great So with regard to insurance, I do have a little bit of experience in this space I've you know looked at the insurance models before in fact I went through a process And I have to tell you I was pretty shocked at the questionnaire You know when I went through the process and questions seemed really simple to me And I was thinking to myself if I'm on the other side boy But I asked a lot more difficult questions here And I'd be asking about your advanced controls I'd be asking about your risk management practices and to be frank wasn't there And so it just kind of didn't smell right and I was thinking to myself How on earth are they gonna underwrite? based on these questions so, you know, I got the thinking and and You know, it just baffled me that whole process that you know How how is this model going to move forward without advancing? I think for those organizations To be successful. They're gonna need to really put tougher questions in there and actually push the organizations to do more in that area I'm gonna kind of blend the second and the third question because to me it's It's all about the sophistication of the threat and and you know, I'll be frank here too I mean when I was in critical infrastructure Three and a half four years ago. We put out a large program around advanced persistent threats. I didn't wait I didn't wait for a document to come out So the problem is here and it's here now and this again was three and a half four years ago So what can we do differently? What do we need to do now? You know, I think you know as I said, I think the framework's a great tool to start with We've got a core set in there But I think you know, they're kind of wanted to avenues here You know the court itself needs to be strengthened sort of turn on its head a little bit to add some of these more advanced controls Or it needs to be sort of feathered out by maturity level so that an organization can pick it up and say oh Here's what those advanced organizations look like and then the board can say you this is critical infrastructure This is where you need to be and so those are my thoughts in that area And I do think we can do a lot more now And there's a lot of organizations that aren't moving forward already. They're not waiting Just one brief point on the insurance market because I agree basically with both my colleagues here I think there is not sufficient actuarial data I do think that we see a lot of progress on the data breach side And so if there's data breach and leakage of data, there is an insurance market that's forming more clearly around that I think that the framework will help establish a part of a standard of care But it's still hard to understand the consequences of an attack if it's not just a data breach And so I think there's still a long runway. I think it will develop over time But the runway is a little bit longer than We would like it to be necessarily When we talk about the framework doesn't go after APT. I would say yes and no I go back to this idea of Horizontal there's things that everyone should be doing and then on top of that I think about there being a vertical piece that is there are organizations that have higher risk Profiles based on the national security type threats. They are facing and so I don't necessarily think that the framework is the best way And it's is dynamic enough to be able to deal with that, but I think the policy Approach both here domestically and internationally needs to have a place to deal with that I just don't necessarily think the framework is the best place to deal with that vertical Inside of the vertical though what I would say is there are some key practices many of which have been talked about Here today under a bucket that I would call automated collective defense. So we really do need to be moving towards Better telemetry from products and services being able to derive threat intelligence from those being able to collaborate among communities Protective communities to be able to work those issues and then having an automated Automated policy environment with actions that can be taken because ultimately I think in that more advanced space We're just gonna have to be a lot faster And then when we talk about You know when are we gonna move out my my only issue with that is it sounds like it's a start line And then we're moving out and I would really agree with Craig There's a lot of folks who've been moving out in this space for a really long period of time ourselves included whether it's doing things like Botnet takedowns, which is not on the technical side, but as a campaign effort But I actually in prepping for this particular meeting reached out to one of my folks in network security And I wanted I was talking a little bit about this exact issue and he said, you know One of the things here Angela is we really have to remember the multidisciplinary aspect of this approach and that They're going to be things that are campaign like they're gonna be things that are product improvements There's gonna be collaboration among communities, but it's really a multidisciplinary effort that has already started and will continue to move forward I think again what we're doing here is we're kind of raising the awareness And what the framework can do is it will make the APT a little easier to see? Right and so if we can reduce the signal to noise ratio We can get rid of some of those everyday things You'll be able to see the APT and then those who are moving out will be able to work in that more dynamic way to help address it Okay, and the first question on insurance I spoke to it a bit earlier. I do remember about a conference in July of 2003 in New York where in free insurance market came together It's start to talk about cyber and and I I don't think a lot of progress has been made And what was missing was the framework? And so my suggestion would be is for the insurance industry to work with innovators and the tech and tech industry to try to take what is encapsulated in the 20 controls and in the framework and put together The a package that might ultimately be rolled out to the private sector and To certain industries to begin with I think starting with the critical infrastructure is probably not the right place to start because it's overly complex to the second point on On the framework what it is and what about the day after and my simple comment would be to inoculate everyone It would be Important to put a note up front in the framework as to what it is and what it is not It and that's you know helps say it's almost a poison pill For legislators and and policy makers to say we'll just legislate this this framework And we need we need to be clear about about that in the third point on As far as moving out I would agree with what everybody else has said I think a lot of people have been moving out and The private sector in many ways has been moving forward and once again probably creating controversy the irony of the Snowden affair And everything that is unfolded with government may be that private sector will now take security more seriously We see a lot of chatter in the in the paper about lots of big companies thinking more seriously about Encryption and how they actually secure and that's really demand side generation Customers will now be saying how are you going to save me from government? And I don't think we ought to single out the US government here frankly there are a lot of other governments that Will be developing these kind of capabilities if shit if Or we'll be thinking about developing these capabilities and for social management or whatever the issue may be Yeah, I think one of the Few benefits of the Snowden affair is it's one revealed how In unsecure the global networks are and to perhaps inadvertently it's created an incentive that didn't exist before so Looking on the bright side, but it's a bold agency that would close its comments on February on Friday the 13th and so for that reason we'll give we'll give Adam the last word Thanks, and and I appreciate Jim hosting us I appreciate all of you taking the time I appreciate your organizations that have taken the particularly they've taken the time to come to our workshops And that includes I think Angela gets gets a t-shirt for coming to all five of the workshops So I just wanted to thank everyone for that and really you know, we wouldn't be here if we didn't have that sort of That engagement. I think Ashley gets a t-shirt to That level of engagement and folks really working and thinking about these problems in terms of the the questions I mean what the panel said I really agree with so much of it I do think in terms of helping people understand what it is and what it is not It's something we've thought about a lot in terms of what's the appropriate language, but and I also think that there is a place for Legislators in Congress to help us with that. I think that's one of the goals of the Rockefeller Thune bill I'll also say you know on the insurer the the conversation with the insurer insurers has been really interesting I think we've talked about there. There is there is a market there So it's a part of it will be leveraging that and they've really I thought the panel that we had in Dallas where we had the insurance companies talking about this and how they view it and How they view risk and how they help their customers manage risk was really interesting I didn't get opened a lot of eyes for the critical infrastructure companies that were in the room So I think that will be part of the ongoing discussion because I think they are going to look really closely at this I really liked what Angela said about Ashley's question about the APT. I think there are a range of things that an organization are going to do with that We did put out When you think about sort of next steps, we did put out in August You know to try to illustrate if the the framework was truly extensible We did put out some sort of threat models where we said, you know, this is what you're trying to protect here Some ways you can use the framework. So I think that's perhaps an important thing to look at But you know, again, we'd love comments on that topic anything we can do to help improve the framework will take And I'm moving out. I mean, I think we're ready to go and I think we are moving out I think there's a lot of great work going on both in the private sector and in government and You know, it's this is part of the process. It's important one and we'll continue to do this work What's the link for submitting comments? Have you go to nist.gov? You'll see it right on the right hand of the screen in big letters Please join me in thanking the panel and and thank you very much for coming to this event. Thank you