 Yeah, it's a little deep. Disappeared to the sofa. Don't do that, Michael, we need you. Up front and center. So good morning, it's really a pleasure to be here as part of this important conference that New America Foundation has put together here. It's quite an impressive lineup, and it's really a pleasure to be a part of it. They asked if Michael and I would just kind of unmoderated, which is dangerous, skit up here and have a conversation, both about our time in the Obama administration, and some thoughts going forward. And so we thought we would do that and follow the instructions of our sponsors. So, yeah. We've gotten used to taking orders, now we're out in the real world, we don't have to, we're going to anyway. So I'm gonna start by introducing Michael Daniel, who really doesn't need an introduction, but as a reminder, who was most recently, of course, the cybersecurity advisor for the president at the White House from 2012, until noon on January 20th, 2017, where he led, of course, the policy development and helped with the overall approach to cybersecurity across the United States government. And he's gonna talk about some of the things that he's most proud of accomplishing in that job. Prior to that, he spent 17 years at the Office of Management and Budget, which really set him up very well to come into that position in the White House to understand that it's not just about policy and strategic direction, but to get the execution right, you've got to understand of the budget and how that works. And so Michael was really well positioned for that job. And so I thought I would start by asking, what are some of the things, as you think back on it, this is an opportunity to talk about some of the things that you're most proud of or feel most satisfied about having accomplished? Well, thanks, Suzanne. And I appreciate the opportunity to be here and chat with Suzanne, who is always one of just a fabulous partner in during our time in government. And I think the thing that I can say is that we truly moved the ball forward down the field in a number of different axes. And we are actually considerably better off than we were. That's not a message that you actually hear very much in cybersecurity, because the problem is so big and the threats are growing and the threats are so dire in many ways. But the truth is we are in much better shape now than we were seven or eight years ago in order to both as a society and as a government be able to tackle the cybersecurity problem. I think to highlight some of the key things though, I would say, obviously I have to start with the NIST cybersecurity framework. That was truly an innovative and landmark development in this field. And it continues to pay dividends through today and will continue to pay dividends going into the future because it really gives non-cybergeeks a framework to use that term, but a hook to actually hold on to and think about how to deal with cybersecurity from an organizational standpoint. And that's just absolutely invaluable out in the private sector when you're dealing with people who are smart, they run businesses, they're really motivated, but they're not cybergeeks and they don't need to be. And the cybersecurity framework really helps with that. I would also say the legislation that we did work with Congress to get passed, that was very important as well and Congress deserves a lot of credit for stepping up and getting that across the finish line because that was no easy task. The tools that we put in place inside the executive branch from a policy standpoint. So the various presidential policy documents that we put in place, the executive orders, like the one that provides the authority to impose sanctions on malicious cyber actors overseas. That was a completely new thing. And we finally exercised it, right at the end of December of 2016, right before we left. I would also say in the international space, the partnerships that we developed with our Five Eyes partners, the much deeper partnerships there are really impressive and there's a lot that we're working, that I believe we're continuing to work on. But also some of the arrangements we were able to reach with some of our countries that are competitors, including the agreement we reached with China to endorse the cyber norms and to agree to not use their intelligence capabilities to steal intellectual property through cyber means. That was a pretty big agreement to reach with them. And believe it or not, even with the Russians with the hotline that we set up and the confidence voting measures we agreed to and agreeing on the norms of behavior that we have been promoting around the world, getting agreement on that within the US government was a challenge and then trying to establish that globally has been, I think, a big advancement in the conversation. So those are just some of the things that I could highlight for you in a number of different areas as we have really improved the capabilities that the US government has. Obviously another one of those would be what actually happened at DHS. And I think through Suzanne's leadership but also with a number of the people that worked for Suzanne, including Phyllis Schneck and Andy Osment and other folks that are still there, we've made a lot of advancements. So, and I think DHS is in a much different place. So I could ask you the same question, Suzanne, sort of what are you sort of proud of from your time over there? So you're absolutely right. And among those who are still there is Jeanette Manfred, of course, who you'll be hearing from later today who is performing the duties of the Deputy Undersecretary at NPPD very competently. And I think what we did, Michael, was to take the strategic and policy advances that you were able to make and turn those into operational advancements. And vice versa. I think the, so for example, PPD 41, not only advanced the ball in terms of delineating the roles of the various federal players in cyberspace and helped explain how we will coordinate in the event of a significant cyber incident, but it also reflected progress that we had made at the working level in coming together and collaborating much more effectively than had been done in the past. And so you could never have gotten, we tried for years, as Michael would testify to, turn the so-called bubble chart, which was a picture that tried to roughly capture the delineation of responsibilities into text and failed, but ultimately under Michael's leadership and with goodwill and hard work across the board among the players at FBI, DOD, NSA, DHS, were able to come forward with this PPD 41. So again, a reflection of progress in addition to providing a platform for greater progress in that important collaboration and great collaboration cooperation with the sector-specific agencies, particularly Department of Energy and Treasury, for example, in putting together exercises and coming together to understand interdependencies. I would say the deployment of the tools that DHS put out there, Einstein, which of course is monitoring civilian.gov networks and doing both intrusion detection and intrusion prevention and provides a platform for new tools as the private sector innovation creates them. Einstein is a very good platform for implementing those tools across the government that we did not have before. CDM, which is a suite of tools helping you understand your network to begin with, what's on it and then what's happening on it. And those, importantly, provide data across the federal government and a dashboard and provides a common operational picture across the.gov, which was not there before and that's critically important. We know that as we go forward in cybersecurity, data and data analytics is going to be key and that's a big part of what these tools across.gov do and we use that information to help the private sector. Automated information sharing, sharing of cyber threat indicator information, which is the government's version of what Michael is now doing in the Cyber Threat Alliance and Michael, I hope you'll talk with us a little bit more about that, but that's another validation of how important this is to share these fundamental cyber threat indicators in real time in an automated way so that we can have the equivalent of a BOLO alert for cyber, but taking it to the next step with that data analytics to be able to detect and stop things we've never seen before but which walk like a duck and quack like a duck, right? And then finally, the holistic approach to critical infrastructure protection, recognizing that you're never gonna solve the cyber challenge through IT alone. And I know Tom Bossert gave a speech recently in which he talked about the importance of a holistic enterprise risk management approach to this and that as any of you who've heard me speak for longer than five minutes know is something I feel very passionate about and we were privileged at NPPD to have both physical and cyber all hazards approach to critical infrastructure which allowed us to understand how to assess priorities for cyber based on its impact to the business or to mission essential functions because we had people who understood the business and understood already what kinds of disruptions could be existential and how to mitigate them, not just through IT. So and that played itself out in a number of contexts. So those are some of the things that I am most proud of that we were able to make advancements in that both institutionally and in the way we did section nine the catastrophic consequences entities the way we did the electricity sector coordinating council the financial services, analytic and research committee FSARC the way we did high value assets the way we did the election infrastructure our approach to making sure that we helped protect our elections against foreign interference all of that brought our physical and security and enterprise and business folks together. And I know that with all of that we look back on our successes there are also some frustrations and disappointments and things we almost got over the finish line but not quite. Michael, I know you've got some of those. Yeah, I think to me the one of the biggest frustrations was well ironically we made a lot of progress on the federal civilian cybersecurity side. We got in place certain things like creating a government wide CISO over at OMB and beefing up DHS's capability in this area. Certainly we got a lot more focus at the cabinet level from folks on cybersecurity particularly post OPM. Nothing like something like that to focus the mind. But yet it still was just incredibly hard. And we just did not make the progress on reforming what needs to be done on the federal civilian side that we needed to. And that is really a project still very much in progress. And one of the lessons, one of the fundamental lessons that I took away from that and I see this over and over again is that it's never the technology that's the hard part. It's all the other stuff. We didn't spend years arguing over things like, we didn't spend years working on the technology behind Einstein, we spent years arguing over how we could legally deploy it. That was where the grind really was. And so that was a big frustration. I think one of the other frustrations that I had is we still continue to have a disconnect and a sort of focus on cyber versus cyber. And particularly in people tend to think of if we get hit with a cyber event we've got to go back with a cyber attack of some kind and that the only time you would ever use your cyber tools is against a cyber adversary. And neither of those two things is true. And sort of breaking out of that paradigm and really trying to think about how we actually use these tools in a way that is lawful in a way that is consistent with our values, but nevertheless makes use of those capabilities is I think really, really important. And then the last thing I would just say is we continue to have the sort of Hollywood effect of everybody thinks a lot of this stuff is easy because in Hollywood, okay, we've now hacked into their system and it's never completely that easy. It takes a lot more time and effort to do those sorts of things. And as a result, we often end up spending time talking about low probability events that might have a very high impact but not focusing on where the real threats are. And I know you share some of those same frustrations. Absolutely, and you talk about, we think of it as cyber versus cyber. And one of the ways I talk about that is we think of cyber in a stovepipe still as if you can put it over here with all your cyber ninjas and understand and solve the problem that way. And again, I think that is just, we're not going to succeed in that way, both in terms of understanding the nature of the threat. Again, I go back to prioritize your cyber efforts. You've got to understand consequences and those consequences are not just going to be to your IT network and system. It is then what are the consequences to your mission essential functions if your government or to your business operations if you're a business. And your IT people are not gonna understand that for you. Any more than the electrician that you call in to fix your problem with your electricity in your building is gonna understand the consequences of you losing power. You've got to have your business folks at the table and very much integrated part of that conversation. Similarly, on the back end, as Michael says, cyber doesn't just, a cyber incident doesn't just have to be addressed through cyber whether it's in an adversarial perspective or simply internally. A lot of times the most cost effective way to mitigate the consequences of a significant cyber incident will be to put in a hand crank. It will be to have some sort of physical redundancy. It will be to have plans for how you will mitigate the consequences of a successful cyber incident and not just how you're going to respond and recover in your IT network. In Ukraine, when they had the cyber attack on their electric grid, they did not, their cyber ninjas did not get the adversary out and the lights back on. The people who got the lights back on in six hours in Ukraine in the dead of winter were the guys who knew how the grid was laid out and where the breakers were, it got in trucks and went physically and put them back in place. So putting cyber in a stove pipe is gonna be a big problem. One of my greatest frustrations was that I failed to institutionalize that holistic approach more broadly across NPPD. Again, as many of you have heard me say many times, when I came in I thought, NPPD National Protection Programs Director is a terrible name. We ought to at least be able to come up with a name and change the name to something that says what we do, which would be Cyber and Infrastructure Protection Agency. You would think that'd be pretty simple and straightforward, no, require congressional action, which was not forthcoming. We had, despite years and years of effort, we had also developed a plan to bring greater unity of effort across NPPD to bring this fully integrated approach to both dot gov and the dot com world that also required some congressional action which was not forthcoming. So those were two of my greatest frustrations. And finally, to again help with that unity of effort, really was trying to get NPPD out of being in 11 different buildings in the National Capital Region into one. Sounds like a pretty petty bureaucratic issue, but in fact, those of you who have tried to manage complex organizations will appreciate how important it would be to be co-located. So those are a couple of my frustrations and begins to point toward advice I would give or things I would like to see in the next administration. And what that might, might that be, Suzette? So I do think, you know, it starts with some don'ts, right? So don't give the entire cyber mission to DoD as I've heard some rumblings about out there. I don't think it's a realistic, I don't think it's likely to happen and I'm particularly encouraged again as I say by Tom's remarks the other day. But I think it's really important that we continue to have a civilian agency and it sounds parochial, but I do think it ought to be DHS who is really leading the way in particularly the relationship with the private sector. Years and years of protection of privacy and building trust with the private sector as well as understanding their business, which is again a key part of what needs to happen here. Don't stow pipe cyber, it's a lot of talk about creating a cybersecurity agency, pulling the various cyber parts out of the various places in the government with the idea that somehow this would alleviate the coordination and collaboration problems that we have. And again, I think it's a mistake. Law enforcement is not the same mission as the preparedness and mitigation mission and it's not the same mission as foreign intelligence collection. So these are distinct missions and as I say, I think we've made great progress in the collaboration and coordination across the board and I think it would be a mistake to stow pipe cyber into a separate agency. I was pleased to see the budget it indicates an increase in funding. I do think we need to put our money where our mouth is and we need to increase resources particularly again for the civilian side for DHS and for NIST. I think the work of that standards body is incredibly important and really a critical place for us to focus. Finally, don't think of cyber as one thing. It reminds me of the early days with WMD, weapons of mass destruction. We talked about weapons of mass destruction as if that was one thing and we couldn't figure out why we couldn't really make progress on it until we finally broke it down and said really chemical weapons are not the same as nuclear, not the same as biological and until we broke it down into chemical, biological, radiological nuclear that's when we began to get some traction and make progress. Cyber security similarly, we need to get more granular and more sophisticated in the way we think about it and talk about it. Intellectual property theft is not the same as an attack on industrial control systems and lots of other things in between and until we break that down and make dedicated efforts in those directions and we've begun to do that. Of course at DHS, we set up a separate cert, computer emergency response team for industrial control systems and they are the best in the world but those are some of the directions I would go. Michael, I'm gonna give you the last word before we go to Q&A. Well, I would say that I agree with all of your pieces of advice. I would even broaden it out a little bit of particularly what you said which is we have worked very hard to establish the roles and responsibilities across the federal government and you can spend your life in Washington D.C. refighting those battles if you want and it's just not a very productive use of time. We constantly talk about how in the private sector you need to integrate across your organization and get out of having cybersecurity be something that you relegate to the geeks in the closet and integrate across your entire organization. The federal government is no different. We need all of the different capabilities that all of the different departments and agencies that have a role in this can bring to bear and the goal should be not to try to like to the nth degree define all of who does what to whom but to actually get working together as a team. So that would be my number one piece of advice. Second, you can't do this by yourself. You need to be in partnership with industry and we still don't have a good vocabulary for what that relationship looks like because it's not a contractual relationship where the government's buying goods and services. It's not a regulatory relationship where the government is telling the private sector what to do but it's actually a true partnership and so working that out and what the relationship is gonna look like is really one of the key tasks. And then internationally we've got a lot of unfinished business about exactly how we're gonna operate in this new domain. What are the rules of the road going to be and what kind of behavior are we going to consider acceptable, what kind of behavior are we gonna consider unacceptable and how are governments actually going to operate together. And then the last thing I would say in this area is just as Suzanne said, we've gotta break down the problem. How you deter and address hacktivists is very different than how you go after nation states and their intelligence and other military capabilities. Those are very, very different problem sets and until we move past that, we're not gonna be able to really come up with some credible approaches. Great, thank you. Not surprising that we're in agreement on so many things, right. But we do wanna give you an opportunity to ask some questions. I think there are microphones. Yeah, so we've got a question up here. Hi, this is Rick Weber at Insight Cybersecurity. So both, for both of you Suzanne and Michael, so you talk about the NIST framework. You both were very involved in the development of the framework. There's a lot of talk now about metrics and audits or the next steps for the framework. Can you talk a little bit about that? Yeah, I mean, I think that one of the clear unfinished pieces of business we have more broadly on cyber security is the issue of how do you measure good cyber security-ness. Right, how do you actually say as a business, my marginal dollar is best spent in what part of the framework? Should I spend more in response, recovery, more in detect, protect, identify? How do you even begin to figure out where to spend your marginal dollar? And we still just really don't have, there's some budding ideas, but we don't have really good agreement on what the best metrics are for organizations to measure their cyber security and how they should allocate resources, either at the organizational level or frankly at the society level. And again, to be to dead horse, but I think again, part of that problem is that we are looking at cyber in a stovepipe when we try to do that. So I think coming up with metrics is gonna make much more sense if you're looking at the entire, across the enterprise and starting with consequences. So what you really are, when you're looking at, if you're just looking at how do I measure my IT security and how do I prioritize my IT security, that's gonna be very difficult because we know there's no perfect security, so then it's how much is enough and if you're just looking at the IT network system, you're gonna have a hard time. But if you start with what are the kinds of disruptions that could really have a significant impact on my department or agency or my business. Now, which of those could be caused by cyber? What does that tell me in terms of where I ought to be spending my marginal dollars? Are there, for example, are there other ways, non-IT ways in which I can address that potential consequence, devastating consequence, right? And then your metrics become, have I reduced the risk to my department or my business? Have I reduced the potential for consequences that could have a very significant impact? And it's not just about what you've done in your IT. Ann Marie. What, okay, thank you. So I love your final point about breaking it down and as somebody who's a veteran of WMD or the fights over WMD through the 80s and 90s, I wonder if you have thoughts about what those categories might be. Because we took WMD and, of course, we made it nuclear, chemical, biological, we broke those down. This is one of the areas where sort of just conceptual thinking can really help to create a typology and then people can work within it. Get us started. I mean, from my perspective, I think there's a couple of different axes that you wanna break the problem down. One is the actors, who are we talking about? You know, and some broad categories there even are just the hacktivist groups that are promoting various ideologies. There's terrorist groups. There's the criminal organizations that have moved in here in a big way, but of course they're motivated by completely different things than those first two categories. And then there are the nation states. At least at a starting point, that's I think one set. I think to Suzanne's other point, then you also have to look at then what are the target types that we're talking about? What is the activity that's actually occurring? Is it the theft of information and the different flavors that that can take? Is it disruption of societies? Is it disruptive activity or is it destructive activity on physical systems? What kind of impact are you talking about? So I think you have to start at least with those two different axes for how you think about the problem. And there's probably a third and fourth dimension that could be broken out here as well. But I think at least you need to start with that. Right, and one of the things that I, when I talk to boards and CEOs about how to think about this as non-cyber experts and the consequences is, think about data reliability. Can you rely on the data confidentiality and access to that data? And how a failure on any one of those could disrupt your organization? And then finally, that industrial control system physical aspects of your organization that are networked and rely on are cyber dependent. So we are out of time, but Michael, thank you very much. It's great to see you again. And we didn't give you as much time as I intended to talk about the cyber threat alliance, but really important work going on there. And thank you, Anne-Marie and New America Foundation for the opportunity to have the conversation. Yeah, thank you. Thank you.