 And right now, I'm just going to hand you up to the speaker for Rob Joyce. Again, let's give him a big def go. Welcome. Good morning, everybody. Thanks for having me here. For those of you who don't know me, Rob Joyce. I'm from NSA, 29 years at NSA. One of the proudest things I did was I had the chance to lead the hackers of NSA organization known as TAO. But I've also worked on the defensive side of NSA as the deputy director for information assurance. And for 14 months, I was down at the White House leading cybersecurity policy for the nation. So it's with kind of that background, I'm going to talk a bit today about where we've been in cybersecurity, the things that are on top of my mind and the things we're focused on from NSA. So last year I didn't make DEF CON. It was the first one I've missed in several years. I was really disappointed because I thought there was something important going on here, and that was the election hacking village. And I will be out there tomorrow and make a chance to go see and learn and focus on that. That's one of the reasons I've really stayed, tried to stay connected with the DEF CON crowd and come here every year is to be a part of some of the creative ideas, the innovation and the things that are uncovered and learned here. It's not apparent to everybody in the outside world, I think why it's important to break stuff, why it's important to focus on it, find those flaws and then talk about it. And I know even today there was some discussion with the states about whether we should be doing the election hacking village or not. Believe me, there are people who are going to attempt to find flaws in those machines, whether we do it here publicly or not. So I think it's much more important that we get out, look at those things and pull on it. So that's, thanks. So the other reason I'm here is, again, to be part of this community. So what you will find, whether you know it or not, there are and have been and will be NSA people involved in DEF CON throughout the years. You know, I'm up here, there's no horns on my head, I'm a real person, I'm a technologist at heart. If you wanna see kind of the things that get me pumped and excited tomorrow at noon, I've got another talk, I'm talking about my house and a Christmas light display I put on every year, building absurd Christmas lights. That one actually might be more interesting, I shouldn't say that as you sit here about to listen to this talk, but that's a cool talk. So come over to DEF CON 101 tomorrow. It's not in the program, it's an ad. So I'm happy to be able to do that. So why am I and others at NSA? It really is to focus on that technology and think about serving the country and providing a way to make this place safer. Because as I talked throughout this morning, you'll hear and I hope you'll agree that there are some really bad things, some bad trends going on, on the internet and through cyberspace. And I feel like each and every day I get a chance to push back on some of that and really try to drive toward better uses of that internet community. So the year after Snowden, when it was a no feds allowed, I came to DEF CON. I didn't come on NSA's nickel, I didn't come as a fed, I came as Rob Joyce, but to stay involved in the community. I think it was interesting that a lot of people knew me, I'd done Meet the Fed panel the previous year. So they looked at me and knew I was from NSA, but I was still greeted and part of the community that was going on here. So that was pretty cool. It was notable that I think it was two years ago I was going up in the elevator to a sky talk and somebody turned and said, hey, aren't you that NSA guy that talked to our cybersecurity club? So all the heads in the elevator snapped around. They looked at me and I owned it. I said, yeah, absolutely. And we started a conversation with the whole group there. And that's what I want you to see. If you see me in the halls, come up, have a conversation. We're real people. I know not everybody at DEF CON agrees with NSA and our mission and the types of things that are said about us in the press sometimes, but I'm hoping I will show you that there really is noble intent. There's important things to be done and we're invested in doing those kind of things. So let me hop straight into it. For those of you who are unfamiliar with NSA, there's really two sides to NSA. There's a signals intelligence mission that's focused on getting out there and producing intelligence on threats to the country for decision makers, for the law enforcement, the military and trying to pursue that. And then there's the information assurance side that does cybersecurity for our national security systems, the highest levels of information that we in the US government has to protect. So with that as a basis of kind of where I'm coming from, I'll jump into where I think things are going. So I'll start here with this slide that talks to the inflection points of the technology landscape. Some big pieces of that one, wireless devices abound. They're really, really growing and exploding in the ecosystem. Another big thing is people are choosing to supply their data, choosing to supply their data to those massive social media sites. And what's happening there, it's funding much of the web and shaping the ecosystem with that big data analysis and the advertising that they can serve up. And so you got to kind of think about that in terms of the direction technology is going because that's what's funding a lot of this, right? And so if you don't acknowledge and understand those components, you're going to miss the boat about where technology is going. So if you look at the technology explosion from the 2000s to today, there's some really big points. In 2015, we broke through the point where half the population of the world is online. That is all the people in impoverished countries to the wealthy and sophisticated high-end technology companies, half of those people have access to the internet through some means. And in 2014, mobile internet surpassed the fixed internet. So we're living on that cell phone device. And if you think about it, those mobile devices go with us, they know where we are, they're connecting us, but it also, they're really powered by the back-end big data that exists in the cloud. And that is a feature and an aspect of where we're headed in technology that you've got to consider in how we're doing security. So for us in the US, more than any other nation, I would say, we depend on the availability, the integrity, the authenticity of the information on those nets. And really, unfortunately, those vulnerabilities to exploit those networks are being exploited by criminals in nation states. And so we've got to think about what we need to do to change some of those dynamics. I talked to the technology, now let's talk about the environmental changes. Key aspect for me is in the nation-state arena, the focus has moved from using the realm of cyber to steal secrets to using that realm to impose national power. Notable big incidents, last couple years, we had huge data breaches. So on the slide, I kind of moved from espionage-focused ops to a growing trend of large-scale destruction. And you all lived through a couple of the big notable internet events in the last couple years. I also think it's noteworthy how numb we've become, right? When a cryptocurrency exchange in Japan lost $530 million in a cyber theft. So this may be skewed because of the audience we're in, but how many heard of that in January when a half a billion dollars of cryptocurrency was taken? So it's a pretty heavy group in this room. If you ask the average person on the street, not as much. And I think it's incredible that a half a billion dollars walks out. If that happened and it was a truckload of gold stolen from a bank, imagine above the fold headlines. But Hoham, another cryptocurrency session was hijacked. They lost a half a billion dollars. We're really growing numb in that. There were also pretty concerning reports back in January that caught my attention. That was the Triton Mailware reports. If you think about the targeting of safety systems and industrial control in big industrial processes, that's activity that you've got to start to wonder about the judgment. That came to light not because somebody was really investigating and found it through extensive cyber sleuthing. It came to light because they weren't doing a really good job in that safety system and cause notable outcomes. So to me, the judgment of those people who thought they should be screwing around with a safety system without the knowledge and capability to actually manage it and shape it in the way that they were seeking just shows how dangerous they are in that. And it probably should scare each and every one of you. I know it scared me to think about the folks going into that. So we're also seeing countries use their national power in other ways. China is using its cyber infrastructure to establish a social control system. Social credit system that they're rolling out. So that's another way to use the power of cyber technology again for some of the national aims. And so for me, the way it compares and contrasts the free world to some of the totalitarian regimes is how we're using those elements of technology to either defend against abuses or prop up some of the social injustices that are happening. So you got to be aware of the new threat landscape. The tools are available, the data's out there, the intent exists, and that's intent, the trajectory of that intent is a piece that worries me. I think nation states and criminals opposed to some of our basic social order are having their way in the digital domain and I would lump the election hacking and other things into that as well. So make no mistakes. A big concern is the chance for miscalculation is huge and whether it's trying to influence our elections or intrude on the safety systems of industrial plants, that's something we as a community kind of have to rally against and deal with. So continuing on the cyber threats, I'll talk about four major trends that are on my radar. Criminals and foreign adversaries constantly prowling this digital domain. They push on America's digital infrastructure continuously and those of other places in the world. First area, high-end sophisticated actors. There's really been a fundamental shift in the nation state activities opposed to free and open societies, aggressive, disruptive cyber operations, asymmetric intrusions, inflicting damage, rapid weaponization of disclosed capabilities. These state-sponsored actors are continuously building on the techniques so what we'll see is some elite folks at the high end of that coming up innovating but quickly propagating that down at scale to other folks who can use it and turn it. People fear zero-day exploits but really hiding in the account of an unauthorized user is much more hard to ferret out over the long haul using authorized processes in ways that weren't intended and so that expertise of the high-end folks who can figure out how to insidiously get their selves into your processes, your data as an authenticated user makes it really hard to deal with. So we're seeing those big splashy cyber events with increasing frequency, kind of reinforcing that numbness and as I discussed earlier, just getting commonplace. Second area, the level of expertise is just decreasing. The quality of tools released, the ability to get and build yourself on the shoulders of others and get out there is really a leverage factor in enabling bad activity that's going on. So most advanced members of some of these overseas groups create the tradecraft and then again, bring others along unhinged to responsibly guide the use of those activities. Third area I'd highlight, the move from exploitation to disruption. So the last two years, a number of destructive attacks top of mind for me is Russia targeting Ukraine in their ongoing conflict. They wound up inflicting on the world with wanna cry, right? It was aimed at Ukraine, did a supply chain exploitation in the Ukraine, but it quickly propagated to the globe and if you look, a significant number of maritime ports were shut down, the shipping channels disrupted. That's real world physical impact. The supply chain of our modern businesses rely on those shipping channels to follow a predetermined predictive timeline and by shutting down those ports, impacting those things, it had huge impact around the globe. There was a non-binding resolution out of the UN in 2015 where a group of governmental experts said, hey, one of the important norms we have to establish is that we won't intentionally damage critical infrastructure. We've seen disruption of civilian power, we've seen financial institutions knocked down, we've seen a lot of preparatory activity in critical infrastructure and that stuff has no purpose other than preparation for these types of attacks and so that's a trend line of the cyber threat that continues to worry me. Fourth area, the growing use of information operations leveraging cyber intrusions and so that's the story of where you can get a hack, grab data, weaponize that data and then make outcomes from that data. Every single day we've got adversaries producing campaigns, pursuing campaigns to achieve those strategic outcomes and many of those campaigns have cyber components. When these people take our intellectual property in a big campaign, any given single theft is just that, it's a theft, but when you look at it as strategic intent over timeline, that really is a cumulative effect on our national economy and national security implications really are undeniable of campaigns that are looking to affect our intellectual property and business chains. So in America, we have the luxury of thinking about national security as an away game. I think by that I mean many of the conflicts, wars and activities in our lifetimes have taken place overseas. We've been insulated from that, but cyberspace has made it clear that we're no longer in an away game. The threat has really come to us and that as cyber professionals or people interested in the technology has to be a fundamental truth that you've gotta absorb and think about how that changes the way we need to react and work. So the new threat environment, not only has that threat changed, but the environment around it has changed. I talked in the very first slide about information technology game changers. The connectedness of our life is exploding. Sensors abound around us, if you think of the internet of things, the amount of data that's pulled together and we can expect criminals to go ahead and look at exploiting and weaponizing that environment. So it's an escalating threat environment that has me motivated to fundamentally look at how we protect ourselves and protect national security in that space. So I hope I've set the table for the background. I don't think any of that's hugely surprising, but what I wanted you to do was kind of walk with me thought to thought and get a sense of where that's been and where that's going. So at NSA, we're lined up with those two missions I explained earlier. What we found is cybersecurity benefits from the union of those two things. The signals intelligence mission goes out and gets unique insights into foreign threat actors, ensures that the national security systems are equipped to defend against those kinds of trends I talked about. Signals intelligence really is at the core of NSA's fundamental advantage in doing security and so we can take and discover threat intelligence on foreign adversaries. We can inform our partners, DHS and others to go out and take action in that space and both tactically counter the day-to-day malicious activities or support the entities that can go out in a more strategic environment and to create and defend against those who go after our freedoms and our institutions. So we focus on providing deep expertise to the US government on the targets, the technologies, the cyber defense tradecraft we have to work and there's a lot of partnerships in that. So in the US government, we approach these threats as a team, DHS provides a mitigation role, FBI does an investigative role and then we underpin both of them with support and the expertise in the nature of the foreign threats. I told you I'd walk a little through history too. Cyber security at NSA has been on this journey for a while. We've been working on the information systems and the comm systems of national security since 1953. So over 60 plus years, we've not only produced the security policies but we've done that hard work of deploying and developing the secure products and services that implement those policies and that's kind of a unique place to be not only just writing policies but being a practitioner of it and that's one of the things that helped me at the White House was knowing that what we do and how we do it and what others do against us on top of writing that policy was so beneficial. So in the 40 years ago, we were in the security business, it was communication security or comm sec. That was really almost exclusively about protecting classified information as it traveled between two points. So we wanted to keep it from unauthorized disclosure. We did that by building very secure black boxes, right? Goes into unencrypted, goes out encrypted, high grade encryption, careful engineering to protect that information. In the 70s, even the early 80s, the advent of the personal computer came around. We had a new discipline for computer security or compu sec. That was still focused on protecting the information from unauthorized disclosure but it also started to address additional challenges. The injection of malicious code or the theft of large amounts of data on magnetic tape, it was really a transition into that new information realm. We saw a big compu sec contribution back then. It was the rainbow series of books. These were descriptions by the government telling everybody how we could protect trusted systems, evaluate them with guidelines on things like passwords, audits, network databases, risk management. It was stuff in the 70s and 80s we were talking about doing. All the same problems we're talking about today, right? I think it was the time where we were first questioning how you could be sure a computer was doing what it was supposed to do and nothing more. That was a surprising question back in that day. It is not today, but it really was a fundamental pivot at the time. We realized separately we're dealing with compsec on one hand, protecting information as it transited and compu sec, the computers and the security there. And doing those separately was no longer feasible. So we started working info sec and quickly realized that that also was not enough that we had to worry about unauthorized modification of the information data integrity on top of just the confidentiality, positive identification, authentication. And it became really important that after a transaction somebody couldn't say they were not part of that transaction. So the non-repudiation portion of that. I think finally eligible receiver was an exercise the government ran in 1997. That was a key moment in the US government's recognition of the vulnerabilities of cyberspace. A red team playing as foreign adversaries were able to target significant US critical infrastructure and it shocked the national level leadership. And you saw a big pivot on the way we do security. And even to the point you can probably point at that and say that was one of the fundamental justifications for the establishment of cyber command back in the day. So as we evolved we got changes even beyond the information assurance. NSA as I said, leveraging the power of intelligence and the defensive folks. We integrated those two missions at NSA. Information assurance didn't go away. It really is an integrated whole of effort where we inform from one hand and then act with the other. So the SIGINT system was also set up to work in operationally relevant timeframes where we had to produce intelligence for the war fighters. Time and speed mattered. The information assurance mission was often about standards and evaluations and other things that weren't time bounded. And so that's where we are today is cyber security on top of information assurance has gotten to the point where it needs to be in an actionable timeframe. So let me pivot over to back over to the nation states. There's been a lot of talk about cyber norms. I mentioned earlier, 2015 there was a UN group of government experts that recommended a set of cyber norms. Those norms were non-binding but they were a significant step because nations came together. They debated, revised and endorsed those norms. But norms are only norms if people follow them. So norms are only norms if people agree and are executing that way. I think though with all the discussions about norms people should generally agree that most nations are behaving reasonably on the internet. I think there's four notable exceptions. I'm naming names here, calling them out. And I don't think they'll surprise anybody at this conference and I deal with them on a daily basis to the point they just flow off my tongue. Russia, China, Iran, North Korea, it's that easy. Russia, certainly their use of cyber and election interference, information operations, military operations doesn't need an introduction. Russia has used cyber tradecraft since at least 2008 in a big way when the Russian incursions into Georgia were accompanied by a denial of service attack against the Georgia internet service. We saw that internationally as a component integrated with the physical activities they were taking up. So at NSA, we've had a front row seat to a lot of these activities. Our former deputy director, Rick Legit, talked about a Russian intrusion into the State Department. He called it hand-to-hand combat. Because unlike a lot of places where you get an intrusion and you've got an incident response and the first sign that somebody's following up on it, the attackers kind of disappear and lock and run. What we saw were these guys retarget the system administrators, try to lay down new tools, new techniques in places that they hadn't been to hold that ground. So it really was an interaction. It was kind of bold and audacious as they fought to stay in that network. And what we see is that is often the same way they as a state have responded in the physical world. So they mimic that behavior. A good example of the insidious nature of some of that intrusion, I would point to the May 2018 VPN filter, where Russia actors were targeting SOHO routers. We saw APT-28 Fancy Bear called out and the Cisco-Talus reports estimated hundreds of thousands of devices were affected worldwide. So that malware had really creative ways of interacting. It went out and it was trying to get Instagram pictures to decode out of the metadata, the IP address that it should use as command and control callbacks. And if those Instagram photos were unavailable, then it had a domain name to knowall.com that it would call out to and look for instructions. So between the government, the commercial industry, that was disrupted, those channels were disrupted. And you even saw FBI warn everybody, reboot your devices, disrupt the Russians command and control of this large army of bots we were gonna see. This was effective at knocking down that command and control, but, and that's the big but that I haven't seen really talked about much. It's that there was a persistent stage one on all those routers. So what it did was knocked it down, if it was at a stage two or stage three implant, it knocked it back to stage one, which was power reboot persistent. And then at that point, it couldn't call back out to those two, via those two methods to reestablish command and control. But the Russian malware is actually still there. It's still on those routers. And if you know the secret handshake packet wakeup knock, you can go in and you can still talk to and control those routers and put a stage two or three back on. So I guess I'd ask, what do you think the odds are that the actors in Russia who put those down kept a list of the IP addresses that they put their malware on? I think it's pretty high. So what we really need is we need industry looking at it in an easy way, because again, these are consumer devices. These are deployed that people can check for that persistent backdoor and that they can knock it off their machines in a real way. That's the kind of thing we're up against. And that's what I'm talking about in the way that they're massively controlling and chasing big parts of the internet. I think what we and the government did with that FBI warning, that was a battlefield medic putting a tourniquet on something. We need to get him into the hospital and do surgery and get him out of there. So I'll leave that as a thought. So summing up Russia, they seem the most willing, perhaps the most skilled of our adversaries in seeking strategic outcomes and in and through operations that involve cyberspace, but it doesn't have to just exclusively be cyberspace. When we're talking China, the immense volume of operations is the thing that I think comes to mind for most people first. So we see them stealing intellectual property and back in 2015, we got together. Our government and theirs to reach a commitment that we wouldn't steal for commercial profit. We would start some cyber dialogues about the security in the space and FireEye said we saw 90% drop off in cyber intrusions after that agreement was reached. From the government, we don't have the specific stats but we saw a similar comparable drop off. But however, kind of like other intrusions, China continues, what they did was certainly became more refined and even improved their tradecraft after learning from some of those compromises. I would point to recent activity like the cloud hopper intrusions where they're targeting managed service providers to get in and underneath a whole wide array of businesses and then be able to move through those MSPs to exploit the businesses above them as a sign that they're still in this space. So we're focused on cloud hopper with other allies, other like minded nations and trying to work out ways to get them out of the infrastructure layers that gives them big advantage. Same thing, the social credit services that they're setting, the social credit system highlights the totalitarian natures of the way they look at the internet and the technologies and even a long-term focus on the technologies that are gonna be pervasive, whether it's 5G or artificial intelligence, quantum, those things show that they're trying to stay ahead and use that tactical advantage. And then even perpetuating a domestic legal regime where they have to have intellectual property transfer from companies who wanna do business with or in China. So participating in the Chinese markets using their national laws to kind of edge people in ways that are not exactly cyber but aimed at some of the same outcomes. Again, theme here that a lot of countries are looking at more than just cyber operations but cyber in part of a bigger, larger activity. So Iran, as US and Iran entered into the nuclear agreement framework talks, 2015, Iran really seemed to curtail a lot of the destructive cyber activity against Western interests. They still went against Saudi, Saudi Aramco and other big issues there. But what we saw was denial of service campaigns that had been disrupting the banks and targeting the banking industry dropped off a lot. They even came after at one point, a Las Vegas casino, where the CEO had made some public statements about wanting to attack Iran. And so they came back and defaced the casino's websites. So what we see is Iran willing to use their cyber influence in real world operations at a level that is targeted and malicious that we don't see other states behaving in similar ways. So I think of note, when bilateral relations between Iran and Saudi Arabia decreased, we think that was a major factor in the January 2017 dated deletion attacks in Saudi. And so as we move to a point where the US has just reimbursed sanctions on Iran, there's a lot of focus on how are they going to respond. And one opportunity they have is certainly in the cyber realm. So I know as NSA, we're gonna be very vigilant and watching closely in that space. The world needs to join and be ready to push back if that's a tactic and a way of acting that they're gonna take up, we've gotta be on guard. And then over to DPRK. So I would say DPRK has been some of the most consistent. They haven't shifted. They've always seen cyber as an effective tool of state power, but what we have is confirmation that they're going to use cyber in every strategic activity they have. When we put Michel defense systems in South Korea, one of the plans of the US government was to expect cyber probing, cyber efforts from DPRK to come at those systems as they went on the peninsula. I think the disruptive attacks they've undertaken against Sony well understood, but they've been hitting South Korea, banks, infrastructure, government for several years. We would expect that to continue. They don't have a lot to lose and there's not a lot of levers we can use in that space to discourage it. The biggest place that they stand out amongst nation states is looking to steal hard currencies. They hit the Bank of Bangladesh for $81 million in a theft through the SWIFT network. They continue to do crypto currency thefts, continue to target banks looking to get hard Western currency. And it really shows the way that as a nation state, they're engaged in criminal behavior. So where do we need to be? Cyber security really is a team sports. We in government absolutely recognize we can't do this alone. If you look at strengthening cyber security, it can't be something driven out of Washington DC. I think the greatest progress happens from the bottom up, not the top down. And that as I talked about the election hacking village and other things that go on in forums like this and in companies in our industry, it's a place where we've got to be prepared and figure out how to let that drive from the bottom up. NSA has some unique expertise and capabilities that we track and coordinate across the overall federal efforts. There is sector specific knowledge across the federal government, places like Treasury who understand the financial industry, energy who understands the power industry that we can supply cyber expertise to and they know domain issues. So that coordinated cross sector government industry to protect is vital. A good example of where industry brings knowledge that we just don't and won't have. Microsoft recently highlighted intrusions against a senator's email. And what that shows is these companies, like Facebook taking down Russian troll accounts, they are going to know things on their platform and be able to see things that the government couldn't and shouldn't. And so there's got to be this connection as we work against those government nation states that are behaving badly at the federal level and they see some of the things on their platform. There's got to be this virtuous cycle where there's engagement. I think in terms of social and our society, the Russian election meddling, the Russian divisiveness they're trying to sow is top of mind. I think many of you heard the director of NSA, General Nakasone, set up a Russia small group. He brought together people across Cyber Command and NSA to focus in on this problem and use the intelligence capabilities we have. I think it will be a key component of informing DHS who can contract to the state and locals what threats they're up against that, sharing relevant threat intelligence with DHS, getting into the multi-state information sharing and analysis center, the election, ISACs, other places like that. So when it comes to Russia, the recent policy pronouncements really have made it clear that any attempt to go after and interfere with our elections, successful or unsuccessful, really is a direct attack on our democracy and it's unacceptable. So I think we're resourcing that. The national defense strategy laid out a role for DOD and you're gonna see us resourcing and pushing into that. So I've talked in the past about the intrusion lifecycle. I laid out as an attacker, the mindset of an attacker, how you go against a well defended hardened target and you can argue about the way stops in between but you basically have to cascade through these activities to find your way in and exploit and make use of systems. So there are points on these steps that we've gotta think with a defensive mindset about breaking up, interrupting and disrupting their ability to have success. So it's worth your time if you're involved in any sort of defensive activity or you even wanna protect yourself at home to go after and look at this kind of model and think through where are your weak points and where can you harden? Because what you need is layered defense. One thing I can't state in strong enough words, really. The basics matter, right? The basics totally matter. Ignoring the basics gets you caught up in massive cyber casualty effects like wanna cry. Ignoring the basics, let a nation state adversary tear into you, get to their objectives easily and likely root into the point where you're gonna be hard pressed to know if they're still in after you remediate. The basics give a beach head for much harder follow on exploitation. Ignoring the basics will give you a beach head for that harder to follow on. So I think regardless of the changes in technology and there's some really cool innovative defensive things coming on, if we don't have the discipline to stick to the basics, the cool stuff really doesn't matter. Past, present, future, it's going to rely on doing the basics because no matter how skilled an adversary is, my time in the past at NSA, you only use what you absolutely need, right? Attackers are gonna limit the sophistication of what they're doing. So in that, there's a host of things that we need to do where it runs from no kidding, get that multi-factor authentication out, it matters. Enabling, logging, doing the analysis, paying attention. And for me, I harped a lot in the USENIX talk I gave years ago about knowing your network because attackers don't care what you think you have connected. They go after what you have connected and if the shadow IT in your infrastructure gives them a way in, they're gonna take that, they're gonna use that opportunity and go on. So the reality of where we are is that we're all living on commercial technology. In the past, the US government used to build black boxes, used to have isolated government networks. We now live on that same commercial technology that banks, industry, critical infrastructures, and we as citizens live on. And there's a reason there's a patch Tuesday, right? The day that stuff comes out is the day that people start picking at it and poking at it. It's going to over time erode and people are gonna learn flaws. So it's important we keep up with that. It's not that this tech is either good or bad. We've gotta have a way where we deal in that gray zone and if you're a static target, if you're not improving, evolving, and moving with technology, you're gonna fall. So don't be that victim that doesn't move and gets to be a victim of technical debt. So for me, asymmetric advantage is the private sector and the government working together in that because what we've found is we used to have a lot of expertise on this stuff. You think cryptography, the government had that monopoly. Today, we do not. A lot of the innovation, a lot of the brain power is gonna be on the West Coast. It's gonna be an industry, it's gonna be in private companies and increasingly, it's gonna be international too, right? There's not gonna be this huge dominance in Silicon Valley. We're gonna see what the investments going on in Asia and elsewhere. We're gonna see others rise with really important things that they can contribute. So a coordinated approach is important. So I'll leave you with a thought here if you like this challenge and it's gonna be a challenge for years and you wanna be part of doing hacking for good, you wanna see using some of these skills to defend the country. Come to NSA, you can be part of that. I can tell you, you can come back to DEF CON and not worry other three letter agencies, maybe waitin' here for you too, right? So we can do that hacking stuff in a legal way. You gotta be clearable. So sorry, I know there's a heavy foreign contingent in here, but we'd be happy to talk to you. Go to the NSA website. So where cyber going? It really means that we've gotta innovate. We've gotta be engaged every single day. We've gotta counter the people who are engaging us at a level below the threshold of war and the people are gonna make the difference in that. So I've run past my time, they're about to give me the hook, but I'll wander, I think we're gonna go to the chill-out room if somebody's got a specific question. Thank you for your time and attention.