 Good morning. Oh, man, tired. I know. It's early. It's early. Thanks for coming out. My name's Tony Sager from the National Security Agency. Yes, I'm a Fed. I look like a Fed. I talk like a Fed. Dress like a Fed. It's a long career for me in federal service. I'm proud of it. Thanks for coming out to see me today. I know you've got a lot of options out there. You could be waiting in line for your badge. You could be at one of the other talks. I also know it's kind of early for some of you guys, so I doubly appreciate you coming out to see me talk today. I just came from speaking to Black Cat. Did anybody catch my address over there, the keynote address? This is all fresh and new, except for a couple of folks. I'll try not to bore you with something I said before. There'll be some repeats. I'll talk a little bit more about some other things that I think is more appropriate for this audience. It's a real honor to come out here. Nico Cell asked if I would first come to Black Cat and then speak here at DEF CON also. As you know, the NSA guys don't get out that much. I get out more than most to talk to people about the business that we're in. I'm going to tell you a little bit about what we call the information assurance business at NSA. This business of dealing with the world of vulnerability and network security. There's all kinds of names that have developed over the years with it. It's a really challenging, important national problem. You guys are part of this community that deals with this kind of stuff. My goal is I'm going to tell you a little bit about our mission and what we do, the kind of business I personally have been in for a long time. I hope that you'll be number one surprised that the NSA has been in terms of the work that we do and the stuff that we give out into the community. I also hope that by the end of this you'll figure out that we might have a lot more in common than we do different. Aside from the way I dress, of course. Just for a little bit of context, I really am a lifer at this sort of business. I started at NSA. It will be 30 years next month. I actually have a little over 30 years of federal service. I spent a couple years in summer jobs for the Department of the Army as an mathematician, kind of junior guy, mailroom clerk and all that kind of beginner sort of work that you do when you're starting out. And then wound up at NSA starting in September of 77 after I got out of college as a young mathematician. And little did I know where I was getting into, very exciting career, a lot of great opportunities. And the context is kind of this. In fact, the world is so different now than when I started. In fact, the world I started in, a lot of you were not even alive at that point. The world of technology was entirely different, right? When I started, my kids love this, I took the only computer course that my college offered. I mean, that was it. There was one Fortran programming course, linear programming for math geeks, and I thought, man, who wants to do this? Punching cards, I know, that sounds like a dinosaur stuff, right? Punching cards, carrying card decks, trying to talk the guy at the computer center into giving me more than two runs a day, screwing up the JCL cards all the time. It was a really different kind of business. I thought, who would ever want to get involved with this? It was a professional life at NSA. It was a lot different. But the world of security and secrecy was so much different then. So the way I was offered the job at NSA, I got a phone call. My summer job at Aberdeen Proving Ground was running out. It's about middle of September, and I had applied a year before. That's how long it takes to wind up working in a place like mine. I got a call. If you can report in two weeks, you've got a job. Great, because my job ran out in about two weeks. What is this job? It says, well, communication security intern. I said, great, what is that? And he said, well, go to the Encyclopedia Britannica. Back when there were paper, big books, you know, a whole shelf. Look up the article on cryptography. And it was written by a guy, again, I think this predates most of you, a guy named David Kahn. We'd written a book called The Code Breakers back then, and it was the first really large-scale public discussion of cryptography and the mathematics behind it and the historical figure in this article for the Encyclopedia Britannica. So that's literally all I knew going in. And so 30 years now, close to at NSA, so in my entire career at NSA, and what I do at NSA is not what you think of when you think of NSA. I spent my whole 30 years in the defensive business at NSA, which is a bit unusual for a career like mine. In the defensive business, and many of you probably didn't even know, NSA had a defensive mission, they were vulnerability finders, the people whose job it is to figure out what's wrong with U.S. information systems preferably before somebody else does. So my early days were as a mathematician, doing analysis of systems and the mathematics, when I was a mathematician, then mathematics is obviously the most important thing going in security. And then when I switched to computer science, obviously computer science is much more important than the mathematics. But in the early 80s, some of the books are involved in, and I know this is another dinosaur moment for you guys. When I realized the government would buy me my very own Apple II to have on my desk, and that was the coolest thing I could imagine. That was a $2,500 computer, $48K RAM, $16K you could add with a language card to a program Pascal, I see some great builds out here who are there with me. That was a hot machine back then, that was pretty cool, and you know, very simple 8-bit processor, but really a fun way to learn about computing. And I thought, this is pretty fun stuff. I'm really interested. So that triggered the majority of my career, what I call looking at other people's software for a living, and doing analysis of systems and kind of the skills that many of you have out here. But the business that I'm in is really different than it was back then. So back then, the output of our business and these phrases ring for you, key generators and link encryptors and voice scramblers and things like that, that was the business that NSA was in. Protecting information and we could identify the links that were at risk, we could think up the mathematics to protect it and we could build it, and we could control the whole thing. We could decide how those circuits should be laid out down to the gate and transistor, if there was software in it, which was not true when I started, but we could do them an own test and validate and protect forever a line of code that was going to drive that device. And the kind of analysis that I did was the stuff we did in the back room. We built stuff, but we did analysis so that we built the right stuff. We didn't make mistakes in that part. That's a different model. The government was a monopoly provider of security services. Only the government really understood, the government cared, and we could design a way to risk. We could decide and control and control for the lifetime of that equipment or that system. That's a different world than we live in today, right? We're all connected, we're talking to everybody, technologies rolling over faster than we can understand it and we're connected to everybody and the thought that we'd be sharing the same network, friend and foe, my kids, everybody in this room was just, it was not even in our consciousness at that time. So a really different world and so the role of the government I think really changes in that kind of scenario in the world that we live in today. And some parts of the government have figured that out and some parts have not figured that out. But some of the things that I've concluded and kind of the way we talk about these things strategically in my part of the National Security Agency is less about the government stuff that we build, more about the commercial technology that we all live in. And some of you are involved in both building that and analysis of that. But when it comes down to it, it's touching all our information, moving it around, storing it and really protecting it the other day. And also we can no longer confine security to individual things protecting individual links. Information has to be protected wherever it is. It's a really different kind of problem. And when the government has a monopoly, that's a little, that's a much different business than where we are today. So today, and I'll talk a lot about this later, I view this as a business of influence, not as a business of control. The government doesn't have the authority or the money or the ability to control all the angles of this. Security is just too pervasive and too widespread. So you have to, and you cannot separate out the assurance or the security problem from all the things around information technology and all the things around the way people conduct information or deal with information or in my world conduct military operations. So there are lots of stakeholders in this business. I use these five categories I'll talk about in a moment. So it's not about do we have the right technical solution that we built. It's about all these other things, the policies that govern us and the compliance things that we have to deal with and the way we purchase our IT and the way we configure it and we operate it and run it and the way we report about problems and share them. That's a dramatically different problem for the government to deal with today than we've had for the last ten years. And so building, again it's less about the government products more about the knowledge business. So the stuff that folks like me used to do in the back room the output of the mission now at NSA you know an understanding of the vulnerabilities in technology is really more important than taking apart any one thing and then building the best possible thing because frankly the way the government builds things you know we have lots of restrictions on funding and oversight and all this we cannot build things fast enough nimble enough responsive enough to the marketplace. It's just not possible anymore. Not everyone understands that in the government but it's just not possible anymore. So you have to think of this as how do we get in the evolutionary path for all this technology to help people improve things as we go. And then the last is about the conclusion that I would draw about the way our mission has changed and where we're going is about this dynamic operational focus. So that is in the late 70's and 80's we could build a thing, put it out there if we found a flaw later we could fix it. But it's a different world right and now things happen so fast and it's less about protecting the information for decades from you know big monolithic adversaries it's about protecting information dynamically constantly and so if you see the things that my folks do today we're much more I'll say in the fight supporting warfighters directly in risky parts of the world helping people with you know issues around IT configuration and how do we operate things right now and new technologies rolled out and new vulnerabilities been found by maybe somebody in this room what do we do about it it's a much more dynamic much more focused business across a wider range of topics never before so it's very challenging but it's also very exciting because it brings us into this community with folks like you so again I've got three decades of experience in this and you know the work has a couple decades before me and so my boss a year ago reorganized some things creating a new group called the vulnerability analysis and operations group and asked me to run it and it's very humbling because it's a really amazing group of people and a great mission at NSA I often say that I have the best job in the US government in network security I mean it just doesn't get any better than this and what he did was bring together all the vulnerability finders that worked for him everybody in the defensive mission at NSA whose business it is to find vulnerability for a living okay so that goes back and some of this work is literally four or five decades old if you've read any of the later histories of World War II the cryptanalysis the mathematics underlie security right that's part of my group all the way up to the testing of the security of individual products you know kind of what's happening the buzz and the latest vulnerability in the latest piece of IT all the way out to the field and operational testing things like red team testing, blue team testing if any ex-military are out there concept monitoring or jikma if that rings a bell we have that full range of things those kinds of skills those kinds of missions from again the most conceptual most esoteric out to the most live current operational and we have every skill that can go along with that you know I don't give you a wiring diagram it's sort of everything you can imagine throwing at this that's part of my group so it's very challenging and also very humbling and also a thing to point out is that you know for us the vulnerability business we loosely use this term as a full spectrum business right at the national level as a worldwide problem this is much more than flaws in IT you know the latest bug in a piece of software that's a really important part of it but you know at the national level everybody cheats people will do all kinds of things and we have people that worry about the security of the physical spaces and the signal environment and the hardware and all these other kinds of problems so we have a very diverse set of technical people and skills and laboratories and relationships to help us pull that mission off so a lot of exciting opportunity in that so you know I'm a good bureaucrat when you start an organization you have to have a vision statement I got a vision statement you can take it for what it's worth just a few words for you just a context and I think it will help you understand where I'm coming from the nation's most capable, influential and trusted so those three words capable is in part about technical capability and I think most people would grant and to say a lot of technical capability if you've ever dealt with us before but capability is also about mission capability can you deliver those skills at the right place in the right time to do something useful that's a really different kind of problem dynamic this world is now and so it's not about studying and coming up with the best technical answer most technically capable it's about figuring out how to deliver that kind of skill at the right time, the right place that leads me to influence how I see this as a business of influence now which means helping people all across the spectrum of stakeholders in this business, how to approve now I'll walk you through an example of that again to give you an example of the way that we've been thinking about this in the current bureaucracy from NSA operates and then the role that we play in the community and then trusted, thank you no one laughed when the NSA guy said trusted I really appreciate that because I don't always get that but trust is a challenging thing to get a hold of I can't declare trust you have to earn trust trust is about behavior, the way you operate and in our kind of business trust is about sort of what you bring to the table what you deliver, what content you create and bring to part of the community I really saw this shifting in the late 90's and early 2000's it has become no one in this room probably is impressed if I walk in and say we're the government we're here to help you right we're the government we're here to organize you the model is a lot different there's a lot more openness and connectivity in this community so people that really don't care whether you're a government agency a multinational corporation, Joe's garage shop or somebody working out of their basement a virtual property that didn't exist that's critical to everybody else isn't that a really weird exciting world and so you have to think of trust as what do you bring into the table and people really don't much care about the authority or the oversight or whatever at least not the world that I live in anymore so trust is about behavior so again I'm not asking you to trust me I'll give you examples of what we have done and how we operate to bring ourselves and the content that we create into the community and you can just decide for yourself just a quick sentence on that or so when I started you can make a great living just poking holes in other people's stuff it's great work if you can get it but you know I tell people if knowing about vulnerabilities was the solution to the national problem then we were all out of work 5 or 10 years ago we're awash in vulnerabilities we find them all the time in every product and every aspect of the way we operate today the challenge is can we take that knowledge put it together in some useful way that's actionable that is that changes things about the environment that improves security and so again I hope I'll walk you through an example of how we've done that so real quickly this is kind of my simple management model about thinking of the information assurance business and its pervasiveness I mentioned earlier the stakeholders in this business and so for my simple purposes it's useful to divide all the stakeholders around security into five bins you can see them up here and why do I do that number one because if you think of all the folks and organizations and oversight things that are involved in this it just makes your head hurt there's no way to get a handle on it so you have to simplify the problem a bit and you have to think about this problem and the reason you need to think this way and categorize things is that here's the challenge in security every one of these categories of stakeholders thinks of this problem differently you know how to solve the national problem and each of them speaks a different language which is dramatically different so if you're going to have influence or change behavior you have to speak to people in their language as the bottom line and you guys are in the practitioner category by and large I'm sure that is you live and breathe and think security this is your preoccupation or your profession or whatever you bring to the table in this and most I know it's a shock to some of you but most of the world could care less because we speak our language it's not that important to them you can give them infinite list of vulnerabilities that's not actionable in their world they can't change behavior based upon that but this is the world that we live in as practitioners what is vulnerable and the technical issues and configuration things and basic flaws and conceptual problems and algorithms and so forth this is the world that we live in and it occurred to me just for a quick historical story I'm telling the story more simply than it happened and a little bit more linearly than it happened but you got to have a good story as I say it's good enough for government work so I'm going to tell the story this way because I'm telling the story but it's a really exciting change of perspective for the government and I didn't think of this all ahead of time this kind of evolved as we went along what happened is in about late 99 or so my group at NSA I had a group called the system and network attack center NSA, good guys I promise you and one of our businesses was blue team testing that is going out to help system administrators and the defense department scanning for vulnerabilities, configuration analysis of the components that are out there and so forth and give recommendations that sort of thing and we produced a very successful what we call a security guide for Windows NT as a byproduct of that it wasn't a specialty product it wasn't like a separate group of people doing it it was just a byproduct of how much to train our own people as we did for anything else and it became kind of a giveaway something we could give to customers when we left these are the criteria we test against this is really nitty gritty registry level kind of advice that we would give out and it became something we could give to people when we didn't have time to go visit them we didn't have the resources to visit everybody that wanted us to come visit but it became very successful and a number of folks kind of grabbed it used it, adopted it maybe kind of the official DOD standard now DOD standard does not mean what you might think it means it doesn't mean if it's a DOD standard everyone at DOD is using it it's a lot more complicated than that but it was very successful and around that time then Windows 2000 started to emerge as an issue for the defense department when should we roll over how should we operate it and so forth and my team who had worked on Windows NT these were the blue team folks this wasn't some big gigantic room for the folks who were thinking about configurations and writing guides they showed up and said, Tony we've looked at this Windows 2000 thing and it's a lot bigger than the NT and we're not sure we can do this what are we going to do we don't have any more people we don't have any more resources we thought about it for a few seconds let's go call everybody else in the US government that has a similar Windows NT security guide and see if they want to work together and that's kind of what the state of the art was and we did a quick count there was something like mid teens unique Windows NT security guides floating around sponsored by some government agency somewhere that's the best count that we could come up with and we literally tried to get a human beings name and called down the list and say hey you worked on NT you're going to do about 2000 do you want to work together and here's an inside secret about the sad life of government bureaucracies literally in an afternoon that list evaporated it didn't happen people had been reorganized lost the mission and funding whatever that just kind of evaporated some of them had adopted our guide or we're getting their advice from somebody else so this big market of unique products had kind of dissipated so at the end of the day it was us our friends at Odessa and I think Navy Spade War had a little bit of help that they would offer and another belief was kind of building in my mind at that time I said now why is the defense department's problem any different than the rest of the government's problem and why is that any different than the rest of the community's problem and I couldn't come up with a reason why they were different so I started calling other people that were kind of in a security advice business like the Sands Institute I'm sure many of you have taken classes from them and then there was a little non-profit starting up called the Center for Internet Security and I said so what are you guys going to do about Windows 2000 I don't know what are you going to do and then it all came down to how do we do something together so that started an informal movement of folks practitioners to get together and develop security guidance up front and we called this the consensus guide for Windows 2000 I wrote an open press article that's floating around on the web at various places if you're interested there's a link I could tell you about and what I wanted to explain was from NSA's perspective why were we doing this both as a resource thing and a community thing at the start of a partnership that's lasted through these years of practitioners working together so my goal was to bring the advice givers together why should our customers have to decide do I trust the NIST guys more than the NSA guys more than the DISA guys or the CIS folks it didn't make any sense to me why don't we work together up front and we can come up with maybe something even better than anyone of us would have on our own and by the way we could share labor so that made a lot of sense to me so practitioners this has become the model now and you may have seen some of the press about us up front on the shipped security configuration for Vista I'll mention that in just a bit so rethinking the way we did this up front it's a big change for a government bureaucracy especially a really secretive place like ours in parallel some of you may have noticed that we started a campaign to release all this to the public okay so it got permission to post on the NSA public web server the security guys this was major culture shock with the NSA just take my word for it but really was quite a dramatic change in the way we thought about this problem and again for me and my team it was based on this belief that we really couldn't see what our baseline security problem why was it different than everybody else's problem and by the way since everyone's connected to everybody there is no stand-alone defense department to be protected anymore we're hopelessly interlocked with our business suppliers and our partners you know all these communication systems and weather data from public sources and all kinds of stuff so our security now depends on everybody's security right so you don't get better in something as vast as a DOD unless everybody gets better I mean that's what I believe and that's the way we've acted right so that says you have to push outside your borders and think about how do we solve this as a national level problem so lots of folks started banding together lots of folks started to adopt this stuff it became kind of a thing okay we'll take our checklist from I think NIST calls them checklists CIS calls them benchmarks we call them security guides and by the way when I say consensus no one's in charge everyone kind of works together it's a classic hurting cats kind of exercise every once in a while the cats wander off this way or that way you all know we're in a business filled with really really bright really really opinionated people who have fallen their sword for the last registry setting and the last recommendation so keeping everyone kind of heading in the same direction when no one's clearly in charge and no one's paying for it all is a great challenge but it's a really satisfying challenge and it's actually I believe more powerful than trying to put a government bureaucracy in charge because people are there because they want to be there and they want to bring their content they want to get the recognition of their peers and so forth but believe me it is hurting cats so people started to adopt this interesting twist every one of us so NSAID, NIST, DISA everybody has their own constituency that came to the table and there came to be this sort of you know bringing together of opinion and bringing together of operational usage and another thing started to happen that the buyers started to take notice notice of this well wait a minute our folks want to operate our clients and servers this way why don't we buy it that way why don't we tell the vendor that we want it preconfigured operated this way from scratch but for something like defense department do we really expect that the way to deal with network security is to ship wide open systems to everybody in the DOD ask every poor person under trained, under loved, under resourced person in the DOD to configure a system from scratch to be secure that's clearly a losing strategy so why bother with that we are not going to train everybody in the DOD to be a security wizard the knowledge that folks like you have is pretty precious stuff and it's not something that comes for free and so you've got to think of this how do I make it easier for those front line operators of the IT because our security depends upon them so Air Force has really led the way for the US government in this and we've worked very closely with the Air Force on their baseline configuration this has gotten a lot of press in the last two years or so if you follow this sort of thing but basically Air Force in order to simplify their network management problem primarily decide to standardize on a very small number of desktop loads plus the applications and then servers and the idea is we've got to make this simple we've got to be able to manage this we've got to be able to automate it so we work with them very closely on defining what constitutes a usable prudent security baseline that has become a really important thing and we work with them very closely to track breakage that has become the model now for the entire US government by mandate from the Office of Management and Budget that is every government agency is expected now to follow this basic model tighten up your systems integration, work with the security community to define what that means tell the vendor this is what we're going to buy from you we get to now we are now reversing the security problem why do things break every time you tighten security applications break why do they break? because we never told anybody what our runtime environment would look like in terms of security so vendors write in whatever manner use whatever ports, whatever services are convenient for them but now as you can define what a DOD system looks like when you look into your software developers and your suppliers and say before you sell to me you need to certify that it will run in this environment and MySQL offers a participant by creating virtual images by which vendors can test and certify their products will run in this environment so it's reversing the security equation so it's a tremendously powerful idea that is now becoming the model for the entire US government and I'm sure that will spin off if you're a supplier to the government the government is worrying more and more about hey if you're a supplier to me the same level of network security in your network when you're handling my data and my resources as I do for my own so prove it to me show me that you are operating by the same technical policy that I am so of course the vendors have to be involved in this the big OS vendors and everybody else because frankly if this is the way the buyers want it then they will deliver it but what they don't want is some gold-plated standard that breaks things and causes them lots of expense to answer the phones and troubleshoot and so forth so they say well it might as well work with the security geeks up front so that they make the right kind of trade-offs in operational of capability vice security and so that has led to close working partnerships with the major OS vendors and so what constitutes a baseline security configuration and then the whole market has popped up in the last several years partly through personal lobbying and partly through just people seeing this opportunity for themselves well wait a minute if these enterprises want to run their clients the NIST way the CIS way or whatever well I'm going to tweak the tools I've already sell into the market to measure and report on compliance against those benchmarks so it has become an emerging market in a pretty big way now so you can say well I'm going to run my clients this way their measurement tools from any of the major IT security tool vendors and they operate in kind of this fashion and what happened even better was folks who are authorities people who have sort of regulatory right things and all of us are governed by something in our lives you know FISMA, HIPAA Sarbanes Oxley and all these large-scale well-meaning you know handle data responsibly kind of things that are happening today this is where a lot of the money is going that should be spent on security and they all want good security but they don't really define it very well so what's happened with all this sort of security guidance and stuff that I talked about NIST has really leaped on this and said hey you need to you need to do something today and that will take you down the right automation path for FISMA compliance specifically for the U.S. government so a very powerful idea that takes away a lot of this sort of Wild West well how do I comply well I have to write a lot of paper to convince somebody that I complied with FISMA that's kind of how the world works today and when you see you know government agencies so and so including us gets a D or an F on the report card that's what the reporting on is the quality doesn't link very well to the technical what did they actually do so that's what's happening now is NIST in particular is leading a group to make that linkage and to clarify that so that everybody individually across the U.S. government doesn't have to make that case for themselves which is just a tremendous waste of resources okay so I kind of walk through this quickly this is a multi-year story that really started from a pretty modest beginnings right thinking about testing systems what would my recommendations be let's go find the people that kind of work in the same sphere with us the practitioners people that start to adopt it and the real breakthrough was creating guidance in actionable form so I mentioned earlier about speaking other people's language is the only way to change things so the security people want to say well we ran all these scans we found thousands of vulnerabilities and this is really cool and we give it to the operator and they go what am I going to do with this it doesn't tie into my trouble ticketing system how should I configure my systems in the registry settings to stop these vulnerabilities see that's the language they speak they speak the language of system administration so the vulnerability finders up front right the blue teamers that kind of stuff until they create something in the language of system administration there's nothing actionable to work with well the buyers they don't even speak that language right they don't speak you know configurations and file permissions they speak whether I put in the request for proposal or in the specification what should I demand of the vendor so I get the security up front so we have to create the language that goes into the buyers hands to make this much more powerful for the vendor to operate with right and when you work with the vendors you've got to sit down with them and look at the trade-offs in how things break you know because they are much more aware of their very broad customer base and the complaints they get over the phone and you know the things their field engineers have to deal with so they say hey you government geeks thinks this is the most important thing in security but as soon as we tighten this you know 5,000 applications break over here that's their language that they want to operate in and they pay attention when the buyers demand and then authority folks you know bless their hearts they're the least technical of all they don't they they're just looking for good things to happen at very large scale and they write laws or directives of policies and that's the world that they live in so you have to give them some content in their form so that they can make changes across scale right so you the way to achieve success at the national level is to think about this this is not purely a technical exercise or an Uber geek exercise this is about translation helping people all across all the stakeholders in this business think about this problem speak in their language give them something to work with and they all want to do the right thing that's really important they all want better security but they just don't know how to ask for it until you help them and it's communities like this right we would speak the kind of language of geekdom that really have to help create that because it really starts from the analysis and the understanding of the technology or we just never get there what's happened behind the scenes in parallel with that story over several years is the development of what I call the the plumbing of vulnerability if you'll pardon me okay so what does that mean well it all comes down to content right what is the content of the world that we live in of vulnerabilities well it's knowing about new flaws in pieces of software IT vulnerabilities it's security guidance that's unique content created by people security guides benchmarks whatever you call them things like red and blue team reports we go out and test the system and we found all these problems that's unique content test of individual products tracking of security incidents you know bad guy so and so has done this to this IP in the Defense Department etc etc this is the unique content of vulnerability in my world the problem is this content is often generated independently by different people speaking a different language even within the technical community and so the challenge is you know we don't want to do red teaming for red teaming sake right what's the reason for testing these products we're doing operational tests so we can fix these problems and operate our ITs more securely or get it more securely from the vendor to start with so you need a way to move this content from place to place that's what I call the plumbing so I'm sure some of you are familiar with some of these terms here these are standards that are out in the public that we have either developed at NSA or sponsored at places like MITRE or done jointly with other folks like NIST okay so it's not all uniquely from us but CVE is the standard naming scheme of vulnerabilities out in the public it's kind of the universal language of how we refer uniquely to flaws in IT Oval is a standard test language CCE common configuration enumeration CPE these are two kind of brand new things common platform enumeration unique ways to refer to specific versions of software running out in the environment so forth now these are standards that were developed mostly independently they're coming together now to constitute what I call the plumbing of vulnerability that is the ability to move information from place to place in a way that poor human beings don't have to do all the moving or the integrating or the analysis and so what we've done is reframed a lot of the work that we do so that our output speaks these standard languages I'll mention one in particular XCCDF I hope I get this right, extensible common checklist descriptive format I think from NSA it's now in the NIST standards program you can look it up on the web and I'll give you a link to it later but it's become it will become the new universal language of security guidance for us in the old days security guidance was literally narrative we recommend you set registry to your key to so and so that's the way it worked and then we produce plugins for specialty tools what's the new universal language of guidance same as everything else an open standard XML XCCDF will become by consensus among all the advice givers the new universal expression of security guidance so that's machine processable so it allows you to move kind of recommendations from the minds of the analysts out to the tools that measure and report and so forth so this is a really important activity now not all these standards are fully mature they're still underway you have plenty of opportunities if you want to help turn these into the right kinds of things but it's maturing very rapidly and there's a major activity around this both with us and with NIST there's a major workshop in mid to late September that you're welcome to be a part of if you like this kind of stuff and the little dots on the bottom there's plenty more standards and the fixtures if you'll pardon me straining the metaphor just a little bit longer so to move the content via the plumbing to something useful right so multiple tools to measure and report and you know I live in a world that's very complex you know the army does not have one vulnerability tool it's got many tools because it's got networks that are worldwide fixed plant it's got networks that's running out of air-condition tents and really nasty locations and everything in between no tool single tool runs that no single vulnerability scanner can deal with that you inherently live in an environment of great complexity and lots of mixing and matching you have to be able to bring information together across multiple tools to drive multiple remediations and you cannot count on poor human beings to make that integration happen right you can do it a little bit if you buy every tool from a single vendor you know because vendors are buying each up buying each other up vertically faster than we can count but the real way to deal with this is through an open standards kind of way so we're very strong on that so integrating the reports why are red team reports separate from blue team reports when they in fact give really complimentary information but again you can't the humans can't wade through enough narrative and written pages to correlate this because red teamers talk about password things differently than blue teamers do so you need to again work the plumbing issues record the data differently up front that's a really important part of this and then all these other things policy compliance to let us link so I want to give you a hint behind the scenes a lot of technical activity this is not just about guidance and good advice but in fact it's a really massive activity to build plumbing that is to move data around in a mostly automated fashion this is all this this gets bundled together in an umbrella program that's honed at NIST jointly sponsored by us Dyssa a little bit by DHS and NIST called the security content automation program which encompasses all those standards into kind of a nice neat bundle links it to physical management compliance again not everything is mature in this but it's looking really really really promising and the interest from vendors and the acceptance with them is very high so I think this is an opportunity to make massive progress for the entire nation in this urge you to take a look at it again there's some events coming up if you like this kind of stuff you have an opportunity to get involved in that real quickly I run through another quick anecdote here that again gives you my perspective it's a parallel and related story each of these little cylinders on the bottom is a job among many others that my group does so red team testing, blue team testing OPSEC is operation security what do we give away for free to the bad guys through websites and that kind of stuff COMSEC monitoring is the by permission we have a group that is a very military thing primarily monitoring plain text communications primarily as to protect the forces in the Middle East what are we giving away for free to bad guys you know about when the fields go in and where the convoy is going to go and stuff like that can we find that turn it around so we can change our plan so that the bad guys don't take advantage of that technical security is about physical spaces and signals and so forth going out to sweep and look for things each of the reason I draw them in these little little boxes here each of these jobs evolve from a different management structure at a different time different funding lines different skills and so forth right they started independently and they optimized independently and when you're operating jobs like this then your concern is about how efficient can I be can I do 20% more jobs next year than I did this year all my people trained and so forth and this becomes the manner that you operate right and in the polite to politically correct these are all developed in a stove pipe as the way I might have said it before well they're not really stove pipes now we call them to be politically correct cylinders of excellence really important cylinder of excellence in fact some of them were very highly polished cylinders of excellence so that nothing could ever leak from them that's the way these things evolved and the world is different now so a few years ago I started to refocus all these folks that worked for me in particular to say you know remember that practitioner word I used we're not the only red team in the DoD or the only blue team or the only tech sectors go find the community of people that are like you right go find the rest of the community in most cases all modesty implied NSA we usually have the biggest and baddest team that's just the fact of where we happen to sit but you know when you're the biggest and baddest in the community then your responsibility is to go help others so go find the others let's start acting like a community what does the community do well they get together they share lessons learned they swap tools right they come up with ways to record the data so they can be correlated they host events they train people they certify organizations and so now all those things across every one of those communities or within every one of those communities some formalize for the DoD some less formal so that became a big theme for us maybe six seven years ago and we are I think it's been a very successful way to think of this but it's still not not really good okay and and what I decided several years ago was that you know in something like the DoD you're not going to test your way to security one network at a time it's just too big that's not why we're doing this so why do we do red team and blue team so I said our new our new business model is sampling yes we want to help the customer that asked us to sample their network right to test their network but the real value the long-term wing for the DoD is sampling that is what can I learn about this that will help me solve this problem at scale alright so remember seventh grade science what's the purpose of sampling is so we can draw conclusions about the underlying population so that became our model let's learn from not only red but blue and all these jobs what conclusions can we draw about the real problem what's most common what's the root cause where's the best place in the life cycle to solve this problem that really became the mantra okay this business is about sampling the environment then deciding where among those stakeholders is the optimal place to solve this problem do you see how this parallels as we put this together then we have a buzz phrase for this called you know integrated analysis and reporting that is you know if I learn it in the blue team kind of world I need to know it for the red team kind of world right it's interesting blue teamers do things by the way people don't realize you know they're very quiet and stealthy and narrow they don't really get to see most of the network because they don't look around because it's too noisy right so they demonstrate exploitation scenarios so it's interesting to me that the red team succeeds but until you match it up with like a blue team view what's behind the firewall and how the system is configured and what critical things are dependent upon that piece of IT you don't really have a very good picture as to what's going on and then comes like monitoring you get to see we just gave away really important information that's a critical part of red team scenarios that bad guys could be exploiting right so integrating this information across multiple sources is a really important thing but you can't do it the way we were collecting the data before right because how would you do it you put your best human beings in a room with literally thousands of pages of narrative trying to sort that out it's impossible so you want technology to do the grunt work so you need the same plumbing that I talked about before right let's record our data from red team and sensors and blue teamers up front differently and then we can bring it together let the technology do the grunt work of the 60-80% you know just the simple stuff let the human beings lose on analysis because that's really what I want then so that became a whole different way of thinking about this problem so I want to share this story really briefly with you because it's in parallel and it shows you kind of the way we've been thought about this the way we've reached out to a much broader community of folks beyond us in a sort of different than the guidance stuff okay so I'm an old guy in computer security I've never had another job some people say I've never had a real job but I've learned a lot of things so let me just show you a couple things to wrap up to gain assurance in this business these are kind of my lessons learned from decades of this and thinking about this as a government kind of guy that you have to think about this problem at scale if we're never going to solve it at scale nationally so these are the way I think about it you've got to organize the content generators bring them together to think about this problem together the people that create new intellectual property that tells us some kind of insight lets us draw new conclusions you have to bring them together to bring them together you have to go out and be among them and that's why I'm out at places like this you have to standardize the raw data otherwise we're just going to wade through pages of narrative hoping that brilliant sleep is out there for us it's not going to happen so we've got to think really hard about standardizing the raw data so we can bring it together and we have to think about storing that data in a way by the way that's in the blues and all these kind of friends but the challenge is people that need this data are often not people I know or ever met before so we have to record the data that's discoverable by them in their context that's the new model of information management today you have to translate into something useful upstream so we speak a certain language as I described but the people are going to solve this problem at different places in the life cycle speaking an entirely different language so our responsibility is to help translate what we know into something else preferably in someone else's language and then the real key has been to link to other business areas and again we live and breathe this sort of stuff but other people are just trying to manage their networks better or they're trying to arrange for compliance so that data is well protected so you have to link what you know to these things or we'll never make great change because by the way things like compliance that's where the money goes you can show up with a list of thousand vulnerabilities and people just scratch their heads and they say they give it to the poorest this admin for okay the money goes for FISBA and Sarbanes Oxley and so forth so you've got to help people do the right thing there technically so that we're not overwhelming the poor frontline defenders okay and then a last thought about the role of the government and working with the private sector you know a couple of things for me again we're the government we're here to help you nobody cares okay that's a different world my experience has been you know bring content to the table that's why we released you know our security guidance to the public go if you go there I promise you you'll see stuff that you won't believe came from NSA bring content people know you're serious to pick at it they'll find flaws they'll come up with improvements what you're gonna do you're gonna thank them right you're gonna thank them and improve it that's the way things work today put your content on the table let people chew on it and it'll get better that's really not the way government agencies are used to operating by the way so that makes a difference and bring good people okay that's why my folks are out of places like this I'm sure you'll spot something when you meet some of my folks you realize these are your friends these are your peers they have similar technical skills they want to do good things they're trying to protect the nation they love technology they live and breathe this stuff and again they're not bureaucrats here to organize you they're folks that are here as part of the community we step out and become part of you guys if you're gonna work across government about the different classes especially the buyers their money talks and then abstract the interfaces I know it's kind of geeky but it's how do things talk to each other how do how the blue team reports get used to change system behavior right how do I move information from place how does a vulnerability scanner talk to a patch manager places like that these interfaces are places for the government to take a hard look and say should that be open should that be a standard should that be something that's an open standard that's why I'm always looking for opportunities to open the business up to other folks okay you may have seen us in the news we are as far as NSA technical centers go we are head and shoulders the most publicly known the vulnerability analysis and operations group and so I spend a lot of time talking to folks well there aren't too many folks like you but the audience is about our work looking for new friends partners and telling you that we're okay we're bringing our content to the center and then some and you can track all this a lot of this broke this is just places we've been mentioned in the last six months or so six seven months a lot of this was triggered by Microsoft's public announcement that they worked with us on the ship to configuration for Vista just to let you know and I was interviewed in Washington Post sometime during the winter lots of stories followed from that and you can you can go look to stop if you're interested we also received two unsolicited awards from you know you don't come to NSA because you love public recognition believe me but it's really been a change of heart for us it really has a dramatically changed the perspective but it's also really gratifying to my workforce right because they want to be a part of a community like this and work with folks like you because we want to solve this problem last thing to learn more these are some links where you can track some stuff down NSA security guidance again the security content automation program there's lots of information on the center of security you can find a reposted article that I wrote in the early 2000s about this it's a little dated now but it tells you why NSA moved out into a much more public sphere and with that I want to thank you before I get to hook this is the number one is the only conference where I have to go before I hit the on button on my laptop and then number two where there's actually a bouncer to kick me out of here when the time is up so the time is really valuable and it's great to come talk about our work here so thank you all very much