 Hey, is that Finish Kid Miko here at all? The Finish Kid? Yeah. No, I guess not. Foundation, that's a sad state of affairs. Welcome to Linguistic Platform for Threat Development, the extreme hangover edition. I'm Ben Kurtz, I work for a company called Perfect Networks out in Burlington, Massachusetts. Today I'm going to talk to you about what I have cooked up over the past year, why I did it, and how it can help you do what you do. Well that's all well and good, but basically I'm going to talk about a design pattern for a, I'm going to give you a view from like 30,000 feet, it's just going to be like a broad overview, real abstract, I don't really have the time to go into the sort of details I'd like, but let's see. By applying some often overlooked corners of programming language theory to the development of network attacks, we can create a new type of threat platform which can handle every conceivable network protocol and run readily on any architecture that can handle embedded Linux. The ultimate goal is to reduce the total time to develop a threat to the time it takes to research and understand its behavior, you know, plus two minutes or so. Once the threat is implemented, we also want the ability to store it in a database and play it back whenever. This presentation is divided into four sections roughly. First we'll look at the motivations for a new threat platform which is hopefully not going to be too dull. Second we'll take a closer look at what we need threats for and how the threat development and testing cycle works. The reason I'm going to go through that is because many people here look at threat development and threat testing from an enterprise standpoint or a pen testing standpoint and I'm kind of coming at it from a slightly different angle. I need to encompass a broader spectrum of threats in a single system. I'm looking at it more from a manufacturing testing and regression testing standpoint. So I'm going to take a little time and reframe what threat testing is in terms of the talk. Next we'll summarize the design goals of the new system. Then we get into the good bit. We'll look at some ideas from programming language theory and how we can use them to get what we want. Why do we want a threat development platform in the first place? Simply we want to break stuff. We want to break stuff and we're incredibly lazy. We want a tool that makes it easier to break stuff quickly. We also want to minimize the time from the first rumor we hear of a new threat to the ability to play it back on the wire with the touch of a button. Being up to date on new threats is a foot race and anything that helps us go faster is our friend. Why do we want to break stuff? There's two main reasons along the standard black hat, white hat divide. What I'm focusing on is we want to break network equipment and services that we develop and use so we can assure our customers of some degree of protection and reliability. There is no trust without testing. A lot of things are advertised as being secure and that's never been verified by anybody in many cases. And on the other side we want to break stuff because we have malice in our hearts. Like any powerful technology, a threat platform can be a tool of assurance or a weapon. Since the malicious uses are fairly obvious developing a threat and executing it quickly with a great amount of throughput, I won't really revisit this. Many security products such as intrusion detection systems are developed and brought to market with little or no testing against real live threats. Usually they're tested against a small set of handcrafted sample traffic that is designed to trigger existing threat signatures in the IDS. This means that the test cases are usually written to test what the IDS already does rather than to test its ability to protect against real threats. At some point you're going to have to test your network security products against real attacks rather than, you know, just handcrafted stuff. Otherwise you're providing your customers with only the illusion of security. There are numerous places that release vulnerabilities or exploits to the public. You probably all know most of these. You got bug track, security focus, full disclosure, Mike Lynn, that's a drink. As soon as an exploit shows up on the net, every potential attacker has access to it. Me? I've got to put a stop to that right now. You need to know that you're safe from each new threat as soon as it becomes public. Private threats are kind of an unknown quantity but you can definitely keep up with the mailing list. You have to do this partly out of prudence and partly because your boss called you in a panic because his boss called him in a panic because the CEO called him in a panic after hearing about some worm on the news. The threat developer needs to stay on top of the O-Day threats mostly to be able to say, yes we've already accounted for that and we're testing against it. You could just say this anyway but you're still going to want to actually go out and do the work before anyone checks up on you. To be sure that you can handle all publicly known threats, you have to keep up the mailing lists. Zero day protection requires zero day testing. Honestly there's only three steps here but it looks better with five. We're going to spend a few minutes talking about threat testing. This is from a manufacturing network equipment manufacturer standpoint rather than from a pen testing or enterprise security testing standpoint. For each new threat that you want to develop and test, a threat developer goes through several steps and that's three, not five. First the threat must be researched, you have to figure out what it targets, what the packets look like, what protocols it uses, how it behaves and so on. Now next you have to implement the threat which means writing the pearl code or the C code. And finally your service or equipment is tested against the new threat to see if it's vulnerable. Then you begin the cycle again with the next threat or sometimes developing one will lead you to another like a variant. All these steps can be very time consuming to the point that many QA departments don't bother. The time period from when an exploit first enters the public domain until you release a patch to defend against it is your window of vulnerability. The testing side of the equation is at a serious disadvantage from the start since a would be attacker can usually just download sample code for an exploit and go to town. If it takes two weeks to research, implement and test a threat, that's including all your QA guys, vacation times, coffee breaks and so on, mind sweeper. But if it takes you two weeks to research, implement and test a threat, your window of vulnerability is two weeks where an attacker doesn't need more than an hour or two and that's a pretty conservative estimate. It's easier to break something than it is to fix it. So if you're on the defense you have a disadvantage. So what are your options to speed up threat turnaround time and to minimize your window of vulnerability? Your first option is to hire more people to handle the daily updates. Some places have upwards of 20 employees working on researching and implementing new threats as they come out. As some places. No names. This approach allows you to deal with more threats concurrently but each threat must be developed and tested in parallel using the same time consuming tools. So for any single threat, your window of vulnerability remains the same. It's kind of like you know, you can't parallelize baby creation. From start to finish it takes a certain amount of time given a certain set of tools. Alright, I'll drink for that one. The second option is to improve the tools that threat developers use to speed up the process of developing a single threat. And that's what we're going to talk about. By automating the common and repetitive tasks associated with threat development and by only forcing a developer to describe the unique aspects of a threat, the threat turnaround time can be greatly reduced. This is probably not a popular idea. I love my job. My inclination is to say, hey, we'll hire more people. We'll give more people jobs that they can love. But this is an expensive proposition. You suck. I'm trying not to. Pure pressure, you know. That was your fault, man. You have a drinking song? He just did it. This is going downhill quickly. I'm Ben Kurtz. This is this is tequila, you know, very good. Well, I got to press on or I'm going to run over. I can't actually focus on my notes at the moment. Alright, so where was I? We're talking about hiring more people, right? But it's not hiring more people, right? It's an expensive proposition for a smaller company and a difficult proposition for a larger company. If you look around this conference, like you guys and your friends back home are pretty much the hiring pool for our entire country. So, yeah, it's a problem. There aren't really any college curriculums that teach this stuff and it can be incredibly difficult to find the sort of employees with the passion and personal drive to learn it on their own, which is you guys, really, so. Even if you do find a skilled security guru, you're going to want to maximize his or her output. This means we need better tools. There are a couple of existing threat platforms. Metasploit is pretty cool. It gives you a mix and match ability with exploits written in Perl and various types of shell codes. The downside is that the scope of the attacks are limited and it's difficult to add new exploits quickly, even if you know Perl. It doesn't provide an easy interface or any regression testing tools or reporting. There's also Canvas, which is similar but in Python. Metasploit and Canvas are good at what they do, which is primarily running exploits that will result in control of a box. It's more of an enterprise pentesting type solution. They don't really deal with denial of service or worm simulation or man in the middle attacks or fuzzing or a variety of other types of threats that concern IDS vendors or network equipment manufacturers. We also have Nessus, which is primarily an auditing tool and is designed to report on vulnerable version numbers of services and to avoid running an actual exploit as much as possible. It's great for use on a live network where you don't really want to break anything. It's not so great for doing testing with live threats on network equipment, IDS systems, stuff like that. Historically speaking, the most widely used threat platform has been Perl or C with some escaped shell code. Many threat developers are of the opinion that if you can't write an exploit in Perl, you have no business messing with it. This is kind of a societal safety valve but it stands in the way of speed and efficiency in threat development. IDS vendors and network equipment manufacturers just want a QA guy to check off a list of threats and run a test, see what happens. When the CEO hears about a new threat on the news, he wants assurance that his product copes with it quickly. There isn't a lot of time for inefficiency which means mucking about with Perl. So a new threat platform has to provide the versatility to handle all types of attacks, not just shell code exploits and has to include tools to develop new threats quickly and the ability to run live threats. Not just to check to see if something might possibly be vulnerable. You want to know yes or no. So we're past the boring bit. We're on to the good stuff. We've talked a bit about the problems we're hoping to solve with our new systems. So now we're on to our wish list, the design goals of our new platform. We want one tool that can handle all possible threats including exploit shell code, denial of service, man in the middle and fuzzing. We don't want to limit ourselves to running on a particular hardware architecture. We might want to run from a laptop or from a blade server or from every blade in a bunch of... Wow, sorry, tequila moment. Rax? That's the word. Good deal. Yeah, one more. We also don't want to limit ourselves to a particular network medium. We want to be able to handle copper pairs but we also want to do fiber and wireless. Additionally, we want to be able to target anything regardless of what it is or where it is on the network. We're not limiting ourselves to services. In fact, we're focusing on hardware level testing. Since it's often the case that a developer gets a packet capture of an attack before it's even identified, we need to support pcap imports into our system so you can import a pcap file, edit it and play it back. To accurately reproduce some types of network threats such as a man in the middle or a DDoS, we need the ability to support playback of network traffic from multiple sources on a network. So we're going to require kind of a distributed architecture where we can have a number of different threat engines on a network that can all play back different roles of a conversation like something you import from a pcap file. To accurately reproduce some types of network, oh I just did that one, our tool is going to have a few bells and whistles that make it easier to use as a threat testing platform including the collection and aggregation of a variety of statistics. But statistics are insanely boring so I'm going to skip over that bit. Since the execution of one threat might involve playback from different network nodes, the statistics have to be aggregated. And that will happen, we'll get into that in a little bit. I got to take a moment, you guys suck. This is harder than it looks guys, come on. Now we're going to do a quick review of some of the concepts of programming language theory we use in this system. By quick I mean that this is probably going to go right over your heads if you're a foundation of CS class, but the reason I feel comfortable kind of skimming over it is that this, there's tons of books on the subject written by people much smarter than I am. So this is a, if you haven't taken a foundation's class this is basically a list of words to put into Google or Wikipedia. The bits we're interested in are all related to languages and translating one language to another which is pretty much all of programming language theory. The most important concept is the idea of a grammar which is a set of formal rules describing the structure of valid expressions in a language. Grammars serve two purposes and they can be used generatively to produce valid expressions or they can be used to determine whether a given string is a valid expression in a language. So it's kind of like an input and output. You can use a grammar to generate an output or you can use a grammar to interpret an input. The most common method of describing a grammar is the extended Bacchus-Norform or EBNF which we'll touch on briefly later but that's something I have to Google. Compilers. A compiler is something that translates a valid expression in one language to a valid expression in another language. It's basically a language translator. The compilers you're used to translate from a programming language like C to a particular architecture's instruction language or another one that many people are familiar with is the Java compiler which translates from the Java language to Java byte code. A compiler makes use of two grammars. The grammar of the input language for Java would be the Java language spec and the grammar of the output language, in this case byte code. It uses the input grammar for validation and the output grammar for generation. Compilers work in four stages. I'm just going to skim through this. First is lexing where the input expression is broken into tokens based on white space and other syntax. Next these tokens are parsed. During the parse the input grammar is used to build a parse tree which represents the meaning of the expression. This meaning is stored in an internal data structure called the intermediate representation. And finally the output grammar is applied to the intermediate representation to generate a valid expression in the destination language that has the same meaning as the original expression. So this is the key part, the parser generator. A parser generator is just a special compiler that can read its input and output grammars in dynamically at run time. So it's sort of like a general purpose compiler that you can use to translate any one language to any other language. You may have heard of yak or java cc. Bison? Yeah, bison yak, lex. That's sweet of tools like java cc. If I had any books left I'd give you one. I should have held out. I'm switching to water. Please. It's true. Yak and java cc predominantly just generate parsing code for ebnf grammars. They take an ebnf grammars in as an input and then those two tools are fixed for their output grammar. Java cc puts out java code that parses an ebnf spec or not the spec but the grammar specified by an ebnf file. The advantage of a parser generator over a compiler is that it's very easy to modify and update the input and output languages without code changes. You can simply update the grammar specifications and reload them. Because it's so easy to add support for a new grammar to a parser generator they're good for situations where you have to deal with a large number of different languages and constantly add support for new ones. An example of such a situation, network protocols. One of our stated goals is that we want one system to handle all possible threats. We want to be able to fuzz each field of every protocol. We want to deliver a malicious payload in every protocol and we want to be able to intelligently DOS every protocol. And we want to be able to import pcap files in any protocol, edit them and play them back. I am way too lazy to code and debug all the support for every possible protocol. Fortunately there's a shortcut. Network protocols are languages that are designed to be easily handled by low level code. As a result, most of them are very simple. In fact, if you exclude things like check sums and internet header lengths, network protocols can be represented by the simplest subset of grammars, normal grammars. This allows us to make a lot of assumptions in writing our parser generator, which simplifies the code and speeds things up immensely. We deal with check sums and internet header lengths at the engine level, which we'll get to in a little bit. So we're going to use grammars to represent network protocols, both to understand incoming traffic and to generate outgoing traffic. Since there are an insanely large network protocol out there, both public and proprietary, the parser generator will allow us to easily add support for new protocols without mucking about with our code. Each network protocol we specify is essentially just an ordered list of fields. Most protocols allow other protocols to be encapsulated in them. We handle this encapsulation, I'm going to call it stacking, of protocols by using a special rule, it's kind of called the payload rule in each of our protocol grammars. You will see this in a little bit. Each protocol that allows encapsulation will have a payload rule that can be made to point at either another protocol's grammar or to a chunk of arbitrary binary information like a shell code. This is the simplest example of a protocol specification and it's for Ethernet. The start rule goes to just Ethernet and the Ethernet production rule on the right hand side is just the list of fields in Ethernet. You've got SourceMack, DeskMack, Packetype and the payload rule which is what would point to whatever you're encapsulating in Ethernet. The various fields go to regular expressions that you can use to validate them. You've got hex fields from one bit length you know, some of them are like 13 bits. They're very strange sizes, up to like 64 or 128 bits. But some of the data types people are used to seeing in human readable form. IP addresses, MAC addresses and stuff like this. So you can keep your regular expressions for those types of data type validations in a common location and refer to them within your specific protocol definitions. That's right. One thing that's not on here but that's important to mention is that we need to, we can't just use a normal grammar like this. We also need to decorate the grammar with a little bit of additional information which allows us to generate packet streams correctly and to correctly deal with pcap imports. The most important thing is that each field needs to be marked with its bit length so that you know how many bits to read in, how many bits to put out. So each of these protocol definitions is a template for creating a series of packets. The user will be able to stack these templates to create a representation of any type of encapsulated network traffic. A common protocol stack involves combining three different protocol definitions Ethernet, IP and TCP. Ethernet's payload points to IP, IP's payload points to TCP and TCP's payload will be it could be empty or it could be a shellcode or any chunk of binary. Some protocols are a little bit more complicated and can have a number of different forms based on a type field. Examples of some of these protocols are ICMP, DHCP and OSPF. We can still deal with these protocols by splitting other definitions into separate chunks. It's kind of like making look ahead tables if anyone's written a compiler before. I'm so good with DIA. Here's an example for OSPF which has several different packet types. All of the different OSPF packets share a common header which contains the type field indicating what the rest of the packet looks like. To handle this situation we can simply create a protocol definition of the header and a separate definition for each of the packet types. What I have been calling protocol definitions is actually somewhat inaccurate because you may be using more than one of these grammar chunks to define a particular protocol. OSPF I think has six different packet types. Is it five? So you'd have the header and then sub types that would follow from that. This is something we do in order to keep using normal grammars so we can keep our nice assumptions about how to write our parser generator. Does anyone in here know about the LLK algorithm? LLK? Right, LLK is basically just keeps increasing the look ahead until it finds a match. So it's pretty inefficient at some point. And using this scheme basically breaking off the header allows us to avoid going into LLK territory which keeps our code very simple. That's tequila. It's getting on top of me. The basic flow of creating a threat file goes like this. Is it? No, it's not. Thank you. A bunch of sadists. The protocol definitions are loaded into a GUI. The GUI examines the protocol files for information about fields and what sort of data types they're expecting so that it can represent the values of the various fields in the way that the user is expecting. This is really only an issue for like MAC addresses and IP addresses. For your 13-bit hex field, no one's expecting anything other than hex. The user then creates a threat file by selecting which protocols to use and filling in the fields. This information is fed into the parser generator which uses the selected protocol definitions as the input grammar and outputs to a threat file which can be stored in a database for later use. The GUI provides the user with three abilities. First, the user can select and stack protocols to create an arbitrary encapsulation of network protocols. Second, the user can arrange any number of these stacks in any order and define the behavior of the threat in terms of iteration or looping. Finally, the user can assign functions, variables, or values to any field of any protocol. Here's a sample screen shot. It's an example of how a GUI can be used to create a protocol stack. All of the protocols or subsections of protocols that we have defined appear on the left and your stack appears on the right. The interface must allow you to create any protocol stack you want even ones that are complete madness. If you want to do three levels of ethernet or something, we can't limit that in the GUI because you have no idea what people are going to want to do. You're trying to create bad traffic. You can't hold the users of your interface to any kind of sane standards. Common sense no longer applies. The interface also allows you to add a series of outputs of any combination of protocols. It's pretty simple. In this example, the left panel shows you a list of the available protocol stacks and the user just chooses a stack and adds an empty instance of it to the series on the right although there's only one thing in the series. It's kind of like ethereal, actually. You all have looked at pcap files in ethereal, I'm assuming. When you see a packet capture in ethereal in the middle panel, you see this expandable tree and it's broken out by ethernet frames. You see your first frame, your second frame, your third frame. When you open those up, you see all of the protocols that are in that stack, essentially. What I'm calling a stack is like everything that's encapsulated in one ethernet frame in ethereal. What I'm calling a series is your list of all the frames. Basically, our GUI allows you to create these structures rather than just viewing them. We'll also be able to import from pcap, edit, boot it back out. Then, with your empty instance, you just fill in the values you want for each field. Values that are left blank should be filled in with typical defaults at the time the thread is executed. You'll notice that some of our fields are set to values, such as internet header length and the flags C, E, D, F, and MF. Our GUI understands that false is one bit, zero, and true is one bit, one. Some of our fields are variables which start with a dollar sign. These values, like destination IP, can vary from run to run. Every time you run the threat, you might be targeting a different machine. We kind of just put little tags in here that say, hey, when you run this, we're going to ask you to fill these values in, because they're likely to be different from run to run. It's just a little convenience thing. We've also set several of our fields to functions, which start with the at symbol. The only function we're using here is random, which is in source IP and source part and sequence number and source MAC. That just tells the thread engine to put in a random value each time, for each iteration. It just picks a random value from the minimum to the maximum and puts that in there. We have decided on a minimal subset of functions because functions are things that get passed down to the engine, things like timestamp, random values, or ranges. It's something you want the engine to handle at the lowest level, but we want to keep our engine code running quickly, so we want to minimize what we're doing with functions. This syntax doesn't matter at all. Like, you could pick, you know, any way of specifying functions, any way of specifying variables. So here's our minimal function subset. From this, you can do most every operation that you want to. I guess it'd be like atomic operations. Some of the most useful are range, which will go range from like a minimum argument to the maximum argument. So you could target like a range of IP addresses from like, you know, 192.1680.1 to 192.1680.255 or something like that. Random, I already talked about. Random string is something you might want to do for fuzzing. You can generate a random ASCII string of a particular length. But more useful for fuzzing is the homogenous string of length X, which is kind of like a fill function. So you can use that with range to say, make a series of slashes in a URL request that just increased by one each time it loops through. Also, you need concatenation for a closure and check sums are best handled as a function that is something that's dealt with by the engine, rather than at the designer level. I gotta hide this. I can't even smoke. You suck. Alright, the threat file just contains a representation of your threat, including the stacks of protocols, the series of stacks, and all the variables and functions and values that you're using. One useful thing to do with threats is to include a good amount of metadata, metadata descriptions and target info so you can easily sort them into a database. The threat file is just a serialization of all the info from the GUI, so it's pretty much just the information you saw on the screen before dumped out into some common format. You know, XML works. To execute the threat, the variables will have to be substituted for real values and the functions resolved, which brings us to the next part of our platform, the binding and execution of a threat. This stage runs a threat file in a repository or your database back into the parser generator, but this time we're using the appropriate protocol grammars as the output and the threat file format as the input. This is also when the variables in a threat are substituted for actual values. This produces a chunk of binary that closely resembles the final packet stream, except for the functions which have to be resolved a little later. We call this the bound threat. The bound threat gets passed over a management interface, whatever you're using, from the design machine to the threat engine. The threat engine is like a virtual machine for network traffic. It must be a small, fast piece of code, which is why we wanted to keep the operations that we do with it fairly atomic. Its sole purpose is to read the bound threat in, evaluate the functions and boot out a packet stream. Another important aspect of the threat engine is that it must allow clustering so that the management machine can initiate a threat execution on several engines at once. So you can do like a man in the middle or a multi-part conversation. By allowing threat definitions to contain different roles for different nodes in a cluster, you can do fairly complex stuff, like the man in the middle. Our last design goal, import from PCAP. Once you have the parser generator and the protocol definitions, this is pretty straightforward. All you need to do is write an importer which strips off the PCAP header and gives you the binary of your packets. So you basically just get binary of all of your ethernet frames in a series. You can then use your parser generator with your protocol definitions as the input grammar to decompose the PCAP file and translate it into your own intermediate representation. Thank you. I'm Dian. I'm going to get a few people in your Qo10. Oh, crap. Okay. So this time we're using the protocol definitions which have each field for each segment of our protocol stacks broken out by bit length. So you look at your list of fields in bit lengths and you can decompose a PCAP file into which like protocol objects, you can pull out the fields and all this. Once you have it loaded in the GUI you can edit it like any other threat and boot out a threat file. A common thing to do is to substitute some of your fields with variables rather than just playing back the PCAP exactly as it was. You may want to put in a different destination MAC address or a different destination IP. If you're going the clustering route there's an additional step for PCAP files with multiple traffic sources. The user will have to explicitly assign each source to one of the clustered threat engines. So if you're going for a multi-node playback a PCAP import actually requires some user interaction. And here's some amazing word graphics about the general flow of a PCAP import. We take our protocol spec and the PCAP file and boot out a threat file. So to sum up the task of developing network threats is essentially that of writing a series of expressions in a variety of languages. By using compilers or parser generators we can automate a good portion of the grunt work and provide integrated development environments that can speed up most of the common tasks. We can also create more complicated and realistic test beds in the same integrated environment. And we can edit and playback protocols and deal with any current or future protocols. We're going to do some questions which should be rather interesting because I have no idea if I made any sense at all. But does anyone need clarification on something? We kind of took a pretty serious shortcut there. In our protocol a lot of protocols will have some sort of field which says what's encapsulated within them. Ethernet has the packet type and so on. Some of them don't. So like TCP and FTP for instance it's like you don't know by looking at your TCP fields what if any protocol is encapsulated in it. In our case we just added that to the protocol definitions as kind of a hint rather than trying to figure it out automatically. Cool, yeah. That's good. Anyone else? Shoot, no demo. Sorry. Yeah. It has to be another... it's something you add to the threat file. You have to manually add something to the threat file and you have to manually add a case to your engine to deal with it. So you say well we want a fragment and then the engine has to accommodate that. You mean by streams you mean fragmentation or like sessions? I don't think I can... Are we done? Okie dokie. I'm going to clear it out.