 Actually, wait. I'm standing in for Randy Vaughn. My name is Paul Vixie. Randy can't be here for family reasons and he knew I was here and that I have kicked Gotti while he was at the microphone before and that I would be okay for substituting for him on this talk. Many of you know Gotti Everon. He's working on downloading presentations because my laptop decided that this projector was evil in some way. So, we've already got a hand up. I wish I'd thought of disabling your microphone, but I didn't. Okay, just for clarification, this guy knows absolutely nothing about DNS, which is why we asked him to come here and fill in for us because if we know nothing and he knows nothing, it's pretty much the same level, okay? I'm not going to go over my whole bio, but what it says in the book is the... I will go over his bio. Paul Vixie. No, no, no, shut up. You don't work today. Paul Vixie, to make a long story short, either... So, what it says in the book is that I hold the record for most cert advisories due to a single author. In other words, since you wrote a book, you don't know shit. Which actually is true if you can't send mail and cron and Wu FTP and all the others. I did stop writing code, started writing paychecks instead, so I now sort of don't wear a suit anymore, but I do just management rather than programming, and you should all be thankful for that. In any case, during the years I wrote a lot of code. I also got to know a lot of DNS operators, so they still call me when they have trouble. I like to track attacks. Sometimes I will ask someone or invite someone to be hosted inside of ISC so that when they are attacked, I'll be in a better position to help track that attack. That's not what happened here, but... On with the actual topic, which Gotti will eventually get slides for, DNS amplification is a little bit like the old Ping Flood, where you spoof the source address of your victim and you send it toward some middle box, some reflector who just sees that as a normal ICMP query or DNS query, whatever it happens to be, and responds to it. Of course, the attractive part of this, there are two attractions to doing a reflector attack when you want to ping bomb somebody. One is that the traffic that is hitting the victim is not coming from the attacker, and so it's a little bit harder to track down. They can't come immediately back to you and say, I see that you're sending me the gigabits' worth of traffic and I see your IP address in the trace route and so I'm going to come after you with guns or lawyers or whatever it is. Instead, the victim has to go through a large number of these middle boxes and see if they can get trace routes done from there, but the trace route won't help because the source address was spoofed at the time it reached there. I do have a USB dock. Yes, you copy, I'll talk. That will work. Hi. So basically what happened is that we have the presentations, but Paul's laptop was supposed to be connected. And, well, that's a story, basically. So right now, Paul is skillfully moving over his mousepad and trying to mount the USB drive, the dongle, okay? And now that he's done that, he's moving the window around very skillfully. What? He's kind of shy, which shows that he probably knows what he's doing. Okay, a little bit about Paul. I don't really have any fewer things to do except for bad jokes and more bad jokes. Okay, the enosmplification attacks. There has been a lot of confusion about the enosmplification attacks and when it comes down to it, nothing is new about the amount of service attacks on the internet, brute force and the amount of service attacks. What does change over time are the techniques used, whether they are indeed brute force or in fact exploiting some sort of vulnerability in whatever space that may be, and how effective they are for how much you spend, which means if you send one packet and you get back 11 to a potential 255, say smurf, that's pretty impressive. And with DNS, DNS attacks basically have been around for a long time. Paul, how's it going? Going well. Going well, okay. And the point is that instead of sending one packet that is very, very difficult to filter and the DNS server would have to receive, whether it is legitimate or not, again, this is UDP based, you can send more. That's the basic of it and when you can send gigabytes of traffic amounting up to even 10 gigabytes at a time, that can be kind of significant. Anybody ever heard of blue security? No? Yeah. Okay, people heard of blue security. That's great. The what? The brute force. The brute force what? The brute force. What? Okay, great. You can go now to Mike. All right, as Gotti was trying to say before his... Very skillfully. Humor got in his way again. The important part about these attacks is not just that they come from someone who is not the attacker and are thus very hard to track, but that the response that are sent by these middle boxes are much bigger than the attack that is launched by the attacker. So, you know, yes, we know that you can assemble a botnet of 10,000 or a million or whatever nodes and send somebody one packet each from each of those places, but that's not artful. So, it's much more artful if you can get a couple of hundred of these things and generate gigabits' worth of traffic toward the victim. And that's what happened in this case. I have two sets of slides. One was generated by Randy and Gotti from the paper that they published on this topic. Another was released by Verisign. The other set was given by Verisign at a Nanog meeting and I know the people at Verisign. So, even though Verisign as a corporation has sued me, or at least they named me as a co-conspirator during the Sitefinder Wars, and I have no love for the corporation as a whole, I have to admit that they have some very smart people working there and some of them are my friends. So, I have his slides that were given at Nanog and there's some pretty juicy graphics on that one. So, what we'll do with the time we'll have left if we ever get the AV situation straightened out is run through both sets of slides quickly. And in the last 20 minutes or so, I'm hoping we'll get to kind of a question and answer about, well, what does it all mean? What can I do about it? What can I do to avoid being affected by it? Probably we will get around to why are we using a laboratory-grade internet for, you know, production traffic because that usually comes up in this type of top talk. Gotti. Yes. What are you doing? Um, wasting time. I think he's reading his email. Um, all right. Something not covered in the slides, so I won't be boring you when I go over it again later, which I won't. DNS had originally come of age in a time when you could only have 512 bytes in a response. Uh, that was irrespective of the headers, the IP headers, UDP headers and whatnot. You could include 512 bytes, and that seemed like a gargantuan lot. Uh, sort of the way that 640K must have seemed like a gargantuan lot to the people who invented the computer that had that as its limitation. Um, eventually all sorts of new things went into DNS and 512 bytes started looking pretty small, especially the crypto stuff that goes with DNS security. So, um, the IETF stood around and rung its hands and, uh, stared off into space for a long time, and I eventually came and made myself less popular. It's sort of an incremental process, uh, making myself less popular by coming in with a working solution that's already implemented and bind and has a, you know, RFC that documents it and so forth, and so they were left with nothing to do, but say, well, okay, well, let's do it that way. And the result was, uh, RFC 2671, so-called EDNS, extended DNS, and it adds a number of interesting protocol features to DNS, but the important one for the purpose of this talk is that it relaxed that 512 byte limit so that when you are sending a response, uh, if the person who sent you the request indicated with the EDNS signaling extensions, if they indicated in their request that they can take a bigger response than the standard 512, then you'll send them a bigger response. Um, and the fallback, if you can't fit your response into the available space, is that the requester gets told that it wouldn't fit, and the requester has to start over with TCP. Now, if you are sort of watching as I do a root name server take, uh, 10, 20,000 queries a second around the clock, uh, you know that, uh, UDP, which is two packets, one request, one response, is a lot cheaper than TCP, which is a minimum of about seven packets. So, we try to avoid overflowing that 512 bytes as often as possible because if we didn't, then the internet would stop working, although I'm not sure how we would tell. Boy, they're having more fun than us. Um, so, um, so what you do- Any experts on NVIDIA drivers here? Oh, God. Alright, we're winging it. It was supposed to be your laptop. No, it was supposed to be your laptop. We were going to use mine because you couldn't get yours working. I agree. Let's hook up a Mac. I've got it right here on a USB dongle. Somebody bring a Mac up here. That's not a Mac. That is a Mac. Okay, so, um, naturally, when you want to attack, if you want your, the, the middle box response, the actual attacked flow to be bigger than the triggering flow that you're sending from your little botnet, you need amplification. And is Mike Schiffman in the room? Mike, stand up? No? Okay. Um, Dan Kaminski in the room. Oh, man, they're, they're, they're, they're with the fun people. Um, okay, so those guys have done some studies on a form of amplification that is just continues to mystify all of us, which is where you send one packet to something that you think is a DNS server and you get anywhere between 20 and 50 responses to it. Uh, so it's some kind of, uh, broadcast amplifier. We don't know exactly what it is. Um, but there are a lot of them. Uh, it's a common enough thing. So if you do what Dan Kaminski does, which is to sort of sweep the IPv4 address base with DNS queries, sometimes you get nothing. Sometimes you get a response from an address you didn't send the query to. Sometimes you get a lot of responses. Um, these guys, the, the ones who launched the attacks that we're going to describe if we ever get this working. Hey, I see a Mac. That's going to work. Okay. Hey, they think they've got it. Um, so the guys that launched this attack... Fuck this shit! Woo! Woo! Apparently there is an NVIDIA toolbar. What the fuck? Okay. Paul, which one do you want first? And thank you, whoever you are. I don't know you. Whoever you are. I'm dissing you after you try to help me. I got my hands full. Yeah, kind of. Can we have a Randy's slides please? Um, which ones are those? The PowerPoint file. Oh, figures. Yeah, pretty much. I need my Linux. Had I made these, they would have been ODP format, just so you know. It's supposed to be, yep, here it is, F5. Here we go! Woo! Can you hear me now? Okay. Um, so the fellows who launched this attack, I assume they're guys, because women are usually more polite about this kind of stuff or sneakier. Um, the, the folks who, who launched this attack did a little bit of research to find out where the, uh, open recursive name servers were. In other words, name servers that were willing to respond to queries that did not come from their lands. So, they either got a hold of Dan Kaminski's dataset, or they just did what everybody else does, which is to add to the cosmic background radiation on the internet and send a packet to every possible address and see what answers. Anyway, they had a, uh, an idea of where they could send this stuff. Um, but they didn't use it. At least in the, the attack that we heard about through Verisign, it did not get used that way. Uh, what they did instead was they let their bots send their DNS traffic to whatever was the, the victim PC's normal name server. And we just, what we discovered by analyzing the attack is that those name servers are willing to bel, believe spoofed traffic that comes from within their own ISPs. Um, it is a mystery to me how the internet works at all. But I guess I've said that before. So, on with Randy Vaughn and Gadi Ivran's slides quickly, and then we'll get on to Verisign's slides, and then we'll get on to what I hope will be the real meat of this, which is Q&A. Okay, you're wondering, what is all that crap? Okay, the thing at the far right, although you can't read it, is TXT. These are queries for text records. Actually, these are responses about text records. And what the rest of it is trying to tell you is that these are frags. That the real message that you're seeing the pieces of is about 8K in length, and that you're seeing 1,500-byte frags because there's some sort of path M2U limitation between the middle box and the victim. One of the ways that this is sometimes mitigated is by turning off the reception of fragments, UDP fragments, but there isn't enough information in the headers of a fragment, the UDP header is only in the first frag, so there are certain things you can't do. You end up eating the bandwidth for everything after, I don't know, it's hard to mitigate this. I guess I'm getting ahead of myself. Since you can't read this, we'll move on. Here's what they look like up close. You can see that Randy is a gentleman. He didn't want the slash 24 that this had come through to be visible, so you can see that the last octets were dot 37 and dot 46. But really, you're just seeing something that came from port 53, which is a name server, in the destination port, in this case 31,000 and some, which was a random address chosen for a client context, and it's saying, okay, it's a DNS truncation event. What a shock because somebody tried to send an 8K UDP mobigram down a channel that didn't have it, and this is only a small part of it, the packet capture utility could get to. More of the same. More of the same. I guess Randy likes Hex more than I do. So, that RFC number is wrong. We revved it. It's 2845 now, not 2671. What's going on here is sort of, oh, God. All right, here we go. There's a mixture of NX domains. There are TXTs. There's a certain number of any queries. Most of the queries were for type any, and what type any means in DNS is somewhat opportunistic. If you send a query for all record types, which is Q type any, to an authority server, it will tell you the truth. It will tell you, it will give you the entire list of all the resource record sets of all types that are present at the name you asked. But if you ask it of some intermediate caching resolving name server, like the ones you list in your resolve.cont file, or get told when you do a DHCP transaction, it does not tell you the truth. It tells you only what it knows. So, if it has no data for that type in its cache, it will tell you that there's no data. So, type any is kind of useless, but these guys found a great use for it. So, they started out by seeding all of the name servers that they wanted to use as their middle boxes by making a query for type TXT so that there would be something in the cache. And they did this with, you know, TCP as their transport, and they did this, you know, with something that they knew would be about 8K worth of text. And then, man, oh man, they're having it more fun than us. I want to be over there. Can we stop? No, I'm sorry. We're not going to stop. And then they launched the real attack, which was a fairly small trickle of stuff that turned into an avalanche at the victim's site. Enough of an avalanche that, chances are that no matter what they did in their firewall, their upstream link would have been saturated, which is the traditional DDoS effect, is yes, I built a name server that was a mile high and took 10 terawatts of power to operate, but then you sort of blew the fuse inside all of my ISPs, and so I didn't receive the traffic I was paying to receive. This was discussed to death on Nanog, and there were all kinds of people who were theorizing about things, and very few of them. I guess it's the usual sort of deal. I'm sure it's true in your user communities also. The people who actually knew the answer did not speak up. So what the amplification looked like is a whole bunch of, maybe 20-byte packets got sent in, and a whole bunch of 8K Mobygrams came out, and when all was said and done, it was a 63-to-1 amplification factor, which, if you think about the battle days of Smurf amplifiers, 63-to-1 is damn good. Estimates are, and we'll get to this in the verisign slides, estimates are that it only takes about a couple hundred bots to generate a couple gig of traffic if you do it this way. Here's where we're starting to talk about the duplicates. Definitely get your hands on Mike Schiffman and Dan Kaminski and make them tell you about their 580,000 estimate. My estimate is different. Actually I spoke with Dan and he promised to get me better results and it means much higher, because this was written before he finalized his research and we never got done with that. Well, then, like I said, get your hands on Dan Kaminski, make him tell you about this 580,000 number, because it's either far too high or far too low, depending on how you look at it. There are some documents that explain what needs to be done, and here's where we get to the tricky part. What needs to be done is not going to be done by the victims, it's not going to be done by the middle boxes, and it's not going to be done by the attackers, and so what needs to be done isn't going to be done, but I'll tell you what it is anyway. The first thing is that these recursive name servers have got to get some ACLs. There's no reason for me if I am on a land somewhere and I'm listed in the resolve.conf of hosts on that land for me to be willing to accept source IP addresses that did not originate on that land. If they did, then chances are they are either never really were there, they're spoofed, or they got there by mistake, and in any case answering those queries would be a mistake. But that's the default, that's what we all do. And the reason that we do it is that those jackasses at ISC made that the default in the bind, so... Who is that at ISC again? I recommend that we all sort of let those ISC weasels know that they should set up the default ACL. What is the name of the guy in charge of the ISC? Is it Randy Vaughn? Yeah, Randy Vaughn. Send mail to Randy Vaughn and tell him that you want the bind default changed. More seriously, we did change it, and in bind 9.4 the default will be that if you get a query from something that isn't on your local land it will be dropped. And we expect that to make the phone ring off the hook with people who say they liked it better the other way because now everything else that comes to a name server from that campus will not work by default. And we expect that the usual thing people will do when it doesn't work is they'll put in an ACL that says allow anything because they don't want to be bothered with having to add their lands one by one. But I am hoping that that's the minority and that eventually we'll see enough upgrades that things will get better. Now bind, of course, is not the only name server under the sun. There are plenty of others. We have gotten in touch with Microsoft, tried to get them to change their default. But ultimately the people who need to make this change have no incentive to do it, right? They were receiving a very small trickle of packets and they were sending a very small trickle of packets, you know, on the order of tens or hundreds of kilobits. They never noticed that they were helping to fire bomb some poor victim somewhere with traffic that they could not mitigate. And so we're essentially going back to the sort of the spammers paradox of trying to get the people who are not being hurt by it, by the fact that there are open relays in that case, to make a configuration change that was going to increase their costs across the board. But that's what you have to do. So I'm hoping that as you go through life and you find name servers, you'll start asking the operators of those name servers, what's the ackle on that? Because perhaps if they learn that they will be annoyed to death and pecked to death by ducks, they'll decide that that's a higher pain threshold than making the config change. The other thing that needs to be done by people who will not do it is the so-called BCP-38. Paul Ferguson wrote this. Paul Ferguson and another guy, Dan Sene. Yeah. It says that if you're an ISP and your customer tries to spoof their source, you should drop that traffic as early as possible, and you should not fling it out into the global internet. Now, I can tell you as it... Oh, okay. Thank you for your applause, but you might hold that for a moment. I think that Sene and Fergie wrote a great document, but there was no incentive for the ISPs to do that. Because, again, the traffic that was being spoofed was only leaving their network. It was not going to be hitting anybody inside their network. So what you're doing, you're going to these people with an economics proposition that is exactly backward. Would you please spend money to keep your competitor from having to spend money? We want you to invest so your competitor can become more profitable. So they hate that, and they laughed us out of the room. So it's a great document, but it has not been widely deployed. I know there are some examples. Somebody told me that Internet was starting to really lock this down, and if so, God love them. But most people don't do this, even though it's a one-line config change on all of the routers made since 2000. And for reference, I'll just tell you, that means that a mom-and-pop ISP who is buying junk off of eBay that they cannot afford support contracts for, and they're buying extra copies of that junk so they can maintain their own spares, is getting hardware that is capable of doing this. That's how long it has been, the current hardware in the state of the art. But you will still find that inside of MCI, which I think is the new name for WorldCom, which is the new name for UUNet, which I don't know exactly how many other networks are in there, this is not the default. They're a telco. They have a lot of knuckle-dragging union technicians that they would have to upgrade in order to get this done. That's the expensive part. Actually, I just learned something last year. I'm actually pretty offended by this, but hey, do you know what's the name of knock engineers? Reboot monkeys. Reboot monkeys, okay. That's not funny, that's kind of offensive. Monkeys? What? Most of the problems are solved by a reboot and somebody calls them, hey, can you reboot machine AXBC? Sure, thanks. And you're lucky if they reboot the right one. So anyway, what we need is for ISPs to stop allowing these spoofed sources to come from inside their networks and then we need name server operators to put in some ACLs. Neither one is going to benefit from their increased costs, but we as victims would benefit from being attacked less often and that's the way the internet is. I'm going to switch to verisign slides, which are glitzier by... Goddy wants to say a few words. Wow, when you say it that way. The interesting thing here, yes, it's a denial of service attack, a distributed denial of service attack, yes, it is very impressive, yes. It has taken down networks, significant networks, but you should really concentrate on what Paul just said. On the one end, to make this attack work, we need to maliciously put some sort of DXT record, a SOAR record in this case, over at some sort of server you hacked into or put in place ahead of schedule. Okay, thanks. And... You Zoom? Control L, thank you very much. I do not use Windows as my usual thing and Hebrew is not my usual keyboard either. Isn't it, everybody? I should have a problem with that. I can type blind in English when I type in Hebrew, I have a ton of typos. Never happens, but I still have the stickers on the thingies. Anyway, that's the one end of it, so basically the attack on you is happening maliciously and not just by some bots pulling out data. And the other part of it is, imagine you live in New York, okay? And the crime that hurts you is on some other block in the city, across the city, okay? Now, the other citizens over there would kind of... How is the mayor supposed to explain to these citizens that they see a lot more cops in their neighborhood and they get imposed with a lot of regulations and rules about their movements when their neighborhood is kind of free and shiny and happy, ooh, shiny and all that, when the actual problem is across the city, somewhere else that you don't even see. So that's basically the point is the internet has become a huge... What do people like saying? Global Village? Yes? What? Who? Oh, that, right. I'm a DEF CON. If somebody is going to break in, I at least want to see it. You're a DEF CON. If somebody is going to break in, you're not going to see. That's why this is an expendable laptop. Looks to me like you've been broken into by somebody in Redmond, Washington. Woo! Okay, you saw it. So what I'm basically trying to say is that why should they spend money on you? Why should somebody in China spend money on somebody in the United States? Why should anybody be nice network citizens to one another when their attack does not affect them? So basically what happens is one day their attack does affect them, they go down, and then in a nice Dorinian way, everybody gets patched, but it takes a long, long time. So that's pretty much it. Everybody knows your password now. It's blah, blah, blah, blah, blah, blah, blah. How did you know that? Actually no, it's one, two, three, four, five. That's amazing! That's the exact same combination of my luggage. Never mind. Okay, so I told you that the Verisign people spent a little more time on the graphics, and this kind of shows a little bit more of what's going on except that even the Verisign, they don't know how to spell attacker. That's interesting. I hadn't noticed that until just now. Yeah, this is a PDF file. I did not alter their slides to make them look like they don't know how to spell. Okay, so there's this big TXT record. They've got an army of 300 to 500,000, or 30 to 500,000, I don't know. That seems like that was written for the suits. I know there weren't 500,000, but I think that's what it took and you get this... In this case, the DNS server that had the TXT record, the authority server, had itself been captured, right? So this was somebody's zone somewhere that got broken into, and the attacker put an 8K text record in there for the purpose of then seeding all of the middle box name servers so that they could then amplify the attack. And this is sort of part of the tried-and-true philosophy of don't put this... Don't buy that with your own credit card. You don't want the police coming to your door. You want them to go into somebody else's door. And the Verisign folks were very personable, and they went and located the name server in question and got a copy of the zone, and I'll get to that later. So here's a 20 megabyte... megabit per second spike. But this is only a 1% sample, so they were estimating that it was 100 times bigger. But they know that they're... They know how much... how much trans that they buy and how much their ISPs were throwing away at the time, and so they were able to estimate that the total size of the attack was 5 gig. This is pretty much the same attack, or at least the same attackers that Gotti and Randy were tracking. But they were attacking different TLD name servers at different times. And we, as far as I know, there's no publicity around any kind of arrests in these cases. So the folks that are out there doing this... We know who they are. Yes, but we know that they have not been arrested. No, should they? Come on, are we really going to go and arrest them? If I were in the business, yes. If I were in the business? Well, we'll be in a lot of trouble then. Okay, so there's the name... the zone that got broken into is in South Africa. And this one was a 4K answer and unlike the 8K Mobygram that Randy and Gotti were tracking, but it was a 60-byte query, so you see the amplification factor, you see how many reflecting middle boxes there were. Now, usually, when somebody is DDoSing you, they will fiddle around with the IP time-to-live flag, which gets initialized by the sender and decremented by every router between the sender and you. And depending on what games they play, you can tell whether they were attempting to randomize it, whether they succeeded at randomizing it, or whether they were not able to set it. Some APIs don't allow unprivileged applications to set the IP TTL. And so if you get enough of this data, and we've got Dave Dagan here, I see who has studied TTL in detail. So if there's Q&A on this, then we've got the expert. Anyway, by looking at this sample, they were able to tell that the traffic, the actual flow that hit the Verisign name server was itself not spoofed. That actually came from real places that were setting the TTL in normal ways. And proving that it isn't spoofed is one of those expert things you can ask Dagan. Sequential IP addresses? Yes, so in other words, they found these name servers with a sequential scan, similar to what Mr. Kaminsky does. And Verisign and a lot of other people went and looked at the sources of these and said, well, gee, they were sending me these big Mobygrams. Does that mean they're open recursive name servers? Can I do a dig against them for data that is not local to them? And sure enough, yes. And then we started to notice the true scope of the problem. Okay. Yeah, yeah, yeah. We don't care about this. Right. There's this estimate that the attackers only had to spend 80 megabit to get the 5 gigabit response. And that 80 megabit was spread across a bot army of somewhere between 30,000 and these people estimated 500,000. But 80 megabit, you know, host can generate 80 megabit these days. And it would be very difficult to find out which host had suddenly shown an 80 megabit spike because two or three hops away from that colo it would be a bunch of very small flows headed to, you know, 100,000 open recursive name servers or 580,000 if you ask Dan. And I do hope you ask Dan. So the domain in South Africa had two authoritative servers but only one had been compromised. In other words, they broke in and changed something that should be a slave name server to be a master name server and then edited the zone in place. So there was a chance that when the middle boxes went to go get this data so they could seed the cache that they would talk to the wrong name server and potentially not be able to cache it. But even that didn't really stop this from being a very successful attack. Well, okay, it's a verisign slides on the board here and even though they named me as a co-conspirator during the Sitefinder Wars I don't want them to be unfairly characterized here. I don't know if this was successful in costing them any money but it certainly got their attention. Okay, so this talks about the fragments. You can get copies of these slides at the URL that I'll tell you at the end. So if you want to study this in more detail just basically if you are trying to send a UDP datagram bigger than your Ethernet M2U you'll have to break it up into chunks and those chunks look very odd at the receiving end because DNS is normally not large enough to need to be chunked. They were measuring 25 minutes. Again, this is only one of a number of attacks that happened across about a two month period so Randy and Gotti had different spans of time. In some cases these things go on for days because there's really no incentive to stop it. I mean it's not costing you anything. There's no chance you're going to get caught. Hey, he spelled attack right on this slide. Yeah, see the near vertical start is part of what you can sort of tell. This did not ramp up over a period of five minutes. There was a lot of synchronization and getting all the different attack flows to get started at the same time. Okay. Yeah, this was mitigation is always what people want to talk about which I worry about sometimes. How can we live with this instead of how can we avoid having to live with this are the two different ways of approaching that question. So there was this thing. Okay Dan, you've got 580,000 name servers on this list. What if all the authority name servers on the Internet were to subscribe to a black hole list that was seeded from your list and they just stopped answering any queries from somebody who was known to be open and recursive. Well, I think the collateral damage would be too high. I don't see Amazon.com deciding that they don't want to get any queries from known to be open servers. It costs them a lot of credit card transactions among other things. So dropping fragments toward the victim turns out to be pretty effective but as most of you know... Dan Kaminsky on the line here, so hold on a minute. Dan, can you come here to the lecture about DNS? We kind of want to ask you a question. Yeah, that's a good idea. Let's get Dan in here. Oh, damn. Oh, you're at Caesar's for a second. You left some shit there. Okay. So when are you getting there? Too late for our purposes. In other words, get here. Anyway, these are all ineffective because if you decide you're going to drop fragments then they will retune their attack to use the maximum size packet they can send that doesn't require fragments. So all of the things that you could do to mitigate this will be as costly or more costly than just living with it. So at the moment, we all sort of live with it but it's not that different from Smurf amplifiers or other ping of death type scenarios. Anybody can DDoS anybody on any given day on the internet and you just hope you haven't pissed anybody off who's in a bad mood. Well, about that issue of... basically some people said that limiting... I said it originally too and some people including me said originally that... sorry for repeating myself. If you limit the packet size to 512 bytes, you limit the amplification. Now theoretically and practically that's true but before the but if you get to limit one bot sending out packets for one basically set of fragments to have 7 times 7 amplification instead of say 2 times 70 that's pretty impressive, right? It would work but you would at the same time be limiting your DNS server abilities be limiting the ability to use IP version 6 and a lot of other such expansions to DNS and it wouldn't matter and I'll tell you why. This attack is built in a way much like in any security system in any system in any way at any stage of the links of how this attack works from how many bots are sending the packets to how many open recursive servers you are using or connecting to and so on. You can use more bots, you can send more packets you can connect to more open name servers and cause the amplification to be just as large if not larger. You basically control how big this is going to be until the point where you reach saturation. For example... You're burning up the Q&A time. Put the mic down. Here's another long list of things that could be done but would hurt a lot and give very little benefit. None of these will make the attack unattractive so these are just speaking points to the suits. You've got to really study these slides and hear the guy who made them tell you what they all mean. Let's move on to the Q&A because we have about 10 minutes. I think we're going to do that. This is the slide I wanted to stop on here. We have microphones, please use them. Someone? The URL where these slides can be had is public.orci.net and it's a Drupal site and I apologize for that and you'll have to click around a little bit. It's part of a DNS operations forum or workshop that was done so it's your two or three clicks from it. It's public.orci.net The study by Randy and me can be found online. Just do DNS amplification attacks in Google and you'll find it. First question. Are you guys aware of any utilities that have been posted at places like PacketStorm for instance? No. This isn't like nobody's put something together and published it yet? Work for 30 minutes in Python and you'll have it. Nothing public. I've seen the kit but I don't think it's online anywhere. Gatti mentioned earlier that you'd need to for the text record attack go and put maliciously something into text records. I've actually done a little bit of research on data mining text records that are already out there and there's quite a few that are pretty large. I've found ASCII armored encryption keys I've found large shell scripts and even one had a very large chunk of the book of Revelation which would be great for a DITOS attack for Horsemen of the Apocalypse clogging up your series of tubes and that it'd be great. You might not necessarily need to maliciously put something out there just find something and use it. Yes. That is absolutely true. You don't need to capture anybody's name server. You can use normal traffic. The response that you'll get from www.microsoft.com is a lot. As soon as we get DNS security really underway and we have these big crypto blobs attached to every RR set on the internet then once again the ease with which you can generate a gigantic response coming out of somebody else's name server will just keep on coming. Absolutely. One more thing about this is that when you use something that's already out there public servers with huge records really limited because you can use quite a bit of servers in another link in the chain instead of just adding bots or just adding name servers but in this particular case the guy put a malicious record of his own in one name server and it took quite a bit of time to get it down but he had enough bots and enough name servers that were open to generate the attack sites he needed so he didn't really need more records elsewhere. Yes, needless to say had I crafted this attack it would have been a lot smoother. You sound like a virus writer. Actually, I sound like an anti-virus guy. I'm sorry. My mistake. Question to Paul. What is the normal behavior for the properly configured public name server when it receives the query, the request for the an object in the zone it's not a 3084 and from the land that it's not supposed to answer recursive queries for. Is it still responding something or is it just dropping this request and keeping silence? I don't think you meant to but you asked three different questions. So, if someone asks you a question and they're not in your ACL then you could I suppose answer with refused there's an opcode out there called refused but given the purpose of an ACL and that ACLs were not defined by the DNS protocol and this behavior is therefore not not defined I think that the spirit of an ACL is that if you get a question from somebody who shouldn't have sent it to you shouldn't answer it. If you're an authoritative name server and let's say for example.com and somebody sends you a query for you know foo.net most name servers and again I blame those evil bastards at ISC most name servers will give you a root referral. That's what I was referring to. Still a lot of data. It's still a lot of data. It's a big chunk of muck that you can't get any value from and should never have been sent but the document was unclear and so the code was unclear and that's just what it sort of does. One way you can turn that off is to follow my personal recommended practice which is don't run recursion and authority in the same name server instance and on your authoritative name server don't give it any root hints if it doesn't have that data it won't send it. Okay, so the right for the DNS administrators out there the right behavior is to send a refuse. Just drop it I would say drop it because I'm aware of virtually all DNS requester source code bases out there even the Microsoft one I've talked to the developers and so forth no one process is refused so you may as well not send it because it's not going to do them any good even if it was sent legitimately. Thank you. We have three minutes but we have three questions. Okay. Is there or couldn't there be some sort of legal liability associated with running an open DNS server open mail relay allowing originating spoof packets etc. Could there be a legal something whatever about running an open relay mail server? What I'm saying is you know if you're running if you've got a DNS server and it's connected to the internet and it's allowing recursive queries from anywhere that's a legitimate use for DNS we don't have time for this so he is not a lawyer I am not a lawyer there are lawyers in the room they're not going to answer this because they don't want to be your lawyer without getting paid but what I believe is that if you can show that they have been hurting you here's the pcap here's your cease and desist letter then if they don't cease and desist then there might be some civil liability in no case criminal liability that I can foresee I didn't even mean criminal I just meant like tort law is there any reason other than legacy for the text field and if not can't we just get rid of it i.e. block it with a firewall or just remove it from the code? TXT is not the problem as I said you can use DNS security records you can use just normal A record sets that you would get back from any server but to the extent that it is a problem it's one we will have to live with because TXT is a catch-all it is for example being used by the SPF people to contain all of the mail server sort of ACLs that they want to do about what mail servers are able to send mail from a given domain name so we're stuck with it but it's not the problem so it's not bad that we're stuck with it so from everything I've heard so far the possible solution here is ACLs but that's not going to scale when you're using valid SPF, DNS, etc when you're not pre-posting or you're not doing a recursive DNS if we actually implement ACLs as you're suggesting then that's not going to scale because as you said they're going to change the attack as such don't you think the problem is actually the fact that you can spoof the packet in the first place so wouldn't you say that the problem is in UDP and not DNS? Certainly in this laboratory grade toy for production use that's an excellent question and with that I'll turn it over to 10 minute break and then Gotti will be back here by himself talking about other DNS related stuff thank you