 I'm Raven, and it's nice to see a bunch of familiar folks, and is that better? Alright, I will eat the microphone. Not like that. Alright, I'm here to talk about penetration testing for backbone networks, and I had these really interesting great slides, and then Wednesday some guy came and revolutionized my field, and I haven't gone to any parties this year, and I've done a lot of late night rewriting, so thank you Michael Lynn, and the reason we were waiting is that my lawyer isn't here yet. Oh, she's here, fantastic. So the first thing that you really need to address when you're looking at penetration testing for backbone networks is that there's a pretty severe culture gap between ISP folks and the hacker folks, and there's not really similar views on disclosure or vulnerabilities or anything like that, and that can be kind of problematic. So how many of you do work or have worked for an ISP? Joe Hans? Okay, you guys have probably run into this. How many of you were in the routing department rather than the security department? Joe Hans? Okay, I used to work for ISPs. I spent a bunch of years doing like senior ISP backbone engineering and things like that, and I eventually left for a more security focused role because I got really frustrated with just the differences of opinion. A lot of ISP engineers, when they look at vulnerabilities, they tend to have a very, oh well no one's going to do that sort of view of things, and their views on actual attacks, if they don't see it in the wild, a lot of times they are very unwilling to actually treat something as a real problem, and so it takes something like, well, Wednesday to make people sit up and take notice. ISP engineers in general tend to go for a limited tiered disclosure method, so when something is found, you know, sprint in UUNet and people like that here first, and then Cisco's next favorite children here second, and so on and so on, and so if you have their gear, which are a small network near the edge, you will hear late, if at all, and most ISP engineers think that this is good. They believe that, you know, catching the core is the most important thing, and then moving outwards from there will release things in a more slow fashion and minimize the window vulnerability. I have a feeling that most of you don't think this is a good idea. So when you're trying to do pen testing on a backbone, you're going to have to work with these ISP engineers, and you have to sort of be a little bit gentle with them because otherwise they'll immediately say, Evil Haxor and stop listening. So for maximum efficiency and security, generally when you're looking at testing a backbone, I generally recommend doing it in a white box fashion rather than a black box because you can do all sorts of recon and data gathering and such when you're actually doing it in a black box fashion, but so many of the vulnerabilities that there are in backbone here, denial of service, that people get a little pissed when you take down their network. And even if you think you've explained this to them, you've signed the contract beforehand, you make sure that you're legally covered and say, No, look, I could knock over your router. They don't know what that means a lot of the time, and so then they get ticked off that their backup didn't go through or suddenly they can't get to their router that you knocked over because it's offline. And so you want to be really careful when you're setting up your gigas, you want to have a pen tested backbone because a lot of people don't really understand their ramifications. So when I do this, I generally go with a white box sort of testing methodology. So when you're doing it, it's just like any other pen test. You do your initial block of recon, you look at the registrars for the address blocks and the autonomous systems that they have, and any other data that's associated. You can often get contact information for social engineering, all sorts of information about the network that you are intending to test. By doing a lot of these standard pen test techniques, you can identify hosts on the edge, but looking at the other IP addresses in their net blocks is more likely to give you addresses that they're using on their backbone. Now, a lot of people are or should be filtering information on the backbone from things past the edge of their network. In a perfect world, you shouldn't be able to hit the routers themselves from outside of the network unless you are on a management station or something like that. It just traffic should go through, and you shouldn't be able to target. This is really, really not a perfect world. A lot of ISPs fail at that level of filtering, and we'll look at some stuff from the routing protocol security group a bit later on that will show you some data on that. It is worthwhile. There are many public route servers, and it's worthwhile to look on those and internet maps to try and determine who is connected to whom and where the peering is. A lot of peering happens at public exchange points, a lot of it happens elsewhere. You have private interconnects between different ISPs, and those will be crucial points when you're actually mapping their network. You can often also on route servers get information about their BGP routing policy and try and figure out what they accept and what they send to where. Community streams will often be viewable, important information about like BGP and which routes are preferred, and there are many public route servers over there that you can look at things from their perspective. So if the network that you're analyzing is not one that is directly connected to a route server, looking through two or three will give you several points of view into how to look at it on the net. Whatever you get, dump it into Google, and if you get good results on that, dump them right back into Google, and that will really help you build a profile of the network. I believe it's really important not to just take the network maps that their admins give you in Visio or what have you, but also try and build your own map non-destructively. Their engineers aren't aware of everything. Often they're not aware of much, and that can be really important when you're looking at things from a pen tester's point of view. If they don't tell you about a link and you don't look and you don't find it, you're not going to test that. Also, a lot of ISP engineers aren't terribly security aware, and so they'll post a mailing list and say, here's my config. Gee, this isn't working, guys. What do you think? There's a lot of work of information about what they're running, how they're running it, what kind of gear they have, and all of that can be dumped into your favorite friendly neighborhood vulnerability database to build a likely profile of attacks and exploits that you could use against their network. Now, a lot of the backbone is built on gear from a very few vendors, and these vendors are generally not fans of full disclosure. Cisco and Juniper both have on their networks things that are behind the front-end of their web page. You have to have a login. In many cases, you have to have a contract in order to see details. Cisco does release some vulnerability information publicly, but there's a lot that they don't. Juniper releases almost nothing publicly. I think I've seen like six advisories from them ever that didn't require some sort of back-end acknowledgment, and so they believe that by doing this, they're helping to keep it out of the hands of strip kids. If you have a contract with them or if you have a login to their site or something like that, you will have a much greater database of information that's available to you when you're trying to determine what to test for. We're going to talk a lot about vendor negotiation and dealing with your vendor in order to secure the network address as possible. If you are working for a company, they may allow you to use their CCO or Juniper logins to access the bug databases and look for vulnerability information. Also, don't forget that the backbone is more than just routers. There are switches. There's lots of stuff that gets done on Layer 2, and a lot of people just kind of ignore that, so test through your switch here also. All right. Cisco in particular has really complex code trains. iOS has a zillion different branches which support different features and for different hardware platforms and such, and you will need to be aware of what you are running in the network you're testing. If they've left CDP on or something stupid like that, then you can get them to tell you. If their engineers are friendly, you can get them to tell you. You can log into all the routers if they're giving you that level of access and do a show-over and pull the details on that. But something that is vulnerable in one train may or may not be implemented in other trains. It's worth looking with a broad brush to start off with. Something might be vulnerable and then going, oh, okay, it's not a vulnerable version is much better than saying, oh, well, it's only this one version, let's not test that. It's also worthwhile in looking at code trains and where they're derived from. The code is forked all the time, and sometimes you'll come up with an error that was in one version and, you know, years and years down the road, oh, look, it's still there. There was a BSD vulnerability in Telnet a while back that, you know, whacked half the stuff on the internet almost regardless of operating system derived from that old code base. And I have a link here to the vulnerability in Cisco's CADOS boxes. Use that same bit of code and we're vulnerable to that same bit of, you know, exploit. There are some scanners that will incorporate these checks for you. I know Nessus has some Cisco vulnerability information in it, but a lot of it isn't automated yet. And because of the great potential for denial of service, you can see why a lot of places are real thrilled to have that run against them and why a lot of people haven't been too eager to put it in the exploit tools. So this is something that, you know, think pentesting like five or ten years ago. You know, you're going to have to do a lot of this by hand in order to make sure that you've got everything. As with any other pentests, failure paths and trust relationships are important. When you look at their architecture, look for redundancy, look for network robustness, and look for security. One thing that I think is really important when you're pentesting a background is to make sure that availability is a factor. I mean, denial of service is such a big deal for them, and yet so many people build networks with one path where a link goes down and you're screwed, or they have two lines terminating into one card on one router and a hardware failure will cause problems. I think as security analysts, it's our job to point these things out and go, oh, you've got a single point of failure if someone knocks this out. If there's an exploit, if you have a hardware failure, you have an issue. That's what I think about when I present a report to the client. Also, check for the authentication means and see whether those have redundancy. Just as with some of the SSL versions, you know, if you can force it back to the weakest point, then you can exploit that. There was a vulnerability on a radius server in Cisco's implementation of radius where you could give it several ways of authenticating and none was an option. So if you can just say, oh, I don't want to authenticate, no. Oh, okay, then. Things like that are kind of bad. So with things like that, check for what the weakest way is because you know your attacker is going to take the weakest way and if pentesters sell away. Once you're authenticated, it's trust transitive. If I get a log into that radius server, what else does that give me? It's important to look at that, and so as soon as you get every new level of access, every new point, if you try and map the network again from that point of view, you may find extra stuff. One of the things you'll see a lot in one of the ways I've had most success in testing backbone networks is authentication servers. If you have a centralized authentication server, that's a really good point to start at. If you look for a log into that, you can sniff traffic and get someone's login if they're foolish enough not to encrypt it. If you can own one of the hosts that they use for management stations, they can send it up before it's encrypted and sent on the wire with a keystroke log or something like it, and that will often give you the keys to the kingdom. Very many places don't think to harden their authentication servers, and so that's a really common point of failure. Another thing that I have successfully done is if they have fallback mechanisms where they have a local login in case their radius or their attack acts or whatever server is down, well, if you can knock that server offline and then brute force the local password, that will also get you in. And once you brute force the local password, depending on how they're storing it, like for a Cisco box, they use a 7 encoding, which it's not encryption, it's encoding. It's symmetric, it's easily reversible. Alcrypto.co.uk. Cisco has an online decrypter, although you may wonder where that's going. So, you know, they also have programs you can download and run locally. These are saved in the configs. You can just, you know, throw them in the internet, get the password out, and they may be using that in multiple places. A lot of places do not do password management well, and so a local login in one place will get you a local login in another place. And as long as that central authentication server is down, there you go. Obviously, once you've logged into the boxes, if you have enabled, you can change that. But that will let them know you're there. You know, please be nice. Um, let's see. Also, man in the middling against the authentication server is usually a fairly successful technique if you can get access to do that. It shouldn't need mentioning, but physical security is really important. A lot of places forget this. Everywhere you date, have a data center, and especially everywhere you're peering. Please have good physical security. Please audit and make sure you have good physical security. Please don't let me walk into maze with the switch under my arm and a laptop under the other one and plug in. Yeah, bad. A lot of the time, if you have local access, local physical access, that will get you a lot. And many of the places, they have a very clear profile of what they're looking for. So, you know, if I was 40 and had a beard and came in, they might be a little more suspicious of me than if I'm 29 and got my eyelashes and go, gee, I don't know, I'm just supposed to deliver this router. You know, they have poorly placed trust relationships in terms of local access. And if you want to take a social engineering approach to that, then that can work to your advantage. If you don't have physical security, you're screwed. One of the things that I've seen fairly often that's kind of appalling is how many people will leak data about the processes of the cord of their network to the edge. So they'll have EIGRP as their internal routing protocol and they won't turn it off for interfaces that face out towards the edge of their network. They just sort of send those multicast, hey, hey, who wants to be my neighbor out to anyone? And well, you shouldn't let people of the edge of your network establish a neighbor relationship and figure a bunch of route data that people will leave it on by default. They won't turn it off. And it will happily send you all kinds of information about the connected device. It will tell you what it is, what version of the operating system it's running, what sort of hardware it has, device ID, all kinds of stuff. And Cisco has designed this so that it's very useful when you're troubleshooting. I have had cases where I wanted to see what was on the other end of the link and we weren't sure that the person either the central office had us plugged into the right ports. So the other side turns on CDP and you go, oh, I'm not supposed to be connected to a 70 to a 60 XR. Yeah, that's wrong. So when troubleshooting, it's a good tool. But when you're not troubleshooting, turn it off. Switching data, also switch management protocols, things like that will be happily sent out. Hey, welcome to my spanning tree who wants to be my root bridge. These things should not be seen by users and they often are. So that's something for you as pentesters to look for. If you can see the data, you can send things back. You can spoof packets, you can send real data, you can feed them information on who's supposed to be bridging to what or who's supposed to be routing to whom. If you can get a more preferred route and inject it, then that will, depending on the implementation, of course, it may be able to take over the route that's actually on the legitimate internet. A lot of this is when you're speaking internal routing protocols rather than BGP, it's a lot easier to deal with on the local net. BGP has its own separate set of rules because of how it's handled between providers and how you set up neighboring relationships. But on the local network, it's often not that hard. If you want to do a really sort of not high-skilled but very effective DOS attack, you can tear down links with spoof packets. A lot of the time they don't have authentication on there. And so many routing protocols support the capacity. Even something as simple as an MB5 hash, it's often called BGP password, although many other protocols also support the same thing. I know you guys have heard about MB5 fairly recently, but it's still a whole lot better than nothing. Similar to the period of rogue access points, if you have visible access to the site, you can just walk right in with a router and say, yeah, I'm supposed to install this. Some places aren't very good about saying who the hell are you, why didn't I know you were coming, you know, go home. And you can plug in a router, you can put it into the peering switch, you can just start sniffing traffic. If you can manage to get the switchboard set promiscuous or something like that, then you can see what they're speaking and start to speak it to. This also is obviously bad. Depending on what your contract is with the client, you can even go in there and say, oh, yeah, I'm from this company, I'm supposed to work on their router, yank out theirs, put in yours with the configuration of whatever you want and hey, as long as you pass it through to them they may not even notice it's up for the 10 minutes of reboot time. A lot of places do not have good passwords. It's better than it used to be because it's spammers. There have been cases of spammers taking over Cisco routers and then using the telnet function to send spam out of the router itself, one character at a time. So, yeah, that. But a lot of places have gotten better about not having, at least, pathetically default passwords. However, if I get called in and someone blocked out of their router, I always get Cisco, Cisco, a try. You know, maybe one time before it works and it shouldn't. You can redirect traffic through a GRE tunnel. I've seen that in a while and people were a little bit confused about what was going on there and I don't remember setting that up. Bob, did you set that up? Meanwhile, it's terminating Moldavia or somewhere and there go all your packets. Another serious concern is that, especially when you're speaking of BGP, you may be able to advertise additional net blocks outside of the net and if your route is preferred to other people's routes, you've just hijacked a net block. And this happened. There was a bunch of attackers that set up a site through an ISP that wasn't doing good security checks about who actually owned that IP space. They announced this block out to the world and their route was preferred in most places because of the way that they did it and how their provider was set up and they happily spent about a week sending out spam and sending out worms and set up a botnet and were owning boxes left and right. Meanwhile, the legitimate users of that net block can't even get online because no one's routing the own site. And so when that was finally resolved at the ISP level, then the original owners are really left holding the bag there. Hi, you're on every blacklist on the planet. You have a whole bunch of angry people pounding down the door at your abuse desk and you're offline for a week. And because some ISPs do not do good checking when they're hooking up a new client, that do you really own this block? Is this legitimate? That can happen and it shouldn't. There are really good people working ISP security that are cooperating between ISPs to try and prevent that and more power to them. We need more of that. A great place to start when looking at the actual configurations of the background devices is to look at the secure BGP template, iOS template, Junos template and Junos BGP template that the Khmeri people do. These are very, very well done documents. They have Bougon lists of routes from IP addresses that haven't been allocated or don't exist. They do strong filtering so that you don't have like, you know, RFC 1918 addresses speaking to you. You shouldn't allow people to advertise, you know, your own net blocks to you, for example, things like that. They have good checking for the strongest possible authentication methods. They try and turn off things like, oh, hey, let's speak finger. You know, ancient insecure demons that you have no business really needing but I can't recommend these enough. Those guys do really good work and it's also handy for a beginning place to start with an engagement if you're doing a white box design analysis. Peering security is particularly important. When you look at how someone is peering, look at the routes they're willing to accept and look at the routes that they're willing to send out. There was a case several years ago where someone actually advertised 10.0.0.0 to the whole internet and claimed that they owned it and that got a lot of people a little bit irate. It broke a lot of people's private network implementations and it broke a lot of VPN stuff. So that's worth checking. Again, route servers are your friend and when you do peering changes, make sure you're doing good authentication. You don't want just anyone to call you up and say, hi, I'm Raven from ViewUnit and yeah, I want you to change this config. Can you take out this net block of this guy who, you know, stole my girlfriend last week? You want actual passwords, some sort of validation method to make sure the person who you're talking to to make these major affecting changes actually is the person they claim to be. Also, make sure that the machines that you use for your network policy and your network admin are well secured. I have worked at a job where we had a couple of management stations and they were Windows boxes and wow, they got owned. Really dumb. If you have only a few machines that are allowed to access these crucial backbone devices, make sure they're hardened. Make sure they're patched. Make sure that people can't get onto them and, you know, in that case, I think what happened was these machines that we did backbone stuff from were the same machines that we did abuse desk from and so you're investigating abuse case and oh, our users surfing porn, go to the porn site, porn site's full of malware, leave, boom. Don't let that happen. Filtering is also very important, not just what you allow to your router for management traffic, but also what you allow as far as routes. You want to make sure that you do not allow internal routes that should stay internal to your network outside to the world. Yes, aggregation happens, but more precise routes are often preferred and that can be a problem. It bloats the routing tables, it uses up memory and it can be revealing more details about the interior of your network than you wanted. The RPSet Working Group did a small survey of people in the field and they surveyed people who were appearing at an exchange point. So these are some of the more aware, more security conscious people out there and they still weren't using everything that was available. This may be because for lack of a better term ISP culture, they view change management in the time that it takes to do good password control and to change all these policies as more of a threat than the security implications. And so you've got to keep track of who's allowed to do what and who's filtering where and oh, we screwed up our appearing list and now there's downtime because I wasn't allowing your router, I had too many prefixes and so they think that the hassle of management is often not worth the additional security. And from an ISP point of view I can see why they take that approach when you're driven by the bottom line and you're a network engineer with too much work and too little time. It can be difficult to deal with these sorts of things. Nevertheless, I think this is going to become increasingly important as more attention focuses on the backbone as, you know, started this week. The RPSet Site Security many people are actually doing an incoming prefix filter for each BGP customer which is good. You should only allow people to advertise their own netlocks to you. You can start advertising your netlocks or someone else's netlocks or that guy who sold your girlfriend's netlocks that can get you in legal hot water. A lot of them also are checking for incoming autonomous system paths. A lot of people use community strings in order to control this sort of thing. However, very few people are using an outgoing AS path filter and in that case it really depends on who you are. If you are a small ISP near the edge of the network, you have your AS and you have a few others that are your customers, that may actually be a good idea. If you are a tier one and you have every AS in the world flowing through you, that can be a real change management challenge and probably operational analyze. Outgoing updates towards your peers and a lot of people use communities on that but not everyone and there is the URL here for the actual survey if you want to look at the results from the RPSet group. I didn't think them all up. So, when you are looking at actually doing a pen test where do you get your background vulnerabilities? They are not all on the scanning tools we have established that earlier and Cisco and Juniper may have you log into their site to actually look at it. Mailing lists will have public vulnerability announcements. I have often found that it is a good idea to check the mailing list against the vendor site because hey, people change their web pages all the time. Mailing lists archives are usually more static. You can at least see that we saw this happen this week when Cisco announced their IPv6 vulnerability and if you look on their website now it is 1.1 the original one that went full disclosure is 1.0 there have been some changes made and it is valuable as a pen tester to be able to see both versions. There has been in the last five years increasing interest in this field by hackers who don't come from an ISP background. FX and Paul Watson and Mike Lin have all made valuable and important contributions that have really caused a bit of uproar in the ISP community and after this week's uproar I think there will be a whole lot more people looking for it. So FX has done a lot of stuff in this field that is really useful. There is a EIGRP spoof packet network neighbor saturation bug where essentially if you send it more than 256 neighbors then it just falls over. EIGRP works by sending out like multi-cast hey here I am who wants to be my neighbor and so if you say me and me and me and me and me you can essentially overflow the amount of memory that it allocates for that and you know it is at least a denial of service it could be worse. Similarly for the CBP neighbor announcement if you send Cisco discovery protocol packets with device IDs that are really really long you can fill up the memory tables and possibly do evil after that there's been GRE attacks the TFTP server advisory and a bunch of other stuff if you look at FX's site there's tons of stuff and there's actually a tool for pen testing that was really specifically to address these kinds of things it only addresses the vulnerabilities that FX actually found but it is well worth looking at. There's also been a really interesting paper about Cisco memory mapping and I am sure any of you that were at Black Hat on Wednesday saw a similar work in my Glenn's talk the memory structures are still the same the magic numbers are still the same they're all online and it is very useful if you want to dig into the assembler. Then about a year ago any of you that were at CANSEC probably heard the storm and fury surrounding Paul Watson and the TCP window. Essentially TCP stacks on background equipment and many other things were not correctly looking at the window size and anything that hits spot on just anything in the general neighborhood was good enough so if you send a reset that's in the general neighborhood and you're able to correctly guess the IP address as important numbers on either side you can knock down the session with the scoop packet and this allowed unauthenticated BGP sessions if you could guess it all correctly to be knocked down and this was really interesting when you look at disclosure and how the whole thing went out essentially what happened was when Cisco started notifying people of this because Watson worked with Cisco and said hey look you know we need to fix this many people on the backbone and in like Nanog and similar ISP communities had known about this for years they just didn't think it was very possible and again we see that attitude of well you know I'm really busy unless I see something in the wild it's not a problem it's not going to happen and so Cisco said okay everyone you need to put authentication passwords on your BGP sessions now okay you need to upgrade now they didn't tell people why and so they told their favorite people first and then their second favorite people and their third favorite people and so on out and so it took I think the better part of a month for it to actually get heard by most people and then Watson gave his cancer talk once the ISP operators found out what the actual flaw was there was a very strong cultural backlash saying oh I knew all about that you know why are you making such a big deal and I was really depressed by going to Nanog this year where a lot of operators were like oh yeah you know we thought that was important but wow we really freaked out and I rolled back I don't have any authentication on my sessions anymore you know stupid hackers and that was upsetting because I really saw the community response to the Watson vulnerability as a strong positive step in securing the backbone and then seeing everyone go wow we freaked out over nothing did not help I think Cisco's approach of we're not telling you why just patch it because we say so no really just trust us I think they hurt themselves because people trust them less than they used to and then we come to Wednesday and what made me rewrite my slides if you were at Black Hat and saw Mike Lynn's session on Wednesday morning he essentially demonstrated what I view as a remote exploit in Cisco iOS Cisco has announced in at least one of their vulnerability announcements that it was just a local I think we have different meanings for the same word there and again the disclosure process here was important because when Mike gave his talk he essentially talked about the internals of how it worked that stack overflows are hard to find in Cisco iOS that the code base looked strongly audited and that you had to work with heap overflows but that Cisco has a check heaps process which will go through and validate make sure the previous and next values are the same so that if you actually do corrupt the heap then it will say oh this isn't okay alright I'm crashing the router because this is screwed up he did not give details of what his entry vector was he was very careful to basically try and not give enough information for people to recreate it now even though the vulnerability has been patched and the old versions have been yanked off of Cisco's website there are still vulnerable versions out there in the wild Cisco really shockingly to me yesterday morning released an advisory to full disclosure that said yeah for the first time ever we have a denial of service and possible remote code execution in our handling of IPv6 packets Lynn didn't tell us that Cisco told us that in the latest version of their advisory they say that it was disclosed at blackhat so that really does nail it down to yes this was the vulnerability and this is really revolutionary for the field because for the first time it looks like you can remotely own the Cisco box you can get it to shovel you back a shell that will allow you to have full enable access and this is a scary thing if you're an ISP operator I think that Lynn was pretty sensible in saying that people need to treat these like normal computers they need to patch they need to be aware of security and they need to be diligent about making sure that they patch through the versions because this is a real threat people we've now seen the remote route now when you look at the Cisco advisory what they appear to mean by local to the best of my understanding and you know disclaimer I don't never have worked for Cisco I don't have worked for ISS I don't speak for anyone but me but Cisco claims that you have to be connected to the same local subnet in order for that to work and that makes it a local exploit right? yeah if you have to be on the same subnet from one remote box to another remote box that's a remote route and then came the cover up the talk's been ripped out of the books the presentation's not on the CD everyone got modified CDs the slides aren't available oh my god they're suing and they're suing and there's the FBI and there's restraining orders and rumors and news stories and all sorts of stuff and Cisco you're really screwing up here you know to give you an idea of the attitude of the security community there are people walking around wearing these I think that gives you an idea of the mentality of what happens when you try and cover something up okay so you're not really happy big pardon? it said fuck Cisco like the record reflect and I really think that Mike Lin was right on when he talked about why he gave the speech and why he released as much information as he did I mean okay the reason he got onto it in the first place again from the best of my understanding I don't know the guy, I don't work for anyone yada yada he said that he found some Mandarin Chinese documents which when translated were talking about how to hook debugger hooks in iOS and how to make those speak to your disassembler and your debuggers this means people are working on it their source code has been stolen and there's now a storm of interest I know that there have been several people who have suddenly been diving into their assembler through to attempt to figure out what's going on hiding your head in the sand is not going to help if you have people who are attacking you that are actively looking for this going no no it doesn't happen quick shut everyone up is not going to help and even if you destroyed all the slides and you aren't going to let things get released well his slides are out there on the internet suing researchers is not going to achieve security I mean only speaking for myself I was in his talk I speak assembler I know Cisco iOS with enough work I could probably recreate his exploits I'm not the only one trying to pretend it won't happen will not fix things and alienating the security community and making everyone think that you're going to be sued into oblivion if you ever speak up that's not going to encourage people to come to you and say hey we've got a problem let's work with you when I was reading the reports of like extant background vulnerabilities it sounded like they treated fx a lot nicer than they treated Mike Lynn fx actually thanks people and if you know elite advisories and says hey thanks for working with me Cisco you guys were really great they don't sound really great right now they filed a temporary restraining order to prevent him from speaking about it so he can't talk about it but I can well his slides are easily findable on the internet and not even the version that he presented at the talk the original slides from before he got fired by his employer I think like yesterday morning they were up on Cisco me and several other sites and this has just been a ridiculous media circus they were put up on infowarrior.org and they got pulled down with a season desist by Cisco and guys just help us pick the cnd for that was from ISS apparently so he had to return his computer he's not allowed to talk anymore and this is really dumb dear Cisco and ISS responsible disclosure is good working with security researchers is good trying to cover up things that are known problems that have been released into the public domain that are on millions and millions of machines all over the internet now is good if you try and shut people up you're only going to alienate the very people that you need to help you alright so what Mike Lynn actually said keepoverflows are way more common in IOS he said a bunch of nice things about their days but he said that the watchdog process of check heaps will check for like heaps corruption and if it finds it it will crash it and most of you that have worked with Cisco routers I'm sure we'll see that if you have a Cisco router and it has a fault a lot of times rather than actually trying to recover it will just crash itself and the really important thing that I think came out of Lynn's research was that the crash process itself is interruptible so you can interrupt it when it says this bad thing has happened my heap has gone badly am I crashing and you can tell it why yes I am okay we're already crashing I don't need to do anything then please continue executing whatever the heck it is you were executing there so you can buy yourself some time that way he did mention that spreading memory corruption on the heap was going to be an issue and that you might have 2-5 minutes before it like exploded all over your memory basically unless you actually handled that there's no code and anyone who's written shell code knows that that can be tricky but once you've actually gained execution there are hardware interrupts that will try and interrupt you but you can disable those so once the CPU is yours it's yours and tell about the crashes and through methods like that you can get the remote box to shovel and enable shell back to your listener which is what Lynn did in his demo enable is the Cisco equivalent of root game over and he indicated that so he didn't give us the specific vector and Cisco did there were other vulnerabilities that you could do this for and here's some of them that might be fruitful grounds for exploration anything that talks about memory corruption that leads to a denial of service is probably a likely factor for this sort of thing simply because since the Cisco routers tend to crash rather than actually trying to recover if you screw up the memory enough to induce a crash it may just be okay your offset was wrong or something like that I really strongly encourage Cisco to look at their code and look at the way that he was able to interrupt the check heaps process and you know, avert crashing because that's the underlying problem the front end vector whether it's IPv6 or CDP or OSPF or BGP that's not really going to be your issue the fact that you can defeat check heaps that's your problem so, yeah, here's a link to the advisory about vulnerabilities in IPv6 processing that Cisco put up and just in bold because this is the first time I've seen it and I've been waiting for years to see when this was going to occur because I believe it was possible but my Glyn shellcode is better than mine Cisco internet work operating system is vulnerable to a denial of service and potentially an arbitrary code execution attack so Cisco did admit it again, just to hammer home the point about local segment versus remote segment, local network segment does not mean that it is not a remote group thank you so look at this for yourself, don't just take my word don't take FX's, don't take my Glyn's, don't take Paul Watson's go and look at the source of these sorts of things read the advisories, read the vulnerabilities and check what Cisco says now versus what Cisco said before read FX's paper about how Cisco handles memory management and there's a link to where these Glyn's slides are and I believe they were still up they were still up when I talked about an hour or so before I got up here they're up, alright so, you network defender people start treating your background devices like computers realize that they will have vulnerabilities that you need to allocate for security that you need to work for this that you can't just sort of set it up like an ancient BSD box and oh, yeah, no one's ever going to hack that it's fine, routing is hard, no one understands that well, I'd like to point out that of the people that I know that are working in this field I think I'm the only one that actually has an ISP background you can't just depend on it's hard and security through obscurity and so, yes, you need to treat them like computers but when you're testing because denial of service is so prevalent treat them like really important computers you know, don't knock them over without your client's explicit consent maybe packet log when I'm pen testing I run a packet log so that if they come and say you know, you broke XYZ, I can go really where here's my timestamps, here's my packets here's exactly what I did so that's maybe a contract issue for those of you who are doing that attacks only get better they don't get worse in time, these test tools are going to become a regular part of pentester's toolkits I would not be shocked if I, you know next year I suddenly saw Cisco attacks in Metasploit or something like that or, you know, more Cisco's vulnerabilities than in Messas we're still kind of in the dark ages of pen testing for the backbone, but keep in mind that this is where things are going and I'm just interested in working on it that it's going to happen and please don't take me off the internet or slap me with a gag order or try to sue me or anything like that there are mirrors, everyone please go out and mirror my stuff, put it on big tour and don't let them shut me up just a few last comments and I believe we have a brief time for question you can reach me at raven at nmrc.org with feedback, additional ideas comments, lawsuits I would like to thank Jericho and TV people for promptly putting all these vulnerabilities up in a public space where everyone can see them and have equal access for research, I would like to thank people who are mirroring these for me and also possibly staring down the eyes of lawsuits and I'd like to thank the EFF for sitting in the front row I love you, new best friends alright, we have three minutes, any questions? beg pardon my updated PowerPoint will be up momentarily I think people are going to run out after this and mirror it, I will put a site online where there can be like a list of the extant mirrors and if that gets knocked down, it gets knocked down also there is a legal defense fund for Mike Lynn, if you pay pal to abaddon ddon.io.com you may need it apparently he will need it yes, the question was given the negative reaction to the Paul Watson TCP issue how confident am I that ISP engineers will actually pay attention to this? well, despite working in this field, I am perpetually optimistic but I think the fact that we have enough people continually saying hey look, hey look, hey look I am cheered that even though I was not going to announce the details of what I was talking about beforehand because I didn't want to be shut up a lot of my ISP friends contacted me over the last two days and went, oh holy shit what do we need to do? so, there are at least some people who care there is an ISP community that's dedicated to security there are both public and private mailing lists that are working on this the problem is not that there aren't some people who care the problem is that most of the people don't care so, beautiful people, be loud alright, one more question yes, in the back that's you, I can't hear you the question was since I've talked about ISP infrastructure and such, can the ISP protect its hosts against zombies and things like that yes, the ISPs can to some degree the question is whether they're willing to a lot of times, to do that level of filtering takes additional resources, you need stronger CPUs you need more memory in the routers and a lot of places are running pretty close to bare bones on that so they don't have a lot of resources to spare and they don't think it's a good allocation what they need to be able to do that is to hear from people and say, no really I want you to do this there is client demand for it because if they don't see operational demand they won't do it, hello not making money off what sort of thing the comment was don't you have to have an interest in keeping spam and zombies and such from happening in the first place alluding to the poor record of many ISPs who benefit economically from allowing such things to happen across their network and you know, I could go on for an hour about that but suffice to say they are following where the money is if you inform them where you want the money to be the more people that inform them the more likely they will be to listen something that we can do to fix what quickly it's already fixed if you go and you update your iOS to the most recent version of firmware in your code train, you'll be safe alright, I think we're out of time thank you very much