 This is only the second time I've given the presentation. I gave it a month ago in a slightly different format with not such a cool crowd. It was at a CSI Netsack event, and they wanted me to talk about disclosure, and I'm thinking, what the hell do I know about disclosure? I'm not a bug finder. I don't really disclose anything. And then I started thinking about it, and that's, well, I've actually been in the middle of a lot of disclosure activity from either throwing conventions, oh, we can get rid of the police by now, it's cool. Whoever's controlling the police, stop them. You know the police never do what you want anyway, so, okay, so the whole idea was that I've been involved in a bunch of these disclosure things, and so why don't I talk about that and kind of frame it as a disclosure debate, and everybody will be happy. So I want to welcome you to Friday of the biggest and baddest DEF CON ever, and like I said in the program, this is the first time I've ever spoken at any of my events, because I just, I don't think that's cool, it's kind of like taking advantage of your, you know, you have an unfair advantage. But I figured you probably want to know what's going on, and it's been a couple of years, so chances of lawsuits are, you know, greatly diminished, and it's an interesting story. But I'm going to try a new presentation style with you today that I've never really used before, and I learned this from looking at a lot of, shall I say, questionable presentations that get submitted our way, and I'm not saying I'm a good presenter, I'm just saying that I've seen a lot of bad presentations. And so I went to securitiesin.com, and I just got a couple of clues there, and the one that really struck me was the Darth Vader versus Yoda version of presentation technique. Let me show you. Okay, here we got Yoda. Yoda's up here, he's chilling. He's got this blank screen, he's out in front, nothing to distract him, it's a very peaceful scene, right? He probably has like one point he wants to tell you. You know, what expects you of man years old, 800? And then we got Vader. Check him out. He's got all kinds of shit up there, he's got like, you know, telling you all about the forest, he's got sub-bullet points about your destiny, and I love the very bottom. For more info, visit my web page at, it's just leading you right in, right? I mean, Vader's just overwhelming you. And so I was trying to think of how do I distill down a very complicated issue that happened, and kind of portray it very simply. So I came up with a style, we'll see if it works. But then after seeing this, I was thinking, hey, what the hell? We got Steve Jobs, right? Steve's up here, he's very chill, he's black, he's blending into the background, it's not about him, it's about what's on the screen. He's got like four icons up there. You know, even a monkey could read, there's only like, probably 16 digits to look at. And we got Bill. Bill's just trying to distill down your digital life. And it's not working very well for him. I thought that was a pretty good observation. So I'm wondering, whoever's building a slide deck for Bill's probably channeling the dark side of confusion somehow. So what I want you to get out of this is, I'm not saying I'm Yoda or anything. I'm not, but I've met him. And for those of you in the audience who don't know who Yoda is, that's Robert Morse Sr., all right? That's the father of the guy that launched the very first internet worm. NSA's chief security, or chief scientist for, I don't know, how many years. He has every crypto system used during the Cold War in his head. Anyway, that guy has done more to protect our country than everybody in this room combined, probably. And he's forgotten more than we'll ever know. Plus he really looks like Yoda. Okay, so I'm going to offer you some insight to what's going to become really apparent is, as we start describing the motivations of all the groups involved, you're going to start seeing everybody pursues their own interests, right? And we all have different interests. So I want to talk about that briefly, and then I'm going to illustrate it. So before we get started, I'm just going to give you a brief context of who I am. How many people don't know who I am? Okay, so this is going to be like for eight of you. At 13 years old, I had this IBM PC2. The PC2 had 640K of RAM, not 512, double-sided disk drives. But soon I tore it apart, made it look like this, because as a 13-year-old, I couldn't afford crap. I had to, like, jury rig the CPU to go from 4.77 to 6.0 megahertz. I was the bomb. So then I hooked it up with one of these, Boeing Surplus. It was 50 bucks, 300 botacus to coupler modem. And I watched this movie. I was like, holy crap, there's NORAD systems out there. And so, you know, then I was hooked. I was totally addicted. And it was really funny, because at the time, I only knew a couple for real-life hackers that were introducing me into the scene. And one of them had been hacking for quite a while. And when that movie came out, he was so pissed. He was like, fuck, now everybody knows about it. They're going to lock down their systems, and I won't be able to break into anything. It's like 2007 now. So then I went to college. I'm a Zag. I went to Gonzaga. But before, they were cool. They weren't cool back when I went there. Our baseball team was cooler. Then I went to law school. And I quickly realized that I don't do well with the law. And I was at law school in this weird time. The dot-com bubble was growing. And there was a maximum number of people graduating law school, minimum people in computer and information systems. And I'm like, I'm going to get paid this. And I love doing that. What the hell am I doing? So I dropped out of law school and started DEF CON. That was DEF CON right there in the car. I mean, that's pretty much everybody. So you got Jackle. You got Noid over there. Look how happy he is. One day, he didn't know he'd be running security. But he's happy back then. There's this character, Dune. If you look at the inside of Neil Stevenson's Cryptonomicon book, you see a thank you to Dune. Man, that fucker. It's like, I said Dune, man, we got to get Neil Stevenson to come to DEF CON. That would be cool. And he's like, hey, he lives local. Want me to try to contact him? I'm like, yes, contact him. That would rule. So he's like, I contacted him and I had lunch with him. I'm like, you didn't invite me to lunch with Neil Stevenson? And I ended up hanging out quite a bit. Neil was like, I don't want to be public. I want to just lurk in the back of the room. I'm not into all this attention. But maybe I'll show up in the back of the room. I'm like, OK, that's cool. Next thing you do, I open up the book and there's Dune mentioned. It's like, I could have been doing that. But no hard feelings. Dune's down in LA now, being a professional massage. Massusier to little starlets in Hollywood. Doesn't do security anymore. There's me in the middle. And you might recognize ALF-1 from a bug tracker and bug track for a long time. So there was, even back then, a lot of people at DEF CON one are still around. Jennifer Granik, I met through DEF CON. I met everybody through DEF CON. So then I started a business which promptly failed. I was integrating SQL servers to payment processing systems to create online stores, back when pretty much the only thing that was buying those systems are porn sites. But pretty soon, $20,000 of custom development work was replaced by $300 off the shelf box system and we went out of business. Boeing hired one employee, Microsoft hired another, and I was done. So I had to go work for the man. I went to Ernst & Young's Information Security Services Group. I lasted 91 days. That's very key because at 92 days, the recruiter can't get his money back. Then I went to Secure Computing Corporation, which I helped start their professional services group. We built a pretty good group, which eventually got spun off to become garden. And they got bought and I don't know what happened to them. But I didn't care. I was busy. I started Black Hat in 97, which I ended up selling to CMP in 2005, just about the time I got sued a little bit later. People have speculated why I sold, but it pretty much came down to more and more stress. Every show, if anything went wrong, I'd be bankrupt forever. And I was dealing with HR and accounting and attorneys and it just, I wasn't having fun. So I wanted to focus on content, found CMP, which was awesome because they were there while Cisco gate was happening. I had, well, I'll tell you what's going on. Now I have the boss. That's the boss. That's my Guinness and that's Converge in the background. I sold them. There were four possible buyers while Cisco gate was happening. I'm trying to have meetings. Well, wait, wait, wait, we'll get to that. We'll get to that. Just talk about disclosure for a moment. I thought this was a self-evident issue that had died like a decade ago. I was tired of it after Jeremy Roush talked about it at Black Hat 99. We were thinking like, how many more times can we hash this topic? It's like, beat, it's beat dead. Then Scott Blake covered it again with the politics of vulnerabilities. That was really cool. Then we covered it again in 2004, then 2006. We're still fucking talking about this stuff. And the panel was this morning. Yeah, you can't get away from this stuff. People love this because it's all about humans and motivation and you can rehash this stuff forever. So we're going to talk about it some more. First I'm just going to go through quick definitions. We've got full disclosure, which a lot of the researchers and enthusiasts and nonprofits love. Responsible disclosure, bug track, vendors, conferences, quote unquote professionals. Limited disclosure with no specifics. There is a bug, please upgrade. But we won't tell you what the bug is. Or zero disclosure, which some zero day and bug finders like. A lot of pay as you go, kind of a zero bay type auction sites, whatever, criminals, researchers, other people also like zero disclosure. Here's how it's broken down. I stole this from Scott Blake's presentation. This was the coolest slide he had. So you can see by this slide where we pretty much play, or I play most of my life, is in this responsible disclosure zone. Because we generally don't release exploits without vendors knowing about it. But we release all the details we can. And if we can, we release details. But I'm not about to try to drop a billion zero days on somebody with no preparation. That might have been cool 10 years ago. But it's not really cool anymore. So I really encourage everybody to at least give the vendors some notification. So that leads to what's responsible. If I say to my speakers, be responsible and disclose bugs to the vendors, they immediately said, cool. What's that mean? And when I ask vendors what it means, they give me all kinds of different answers. And I was talking to Microsoft about this. I'm like, is one day reasonable? And they're like, hell no. How about 100 days? Is that reasonable? And they're like, that's probably too much. So my answer to you is it's somewhere in between 1 and 100. I'm thinking it's like 30 days maybe, three weeks, a month. It could be less, it could be more. It depends upon how pissed off you are at the vendor, how responsive they are, how cool they are to the community. So in deciding to pick Cisco gate, I was kind of reviewing my, this is my portfolio. This is like my reel of what's happened. And I realized I've been close to a couple of different things, each with their own unique situation. The HID IO-active thing was pretty recent. It just happened like last February. But that was between IO-active and HID. And I just chilled on the side with attorneys waiting to be called into the fight. But they never used us. So out of all these, L-COM had happened at Def Con, but they arrested them after the show was over. I was never involved, so that's out. I wasn't really involved in the Apple thing with Dave Maynard, HID. That leaves us Cisco. So here we go, the main event. Cisco gate. To get ready for it, well no, no, no, I want to hold off on that, I want to transition into my new persona. If you were at Def Con, you might have noticed this. Here we go. OK, I'm ready. My Cisco gate shirt. So OK, it's set in the stage. So it's 2005. It's Black Hat's the venue. And I got some clear goals. I'm going to execute on a great conference. It's the largest to date. I'm going to have meetings with these prospective buyers that are going to give me a lot of money for my show. I want to gamble $20 in quarters, which is a ritual I do every year, until I run out of quarters. I make $40. I never make $40. And I always drink a beer or a foofy drink that has lots of umbrellas by the pool. That started from Def Con 1, where I came to Vegas and I was either going to lose all my money, or Def Con was going to be a success. And either way, I was going to have a peanut colada by the pool. So it all starts off. Mike submits us a talk called The Holy Grail, Cisco IOS Shell Code and Remote Execution. And we're sitting around the office like, wow, that's a fucking cool title. We're going to probably accept him just on the title alone. But we probably have to have something to back it up. So that was February 21st. Mike gives us the submission, right? Standard format, date of submission. You'll see it's February 21st. He got this in months earlier than anybody else. Look at the last line and tell me if that's not a cool reason to accept this talk. OK, reason three. OK, reason one, it's never been done before. Sure, FX came close, but I'll be talking about reliable, repeatable, remote execution of arbitrary code. Reason one is pretty cool. Reason two, playing with critical infrastructure is exciting, especially if by playing you mean exploiting. But reason number three is IOS Shell Code, IOS Backdoors, the end of the world, et cetera, et cetera. I'm thinking that's pretty sexy. That sounds cool. And as part of his submission, he has to agree to a transfer of copyright, giving us the right to reproduce the materials online and distribute them and everything. And since he's working for ISS, he has to get approval from his bosses. These are the two operable areas, right? I've obtained permission, and they've granted us permission to duplicate everything. Now, it turns out he received approval three different times over three different months. But that'll come up later. So show of hands or by cheers, would you accept this talk? Yeah. Yeah, hell yes you would. You accept it. It's accepted in late February. Now, we've got mics in the scene. We've got only two players now. Now it's early June. Like months go by, the show's humming along, we're getting everything ready. I get a call from one of Mike's friends at ISS just, hey, can we change Mike's materials? Like, this happens all the time. Somebody wants a slide added, a slide removed. Sure, Mike can change slides up until the deadline to print the materials, right? That's fine. The request should come from Mike, it shouldn't come from his friend, but sure, not a big deal. Sure, yes, you can do it. Now it's late June. As DA can attest, late June is kind of a fucked up situation for us, because we have Black Hat and Defcon happening. We have like everybody in the office going crazy 24 hours a day. I mean, what would you say? Insane? No sleep. No sleep. So this is kind of a bad time for anything to happen. Second call from Mike's friend. Hey, is it too late to change the materials, because they never change them the first time? Sure you can change materials. What do you want to change? Well, we want to just pull a slide or two that shows some screenshots of some op codes and some disassembly. Is it going to detract from the talk? No. Is it going to cancel his demo? No. OK, sure, you can pull a couple pages if you want. But I need to hear it from Mike. So, got to find out for Mike is this cool, right? I also have to find out, can I change the materials in late June? Have they gone off to press? And I find out that it's not too late. There's no increased cost of production. It's going to cost me the same amount of money. Sure he can change it. Then I get this email. Ping is the person back then that was dealing with all the speakers. What's wrong with this one? OK, I'll read it. Hello, Ping. This is July 5th. I briefly spoke with Carl Branson, and he indicated you were the contact I needed to work with regarding Black Hat website content. We are revising the abstract copy for Michael and Cisco iOS presentation on July 27th. And we would like to ask you to remove the content that is currently posted. We will send you the new presentation summary by the end of the week. Thank you, Renee Wagner from ISS. So now ISS is asking to remove material based on one of their employees. So what do you do? How many here say remove the material? OK, now why would you not remove the material? Because it's Mike's material, it's not ISS's material. Righto, good. You're with me on this. So we call Mike. And Mike's like, sure, take it down. I'm still going to do the same thing, not a big deal, whatever. Cool. So we got Mike. But now we got ISS poking around, all getting in my business. So it's starting to get a little complicated up here. And then I'm in the mix now. Because now they're calling me, I don't like being called. But all is still cool, right? At Mike's request, we pulled his online abstract. We've removed a couple of details to please his employer. We don't want to get him fired. They know it's still not too late to update their materials for free. Just send us the new stuff, right? We can do it. Mike still has plans to do his demo, which is the most critical part. That's all I really cared about. Then all of a sudden, we get this sucker. Hi, Ping. It's come to my attention from several management contacts here that both Cisco and ISS want Black Hat programs reprinted at a cost to Cisco, including rush fees. Additionally, I've heard rumblings that if it's not possible to fulfill this request, all sponsorships, attendance, and participation from ISS may be pulled. Please call me as soon as possible. Many thanks, Renee. So I'm like, what the fuck? Renee, what the hell? So now I've got these other players all up in my business, right? Cisco's in the mix. Cisco's like this big company. So now I've got to get my lawyer. So now my lawyer is all spun up. It's like combat. So now the question is, do we reprint it, right? Can we, at this late date, they're going to threaten to pull everything unless we reprint it? What do you guys say? Should I reprint it? Yes or no? No, why not? They're paying for everything. It's not going to cost me a dime. Assuming Mike says OK. So what would you do, right? Reprint or not? So we send this little letter back from Ping. If you've ever dealt with Ping, all I should say is, don't fuck with Ping. Renee, I do not believe that ISS understands that the following items would need to be reprinted. One, printed conference proceedings book that contains all speaker submissions and slides, including Mike Lins. It also contains the old description that hasn't been updated, title and slides. The conference CD that contains a PDF of his materials, the printed program contains his new description. It is absolutely impossible for each one of these items to be revised, reprinted and delivered here on time, Monday, July 25th, for the conference. The conference proceedings is well over 1,100 pages, requires a week to produce, a week to ship freight, is 2,300 books, and weighs 14,000 pounds. Rush production was possible option for the program last week, but since no one at ISS ever told us they wanted to do it, they just kept asking, we made no arrangements with any of our printers to reprint the program. I've received the request from Mike Linn to change the website, and it will be done. Unfortunately, I do not believe that we can change any of the other materials. Basically, not going to happen. It can't happen. Rush charges in Vegas, like 100%. Weekend work, 100%. So you're going to pay 200% more on top of the original printing charges. And they still haven't asked us to do it. So guess what? We're not doing it. Here's the response. July 22nd. Training is like started. We're in Vegas. We're in the mode. And Renee has to say, I sincerely appreciate taking the time to outline the specific challenges of reprinting the materials. I suspected all along that reprinting would be a significant undertaking, especially at this short notice. I know you've been in discussions with David Maynard and Mike Linn, and I've asked both of them to pass along my apologies for the frantic emails and voicemails. I wish you the best of luck next week. That sounds good to me. How many people think it's all cool? It's cool. I put this out of my mind. I'm like, great. Onto the show. Everything's cool. Renee acknowledges reprinting as a significant undertaking and wishes us luck. And then I see this guy, Mike. Mike's a cool guy. Mike Cottle from Cisco. We're chilling. Shows work and everything's going good. 40 hours till the briefing. 40 hours till I hand out the books. Mike says, hey, Jeff, hey, good to see you. Got a book I can look at? And I'm like, you're with Cisco, right? Yeah. I bet you want to see Mike Linn's stuff, don't you? He's like, yeah, yeah, I'd like to see it. I'm like, OK, no problem. We don't hand out the books normally, because they're not all in bags, really. They're happening right now. But I'll get you one just because you're a nice guy and want you to see his materials. So he's like, fantastic. Do you give him a copy of the book? Why not? Everything's cool. Renee said, green light, go. Best of luck. I don't have a beef with Cisco. I've never even heard from Cisco at this point. I don't even know what the deal is. So I give Mike a copy of the book, being a good neighbor. So I show it to him. Yes, I do. And he's like, WTF, OMG. He's like, he flips to the page and he looks at and he's like, ISS told us none of this was in here. I'm like, well, what are you talking about? He says, yeah, yeah, ISS told us all the materials had been pulled and everything was OK. I'm like, they never told us that. If it was such a big deal, why didn't you just call me? Tell me there's a problem. He's like, yeah, yeah, we probably should have done that. So Mike's on Speed Dial 1 to the attorney's back at Cisco. And all of a sudden, it starts. Like lawyers are spun up billable hours, ching, crrrrr. They start going, I'm firing up my attorneys, right? We know something bad is happening. To recap, it's my biggest conference ever. I've got buyers all over the place. I've got multi-billion-dollar companies now threatening to sue me into oblivion. And ISS changes their story. Now all of a sudden, Mike Lynn's materials are not ready. They're just sort of test materials. They don't want them to present because it's not ready. It's not reproducible. It's not authentic. It's just some preliminary lab test work. So it's too late to replace the materials. We can't reprint anything. But it's not too late to remove them, right? I've got scary lawyers calling me. And so do you remove the materials at this point? They're threatening you, right? We've got a federal judge on the line. They're filing a temporary restraining order against you in federal court. They got a lot of money, and a lot of people running around. And my biggest threat here is that if the judge believes Cisco, and they shut down the show for a day, I'm bankrupt, right? Which isn't good because I can't like move the show forward until the temporary restraining order is over. So hell yes, I'm going to remove. When a federal judge tells you to remove things, you remove them because you don't want to anger a federal judge, especially federal judge White. There's two judge Whites I found out, one better than the other. You don't want to anger the federal judge White that we dealt with. It doesn't like that. So we have to start ripping out the materials. I'm not sure if this is going to play, but nothing plain, good. That's very handy. Let's try this again. I don't think there's any audio, but somebody who's sitting very close to me toward my left took this. And the interesting thing is Cisco guy called on his phone and got like every Cisco employee at Black Hat showed up to start ripping books out. Because we were like, we're running a show. You can pull the pages out. But we don't have any manpower to do it. So they hire attempts on the spot that swoop in and start ripping pages out as fast as they can all night long. I mean, they're going crazy. Yeah, if you can read that proprietary material, more luck to you. So this goes on. I just want you to see the pile here. OK, here, check this out to the left. I mean, that was just a little bit of the bags. So it was kind of a big undertaking. But they were cool with that. And if that got me off the back of a federal judge, I'm cool with it, too. So we remove all the materials. They re-burn all the CDs with his stuff removed. ISS, or Cisco, I'm sorry, goes out and does that. We update the Black Hat website to remove all the links to Mike's materials. We're done. It's cool. I'm on the telephone. I've got every attorney on this conference call, must be like 12 of them, ISS, Cisco. And I'm like, OK, the materials are removed. Everything's good, right? OK, it's been destroyed. It's been transferred to ISS? Yes, OK, good. OK, everything's done. I'm like, wait a minute, wait a minute. My attendees are going to be here and getting books in two hours. And they're going to open it up, and they're going to see this big thing missing. And then they're going to look at the CD, and it's going to look ghetto, because it's silver. And they're going to be like, what's going on here? I wonder what's going on. They open up the book, and it just kind of goes right into this one spot. It's not going to take a genius to find out something happened, right? And I'm not going to lie to my attendees. I can't do that. I'm going to have to tell them what happened. So I want to write a statement out, and every time they pick up a book, they're going to get this statement. What are you guys going to say? I mean, we have the most amount of press. Over 90 press are here. It's the biggest story so far. The show hasn't even started. We've entered a swirling vortex of shit. And I need to know what your game plan is. And the attorneys, I kid you not, are like, hmm, yeah, oh, vortex of shit. Like, well, that sounds like a marketing problem. No, no, that's a PR problem. I think that's Jones Division. No, no, marketing, marketing. Yep, that's a marketing problem. We're the attorneys. Like, OK, talk to you later, then bye. So I make a statement to the attendees, but they don't. So things are getting kind of strange. It's July 26 now. It's a day before his talk. Now Mike's getting massive pressure to cancel his talk, not just have his materials removed. Now they want him to not talk at all. Everybody's asking him to do it. ISS is asking him to. People are telling me, are you going to cancel his talk because his materials aren't in there? What would you do? Would you give it up? Now, why wouldn't you give it up? Why wouldn't you cancel it? Well, I accepted him, not ISS, right? And his materials have been pulled. It's his spot. I'm not going to kick him off, right? No, hell no. If Mike has a backup talk he wants to give, I'll let him give it. We're barred by the judge from talking about it, I mean distributing the materials. I am. Black Hat is. But if Mike wants to talk on something else, great. I'm not going to deny him his chance. And he has a backup talk on voice over IP security. So of course I'm going to let him. I'm going to let him talk about that as much as he wants. I'm next door on this panel on CISO, blah, blah, blah. And Mike gets up there, starts to introduce his VOIP talk, and everybody starts booing him. And they all, like a third of the room, just gets up and leaves. And Mike's like, uh, how many want to hear my original talk? And everybody's all like, woo! So Mike busts loose, right? And I knew something was wrong, because I'm up at the podium asking questions. And all of a sudden my phone's like, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do, do. Like, ah, something's going on. Mike quits ISS and goes on stage and delivers his original presentation. Now the interesting thing there is he gave his revised materials, and then he revised them even more, censoring them a second time, even more, because he knew he was quitting and it was going to be a big doodoo. But he still got up there and did it. Do I stop him? So that's a really interesting question. How do you stop a speaker? I mean, do I have a big hook? And I'm like, and you try to get him? I mean, do you dive, tackle him? I mean, I don't have a goon squad there. So, no, I'm not going to be in the business of censoring what comes out of my speaker's mouth when they get on stage. I mean, if they start insulting or swearing at people, I'm going to turn off the mic. But, yeah, that's a game you can't win. I may as well just shut down my business if I do that, because who's ever going to speak for me again? Like, nobody, right? So it's sort of a business survival thing, too. Plus, it's lame. So, Mike gets to speak. I'm not going to stop him. But now we add Jennifer Granick. And it gets more and more complicated by the day. So the press just goes crazy. ISS goes crazy, changing the stories a couple more times. And Cisco is caught totally unprepared for the media fallout. They're like, huh, who me? And it's sort of like I told you so. So the current disclosure issues are, was it responsible or reasonable for Mike to disclose? Did his quitting help or hurt him? And does it help or hurt future presentations? So, wait, wait, let's see. So was it reasonable for Mike to disclose? I think it was reasonable because he didn't give out weaponized shellcode exploit stuff. He demonstrated it was possible. That seems pretty reasonable to me. The other thing that was very reasonable is Cisco had known about this for over six months. At the point that he gave his talk already for a month, it was impossible to download a vulnerable version of iOS. They had fixed it like a month previous. You just couldn't get a vulnerable version. So when people were saying this was all wrong for him to disclose, I didn't get it. It had already been fixed. Everybody had known about it. But the weird thing is his quitting actually hurt him. Had he stayed an employee, and this will become obvious in probably the next slide, he couldn't have been charged with theft of company secrets. Because he would have been an employer for the company. But he quit. And so it turns out his talk at ISS called the FBI on him from Atlanta. And does it help or hurt the future presentations? I think it helps in the sense that we didn't back down. And he got to still speak. He got his story out. He didn't end up going to jail. And I think if he had backed down, it probably would have been a bad thing. But the fallout was the federal injunction against your stupid and Mike's content went from a temporary to permanent. And the shows over, I've dodged every bullet. I'm sitting there help cleaning up, and I get a phone call from the FBI. And they're like, hi, Jeff Moss. This is a special agent, blah, blah, blah. We'd like to talk to you. I'm like, oh, great, the FBI, I love you guys. How can I help you? Oh, are you around? Can you come up to the registration desk? We'd like to talk to you. I'm like, oh, well, I'm really busy. But I'd love to talk to you. How can I help you out? Well, we'd really like to see you in person up here at the front desk. I'm like, well, I'm kind of busy. And it's not that I don't trust you guys. But I'd feel more comfortable if you told me what it was about. Do I need my attorneys there? Well, if you'd like Jennifer Granick or Jeff McNamara to be present, you could do that. I'm like, how do they know the names of all the attorneys involved? That's a little suspicious. I'll be there right after I speed dial all the attorneys. So I call up all the attorneys. I vector them all in. They call me. We're here. And I'm like, OK, I'm going to come out from hiding behind the screen, literally. And I show up. And I shit you not. There's two guys in like six, two in suits. I mean, they were perfect suits. I mean, they were stereotypical FBI suits. And we received a complaint from the Atlanta FBI office. We have to investigate it, theft of trade secrets. And we're here to gather evidence. And Jennifer Granick's like all up in their grill. She's like, do you need to talk to Mike? We'd like to talk to Mike. Is Mike Klein a target of investigation? Is he a focus of an investigation? Is he under investigation? Has he been charged investigation? About every single way she could ask the question, well, we're not sure yet. Well, we're not quite sure yet. And she's like, OK, my client's not currently available. So then we understand you video record all your presentations. We're going to need to receive a copy of that videotape. Mike, I'm sorry. I can't give that to you. Like, we're the FBI. We'd like to copy that tape. I'm like, I like you guys. Really do. But there's this federal judge, White, and he's told me I have to hand the tape over to somebody else. So I can't give it to you. And they're like, oh, Judge White. Like, you just do what the judge tells you to do. I'm like, yeah, I got that. I got that. So I'm thinking it's over, right? The FBI kind of goes away. Mike's kind of screwed because now he's got the FBI on his case. But from my perspective, I think I'm pretty much done. Half the prospective buyers ran away. They're like, FBI-esque what? Goodbye. I'm like that whole day. I'm like, I'm fucked. I'm bankrupt. I'm not selling the business. Oh, I'm selling the business. No, I'm not getting sued. I'm getting sued. Wait a minute. I mean, the whole day, I was like a total basket case. But now it's over. DEF CON is starting up. It's going to be a great DEF CON. I think it's going to be great. The TRO becomes this PRO. It's very simple to comply with. We'd already complied with all the PRO requirements, except for deleting the remaining material off our machines back in the office. Then, oh yeah, Mike, somebody calls the FBI on Mike. Ooh, I don't like this last one. The Black Hat web server leaks data. So remember how I said we were all stressed out, and it gets really busy, and when we removed Mike's materials off the website? Oh, I said we removed the links to Mike's materials off the website. So I'm like at DEF CON trying to sleep. It's like a spad memory. It's like it's done. And my phone is just blowing up. My phone's ringing. They're like, they're in a car going to the judge right now, and they're shutting you down. You violated the terms of the permanent restraining order. I'm like, what are you talking about? It's like, yeah, the guys at Cisco are downloading Mike's materials from your website right now. I don't know what you're talking about. We removed that. So we tried to log into our web server, which is being nailed by everybody on the planet, downloading the materials, and managed to get in and look at the server logs. And some guy was guessing at the file names, started at one in the morning, about one in the morning. About eight hours later, he hits it. He gets the right combination because it's a predictable file name. And it's like, you look at your log, and it's like, guess, guess, guess, guess, guess, guess, guess, boom, 200 file received. And there's this pause. Like he's like, what? And he's like, and then he looked, and then he gets it again. It's like 200 OK. It's like, ooh. And then like five minutes go by. And then like hundreds and hundreds of connections, like so many connections, you can't even count them. And the whole web server goes, you know, bandwidth crunches. And people drive to judges and phones explode. And everything's really bad. We end up, because that wasn't intentional, we were behaving under good intentions. We delete the materials. And since the federal judge is ordering us to turn over the materials, we had to turn over the web logs of who had connected and gotten the materials. They sent out a million cease and desist orders. Since then, I've turned off web logging. This is why you do not want web logs. So you think it's all over, and I've got a couple minutes left here, 10 or so minutes. So I just wanted to say there's a lot of post disclosure problems that I had not even thought about. I mean, we debate full disclosure and responsible disclosure, but we never talk about what happens after that occurs. How do you clean any of this crap up? I've got orders to remove material off servers back in the office. And just to show you how incredibly complicated this is, well, before, how do you clean up disclosures like this? Well, there's a couple of ways. You fix or make the problem irrelevant. Cisco had already released a new iOS image and removed all the vulnerable ones. So the problem was sort of irrelevant unless you just had legacy stuff. And then you remove all the disclosed materials as much as possible. That's what Cisco did, right? They sent out cease and desist orders. We removed the material, threatened to sue anybody on the planet to prevent further disclosure. That was their two-prong approach. But now I've got to clean up all my stuff. And as you can see, Mike's submission hits our CFP server email gets copied to myself, Eli Ping. It goes to my USB backup. Eli burns it to a master CD. Ping burns it to a backup hard drive. She sends it off through email to the printer. It's in the printer's email spool. Then it goes to the printer's server, the server's backups. Then it goes to the printing presses. It's in the printing presses spool directory. And you have the final books, the spare books, the books and transit on the way back to us. Then they shoot it off to a convention storage and distribution. CDs, same thing. CDs are on the RAID server, the CD manufacturers. It's in their email address. It's in their backup systems. They're on the backup CDs. It's on the Glassmasters. I mean, we're sending out letters and each one of these attorneys require certified letters from each one of these people saying, yes, we destroyed the master. Yes, we destroyed this. Yes, we hunted down that. Yes, we deleted the email spool. Yes, we did. It's impossible. So it leads into this really weird data destruction thing where we had to agree to a forensic data wiping of Mike's materials off our relevant hard drives. So what ends up happening is you're in a room with all these attorneys from Cisco and ISS and myself with all these hard drives and laptops. And they image them for two days it takes to image them. And this is when I realize all forensic software sucks. I take some 16 hours to image a 300 gig drive. They image it all. And now they have these exact copies. And then they make MD5 hashes. And they do all this stuff. And then you sit down, and they start asking you for keywords to delete. And you say, well, his presentation's in the PDF. Maybe there's a PDF spool. It's an email. Maybe it's an email spool. We can delete the original emails and everything. And the idea is we're going to delete all the stuff off these copies. And when we're done, we'd re-image the copies back onto our original machines. So the drive is all the materials gone permanently. And then the ISS lawyers start just going on this fishing expedition. Well, let's just search for the word holy grail everywhere. Let's just search for alternative aliases of Mike Linn's. Let's search for any other email Mike Linn might have ever sent you. I mean, they're just going on this fishing expedition. And we're like, no, no, no, can't do that. Well, you have all this encrypted email. We're going to need you to decrypt it all. We're like, hell no. Because at that point, we had Black Hat Consultant, and we had some defense contractors that we did work with, and all that email is encrypted, and I'm not decrypting the defense contractor's shit for you. And they all lean back in their chairs like, oh yeah, that'd be bad. Yep, yep, don't want to do that. OK, whatever you say. And we go through it all. And it takes over five months to just delete the data and agree to delete the data. And in the end, they want to destroy the hard drives, the ones that had the original copies. So at one third cost to me, we flew in three guys, a court reporter, a stenographer, a guy with a power drill, and a guy with a screwdriver. And I have the video. They took apart the drives and power drilled them, and documented everything. Power drill right through the drives through my lawyer's desk. This is not an operation they'd ever done before. Yeah, it cost me over $200,000 probably in legal bills. Yeah, for the desk, I forget what we did about that. I think he actually swapped it with another attorney's desk when he wasn't looking. But in the end, it was five months to get out of it, very distracting. But now, I've entered a new realm. I've entered what I call the Wall Street Journal Effect. Now my mom knows what I do because it shows up on the front page of the Wall Street Journal. So now, all of a sudden, all of a sudden, my mom's friends are calling, like, oh, I know what your little boy does. And now, that was a big breakthrough. It's really weird. It hit the China general public. So in the end, Mike Lin, FX, and Dan Kaminski kind of got together, had a drink, and we all laughed it off. But it took Mike. So some of you might wonder what happened to Mike. Right after the talk, everybody was swarming Mike. And Mike hightails it out of there. Six federal agents surround him, like real agents, like badged agents with power of arrest, badges and guns kind of thing. And they whisk him off to this back hallway. OK, you've really done it now. Mike's all like, ah, ah, ah, I've got all these badges in his face and everything. They're like, I don't know what we're going to do with you. They reach in their pocket, and they pull out business cards and challenge coins. And they say, you rock. And they give them all these, like, and they say, I'm on the critical infrastructure, blah, blah, blah. I work for the DOD, blah, blah, blah. And Cisco never told us this problem existed. They're like, we have this special partnership with them, and we're supposed to share information. They never told us. So really glad you got to tell us. So best of luck. Here's some business cards. If you ever have trouble like this, call us. I could have helped you if you'd just told me in advance. And Mike's like, well, I didn't have the bat phone with me. I just didn't know who to call. So I mean, yeah, maybe next time. But thanks for the hookup now. And so I was talking to Mike, and I'm like, fuck, what are you going to do? And he's got his hands full of cards. And he's like, well, let me see. What am I going to do? Reporter, reporter, job offer, job offer, legal threat, job offer, job offer. So Mike was going to be OK. It was just going to be getting out from underneath all the legal bills. So that's all I have for you officially. I can probably take a couple of questions, five minutes of questions, and then I'm out of here. So, sir, what happened to the ripped out pages? Yeah. So they went in this big bin that got Saran wrapped up and had guards watching it the whole night. And then somehow they arranged to get some guy's utility truck, and they filled it up, and went to China. Oh, the other funny thing about this was when Mike was doing his original research, he was stuck at this one point where FX was stuck at. And he was searching, trying to figure out how to maybe get past this one problem. And he found his solution in the website of a Chinese hacking group that he translated. So the Chinese hackers had already figured out that one problem. And all he had to do is translate it, and that was like part of his key to solving it. So to think that the Chinese or somebody else didn't already know about this, I mean, other people were definitely working on it. So another question? Question? Yes, sir? Yeah, checkpoint firewalls. Right? OK, so the question was, back in 2000 or 2001, Dugsong released showed some exploits for checkpoint firewall one, which was all the rage back then. And what's the difference between what Dugsong did in this? Well, Dugsong had notified checkpoint, and they had fixed it. And the difference, I think, is that checkpoint had a marketing department that understood how the world worked, and they didn't have crazy lawyers. The one thing I do want to leave you with, though, this is probably the key point I learned in all of this, is that when dealing with both organizations, they negotiated as a whole. The ISS and the Cisco lawyers negotiated as a whole. And I was trying to think, why is ISS involved in any of this? I could see Cisco being pissed, but why is ISS even involved? They've got an employee. Just go deal with the employee. Why are they suing their own employee and me? I don't get it. And I thought about it, and I thought about it, and I asked around. And it turned out from, don't take this as gospel, but this is what I heard, that ISS was trying to do a deal with Cisco where they could get advanced security vulnerability information and include it in some of their tools. So you'd get exclusive Cisco volns in ISS tools. And that was the carrot Cisco was dangling in front of ISS. Get your people to fall in line, and we'll give you this big carrot. In the end, they never got the carrot. And they got stuck with, I heard, over a million dollars in legal bills. My attorneys are much cheaper than their attorneys. Because they have an incentive to keep billing and not stop, or I have an incentive to shut it all down. But the ISS attorneys, complete nightmare to deal with. They didn't know who was in charge. They're all talking over, kind of talking over each other, sending mixed signals, where the Cisco attorneys, ultra professional, totally reasonable, the best people in the world to work with. So if you're ever going to get sued, get sued by Cisco. Thank you very much.