 This is the Meet the Fed panel, it's the Meet the Fed, Meet the X-Fed, Meet the TV feds. So, you know, on the panel we've got David McCallum from TV Show NCIS, okay? And Leon Carroll, the technical advisor to the TV Show NCIS, a retired NCIS. And what we're going to do is, before everybody else gets introduced, we're going to play Spot the Lamer like we do every year. So, priest, if you would, we've got the first three contestants to give me about three or four more. Absolutely. Should I talk about these contestants? Sure, absolutely. So, I get a nutshell for the back row. So, we have, would you like to introduce you or shall I introduce you? Okay. I've known her since she was this tall. She's actually the daughter of one of the panelists. These are her two boyfriends. Who she met at Khan. This one asked me to repeatedly hit him in the nuts. I don't know why. We have it on film. Okay. As priest picks you out, come on up and what we're going to do is, you know, Defqon had been playing Spot the Fed for years, so we've been playing Spot the Lamer. So, you guys are going to vote, and then we all have gifts for the winner. So, we've got to take this stuff seriously. This crew is definitely uglier than the first crew you brought up here, priest. We just have to juggle this. This thing spins like that. There's no way of fixing it. One more? We good? Okay, got it. That's good. All right. So, we're going to go down the line and ask questions, and so the first question that we're going to, I get the first one. Okay, we'll stay with the standard. Where do you store your porn? Okay, David. Where do your boyfriends store their porn? Dropbox. Do you keep for them? Does your father know? He does now. Why are you turning red, Mark? Because I put it all on Dropbox. Father-daughter bonding, you know. Father-daughter bondage, what? My question is for the gentleman in the Irish green on the end there. Have you ever bragged about the speed of your processor and the size of your RAM? Yes. Would you get to elaborate on that? No. Another boyfriend. Well, it is proof that if you do this too much, you go blind. For number three right here. So have you ever bought virtual things with real money? Only with bitcoins. He can tell you about his 23rd level palette if you'd like. My question is for the young lady standing right in front of me. Do you spend more hours using your computer, your iPad, or your gaming console? If the computer is the gaming console, does that count? If the computer is the gaming console, does that count? Doesn't count. Doesn't count. Computer. That was pretty boring. There's no such thing as a girl gamer. It's a myth. Marcus. I'll leave her boyfriend alone. You, sir. On the Big Bang Theory. Don't be stupid on the Big Bang Theory. If one thing shows up in the opening scene of every episode, what is that one thing? The atom. Nope. How about Penny's boobs? What? Wait a minute, what? Do you have like special access or something? Okay, I'm going to have to go slow down every time now. I'm going to have to frame by frame. Okay, right down here. In the movie War Games, when Broderick hacked the school's computer, what was the password? Pencil. Pencil. She's a little young for that, by the way. I was this tall. You were born. About as wide as you are. Jerry. Don't tease, dear. She asked me if I wanted to play a game. I'll take the guy with the hat down here. The guy with the hat on crooked. Battlefield 3 or Call of Duty? That would be you, sir. Don't look stupid. You know what, do I have to pick? I like both. Remember her father's up there, so look smart. The one, uh, Call of Duty. Zombies. You say that like a question, sir. You're not winning points. Lips are pursed. You know, he worked for the president. Uh, what? The response was I, uh, what? Say Mass Effect, sir. Mass Effect, sir. Say herpes. Okay, I'm going to go with the Jolly Green here. No, I'm Jolly Green. Oh, sorry. Do you void warranties? Yes. Regularly. Yes. Michelle. I'll go with contestant number two. God, it's basically not easy. Which of the following are your favorite foods? Your mother's meatloaf. Taco Bell. Brownies made by priests. How about your brownies? I don't actually make brownies, but I do an awesome lobster. I've got the sauce that... Priest, priest, you... Oh no, it's really good. Priest, you pick one. I pick one? Yeah, I pick one. This is an ecumenical question. It'll applaud everyone. Hmm. How about the dude in the water? How about the dude in the wife beater's shirt? The wife beater? Well, there are two dudes in the wife beater's shirt. The one who's trying to hide from me, bobbing his head. This one right here? Yeah, that one. He's dancing, I guess. See, it's just super tight. He can see what's underneath. I already said I'd say he sees what's underneath. He's still holding over. The place for glistening, listening, yeah. What is your definition or your explanation of doing a dude? If someone invites you to do a dude, what does that mean? Not do a dude, sir, you're safe. Do a dude. Do the dude. Do the dude. Slamming back Mountain Dew followed by tons and tons of alcohol. You are such a lightweight. It's all about the Jolt Cola, man. What's the matter with you? Somebody gave him that answer, I'm sure. No, sir, I did not. For the contestant on the And in Green. They love you for some reason. Because I'm old like them. I'm old like them, that's why. In that case, do you believe that Area 51 exists and if so, where? It is 1 to 50. As your attorney did not answer that question. There's a helicopter on the roof. Well, they're all out there in the Nellis facility, so yes, they do all exist. Pass the microphone down, please. You have some guys in red shirts to come get them. We'll take care of it. We promise no extreme additions unless absolutely necessary and you deserve it. So, the gentleman in the black t-shirt here, towards you, right there. So, have you viewed more videos on YouTube or RedTube? Neither. Create my own. Okay, now we're going to vote. You guys are going to vote who the Lamer is. Contestant number one. Let me try that again. Contestant number one. Okay, you can have a seat. Suck ups. You failed miserably. Contestant number two. Contestant number three. Contestant number two, you can sit down. Contestant number four. That's kind of a tie. Contestant number five. Raise your hand, please. Okay, you're down. Six. Right, you guys are out here. You guys are gone. The last one down. Raise your hand. One last time. Raise your hand. Last guy down. Right, you can sit down. Up here. We have the coveted meet the fed t-shirt. We have a DoD cyber crime response team t-shirt. Put the camera. You got your SNL. And if you would go down, we have presents for you. Here's a bag. So we've got an NSA mouse pad and a really cool NSA translucent cup. You can't even see the microphone in it. I stole a bunch of swag from Verizon, so here you go. A card for the coveted Ninja badge. Have a mint. Have a bottle opener. This is a very specially designed and coveted old fashioned glass. I assume you're old enough to drink whiskey, not just like beer anymore. This is etched. It's a collector's item. It's heirloom. I'm sure it's one that you want to pass on to your children. And because it's from NSA, you'll never lose it. Get a famous meet the fed flag. And a book of brain ticklers. We also, we have a special surprise. We have a free vacation. And the other dude... I love that. All right, what we're going to do now is we're going to let the panelists introduce themselves, tell you what they do for a living or not. And then we're going to open it up for questions, so line up for the questions. David. It's always difficult when half the audience has already heard everything you've said once, but I've been with NCIS now for nine years, the television program, not the real guys, but I've also had a lot of time with the real people down in Washington with director Klukey and his wife Mona and seen everything that the real NCIS does. And as part of that, I've met with Jim here, who I thank profusely for inviting me here today. But at the same time, I also had the pleasure of going down to Atlanta and going to the Cyber Crying Conference. And I carry my little notebook because we have a bunch of writers back in Valencia. And they're very, very, very good. And I try so hard to keep everything that I do, particularly in autopsy, to be as real and as accurate as possible. So I spend a great deal of time now making notes, taking them back, sitting down, being with the writers. This year I was on the carrier wasp. I was given an entire tour by Admiral Skolanski. I've been aboard the Reagan and met the skipper back then. I've just done so much with the Navy and the Marine Corps. My wife here is on the board of the Marine Corps Scholarship Foundation, and they've raised $70 million to send the kids of Marines to college, which is quite amazing. So it's quite extraordinary that someone a week head from Glasgow, as I'm known as back home, managed to sort of climb up here and I'm on panels. And at the 50th anniversary of the Marine Corps Scholarship Ball, I was the emcee. I never dreamt in my life that I would do things like that, but it is such a pleasure to have that involvement. And as I said earlier, to be able to give something back, because I get so much from my life right now. Thank you. I'm Rob Joyce. I'm the Deputy Director of the National Security Agency's Information Assurance Directorate. I've been with NSA for 23 years, and my organization is responsible for the protection of national security systems, which are anything that carries classified information, handles military command and control, or supports intelligence. So we do everything from developing equipment to finding vulnerabilities and consulting with departments, agencies, and those in need. Thanks. I'm Leon Carroll. I'm the technical advisor on the TV show that I get to work with Mr. David McCallum all the time. And I am a former Marine officer and 23-year veteran with NCIS. And I sit with the writers, the guys and gals that he just spoke about, and I get to wave the bullshit flag. I try to keep our stories and what we do, whatever the investigative genre. He's from Scotland. Scotch. Actually, we do this on the show quite a bit, so no problem. Actually, every year we have a great tequila bar just before Christmas, and there are a lot of people after we say wrap that stick around and eat tacos and drink tequila shots. Anyway, it's my pleasure to be here. I thank Jim for inviting both David and I representing the show. I work with a great cast, and we have a bunch of great people in our crew, and thank goodness we're the number one drama on TV. I don't know who these guys are, but I love them. So you guys are going to save some of that for the questions, too, right? Nami, if we get good questions, you guys are going to help out the audience. So good afternoon, Hackers. I'm Mark Sox. I'm a retired Army officer. I did 20 years as a Corps of Engineers guy, jumped out of airplanes, blew things up. We built targets for the artillery. That's what engineers do. But along the way, I had a passion for computers and networks and things that talked to each other, and Uncle Sam figured out I knew probably more about computers than I knew about blowing things up. So I wound up running 18th Airborne Corps networks back in the mid-90s, 4th Infantry Division's networks in the late 90s, and was summoned to Washington in 1998 after a series of intrusions that scared the living crap out of the Pentagon, and we stood up the Joint Task Force for Computer Network Defense. We were the first group to really take on this problem of what happens in cyberspace when people are attacking DOD and others. That group is today, the D&A of it lives on in U.S. Cyber Command, and a lot of good things have happened since then. When I retired from the military at the end of 2001, September 11, had just happened. I was summoned by the White House to be an appointee working for George Bush, actually underneath Condi Rice's staff from the National Security Council, where I spent a year and a half at a policy level, but I was a geek in the White House, and I tell you what, that is one cool experience, because everybody else there is political appointees. There are one or two people that have the technical skills, and it is fascinating to work at that level. I would really encourage anybody who has these opportunities. If you can serve your country at that level, please do so. It's an experience that... Just like these guys are serving their country. Yeah, just like these guys. So today, I am at Verizon, so this is why I get free Verizon swag. And if I get a good question, if somebody comes up and tells me how much they've hacked AT&T this afternoon, I will happily give them some more Verizon swag. There's your first question. Good afternoon, Mark Weatherford. Like Mark Sox, I am a retired naval officer. I spent 20 years as a cryptologist in the Navy. Since I left the Navy, I've done a number of things. I'm currently the Deputy Undersecretary for Cyber Security at the Department of Homeland Security. I was appointed by Secretary Napolitano about seven months ago. Prior to DHS, I was the Chief Information... Chief Security Officer and Vice President at the North American Electric Reliability Corporation, where I worked with the electric utilities across North America, helping them put in the policies and programs to secure their environments. Prior to that, I worked for Governor Schwarzenegger. I was the Chief Security Officer for the State of California for a couple of years. And then prior to that, I was in DHS, I was the Chief Security Officer there for a couple of years. At DHS, we have quite a growing responsibility for cybersecurity. We have three major functions. We protect the dot-gov environment. So all of the federal civilian agencies were responsible for working with all of them to make sure that they have not only the policies in place, but the controls in place and that they're able to secure their different environments. We also work with the private sector, primarily with the critical infrastructures, helping them and helping them respond to incidents and intrusions into their environments. A lot of control system work there. We also respond to incidents of national significance. So when anything really big happens, we put teams on the ground working with private sector and government organizations to help mitigate and respond to those. So and like Mark said, DHS is a great place to be. I don't know if any of you heard my speech yesterday, but we are always actively recruiting and we're recruiting for people who can think outside the typical government, federal government way of thinking. So that's why I love the con because I get to meet a lot of people and I've actually picked up very interesting resumes for people that want to come and participate in what we're doing in the government. Good afternoon. So first off, I'd like to recognize any fellow combat veterans or veterans if you just stand up real quick if we have any veterans out there. Thanks again for serving. Name's Jerry Dixon. I was the former director of the National Cyber Security Division at DHS, ran USR, worked at everybody's favorite agency, the IRS. I ran the incident response team there. But prior to that, I ran security for Marriott International. And so, you know, now I'm with a group called Team Cymruy, a not-for-profit security research group. If you get a chance, yeah. Go out to the Cymruy website. There's tools that we give back to the community. C-Y-M-R-U. So if you're not familiar with it. But we have tools like the Mauer hash tool. We provide a lot of things. We try to give back to the internet community at large. We also have research that we're always looking for researchers to help out. Basically being able to take a look at our insight and basically find other ways to develop tools to help a lot of different organizations. Thank you. Good afternoon. My name is Riley Repco. And first off, I want to thank Jim and all of you participants. And thank DEF CON. I mean, it's all inspiring to me every time I come out here to see more and more of these young wizards that are here. I recently left an executive position at the Department of Air Force D. What an opportunity of being able to serve with some of the finest 18th century minds around. All right. I came from the private sector. My role there was to kind of build the bridge between our needs and your solutions. Kind of identify the seekers and the solvers. And as Mike Hayden, former director of the Central Intelligence Agency and NSA used to say, he said, God made the four natural domains, air, land, space and sea. Man made the cyber domain. God has done a better job. I'm glad to be here and look forward to answering any questions you may have. Hi, I'm Michelle Kwan. And I just want to start off by thanking those folks at DEF CON for bringing this panel back. This is my 10th con. Happy to be here. I was, let's start back in those days, I was in 2004 winner of the most innovative antenna, the Wi-Fi shoot out. I then moved on to be the director of wireless at the Department of Justice. I then moved to be the deputy CISO at the Department of Justice where I built the Justice Security Operations Center and from there I moved to be the director of U.S. CERT. And we specialize in security operations centers. I'm Rich Marshall. I feel like a badger at an AKC registered dog show with everybody that's so bright on the panel. I'm a lawyer by training and I'm very proud of the fact that I do information warfare law. I've had a checkered past. I got involved in electronics in the Southeast Asian war games in an F-105 where we were chum waiting for the missiles to come up at us so we could take them out and protect the rest of the aircraft that came in. We were first in, last out. And I learned early on how important electronic warfare could be because you've got your life on the line. I later became a lawyer in the Air Force and had a successful career there and then I moved on to the National Security Agency where I was the associate general counsel for information assurance. Worked for four different gentlemen who happened to lead in that particular position. My proudest accomplishment when I was in that position was being the legal architect for eligible receiver 97 because I demonstrated to the national leadership that information warfare was real. It also gave me an opportunity to learn a couple of very important leadership lessons. When I briefed the general counsel of NSA and my boss, my line boss on what we could do they said well you need to brief the DOD general counsel you have to go up the food chain. I said fine let's arrange joint schedules and they said no you go by yourself. I briefed the DOD general counsel and I think you were in on that briefing at that time and once again it was great job you need to brief the attorney general now I said yes ma'am I'll arrange our schedules and she said no you go by yourself. The life lesson there is whenever you're doing anything without a safety net you always go by yourself because then they have plausible deniability but ER 97 was so successful venture it has a thousand fathers failure is born illegitimate but that was an absolutely awesome experience I subsequently moved on worked at the White House with Mark and then later came back to NSA and did legislative liaison which was just absolutely a fascinating opportunity and then I subsequently had an opportunity to go down to the Department of Homeland Security and I was the director of global cybersecurity management another fascinating opportunity and I retired from NSA on the 15th of December which was a Friday and then the following Monday I moved to a startup that lasted three and a half months I'm suing them part of my DNA and now like Michelle I run a consulting company and really enjoy what I'm doing and I'm just very happy to be here this is my 11th one Thanks, Rich. For those of you who don't know about Ella's receiver 97 I highly recommend it and then the follow on Solar Sunrise, real world event set the course for a large part of where at least DOD is today in cybersecurity My name is Lynn Wells I'm career Navy I then spent 16 years in the office secretary of defense including a couple years as DOD chief information officer some years ago I was speaker here at DEF CON I've been promoted I'm a goon and so things are looking up right now I'm over at national defense university had a small center called the center of technology and national security policy and from this I'd like to ask the audience two questions we did a conference about a year ago on the trajectory of revolutions looking at the Arab Spring and how are things going to work out and one of the most articulate spokespeople in the conference was a White House producer for Al Jazeera English and she was 26 and talked about how Al Jazeera used Facebook and Twitter and blogosphere and other social media to alert the correspondence to the emerging issues and so at the end of the conference I said what's the next big thing and her answer at 26 was I'm too old as the young people so there are a lot of you young people out here and part of the problem is we have usually in our conferences I ask how many people are under 35 and get single digit hands so one of the great values for me at least of coming here is to get your opinions and really value your thoughts on how five years now you would vector people to better source of information the last point is that we're working with DARPA on a project called social media strategic communication and one of the things that my center's been asked to do is look at the risk posed by social media and not just to leak the wrong stuff but that are we raising a generation of things 140 character sound bites and how do you put checks and balances into a governance system where the velocity of information is so enormous again if any of you are interested in working on that I would love to talk to you so I just had to close by saying last month I had the privilege of beginning a company and every morning I wake up and say wow what really interesting thing is going to happen today and if any of you are interested in careers of government I cannot commend it to you hardly enough thanks wow almost time for a golden anniversary then huh? so I'm Rod Beckstrom I've spent most of my life in Silicon Valley doing high tech companies but had the great honor to work for the federal government for a while so I wanted to share that story to those of you who haven't had the privilege to work for the government yet to consider doing that I fell in love with computers when I learned to program in Fortran when I was a teenager and started my first software company when I was in graduate school age 24 in my garage apartment in Palo Alto I got very lucky because we built something that people seemed to actually need in trading derivatives around the world and that company grew to be a global company and went public on Nasdaq and I got to run that and build a company from scratch to a public entity and run that for five years which was a great experience and then got to work with a lot of other good startups in Silicon Valley and helped start a number of companies I was in New York on 9-11 and my life changed like many of our lives changed and was called to a form of service and worked on track 2 diplomacy in war zones around the world that network spread in a viral fashion and we decided to model it on Al Qaeda and by modeling on Al Qaeda we had to then understand it and decode it and that led to a book called the starfish and the spider the unstoppable power of leaderless organizations which describes decentralized movements and networks including Al Qaeda and many other networks like this community because of that book I got incredibly fortunate and Admiral McConnell the director of national intelligence who is an amazing man was very kind and recruited me to be on his senior advisory group and the books were quite provocative about networks, cyber security and social networks and so it was my great honor to serve on that group with him and other leaders from the intelligence community as well as four other members from the private sector and then I was asked to run the national cyber security center which was created by a presidential directive and Admiral McConnell and Secretary Chertoff and sec def and the attorney general asked me to come and stand up for that organization which is a startup so I'm a startup guy, this is a startup in government which was completely different and I was lucky to work with many of the great people at this table we got the concept of operations approved we created the 21st component of DHS the first component was the Coast Guard created in 1790 that then got merged subsequently I think into the organization that Mark's running now and Michelle and I worked together as well on these issues anyway so I was able for the Obama administration left and had the good fortune to be asked to lead ICANN the multi-stakeholder body this time in the federal government gave me the tools that I needed to work at ICANN and to understand not just the private sector but the public sector and how incredibly challenging and important the work are that everybody in the government does and just gave me a tremendous appreciation for the people who do public service some day I'll probably go back and certainly I have the passion for the importance of governance institution be they governmental or multi-stakeholder so I encourage each of you to really consider it in your lives and it's great for the country that we have people like Len serving for 49 years and others so thank you I'm just honored to be here among the giants at this table thank you I'm Jim Christie I'm with the Department of Defense Cyber Crime Center just want to talk a little bit about what we do we have one of our business areas is we have the world's largest accredited digital forensics lab so we have about 110 digital forensic examiners working all kinds of cases from terrorism, espionage child pornography fraud you know you name the homicides you name it we also have a training academy where we train all the criminal and counterintelligence investigators in the Department of Defense on how to conduct a cyber crime investigation and how to do digital forensics there's several other business areas one of the initiatives that we have is the Center for Digital Forensics Academic Excellence the problem we have is we can't find qualified people that can pass our tests so they can work on digital forensics on real life criminal cases so what we've done we've created a partnership with academia to create standards for digital forensic examiners and we're in a pilot phase right now with about 9, 2, 4 year and graduate level universities and we're opening it up to other universities everybody else come September another initiative we have is the DC3 digital forensics challenge so anybody out there taking the challenge alright now we have about 945 teams from 51 countries participating and I would encourage you to get either yourself or your staff or your kids involved in participating in digital forensics course we need good people in the government and we're always hiring one of the other things that we have is the digital crime scene challenge and what we did was we brought we have three suites set up upstairs and all the DEF CON kids are scheduled to go through and process a digital crime scene in 15 minutes and come up with the evidence so kind of fun so if you get a chance if you have kids here bring them through another initiative is an online self assessment on cyber crime investigations and digital forensics so if you go to our website dc3.mil and all the links are there take the self assessment tell us what you think if you guys I would really love to get questions for you guys to add to our self assessment to make it a little more difficult so at that point I want to thank our panel and now we're going to open it up to questions from the floor my question is for Rob and Rich are really any of the three letters do you see a shift in your recruitment to move away from more of the standard of having a four year degree and have the exception based upon experience so NSA is certainly a very talent based organization so to come in and be a technical person in NSA you absolutely do not have to have a technical degree so the recognition of past accomplishments your current work portfolio and things absolutely can get you in the door I think you know one of the big things we look for is that audacity that's one of the reasons we're here at DEF CON you look at hackers often know the networks they're penetrating better than the system administrators know them themselves so we're looking for people with that kind of mentality that really want to dig into the details understand the technology and apply it so that's what we're looking for let me let me add an observation oh go ahead let me go first Rich do you mind? you used to be in my balls I got to say yes you take his turn he's going to sue your ass wait go ahead Rich no that is absolutely right and I talked about this a little bit yesterday in my remarks I know that there's a perception and we talk a lot about the different programs we have that we're supporting at the universities and colleges around the country but you know some of the smartest people I know do not have a college degree some of the smartest people that are working for us right now some of the smartest people that I had working for me in the private sector and I think we as a government and I can it's a little bit of a challenge sometimes for us but we have to encourage that you can have that equivalent experience without a four-year degree and the way I like that and the way one of my really smart friends put it to me is I spent four years with my hands on a keyboard well a lot of other people were spending four years in college and I would like to point out Dr. Dan Mansons right up here in front he's a professor at Cal Poly in Pomona and he has an incredible program there they just ran a cyber camp last week and they had a 60 year old dude in there in that cyber camp and competing in the CTF last Friday so you know there is no age limit and there's no there's no educational qualifications like real experience in my mind the Olympics are taking place in London I don't think this is a news flash for anyone but everyone who was participating as an athlete in that event started out extremely young in their particular sport being trained most of them don't have college degrees I do not like the word hacker your vulnerability researchers that's the way you should present yourself number one number two we need more emphasis on STEM education science technology engineering and math and I realize that plays well but I've long been an advocate of recruiting at DEF CON and I got some push back from some entities that said we're not going to recruit criminals and I said they're not criminals until they're prosecuted so give them an opportunity to get out of their mother's basement and earn a living don't sit down yet one other comment vulnerability researchers in my mind are today's artists when you look at the great painters in the past none of them had a college degree they learned it themselves through experimentation and that's what vulnerability researchers are known to do so view yourself as an artist you've got to have technical skills those painters develop their technical skills as well but they had a fascinating imagination and in their mind's eye they were able to create art so as you're doing your vulnerability research make some great art really loud please so we talk about education and there's more to you know being a vulnerability researcher to just what you know right we have a mindset I mean the brightest people we don't have degrees we don't have certifications we don't need them but we're forbidden also from teaching the next generation I mean a lot of you know these four year universities a lot of us could really teach people now it's not impossible I've found other ways around it but it would be something that not only to get jobs for the future of the people but we need a way to teach and I look back here I see a lot of potential teachers and not just the jobs but the future now specifically to the NCIS staff I would ask would it be I don't watch TV because I want to do stuff for the kids right exactly would it be possible to have an episode where you had maybe even a season where you know there's a big security crime this you guys say this guy was the guy that found it you know and you go back his life about how he got there not just the end point and make a big deal that he was the person that was able to actually find a crime but how he actually you know became his life so you kind of you know how we grow into being vulnerability researchers so I ask if you guys could pass that on as a potential plot the writers are very much open to suggestions and I try to take things back to them what you're really saying is the background to something that's already happened which is always the hardest thing for them to do because you either do it in a flashback or some people talk about it neither of which is instant neither of which is really very exciting on television but it is something you're when you say the background you mean I mean I'm much more fascinated about what you're saying when you say that you can't teach because you don't have university degrees therefore you don't have what we call in England the poem of education and everything well it goes to a point that people have problems there are people that are hackers they don't or vulnerability researchers they don't even know that they know kids that know stuff and can do stuff on computers that nobody can do but they don't have any role models they can base off of right it's very very hard there's no way to measure yourself versus what you know versus what you don't know we need sports athletes and stuff we broadcast that shit all over the place right they know if they want to be an athlete we don't have a way for them to say you know what you're doing the right kind of stuff you just don't know it it's something that we need some kind of role model not only the end point that they fixed a crime or something but so they can identify with who they are well it's definitely an episode that involves McGee who is our computer vulnerability investigator so I think that it's an idea that I can definitely and will put to the right thank you we only have four minutes left so short questions my questions half for the incident response teams and about half for the NCIS crew about like how does are you seeing the CSI effect in regards to your specific discipline because I know that's still a big problem and for the incident response people is that something that's being mitigated with education and then for the for NCIS is that something that you guys try and handle realistically on the show do you deal with it do you have to like shoot down ideas by just saying okay that's just the CSI effect what are you seeing really quick I was on jury duty when the show first started and the attorneys and the judge made a point of instructing the jury that what they would hear in that courtroom would not replicate anything that's done on a one hour TV show 43 minutes and believe it or not NCIS was the show that they used I'll turn it over to the policy guys here but that is something that obviously we look at as well but we're here to entertain and so in 43 minutes of film or whatever you know we're going to show you know a lot of things that are abbreviated for instance Abby can't get the DNA in real life as fast as she does on television or David certainly couldn't do an autopsy as quickly as he does it on TV so I have seen you guys address like I think Abby's called out to one point I'm not I'm not the CSI people I don't work that fast so I know it's been touched on but I'm wondering like behind the scenes but I think for the folks that work I mean I worked in the real world beforehand and when CSI first came out I'm sure it did have an impact to the point that when you go to a courtroom and these folks can attest to that that they're instructed that what you see in real life or the DNA effect going back to the OJ trial actually is not how it really happens in the real world so I'll pass that to somebody that does it you know the biggest problem we faced is the show 24 how can Jack go 24 hours with no recharge on his cell phone magic well I'll say that it's definitely an issue but it probably in a little bit different way than you'd think it's particularly an issue with executives and up the chain in getting approval and a lot of that comes from the difficulty in educating up and for technical people to be able to talk in the language that executives can understand and hear and embrace and we also have issues with our user community not understanding that every cute device they find isn't going to be attached to the network but I would definitely say that we have a bit of a PR problem part of what we should learn along with the great technical things we can do is how to talk up so just real quick to add to that I think obviously media plays a role in expectations of leadership or management just to Mark's example about 24 there's an unrealistic expectation that you can contain a particular incident you can do the forensics in a matter of hours when realistically a lot of cases are things that you work on take sometimes can take months if you've got to look at a terabyte of log data process that you've got to go through so there is an effect to those that are not practitioners but more to the management side unfortunately the goons have said we're out of time so what the panels are going to do is go across the hall for additional questions and answers and priest has a men and women in black panel following this immediately thanks for coming thanks for good