 totally next up we have defending cyberspace by thinking inside the box by Ernest Wong please give him a warm tour calm welcome thank you very much I appreciate just a quick show of hands any military folks or ex-military in the audience would you mind telling me a branch of service army marines okay Marines no Navy no Air Force folks I was gonna give a plug Navy folks because since we're in San Diego I know there's a lot of Navy but I'll give a plug from anyway my brother's in the Navy so it's actually Army Navy games aren't that bad either way my talk is really getting our military leadership to start thinking outside the box for cyber defense I know the title says thinking inside the box but it's really trying to give creativity to our military folks so what for everyone else in here hopefully this gives you a framework for just thinking about innovation in a more structured way that helps you all in your your lives your own lives and maybe for your company and as I go through this presentation I'll introduce myself as I go through the presentation because what I found was that with the keynote speech that Lance gave this morning a lot of similarities I saw myself with him I've only been in the cyber cyber business for two years now so this is my very first Torcon very happy to be in San Diego my brother spent eight years of his military service here in San Diego got chance to visit him coming from Los Angeles so I'm very happy to be here in San Diego and before I get really started on talking about this notion of innovation thinking inside the box not a whole lot of military folks in the audience I'm hazarding guests not a whole lot of folks are familiar with Army Cyber Institute and so seeing I did this nice video about three minute videos hopefully it works gives a little background on West Point itself where I teach and where the Army Cyber Institute located at as a think tank okay it's coming through this yeah that's kind of light but I'll try and narrate it a little bit then it's so each year the NSA hosts this cyber defense exercise and all the military academies participate in it and cadets are working on it's a cyber defense team it's an elective really it's a extracurricular club that these cadets are involved in each year about four of our research scientists at the Army Cyber Institute help out with the oversee administrative coordinate this exercise so that's my boss Colonel Andy Hall is my boss so I am very fortunate to be working the Army Cyber Institute I've been there two and a half years now before then I and I mentioned that the very start that I'm pretty much just a basic before that I was just a basic user right getting the annual training for the military folks you know the online training the hour-long training it tells you what you can and can't do with what you do on your computers and cell phones and tablets and so I see the flawed system in a lot of our education system for our military soldiers and what we try to do in the ACI is promote this notion of multidisciplinary research to solve this challenge known as cyber security and my take on it is that this notion of innovation I I actually don't think innovation as this one monolithic entity I break it up so a little bit of about what Lance mentioned this morning the keynote speech talking about being innovative and being disruptive that's really what I want our military to be in terms of cyber defense as well and so instead of looking innovation as this one entity I've broken it out into several distinctions first on one axis I'm looking at the technological sophistication or complexity of that innovation itself on the other axis I'm breaking up into the target mark this innovation technology device wherever we want to call it is trying to trying to target so if we look on this lower left hand quadrant here this notion of being a low complexity targeting existing markets I call this sustaining types of innovation we can think a lot of companies right they change the colors they're trying to track more market share though that's why I'm thinking of sustaining innovation needing existing customer needs but just trying to grow the business for the high complexity but still targeting existing markets I call this evolutionary right Darwin talked about right all you're trying to do is with evolution right it's the mix that gets you to be one species above some other species and so this is the notion of incremental types of innovation now when we get to new markets and high technology or high complexity I call that breakthrough types of innovation governments tend to be pretty good at this I'm thinking DARPA NASA a lot of government agencies tend to be good at this now what about this thing right low complexity but new markets I'm calling that revolutionary or disruptive types of innovation right hopefully a lot of stuff that we've been hearing throughout the day and hopefully tomorrow will be hearing a lot remotely only what I call revolution disruptive invasions the problem I see here though is unfortunately right most I would say most people in the military most people in the military from what I've encountered don't think revolutionary sustaining invasions as being innovative right they're fixated on this high complexity right AI we saw in that video right machine learning the military likes automating all this stuff because we were tended to be a smaller military compared to write the Chinese and Russian militaries have lost soldiers so in this case a lot of military folks and even some right some people think innovations can only be high tech okay hopefully that's not the case for the folks in this audience right most hackers I think most hackers tend to think otherwise most hackers actually I would hopefully would agree in this room revolutionary sustaining innovations are innovations in themselves hacks hacks themselves right we thought about hacks that's a revolutionary innovation it doesn't cost a whole lot of money to take a lot of research development okay so hopefully that's why I'm gonna convince you for the next 45 minutes here okay I'm a military intelligence officer and so when I think of military intelligence I told you I give you a little background as this presentation goes on the way I think of these innovations and the analogies I use it comes from Hollywood right Hollywood is giving me some some nice little cheap mnemonics from memory mechanisms think of each of these types of these separate quadrants so I think of sustaining innovation I'm thinking Spock okay I'm a Star Trek fan I am a Star Wars fan I'm also a Star Trek fan so I'm thinking Spock right Spock is very logical right he's gonna give Captain Kirk the the course of action that has the highest probability of success even if it means his own life and I guess that was a breath of cotton so Star Wars 2 so he's gonna give the the best course of action based on the bailout information so I'm thinking that's sort of sustaining efficiency gains right thinking Spock on an evolutionary side any mad magazine fans or still remember mad magazine you might remember spy versus spy okay so that's really what I think of evolutionary right for the white spy versus the black spy they're trying to kill each other right so but all they're doing is trying to one up each other and that's a whole notion of evolution on the breakthrough side I am thinking James Bond right James Bond has the entire resources at MI6 the entire British wealth I guess whatever it is after what was left I guess whoever they can say after the the bricks it I guess that's still in place but you know James Bondian types of innovation right he's always got the watch the car the plane whatever it saves the day right this arms and nukes what about this revolutionary side do we have any examples of revolutionary Hollywood icons someone mentioned earlier this morning MacGyver yeah we're all American versus British that's kind of nice way to think of this too right MacGyver he's got his trustee what is it he's always using his Swiss Army knife I watched MacGyver last night for the first time I mentioned this to Aaron earlier I couldn't stay awake long enough to finish it I'm on East Coast time so I think it was on at nine o'clock but I did see him with the Swiss Army knife when I saw that I said all right I'm on the right track I cut them cut the show I don't even see anymore but for the audience no everyone's familiar with yeah I'm pulling this audience now I think everyone should be familiar with MacGyver or the new guy any thumbs up or thumbs down a new MacGyver anyone watching it thumbs down I saw two people in the promo I thought MacGyver was just one person so I'll have to figure that out and now if you're not familiar with MacGyver depending on the audience I think everyone here should be familiar with MacGyver but if you're not familiar with MacGyver you should be familiar with Jason Bourne okay so Jason Bourne right he's using all the tricks of the trade right he doesn't remember like he got all this training but right he's driving the mini-coop the old mini- coop he's right using a stick to beat up the cops he's using a pen right he's using whatever that has fingertips to us to save the day usually it's saving his own life now one of these is not a good military intelligence type of analogy right it's not yeah Spock doesn't really belong here tell you I'm a Star Trek fan but Spock really doesn't belong here right because if we're talking about military intelligence first think about spies right I really am not thinking about mission I'm thinking mission impossible here but I'm not thinking of Tom Cruise mission impossible right because this is the episode where I think it was Rogue Nation where he's climbing the highest tower right the Burj Khalif one of those the towers in in in the United Arab Emirates right with that glove right that's a high-tech equipment right that's a lot of R&D if it were really real right a lot R&D right had glitches right powers off a few times but that's a breakthrough types of innovation what I'm thinking right for sustaining is if you remember mission impossible the original right the black and white it was in color right Paris right letter name would play Paris and for mission impossible right there are tricks of the trade were the skills that they had available to them so Spock spoke like 10 different languages in reality was just different pronunciations of English right it's just very something to all understand it but I remember watching one episode where he played this Japanese ambassador and he was making out like he spoke Japanese usually he's playing some type of Eastern European type of general so again but that was really for all the MI right the MI forces the impossible forces they were doing sustaining types of innovation they were the best at what they were doing right that's what it was the very first sequence you've picked the best people you have with the skills you need you play the con on on this unsuspecting nation now if you are a mission impossible fan I am gonna yes you hear in the background here again I told you this is San Diego I'm gonna play at the Navy folks can you in here that well I'm gonna play it right here we go yeah so you should write your Tom Cruise van I'm not gonna leave you out Tom Cruise was a little bit revolutionary what movie is this top gun I tough gun fighter pilots how can they can maybe revolutionary well this whole notion is that I'll leave it on since it's in the in the Vietnam War the US pilots suffered so many casualties here's here's statistics about for a million sorties from 1965 to 1968 a million US sorties Air Force Navy US suffered about a thousand plane losses that was the the US forces did not like that so we actually stopped bombing from 68 to 69 trying to figure out how to get our bombers to survive more even our fireplates so what the Navy did the Navy they had right they did this report golly six the Navy said we need to train our pilots better we're gonna get this school called the school called the right it's called the fire weapons school they called it top gun right that's what the movie this was actually from the intro of the movie I had to go back and watch the movie yeah they actually did this right in March of 1960 1969 US Navy established the school the Air Force what the Air Force do they went the route of the breakthrough so the Air Force went the route of the breakthrough they said we need better weapon systems we need an aircraft at the time it was the F4 Phantom fighting against the Russian MiG 17s of MiG 21s okay it was pretty much parody from 65 to 68 parody we shot down as many Russian planes as they shot down a bar so that that's why they stopped those runs the bombing campaigns plus the flights the US military said we can't do this we can't sustain our rates of sorties against the Russian hordes because they had so many more pilots so many more planes and so the Navy went revolutionary they said we're gonna start the school and figure out the techniques that makes our maker pilots better the Air Force went the route of better planes better weapon systems improve the F4 Phantoms and amazing this is from a book called transforming the American Air Power by Benjamin Lambeth he's this is amazing the kill ratio for US pilots to Russians prior to the start of the school was 3.701 so for every one American pilot we shot down 3.7 Russian pilots that's pretty good right but we couldn't sustain that over time that's what the after the school was in place for one year when they reintroduced the fighting campaign over Vietnam for the Navy they went to a ratio of this is amazing 13 to 1 so the pilots who went through again wasn't every single pilot the pilots who went through this four or five week long course at that time was a six week long course they improve their skills so much because they're fighting against right they're fighting against tactics that the Russians are using right if you remember the movie Viper and Jester they were the off for the opposing forces right that's what they did they introduced Russian tactics the amazing thing here's the here's the crazy thing about history I told you the Air Force they went breakthrough the actual Navy actually I'll use a lot of principles from this colonel from the Air Force called John Boyd he invented this notion called Oodaloops this notion of orient observe orient decide assess that's what the Navy was doing throughout that course right see what the Navy pilots doing see what your opponent is doing observe it orient towards it assess what he's doing then make some type of decision that gives you some type of vantage if you can do it faster than the enemy that's even better you can their Oodaloop the Air Force their statistics they had about the same starting off about three three to one after the introduction of new new firepower into the F4's new systems in 1969 the statistics were that year that was the first year the Air Force actually stats went down it went down to 2.9 to one so very statistically no improvement so from a three to one to a 13 to one ratio for the Navy to a three to one to a three to one the or for the Air Force again pretty powerful statistics were an anecdote for disruptive types of invasion now I'm not saying that breakthroughs bad right Air Force does great things doing new generations of fighters but it takes a lot of time and one year span they couldn't do it for Vietnam so this not this notion of looking at this this quadrant system for innovation right this is not from a mathematical perspective right this notion is I don't have any numbers here any business majors in the audience business school yeah you control break the world right business school they tell you can break the world in four quadrants all time I'm not a big fan of that because there's no numbers right where the heck it's my scale right so in this case I'm gonna try to give you a little bit of scale so let's give you a little bit of scale by introducing one more axis this notion called offset potential or possibility of impact disruption that's what Lance talked about last week talked about this morning okay if we look at this offset potential now one more axis right now this graph changes revolutionary and breakthrough innovations those are the ones that give us the best probability of changing the environment sustaining an evolutionary in invasions don't do that right the highest probably success for impacting changing the environment comes from revolutionary breakthrough types of innovation now again I still don't have numbers here right I don't have numbers but this is probably a more representative way of this at the scrap look if I add this offset potential now if I change this offset potential and now probably success we should already know what it looks like right now probably success at these innovations actually succeed yeah then that's the same evolutionary innovations that actually have had had the highest chance right but if you look at right all the business books right all the CEOs are writing take chances make mistakes they're not saying make mistakes with these types of innovation if you're doing sustaining evolutionary types innovation you should be getting 60 70 80 90% success rates they're thinking revolutionary breakthrough right but no one's ever quantified that so everyone thinks of innovation is one of any innovation no I'm telling you you need to be taking revolutionary breakthrough innovations if you truly are following the advice of all these business right these business locals are saying tell you take take chances take risks we're telling you make those mistakes in the revolutionary breakthrough side okay so this is really what would look like more representation okay again I have no numbers here right again as a systems engineer I teach some engineering in addition work the army side of truth as a research scientist I teach systems engineering to cadets at West Point and again I tell them they have to quantify certain things not just call qualitative analysis but quantify analysis as well so I used right this notion called the Arab revolt the green revolt the Arab Spring that took place from 2010 2011 it was amazing right this is kind of like a nice case study it started off with in December of 2010 right a street vendor set himself on fire because the Tunisian police confiscated his vegetables right he was street peddler of vegetables and so he took away his livelihood I don't know what the reason was but because of that the vendor set himself on fire right emulated himself and that sparked long with this whole notion of social media right at that time wiki leaks came out with this report of all these corrupt governments a lot of them in the Middle East North Middle East North Africa right from our intelligence sources State Department cables yeah so it wasn't it wasn't very good so a lot of 13 other countries revolted pretty much within the next six months so revolts anyone remember how many of these revolts actually succeeded Tunisia was one so where first started that government fell Egypt the king there was deposed yeah we have someone saying sort of sort of sort of so hold on to that thought we have Yemen yeah Yemen Yemen the king fled there's still a lot fighting there right the king fled so the revolt succeeded but there's still a lot of fighting in fighting between groups so you might debate whether not the success and we had Libya right Libya fell Gaddafi right two years later we killed them well at least we didn't kill them we some rebels killed them we captured on film so yeah we had someone say so so for Egypt yeah I don't really call that a revolution it started off as a revolution but who took over in Egypt military took over so that was a military coup right it started off as a revolution but within a few days it was the military that actually took over the government and so if we look at these statistics right I don't consider military coups revolution right because military I'm in the military the military has resources to overtake a government that's why when you talk about inefficiencies in the government I look at this and tell yeah we learn from the Romans right the Americans learn from the Romans it's not good to have a large standing army right with nothing to do staying by a government that's probably not doing a great job right the public poll is saying the president's polls are so much in Congress's polls are even worse you really don't want to have a military in the States when that happens so again so just tell you just to give me an anecdote how inefficient the government is I've been in the military for 22 years I've moved 11 times the average is three years for an assignment I took a lot of one year assignments so the average so again that's a lot of inefficiencies a lot of inefficiencies with our service security program that's why I'm trying to promote this this talk here trying to get our government leaders to think of again new ways of thinking this so again that's statistics 21 percent chance of successful revolution and that's just looking at this as a case study that's actually pretty good I would have thought I would have thought it was been in single digits for that if I were just throw a number again using this case say about 20% successful okay so now I can use numbers right and use probability about 20% but again Hollywood right this whole notion of Hollywood again the Star Wars fans who are my Star Wars fans here right Hollywood gives us this notion that revolutions are easy right and then by by by associate theory then breakthroughs are easy right yeah we have this notion of right not just Star Wars we have Rogue One the prequel to Star Wars revolution are easy right people died in those those movies but it happens right we can we can fight the force we can fight the Empire the problem is the problem is on the right once Darth Vader gets hold of what's going on with this revolution right the evolutionary side the spy versus spy he's going to try to wipe out that revolution he can right what does he do where's Darth Vader use where's Darth Vader use he uses the breakthrough right from his evolutionary side he comes up with the Death Star I'm not gonna take care of these rebels one at a time I'm gonna use the Death Star if I think there's a rebel on that planet I'm taking the whole planet out that's a that's a breakthrough type of innovation right it takes a long time right took a long time from the build the Death Star right the rebels got the plans I actually thwarted one of them but right breakthrough innovation once you have one you start making another okay so this notion of revolutionary innovation also instead of thinking just the quadrant system I also think of it in terms of time as a timeline so the person that person's actually came to this timeline this framework here are Joseph Christiansen and I'm I'm sorry Joseph Bauer and Clayton Christiansen a couple Harvard Business School professors and the way we were looking at this right it's color code still writes that we have sustaining and evolutionary innovations again it is making progress over time but incremental right very small so if we look at breakthrough it's jumping above lights that's why I call it my subplot for breakthrough innovation is jumping a curve it's jumping what the market expects of some type of device technology equipment whatever it is on the revolutionary side yeah so the Christian Bauer they actually study this notion the failure of leading companies right companies on this green line to stay at the top of their industries when technologies and markets change how is that even possible right if you're on this green line that is a great place to be right you're almost like a monopoly power you're on this green line we dominate the market why would these small rebels insurgents very calm upstarts right in the in the business parlance why would these upstarts ever bother us right we compete with them on every level well this notion of revolutionary innovations and a current Christian and sin and Bauer talked about this specifically is that revolutionary innovations only appeal to a small set of new markets I talked about that that quadrant system it's really an emerging market initially it's really for the individual right as hackers we make hacks because it suits our lives but it focuses on a different set right that criteria that's so important to that individual is not what the folks on the green line are thinking about that the folks on the green line are thinking of things that improve their current customers typically their current customers are the big customers right we want the biggest market share that is that what they teach us in business school right we don't care about the small margins let the small margins go to competitors they can have that market segmentation we want the big customers our most profitable customers for the revolutionary innovation though it is far worse in at least one or two areas usually a lot of areas it's far worse not least one or two mainstream areas but the notion is that if it's successful the value criteria which the guys in green are are ignoring right it gains such a attraction that folks keep buying it to the expense of the guys in green they've missed that opportunity can we think of examples of that this in history from business Xerox versus cannon yeah yeah I'm gonna come up to that good hold that thought what about Xerox first would cannon due to Xerox everyone remembers Xerox yeah Xerox used to have print divisions right they sold these monster that would probably fit in half this room especially like universities large corporations IBM's the cannon cannon specifically who who did cannon target they target the home base business yeah right cannon came up with this copier that could fit on your desktop right desktop printer terrible right Xerox laughed at cannon when it first came out because the quality was terrible right Xerox is on the green line they laughed at cannon right another example IBM versus Apple same thing IBM laughed at Apple when they came up with that right to we saw it this morning Xerox and Apple as yeah well well you know you all well I'm in a room hackers right we all know the history right everyone's seen the movie jobs right Steve Jobs and Wozniak those the two two Apple folks are really famous there are two other guys in the garage no one talks about them guess where they came from it came from Xerox the two other scientists that were in the garage with Steve Jobs and Steve Wozniak they came from Xerox Xerox did not approve those two inventors those two technicians plan for a personal computer so those two guys quit Xerox jump on board Steve Jobs Xerox right Xerox Park used to be the engineering Xerox Park still around but it used to be where if you're an engineer that's where you want to go and these two smart right enthusiastic engineers that hey we have this great idea right target instead of mainframes let's target it for right home-based business they said nah it's non-competitive right margin is too low we don't like that but someone else mentioned auto industry this was a shock though this was not a market reaction it was a shock to the system 1979 what happened to gas prices yeah they went through the roof right doubled really overnight because the Iranian embargo with the the whole notion of the was the hostage crisis that we had they closed down which company is this okay yeah I'm not aware of that but there's no notion of why did folks gravitate towards these Japanese companies what was the one feature that they had fuel costs right fuel efficiency right the big three they focused on big engines right law of accelerations big vehicles right that's what Americans wanted so they they catered to the American customer but with the oil shock all of a sudden the criteria that became value right difference of criteria it was not quality initially right we think of Toyota Honda's the Subaru's as being high quality today initially it was not quality it was fuel efficiency as they moved up the curve right as the Honda's right they kept selling more cars as they then they started to compete with the big three on quality as well surpassing them some instances now who's competing against Toyota's and Honda's today from do you think Tesla's more on the red side or more in the blue side I think Tesla's more on the blue side right a lot very expensive love R&D so who's competing with yeah the Hyundai's the Kia's used to be the day was but they swear fellow I remember I told you probably success right 20% 30% majorly bang average but Kia's competing against the Kia and Hyundai they're competing against Toyota the exact same model that's why clean clean and clean Christmas and Joe's bar study this it's amazing right now the tears and the Honda's they've made it to the green line and now we have the upstarts again the Hyundai's and the Kia's it's the same company really it's a subsidiary of one other who's competing with Kia and Hyundai now we're gonna see this again possibly Tata where's Tata being made India where about a Chinese company there's a company called Cherry possibly we don't know right the Tata's of India the cherries of China they're competing on this lower end right the smaller vehicles but again we'll see right again 30% success rates at best I'm thinking about but it's amazing that this this cycle keeps repeating itself and it's not just car industries right it's disc industry right the floppy drive disc industry the the drive space industry it's the same thing Western Digital Seagate all those very similar we bought something more recent anyone still have a blackberry I have I have a blackberry the government gave me a blackberry I told you in efficiencies right government gave me a blackberry they have their own smartphone now but at the height remember when President Obama was using the called the crackberry 21% I think about 21 22% blackberry rim had the share the market share you know what they have now but 1% it's gonna be tough for them to break out of this right what was the feature that the iPhones had the androids was that one feature it was apps but did anyone remember any anyone over 70 using a blackberry what was that that was the feature of the blackberry right what was the feature of the blackberry keyboard right it was the keyboard or the keypad right so we always saw President Obama typing really fast and they'd have races with the guys on a keypad versus Morse and then so really with the keypad so iPhone said yeah let's get rid of the keypad I don't need it I guess who they targeted it was where their feature was simplicity right apps was one so it made it tailorable customizable but the simplicity right we started seeing grandparents we started seeing little kids with I never saw a grandparent with a blackberry not because of the exterior I think it's just because it's just so foreign again a lot of folks using it but again I'm still using blackberry again partially because of government but again I just want to remind yourselves like even if you're looking at it from a timeline again more failures than success right about 23% same thing with revolutionary just want to highlight that more failures but it's okay it's okay if we recognize this in advance let's see okay so how can you best protect a feature I've advocated thinking about disruptive innovations in and revolutionary innovations is taught but we have to really do all four right the reality is I want to be doing all four innovations especially for the government sector the government sector does really need about thinking about all four the problem is I think we think too much about breakthrough the government's fascinating with breakthrough innovations but here's the risk of not think about disruptive right if you if we aren't thinking about disruptive let's take a look at again military history though let's look at the wars where we either didn't lose or lost right you can debate that start with the Korean War jail MacArthur right we actually defeated the North Koreans we pushed the North Koreans German Arthur and you enforce they push the North Koreans all the way back up to the Chinese border right then these pesky right Chinese volunteers right there a million a million man I'm on it wasn't a million-man army at first was at first they thought it was estimates of 20,000 30 200,000 300,000 Chinese cost the board this is about the October November time from 1953 the problem that MacArthur had breakthrough innovations right MacArthur 19 what is now 19 and remember when it started 1950 was 1953 in 1953 June of 1953 okay after World War two what was the big invention for if you're a spy these are on airplanes you to the U2 wasn't quite ready yet but what film photography right so MacArthur relied on what film photography what's the drawback of what film why you need daylight right you need light what film photography right right photography you need light so whether or not the Chinese volunteers figured this out they operate exclusively at night under penalty of being shot right the captains were told to shoot folks who maneuvered during the daytime so the again intelligence officer we call it the two right the two is the intel officer in the military unit MacArthur's J2 consistently underestimated the Chinese military strength by a factor of ten you can mess it up by a factor of three and still get away consistently throughout the war he underestimated Chinese strength by a factor of ten you can't win a ground force if you're underestimating your enemy by a factor of ten that is difficult to do so I'm actually surprised his J2 never got fired what about the Vietnam War again I talked a little bit about this early but this is a different aspect we the US forces bomb the heck out of this thing called the Ho Chi Minh trail we tried to stop the supply lines right it went through the Cambodia through Laos but really this notion of the Ho Chi Minh trail right it was causing us problems right how are these Vietnamese vietcong rebels right bandits how are they getting Soviet weapons systems that could shoot down right our helicopters that could shoot down right can blow away our tanks so we bomb the heck out of the Ho Chi Minh trail what's problem with bombing Ho Chi Minh trail well yeah this notion was the Ho Chi Minh trail we didn't figure out it was not initially dependent on infrastructure no road networks right no rail networks it was dismounted infantry you can bomb the heck out of a force and where the people do when you see a crater you go around it that's what happened initially throughout the war right we there's Ho Chi Minh trail right all these supply lines we know where the the concentration is so we bomb the heck out of it and we thought that would stop it walked around the craters right if you're the V con so that was problem with the Vietnam War a rack of a gas tank major killer for US forces improvised explosive device improvised right it's even in the name it's a disruptive type of innovation right Iowa station Iraq for a year right it's old they call it unexploded ordinance out there they we used to have a placard in our vehicles to call the EOD units right hurtlocker those explosive ordinance detachments you call them in they'll get rid of them there were so many in the area where we were at right it didn't make sense to do that I don't know why we even had that in our vehicles like this makes no sense right government efficiency yeah this placard tells us do that they're all over the place and so initially a lot of the EODs are made from remnants of the Iran Iraq war right just 60 million more tubes right the duds right essentially dudded but they picked up maybe it might still explode and that's where it was did they get better yeah over time right they used copper plates they use other things but then I started using right the track triggers right so the cell phones they got better over time but they're initially a lot of passive IEDs right command detonated right you stepped on it blew up a lot of stuff from Vietnam again we're about cyber warfare my contention is that nearly all a newly discovered malware very first time comes from the disruptive realm right you might debate me on that but that's my contention that's why I'm telling these generals right most malware discovered for the first time originates in this disruptive space right so we need to be thinking about this disruptive revolutionary space okay so I say okay it originates right most malware originates in this revolutionary space atop left quadrant but again there are sustaining evolutionary right version 2.1 version 2.0 goes on again there's not a whole lot of breakthrough malware right there's not a need for it okay sure there's some right Stuxnet I'm thinking right we don't know who's supposedly released it I can't tell you right I can't confirm or deny who released it but if you watch the movie Zero Days right intimated that's your Israel's and you must see I have to make that that citation it's not from the US government citations from the movie Zero Days but if you look at cyber defense here's the problem with cyber defense yeah whole lot of breakthrough evolutionary sustaining not a whole lot of revolutionary types of innovation for defense except from this room that's why I love coming this convention I'm hearing about disruptive innovations for disruptive defense from this room sect devops right that is disruptive you're using tools that you have available think MacGyver just a new way of putting your people together to get more security right that's you're not training them any better you're not using newfangled technology you're just putting people together to talk to have after mine right this notion of tabletop exercises for cyber defense just bring people together it's amazing when you put people together right things happen you talk communicating you find out what the weaknesses are but just by communicating together tabletop to me that's disruptive innovation doesn't take a whole lot of AI machine learning right that takes a lot of time so when I see this as a military intelligence officer I'm seeing a gap there's a gap here what there is in cyber offense and what there is for cyber defense right this whole notion of we're not doing enough really on defense not enough revolutionary types of focus for cyber defense now so if you agree with me on that premise we're not doing enough cyber defense revolutionary cyber defense right I just want to reiterate that again it's tough to do is okay it's tough to do that's why our military just needs to know that beforehand that's tough to break through innovations right success rates probably why it's same so you might as well do them both right independent now if you really do think you have a way to do disruptive defense right this is what I recommend you want to count it early that's where a little late for that now right though with a model malware and really the vulnerabilities that we have with our system the last the last point there is being an early adopter of product experimentation is probably the best way of doing disruptive or revolutionary types of innovation so if you want to encourage it this is what you want to do experiment right this we learn about experimentation in in grade school right experiment drop down the results figure out your control group figure out your experiment right figure it out and it's got to be cheap though make it fast and cheap and that's my last point here attributes of successful patient if the expensive experimentation you generally don't learn from it NASA is trying to expensive experimentation I don't think NASA is gonna succeed very well if I were advising them I tell them go cheaper you need cheap experiments you learn better even failure and here's the problem if you're not doing any experimentation if we only focus on breakthrough and this notion of evolution is staying invasions the problem is if we don't do revolutionary we don't do this experiment right we lost this opportunity to experiment and that's problem and the other problem is breakthroughs expensive same success rates right 30% at best but we lose this notion of experimentation that's what I think it might be happening or our military right for our military we often say that if you heard journal Paul's talk right he said we don't have a zero defects army anymore right he told you about his failures at second lieutenant right he lost the weapon during one of these field exercises his company commander tuned him out but they eventually found the weapon that's his notion of yeah we're no longer zero defects military I would agree with that the problem is I want to be better than a zero defects and no zero defects army I want to be an experimenting army right I want to be an army that that's doing all this stuff in red big thanks to Atari and the torque on crew for inviting me here having a Fed speak is for me it's a it's I enjoyed giving the talks hopefully my audience is primarily to be military leaders but hopefully this framework also gives you this notion of the experimentation that you're doing as hackers I encourage it because as Lance mentioned right we're trying to get more hackers into cyber defense even for the military right much much Zacko worked a lot it's amazing I work in the Army Cybers to again I told you I was a newbie we they always have these guest lectures right what was introduced to me in the email was oh we have this famous hacker coming to give a talk to us come everyone come down in the conference room and here okay so I learned okay it's much Zacko I have no idea who he is it's not till afterwards right okay I with the research you much Zacko was holy smokes he is not a famous hack he's the hacker right the loft so he's the guy who breaks in the NSA for the first time so it's yeah it's for me it's exciting working in this space I'm new to it I am very thankful for being in conference like these because my benefit is not from giving a talk almost every conference I go to I find at least at least one speaker I really want to write a paper with because the ideas that they bring out it's it blows my mind right because it's nothing the military is talking about it's the first time I'm hearing about it I'm just starting to go through a lot of these sans courses so for me a lot of terminology is all new so I'm a neophyte and admit in neophyte but again look at just some of these talks right stuff that happened today a lot of this is disruptive revolutionary innovations right this afternoon again some of its sustaining evolutionary some might be breakthrough but most of this stuff is in the red red red box right that's why I'm encouraging that's why I'm encouraging our military leaders start thinking a leverage the expertise leverage disruptive thoughts leverage the revolutionary right leverage the revolution right the revolution is with the right I'm gonna I'm gonna use that when Lance Lance is said hashtag is the pound I'm using the pound sign now as a revolutionary sign and with that I only have a minute left and so why don't I disconnect and let the next speaker set up but if you have a question I can feel that while while we're we're doing that transition here doesn't have it anyone have any questions I'll I'll be here for the rest of the conference I'll be here rest today and tomorrow trying to gain again more of the wealth of knowledge on disruptive innovations taking place inside the defense the great thing again I didn't mention was another thing that I've learned from from ten of these conferences that right Lance mentioned this right we hackers not only try to break things but we also try to give back to community what I've learned is that from these talks that here not only do the hackers tell me how they broke it right how they got the NSA how they got in target things like that but almost always they'll always come at the very end okay if target or company X I've done this one little thing how we've gone to a different company we've gone to different site when I hear that right that's what sparks me right they've done this little thing right everyone here any deviant olams talks physical pentesting right deviant olam I show his video on day one of my lessons now when I when I give my classes because to me cyber security was too abstract so I play deviant olams physical pentesting right deviant olam not only tells you oh he laughs right I broken this door so easily I broke this L there so easily but he gives you the 10-second clip that says if he had done this one thing he had gone to Lowe's or Home Depot and bought these champions for 50 cents I wouldn't have taken this door apart right so I didn't hear that that to me is revolutionary innovation right pound sign revolutionary innovation and that's why that's why I love coming to this conference and gaining your insights as actors and and hopefully now that you know this framework you guys are doing you guys and gals right you guys and gals are doing disruptive innovation you guys are doing revolutionary invasion hopefully it's for the defensive side but even when you expose that the weakness on the offensive side you give me that little trick that little twist that tells me okay if you've done this I would've gone somewhere else and that's why I like hearing and that's why I want to promote our military right more of this little tricks that they do this our defense is much better so thank you very much