 Thank you. Thank you for having me. It's funny. There's not as many people Here in the afternoon as there were in the morning As a professor, I'm used to that and so what I'm going to do is I'm going to send around a sheet of paper And you'll sign it and you'll get extra credit for being here Okay, so it's true. I'm I'm 1988 class of from Navy I Did fly the a4 and the f18 so I actually know your backgrounds More than you would think my dad was a Marine happy Marine Corps birthday But I had to look it up today. I'm like it is not November 7th. It is November 10th. So quit trying to fool me Okay, so I'm going to talk to you. I'm going to talk to you today about The autonomous systems the human role and we're going to touch on both some very technical and and what I would call socio-technical policy issues so when I tell you what I do it's something called human supervisory control and It's where you have a human operator in this case a UAV operator and sometimes I slip and use the word drone That's because that's what the rest of the world calls it and we can we cause them and we can have an argument about that later But you've got a UAV operator here. I don't know if this is there We go sort of working and then trying to fly in this case a shadow But it's mediated by computer in the middle all drones are flown this way All commercial aircraft are flown this way Most military aircraft are flown this way Cars are starting to be driven this way when we've got driverless cars coming. So this kind of human on the loop is really the wave of the Future when you really talk about all transportation technologies if you really want to see Another lecture that I give that's probably More interesting in terms of where you see UAVs in the rest of the world You can go on YouTube and you can see where I talk about everywhere else in the world You see UAVs and related technology so but I want to give you a couple of definitions for today's purposes And that is what you hear the word autonomous bandied about a lot. What does that really mean? so first you need to understand what an automatic system is an Automatic system is one that acts according to a pre-programmed script with defined entry and exit conditions This is basically a very specific. It's called a deterministic computer algorithm It's a set of if-then-else rules your thermostat in your house is automatic Nuclear power plants are automatic Lots of things that in jets are automatic right now. That's not the same as autonomous an autonomous system It's really something with a bunch of probabilities in it that it means that you have a system that is independently and dynamically Determining if when and how to execute a task these are stochastic systems That's how you get a PhD is that you both know how to spell and use stochastic in a sentence Okay now and I'm only partially kidding So later at a bar someday. I'll give you the list of words that make you a true academic You'll hear them here. Epistemology is one that if you haven't heard it yet. They'll sneak it in on you So this is basically a set of probabilistic algorithms instead of an if-then-else It's basically saying on this probabilistic continuum. How likely is this set of conditions given some Bayesian? Some prior beliefs you might hear that a lot in these systems and we're gonna make a decision It's kind of a guess. It's a computer's best guess But computers guess autonomous systems are guessing machines So and you know what we hope is that they're good guessers But you have to understand that when you hear the word autonomy in your head the very first word you should think is guess It's a guessing machine and by the way, you are an autonomous system You are a guessing machine particularly in the military when you go into battle the fog of war You are actually given a lot of autonomy and you have to make your best guess going into certain circumstances And it's a continuum. It's not one system is not 100% automatic or 100% autonomous The matrix go watch that. That's autonomous. That's a fully autonomous system. Okay, anything other than that is partially autonomous And sometimes I see in the and in fact, you know, the military all UAVs in operation today are automatic They are not autonomous Maybe the predator and the gray go have Tiny bit of autonomy in them when they do the automatic takeoff and landing tiny bits But for the most part they're automatic systems the x-47 some new Drones maybe coming out of DARPA, you know, they're gonna be more autonomous But I can promise you you haven't seen anything yet in terms of what's coming out in terms of autonomy now, but one thing that I see that gets confused a lot in and around the military is Confusing an autonomous platform with an autonomous weapon Or an automatic platform with an automated weapon, right? The Patriot is really an automated weapon All right, but that's not a platform. Okay, the platform is the vehicle I and I know I'm lecturing to the preacher the choir But the platform is actually in most cases divorced from the weapon that it can fire so you can put a Potential autonomous weapon that can go out and decide who it wants to kill on its own That isn't necessarily have to go on a UAV. It could go on a UGV It could go on a UUV it could go on, you know, it could on on any vehicle a ground robot that That any propulsion device is a platform that could launch a weapon Which but what may or may not be autonomous and that's important because there's a huge debate internationally And I and I speak around the world on this about you cannot necessarily ban all UAVs because they can carry Autonomous weapons. I mean that's it's throwing the baby out with the bath water if you want to address autonomy of weapons That's actually separate from the technology, but let's talk about why we would have autonomous systems There's based three basic reasons cough safe cost safety in performance cost Do we have any Air Force people here? Good I really love picking on the Air Force. I'm sorry I just spent all those years being brainwashed by the Navy and it really pays off later When we talk about UAVs This is a great example of You know that I'm gonna pick a lot on the Air Force right now. I'm sorry for people in the Air Force But let me let me just pick on the Navy a little bit at least the Air Force has a UAV program that they can be picked on Right that the Navy has failed miserably in getting a real program of record for a UAV So that there's a caveat and so it makes some Air Force people happy. Okay, so we've got a U2 and a global Hawk It's only recently about ten years too late did Congress actually decide to enforce upon the Air Force that they need to get rid of the U2 and Use the global Hawk It is so much cheaper to fly the long surveillance the 24-7 surveillance missions using a global Hawk If you've not not ever been to Korea and I did a lot when I was in the Navy a lot of operations with the Air Force Adam Westpac Those guys, you know, I'm a pilot who here's a pilot Who has ever worn those full-length diapers as a pilot? I Saw one and a half one guys like I'm not gonna admit that These things are miserable to wear They're really they're a diaper they go from basically here down to your feet And that's how you go to the bathroom if you're a U2 pilot you basically sit in your urine for however many hours that I mean This is I don't mean to be gross But you know any time that you're sitting in your own urine for that many hours that might be a good time to automate something I'm just saying that's a good That's a good basic rule Okay, but it took the Air Force for ever and you know, I mean I and I lived it right I mean I my squadron got kicked out of Kadena because we were always fighting with the Air Force guys at the club I mean I live that life. I know how what it's like to be an Air Force pilot You really are at the top of the heap and it's in terms of promotion and who's running the Air Force Nobody wants to let a nerdy guy with no real flight experience run the Air Force Nobody wants that they want you know the fighter guy and before the fighter guys by the way it would be 52 guys, right? So it represents for the Air Force a real Psychological stepping point for them to get the man out of the cockpit But by the way General Hap Arnold who is actually one of my favorite Flag officers ever what he was an Army and then an Air Force officer He predicted that they would have unmanned aircraft 60 years ago 60 70 years ago, so he was a real visionary But let's just see you know in a more practical concern for you pilots out there So how much does it cost to have a f-15 or f-16 pilot now run your predator missions? That's that's the word tx. It means transition pilot. It actually cost the Air Force $2,100 an hour to have that guy learning how to fly a predator SUPT is the Air Force word for just out of flight school So if you just go out of baby flight school It only costs the Air Force $500 an hour to do that mission And then they have a new program called the beta program where they just basically go get anybody off the street Who has a college degree to go fly that and it turns out that they're all about equivalent and capability The beta guys are only $150 an hour now We can't put a price on war. Can we? Yes, we can and we have to right. I mean, this is where the government is right now We've got to become more cost-conscious and it makes absolutely no sense if you can basically get effective performance Capabilities to to be taking f-15 and f-16 guys and I can promise you I've sat on a lot of predator trailers Those guys hate it anybody here predator pilot. I'm not gonna tell you how I'm not gonna make you tell me how much you Hate it because we have a great quote from an interview that got the guy says it is just like sheer mind-numbing boring Okay, safety all right, so this is what the Accident rates look like for all aircraft in the United States all the way from commercial aircraft, which is very low thankfully Out to mail hail UAVs. This is medium altitude high altitude UAVs predators global Hawks. This is really the only Significant data set that we have to make this comparison and this is basically we have a full data set from 1990 to 2011 so and then you'll see somewhere in there, you know Military cargo for military flights military cargo is the safest surveillance bomber fighter attack Which are almost equivalent in rotorcraft general aviation. That's actually a big one general aviation unsafe Those people should be turned into drones as soon as possible. Well, let's look at that So but that this graph right here is actually not that informative That's actually like asking what the United States commercial aviation record was like in the 1930s, you know I mean, it's a it's a new industry. It's new technology. So that's not really a fair comparison What if we look at the accident rates in the last couple years? You have to remember that UAVs are basically a self-regulated group in the military Because they're actually the military puts no safety criteria on them per se because they're weapons of war We don't really have to we want to push the envelope that being said the industry has self-regulated in the last two years Something really important has happened. Hello Air Force that in fact In the last couple years the rate you can see it right here The rate has dropped down to below five and by the way, this is measured per accidents per 100,000 flight hours meaning death or over a million dollars of damage What happened here something really big it dropped almost half and not only that what are we safer than now? It is actually safer today for the Air Force to send a UAV on a fighter bomber mission than to send a manned aircraft This is huge. This is huge and it was buried by the Air Force, right? For obvious reasons if that is the case if we should be sending drones out on these missions Then what does that make everybody in the Air Force, right? So I predict that you will see this this is definitely going to this trend is going to go down In fact, if you actually do the linear extrapolation, this means that in 10 years commercial aircraft commercial Planes would be as safe as drones now. I'm not saying that that's necessarily going to happen That's a whole nother discussion about whether or not we can ever truly have a passenger airplane with drunk passengers in a drone I don't think so But but you can also see another big drop another not change was general aviation So that that's a good indicator to you that general aviation is a good metric to see how one safety recorders maintained But as technology becomes more and more innovative we see huge improvements and of course mission objectives, I mean in the end that's kind of why we have these things do we get better mission performance and You know, there's no question that the Global Hawk has revolutionized the way that we can get really surveillance anytime anywhere The and these aircraft are highly automated and even the x-47, you know as much as I'm not crazy about nav air and their inability to actually field a program. I mean they are make it progress and And but it's long and coming, you know, I I'll give it Admiral close yours. I'll give it to the Navy They did they did a great media spectacular about showing that the x-47 can land and take off by itself There's a big media deal by the way, I flew the A4 in the Hornet Guess what's been taken off and landing itself on an aircraft carrier for a long time the Hornet You're not even allowed to touch the controls on the takeoff So, you know, I I will give it to the Navy because they won up to Air Force in Technology, but that's a good story to tell you that the technology's actually been there for a long time We've been since the 90s We've been taking planes off and landing them by themselves on aircraft carriers, right? So so we are actually a little bit behind the curve in my futuristic view as a professor of where we should be But the real Value of these systems is actually not not necessarily that they can take off and land by themselves Is that they basically collapse the kill chain immensely right now? We have vehicles that can not only they can find they can fix they can track They can target they can engage and they can assess this same vehicle and put no human at risk, right? So there's no question that this has been a major game changer I think what what we've really struggled with as a joint service, although the army is actually there They're pretty impressive because they don't have a lot of legacy history to get to get past Still even at that our operations have been revolutionized by the appearance of UAVs now So Lockheed Martin and Northrop Ground did not pay me to get up here and tell you that I mean that's you know I I work a lot with military all three brand up for it depends on how you account them of the service And so I've seen it across the board what drones can do But now let's talk about some problems because I think as they're all automatic remember that we talked about now But as we start to move towards autonomy you could imagine a case where? if we can actually there have been a few cases for example where the global Hawk has lost calm and What happens is it has a set of if-then-else algorithms that it that it defers to to take itself in for an automated landing And we've never lost one and it's actually happened several times that it's lost calm And so it's it's very the systems are very good under some very Straightforward conditions now nobody was shooting at it. I know that the a2d world is big This is actually something that you hear a lot about coming out of the Pentagon now And so we do need to make some substantial changes in our control systems meaning we better up the autonomy if we're gonna work in those worlds But here's some issues. Yes Donna Rumsfeld is right. There are some unknowns This is one of my favorites so you guys aren't nerdy like me So you're not really following the the academic high-tech Silicon Valley literature, but about a year ago This researcher from Stanford was made world famous because he basically developed this artificially intelligent algorithm that could search YouTube videos for cats and so in his video and in his algorithm could recognize a cat in a In just a random YouTube video this and this was all over the media and in my world You know this guy's it's it's effectively like MacArthur coming back for more or two. Yay. This is amazing And I was like well, I don't know about that because I do the same research I'm like that seems awfully suspicious to me because theoretically if you have some algorithms that can watch videos and pick out a cat I mean you could make make the leap to what DARPA would want to do with that So you actually if you actually read the paper and this is actually that that four letter word It's almost like a missing art, you know read it in Details slowly not in a text message What you'll find out is the algorithm worked 16% of the time 1 6% So 16% of the time it was able to recognize a cat in a YouTube video. You're better off flipping a coin And I'll give it to the authors They did not actually over claim it in the paper so much But what what they said was that they made some like 70% Improvement over the state of the art which was some other obscure algorithm, which I will not bore you with right So yes, they made a huge improvement over with the algorithms that people were using before But it was still only correct 16% of the time This is absolutely nothing that we would want to put even anywhere near a battlefield, right? But to hear the media and in fact lots of people in the Pentagon heard Vision recognition is awesome and the Pentagon starts paying out people to do these to do more research And I'm not saying they shouldn't do the research But I start to object in when we start to think that we can operationally field a system And that's kind of what you're seeing over here. So this was a company called Exponent They were working on the small UGV unmanned ground vehicle and maybe a little hard to pick up it right there You've got two twin barreled shotguns. This thing is actually obviously in the field. They were in a village nearby From what you're seeing here. They let it go. It actually has Basically an autonomous feature on it where it can be enabled to use its vision sensors and to basically Determine that it's maybe being attacked and to fire at will so that is a functionality programmed into it The army guys do not trust this at all It's never been in their minds has not been proven to work effectively They send it into this village and remember they're remote and they actually can't see anything because the cameras were covered up They could tell it wasn't moving and then eventually they just said okay Let's go into the village and find out what happened to our robot And this is the scene that they came upon which is so as a mother of a seven-year-old so so so cute Would have been so so so bad had that thing been left on automatic fire and because I can tell you right now Robots do not know children from adults At this point in time and so this could have ended up really bad All right, the X factor Let's go ahead and if we can get that first link Clicked I'm gonna show you Just so you really can get an appreciation for where we are in the state of robotics today So it's something called the X factor that I like it's actually not the cool factor It's actually how fast roboticists are speeding up videos so This is some new humanoid brand new humanoid research. This is a darker project anybody anybody catch that I mean they didn't put it in two small letters What it this is a rope robot climbing stairs by the way really really it's like that. It's like the holy grail of robots All right, anybody so how fast was that is that real time? Four times, okay, let's just and we'll just don't don't change it. Okay. All right now Let's see. Let's play that back the next one, please Let's play that back in real time Okay, so if you're a real robot if you're a real soldier going into battle with this humanoid robot Let's see what that really looks like Same thing Yeah Awesome, I am totally going into battle with that guy And let me tell you okay So let me tell you some other robotics trickery that you don't know to look for right because this isn't what you do for a living Anybody know what this is right here? Yeah, it's a power source. I got news for you. We cannot go into battle with any robots until we've got a much better battery That right now the battery that would run this would basically be twice the weight of the robot So we we have no power source actually to do this Anybody see anything else? It seems kind of off base. Look look at the top. What's the top? What's what do you got? What do we go on at the top? It's actually a couple of ropes to help stabilize because we don't have something called proprioception figured out It proprioception is the way that your brain and your vestibular system kind of know where your arms and legs are at any given time You know so when I do this, I'm not even thinking about how to balance. I just do Robots cannot do that. There's also one little other thing. That's kind of hard to notice Okay, you can sort of well, they're not the guy clean, okay And now you can sort of see the guy that just came into view right there on the laptop He's at the robots climbing is actually being He's basically being tuned by the guy on the road on the computer in the background Okay, now I'm not doing this to pick on Drexel because Drexel's doing a great job. This is research This is six one research But if I wasn't here to tell you to point out all of those things to you It is very likely that you would have missed it and in your mind and in many flag officers mind Admiral, please close your They see this and they're like awesome. We are totally gonna fund this Let's give that company billion dollars and we are gonna have robots going into war You know in a couple of years, right? And we're in and and we'll come back to this issue because it's a it's a real problem And by the way, if you there's the DARPA does these competitions all the time. We call them paint drying That definitely go because that's really boring All right, so And I like to use this Cartoon because it kind of shows you the mindset of a typical roboticist, which is You know, we're gonna we think that we can design the human out But we'll leave the human around just to intervene in case something goes wrong, right? And I would tell you the whole reason I went back to and got my PhD and started a research lab is To get away from this model and to go towards the human and automation autonomy collaborative model Let's work jointly together because instead of trying to replace humans. We need to recognize that there's a place for humans now There's some other issues at this ethics conference. I thought it would be worthwhile to bring up You know, there are some issues with autonomy and what it does to our sense of ethics or perhaps a lack thereof So there is a huge International push pretty much led by the Human Rights Watch and this is what I've put up here This is their most recent report called losing humanity I mean they are really really I mean you just say the word robot forget Rosie the robot on the Jetsons Which I totally want one by the way and we're not gonna we're not gonna see that in my lifetime They are really anti-robot anything robot, which is unfortunate because when you get this kind of passion about anti-technology It's very Luddite a lot gets lost in translation That being said I actually do think that when you look at the core of the issues that the human rights watch people And other groups like the human watch the International Committee for the Red Cross There are some really core issues that we do have to think about right and so I think it's important to separate the passion and a lot of the You know just over the top statements about and of course you get videos like The Russian YouTube video of the UAV firing a machine gun That was not real that was faked right but again people don't get that some things are real and some things are faked So this actually came from a retired Air Army this graph here on the right Army officer named Dave Grossman and he's been doing research on his lab is called the Killology lab That's pretty cool And he's actually been doing research on what it takes for one human being to kill another human being for many many years And he came up with this idea that you know you've got you know hand-to-hand combat killing You know at the physical distance from the target all the way out to max range and bomber artillery And this is actually before drones UAVs came on the market Which you could actually park out here somewhere now and the whole idea is that it actually the resistance killing goes up I mean The closer you are to somebody the less you're likely to kill them And he's gone back through all kinds of wars all kinds of people there was not nearly as much bayonet killing as we ever thought There were of course as a former Naval Academy midshipmen who had to run around those drill rifles kind of resentful about that Those bayonets But but it's much easier for us to kill with bombers Drones possibly and and I do get asked this question a lot I mean I do a lot of interviews and that's the first thing that people say is like but drones make it so much easier for us to kill Yes, maybe but so did the longbow right? I mean this is this is part of war We've been backing up that distance Since the Middle Ages right and so yes It is true that we do want to back up the distance But it is possible that that distance actually it causes something that I call the moral buffer. It actually helps us maybe give up a moral argument for the legitimacy of All the technology that's in the middle and to kind of show you an example of something that I ran across in my PhD research This is an actual and this is approved by the Navy Navy gave me this to show This is actually a strike generation tool circa mid-90s where You know, they're putting together a tomahawk strike and they're putting salvos together and this little cute happy dog says Done now you may select the salvo review it and generate an indigo mission That basically says I'm about to launch all these missiles and kill a bunch of people And that because I was doing some tomahawk research for my dissertation and at first when I saw this I was grossly offended. I mean I had just walked out of fighters I'm like could we put a growl on that dog? Could we put like a studded collar come on? This is war when why why do we do this and then if you actually dig in the literature, you know It wasn't a military person who designed this by the way It was a civilian support contractor who you know found that tool set inside Microsoft and just feels good about it And likes it right? I mean and this is just one small example of like wow, it's just so much easier to take Oh, yeah, this is so cute. I'm not oh, this is just a game You know, it's not really it's not really in a serious warfare and I think that when we start to extrapolate Yeah, you know, I mean I've seen the videos. I've sat in the trailers. You know, I've flown the planes It is easier to push a button when you're further away Okay, it is a lot easier when you're not under stress that being said It's also a lot easier to not make a wrong decision And when I tell people about yes, it is possible that drones because of the distance They might make us it they might make pushing that button easier But what has fundamentally changed is this whole idea of warfare by committee I never had a lawyer in the loop in any of my missions and a lawyer sitting inside the kill chain is very common today Right and so in a way technology has opened up new doors Possibly but it's actually made some doors It's made harder to go through and and it is true that it's Depersonalizing, you know from a f-18 five miles over target. You mean you're in you're out. You're not you're not there You're not emotionally invested. I've worked with a lot of guys who are Overhead a village for months. They learn the people they learn their spouses They learn their mistresses they learn their kids their dogs You know, they're and that's what they call the patterns of life And then they have to do a mission that potentially could end up killing some of those people, right? And so in that way it actually is a lot harder than when I was there And so these are not any examples to say one form of warfare is necessarily better or worse than the other It's just different and we need to recognize that and this is you know The kind of setting where you're here at the War College to do that is to start thinking through these issues How is it different and how can we maybe think about? Doing something about that in the future to make it safer and better Now I'm gonna touch on something that's kind of Or you you'll think it's orthogonal to the whole message today But when I come back to it, you're gonna start to get how this is a huge game changer Drone the the commercialization of drones. So Amazon made this big announcement last year almost a year ago that they were going to start doing package delivery Huge debate as to whether or not this was a publicity stunt. I can promise you it wasn't There's lots of research being done and development Amazon is in fact actually one of my students is at Amazon today Coaching them because we're actually trying to work on an air traffic control system for the future for how we're gonna do this But they weren't the first they not by a long shot There was a company called Zucall in Australia that six eight months before Amazon even thought about making this announcement They made their announcement and they actually have a regulatory infrastructure in Australia to make textbook delivery by drones a lot easier And so, you know, by the way, the United States were never first to anything You know, we do have an impressive UAV inventory, but we really just copy from Israel And you know in this case right here. Yeah, yeah, not the first but but we're the first to commercialize it Maybe probably not actually And then something actually curiously happened a few months later, and this isn't that long ago that really made me do a double-take And that is the big dog anybody ever seen that thing on their YouTube videos pretty cool I mean, I'm a big I'm a big outdoorsy person. I like to do backpacking I would so so so love to have a big dog with me to take all my crap, right? I mean, I just can't wait to see and so you can see why the military has been sponsoring all this research But not anymore who makes who makes the big dog Anybody know Not far from here Boston Dynamics Who's who owns Boston Dynamics? It used to be Boston Dynamics, but it does not anymore. Who owns it? Google at the same time Google bought seven other top robotics companies in the world In one day Google bought up the eight best robotics companies in the world And You know a month later. I was actually briefing some people at OSD I'm like, you know, it tells me you people are out of touch If you the day that happened the fact that you didn't hold an emergency meeting that day to find out Let's let's get our arms around this what's happening, and you know, this is a senior person I won't tell you his name at OSD said we're thinking about scheduling a meeting That's it ship has already sailed because what you don't know about the commercial world And it was hard for me when I did the military to civilian transition You know, I mean you get into this world where you think the military technology. That's where all the money is That's why we have these companies. I got news for you Nobody wants to make military things anymore Because the money is not nearly as good as it is in the commercial world. So Google Did not buy Boston Dynamics to make money off of military contracts That money just simply isn't there at the at the levels that they want it there They bought that Boston Dynamics to make commercial robots that they can sell worldwide because that's where the real market is And I gave this talk a very similar talk to a group of people and I did not know a senior VP at Google was in the audience And I said and here's what Google. I said I do not have any insider knowledge But here's what I bet Google is gonna do Google is going to honor whatever military contracts It has and it's gonna let them run out and it's never gonna do any more military research And he came up to me afterwards and said you are right about that Do I was good because I work with all these robotics companies, right? And I think it is hard for us to think about it's hard for the Pentagon to get their head around it that not everybody wants to Play with them. Not everybody thinks it's cool being part of the US defense machine And in fact what we're gonna talk about even more is it's becoming less and less cool. I'm sorry to diss everyone in the room I don't mean to I don't want you to think that you know I serve my country serving your country is great But from an economic perspective, it's just not the way it used to be You know I was back in the 80s. We had a rush at the Russians were coming across the border. We had a real enemy We take another problem that the government has It's a very common problem particularly with bureaucratic agencies We love things We love hardware We love in our government to fund Things that go Weapons things are really fast Things that make cool noises. That's what we love to fund things that make really good movies You know, and I'm only half kidding here because I actually worked for on our office of naval research as an IPA which is where that basically the Navy rented me from MIT for a couple of years to help them run a robotic helicopter Program and boy did it was a great experience because I really learned a lot I really learned that we've been spending a lot of money in the Navy on something called a rail gun Which is an electromagnetic gun? Which most flag officers cover his ears love they love the electromagnetic gun Most of them are not physicists and don't realize that the size of a ship that it would take to power the rail gun that we think We want would be the size of Bermuda You know, so it's it's a pretty big power source. Oh and our says we're working on it We're working we were researching those issues, right? But we have this country has funded and I poke fun at the rail gun But let me let me bring the Air Force and let me do some equal opportunity poking by the way the Air Force They're equivalent to this is anything hypersonic the Air Force continues to pour Unbelievable amounts of money into hypersonics. I got news for you We know pretty much everything we need to know in terms of the big-ticket items for hypersonics pointy nose make it super pointy uh put a really big engine on it and You know, there's certain altitude regimes that you can work in and it's also really super expensive to fly at hypersonic speeds conquered You know had had its share of troubles But we keep spending money in there because it's really cool and I work at right Pat by the way I and the Air Force funds my research and they are so in love with hypersonic research Okay, now let me just say you're just like so why is that a bad thing? Yes, could we use faster things? Maybe Could we use really cool? Electromagnetic rail guns? Yeah, I mean it's still pretty darn futuristic We're gonna have to solve some serious hard equations to get that to be really operational But what we're doing is we're putting our eggs in the wrong basket So the government pours money into the hardware because it's so cool Consequently, they they do not fund software development and guess what powers autonomous systems Software it's almost all software. There is some hardware underlying it You know, so I'm not saying we should just ditch all hardware research But we need to we need to seriously rebalance our research portfolio because we're not doing enough software development But let me tell you who is All right, so just so if you think that I'm just up here spouting off. This is some public You can go get this off the internet this is a Snapshot from 20 to 2012 where we have the most recent data about corporate R&D intensity So this is how much companies are spending as part of their overall profit And they're putting that money back into research to figure out where they think the next big leaps are So this is the defense industry up here There's UTC aerospace so they basically have an 8% corporate R&D intensity and their average annual growth rate is up here So pretty healthy and it's funny I work with a lot of these companies in this graph and I would actually say my sense is that's correct you oh Thank you. Thank you. Oh my god. This thing is huge. Okay. Oh It's like a lightsaber All right, so so yes, I think UTC UTC aerospace doing pretty good for defense companies Boeing also work with Boeing at least I did before they ran out of money And this is reflective of my experience of Boeing's in the negative. This is bad This is bad Boeing is huge not just for military but for commercial Research commercial air research negative bad bad Raytheon's on its way down bad G bad You know, so this is this is bad and not only okay, so now let's jump over here Well, okay, these are the software companies in the world Google Intel probably a bunch of companies. You've never even heard of Nokia not surprisingly recently absorbed into Microsoft, right? And so now Microsoft is probably even worse probably Microsoft is down here now. Why this is important to you to recognize is yes in fact The defense industry is not holding up with the software industry And in fact, if you superimpose the defense industry, I mean This is this is sad I mean our best person our best company out there doing the most innovative research is We're not even close to what Google is doing Google could start its own military any day now Google bought Titan Aerospace Google is gonna have surveillance capability from a drone that exceeds anything that the CIA can do in just two to five Years you mark my word and you know why that's because my students are not going to work for Boeing My students are not going to work for Northrop Grumman. My students are not going to work for the US government That's not even that's not even on their radar. My students are going to work for Google My students are going to work for Oracle the top brains There is a massive brain drain right now to Silicon Valley and to other places like Silicon Valley commercial The brain drain is not going into the defense industry Which is it's it is just to me phenomenal when I in the 80s 90s You know when I was just getting out of college working for Boeing working for lucky Martin working for Lawrence Livermore This was a man. This is like the dream come true. I'm working for a national lab. I'm working for Boeing Now when you talk to students, they're like, yeah, I guess I've worked for Boeing if I didn't have any other options It's just a whole different ballgame out there and and where's this ballgame being driven? It's being driven at the software level and so now I hope that puts it in the context about if the Navy and the Air Force Keep investing all these resources into there. They're basically investing their resources here And the losers they're not investing their resources in the software development Which is where the game is going to be played which is why Google and or at Amazon I mean Google and Amazon they just started their own lobby for drones. Does anybody I mean when I read that I'm like this is a whole new game because the FAA has not been allowing any commercialization and when Google and Amazon team together. These are again resources that put any nation to shame. It's good games on fights on Okay, my last slide and then we'll take some questions So I know it kind of sounds my message sounds kind of negative and I don't want it to be negative I really I'm here to adjust your perspective About robotics where robotics sits in the world where Militaries do not sit, you know what do and do not sit in the robotics playing field for a long time The military did have all the innovation so not just our military is Israel other countries were big innovators and robotics There's been a distinct shift that distinct shift has gone over to the commercial world and that's going to be a big game changer But I will say one of the things and I talk about the perception showstopper and there's basically two Elements to that when we say perception and robotics. We actually mean How did we make a robot see or sense the world around them? So, you know the really slow climbing stair climbing robot. I showed you what the big problem There's perception. We actually having a robot see the world. It's still the holy grail. So And I love this this this robot here is from DARPA. It's called Atlas. It's supposed to be Does anybody know what this one what they bill this as this is the robot that's going to come save you on the battlefield I'm like the American public sees this and it's like terminator all over again, you know, I'm having this thing pick me up And But what's funny about this scary looking life-saving robot are the technologies in this robot are actually Almost virtually the same technologies that are in driverless cars So driverless cars are actually just as scary from from my perspective from a technologist perspective These two technologies are equally scary because they both cannot sense the world very well around them But if you go out and pull the American public and you say which of these two robots would you like to have? Oh? Absolutely the car in fact I'll tell you that car is pretty darn lethal right now Because we just we're not quite at the level that we would need to be for it to be truly driving you around with With nobody behind the steering wheel. We'll get there, but that day is not today But what I find interesting about this It's a framing bias and that's worth remembering when you're talking about robotic technologies, you know You put that cute you put a cute You know electric car Package around a robot people like it you make it look like DARPA the DARPA scary robot people don't like that Right, but ultimately effectively. It's the same technology And we have to separate and part of that is we need separating the platform from the mission We've got to make sure that people understand that The vehicle itself is not the same thing as that weapon that we're firing I find that most people's hatred of drones and UAVs is really driven by CIA policies of targeted killing They actually the military does for the most part a great job of overseeing those missions And I just sat on a very high-level committee with general John Abazzad who and we recommended that In fact, the CIA be forced to give up drone operations and have it turned over to the military because we think the military operates UAVs in a very responsible way and the CIA is it's not that they're irresponsible But nobody knows because nobody has any insight into their processes and and because the targeted killing Program that this unit that the United States has is so opaque. It's very difficult for anybody to defend or You know to even try to explain what our process is there And that brings me to the DOD three thousand point oh nine Which says that right now in this country? We will not allow a robot to fire at will to kill Offensively we have to kind of make that caveat because in fact the Patriot There's some other weapon systems that potentially defensively fire and everybody seems okay with that The three thousand point oh nine is very interesting because it's a big step It was a great step that the US government took there in there In fact even by today they're the we are the only country in the world who's actually put it in writing That we will not let a robot shoot to kill offensively But of course if you read it anybody read it read it closely it says But there is a loophole There are basically these four people who if they say it's okay Then it's okay, which I think is good because you know, I went to the Naval Academy. There's a clear chain of accountability there My concern is is that I I'd like to sit down with those four guys and say Have you seen this Drexel video? Do you know do you know what we've got going on here? Do you know what's really going on under the hood? You know so my concern is is that because we've got this massive brain drain from the government and the defense industry I'm not a hundred percent sure that if we ever came to that point where we wanted to make that decision that we actually had enough Qualified people in the government to effectively make that decision and that that kind of leads me to my last point This government is gonna have we are gonna have to do something serious seriously soon Or we or not I mean it's funny is it because it's kind of a I had this intellectual Argument with myself. Is it really that big of a deal? I mean maybe maybe the rise of the corporate nation is just kind of where we are in history Maybe it's right that Google and Amazon will become more powerful as militaries than a nation You know, I don't know it see it doesn't set well with me, but you know, I'm like, I'm not sure how I feel about that I'm not sure how I feel about the distributed knowledge You know, I mean you can look at Edward Snowden to figure out when you know with the prison program The fact of the matter is this government could not have done that without the help of Facebook and all the internet service providers and Google and Yahoo when you see that list of people that agreed to do this It just could not have happened without the corporate nations Agreeing to do it. So when you see that kind of shift in power and capability and the shift has moved from hardware to information You know, you have to think about that. So how how do we how do we? It's not even defend against it. I mean, how do we actually? Pivot and it's kind of a definitely an overuse or it how do we pivot the government and the defense industries to understand that? There's this whole new game in information management, and you know, we we can't even get you know, the NMCI Network the Navy Marine China interface, that's what we call it at O and R right, you know, because we can't even protect our own email programs, right? Surely that's not the first time you've heard that joke every day So I think that we have to really think about you know the government structure and and you know it can stem from little policies like you can't pay these people enough to do it by The way, there's this rule in the government that you cannot make I think civilians cannot make something more like a hundred eighty five thousand dollars a year because you can't make more than the Secretary of the Navy That might sound like a lot of money to some of you young pups in the room But I wouldn't accept that salary. I wouldn't accept I'm you know, the government cannot pay me what I can make in the civilian world and so there's also a big change in The money to be made in the corporate commercial world and this is you know I mean if people could make a lot of money in the defense in the government I mean they would they would but you know, it's really not all about money either People will love the idea of going to war and working and serving their country But unfortunately the government is incredibly inflexible now about how to let people Be innovative. I mean, you know my my students literally I've had I have so many students who have already Started and failed and started and failed and started a third very successful company Lots lots of students. So it's just a different world that we live in today that Entrepreneurism, I mean, it's actually majors now you can get majors in innovation and entrepreneurial endeavors in college right now So we've got to really think about incentivizing a workforce keeping a workforce giving them access to technologies and capabilities that make it worth doing Because if we don't we're going to be back Well, you know, we're gonna be at that same place where I mentioned before where if we are gonna start unleashing robotic systems And this in this, you know, I'll make it personal make it a Navarra. I think Navarra is doing the best job that they can With the people that they have But Navarra doesn't have the best engineers in the world who are doing the x47 Those people are at Google. They're at other places, right? And so it limits our our ability to innovate and to be the best defense force offense to force in the world Because you know, those people are going other places All right, so we have lots of time for questions. I think Martin right work. We're good so