 So I believe they're gonna put a polling question up on the screen to start us off because I don't want to bias your own perceptions of what we're about to say and cue you on. So right now please text what is that 223 on this following poll which is which of the following Department of Defense or which which should the the DOD do in order to ensure that they are best equipped to face a future adversary. Nothing. Awesome. A reform the acquisition process, B be a part of the 4% for Freedom Crowd, C both or D none of the above. Okay you can kill it. So it looks like it looks like reform Wednesday day and that certainly is a topic of discussion. You would think that was the only topic of discussion in the Pentagon right now is how do we reform the Pentagon, how do we you know buy things better, faster, cheaper, how do we innovate, how do we turn DOD into Google. There's you know all sorts of goodness there and yet there is a separate component of that the discussion which is our simple ability to produce platforms or major military, major military platforms and this discussion is kind of encapsulate encapsulated two different ways one in Secretary Hagel's 2014 rollout of Forced Future at the Reagan National Defense Forum where he launched a number or at least codified a number of initiatives to include the Defense Innovation Unit experimental and with really a focus on a stupid nuts reform process inside the Department of Defense that has had a number of results some good, some that have been squashed or set aside for the time being and we'll talk about those the aspects of all of those. The other aspect that I want to talk about is McCain's Restoring American Power report that some of you may have read that came out earlier this year and in that he describes the force that we say we want lots of ships, lots of tanks, lots of men and women in uniform and yet the report also goes on to say that no matter how much we desire to get to 350 ships in the Navy in four years we get 59. We say we need and by the way we need 81. No matter how many men and women we would like in uniform now the report you know suggested 50,000 plus the report also highlighted that we get 8,000 a year before we start getting more Chelsea Manians and Bo Burgdolls. We are also short of the report underscores we are short several hundred several hundred combat aircraft so the argument really is multifaceted both from a reform stance as well as a rebuild stance so here with me today to talk about these issues are as Albert already introduced Raj, Jackie and Alana and to kick things off Raj so you're in charge of Defense Innovation Unit Experimental henceforth known as DIUX and you've been there as part of I guess the what I call the DIUX 2.0 the kind of relaunch of the initial initiative. Tell us how is this different from past efforts that had been aimed at reforming or enabling the acquisition process and and how are you guys unique amongst the current panoply of initiatives and organizations out there attempting to attack this problem? Sure thanks Stephen it's a pleasure to be here with with everyone so DIUX our mission is to access commercial technologies and get them rapidly to our men and women in uniform you know there's been I think 78 blue ribbon panels on acquisition reform since the McNamara Times our view at DIUX and our charter is we can't change everything but we're gonna focus and move out on the pieces that we can so what we do is try to solve our customers problems and our customers are the services the Air Force the Army the Navy the Marine Corps the combatant commanders we focus on capability gaps that they have and try to use some of our unique vehicles and access to non-traditional vendors to solve those problems very quickly we so there's kind of three core aspects to the methodology that we've put together one it's fast two it's flexible and three it's collaborative and I'll quickly kind of walk through that so fast if you're going to work with companies that normally don't do business with the government you've got to move at the speed of business so we since last May since we started 2.0 as you said have done 21 OT contracts the represents about 42 million dollars of capital the average time from solicitation to contract signing has been 78 days which I think is still too long our goal is to get it below 60 and so we'll get back there but that's at the speed where a young company can make a decision do I want to go service J.P. Morgan or do I want to solve problems for the Department of Defense so the speed part is critical flexible is the other part so from IP rights to requiring two sets of books we're not requiring two sets of books the type of contract we kind of cut through the things that a young company again can't solve or handle and then finally collaborative right I think one of the biggest things my observation coming back into government is the separation between the operator and user and the person that's going to go buy or build that right so the operator says hey I have this need it goes through some requirement process sent somewhere else and then five years later you get you get it right and and that makes sense for certain types of products that we're going to have very long life but if you're buying software if the half-life of that software is shorter than the time it takes to buy it you'll never catch up so collaborative we take us we take the customer and we take the vendor and we literally sit around the table and negotiate what's going to be built what's the milestone payments how do we get what we need for the for the government and you know as software goes it we can be flexible and continued rate so those are some of the things that that we do and and I'll highlight and I'll preface that everything that I've described we have no unique authorities or powers at DIUX these are things that are available through the entire department and we're just the the first ones pioneering it that's great so a lot of times when it comes to duty engagement with the private sector and by private sector I'm referring to companies and individuals outside the conventional defense industrial base there's a sense that either the private sector or DOD can or should do more to enable their enable their business or their capability kind of like what you were talking about Raj enable the company to even get to the position the starting line to engage the Pentagon right the assumption I think sometimes is that all companies are at that starting line already right some may still be in the locker room like putting on their you know put on on their their game uniform Jackie a lot of the work that you've done with hacking for defense and other initiatives appears to kind of get at this we have other people in the room like Christopher Zember who been involved with this as well how do how do you kind of view private sector engagement with DOD and more importantly what can DOD do to better enable that private sector engagement yeah I think that's a great question and typically when that question is asked you get the same response that we got in the poll today which is its acquisition reform and you know while I think and the acquisition reform is absolutely needed there there's plenty of far exceptions and commercially friendly contract vehicles you've got DIUX you've got other rapid prototyping places that use flexible contracting methods so to me especially because we've staked our business on this in the last three years we opened up in Silicon Valley with the purpose of being able to enable this engagement between government and and and the private sector DIUX is one of our clients and we work across the DOD as well and the IC but what I really see as a problem and in terms of what's happening and not at DIUX I think DIUX has us right but what I what I see in other places when they start doing this engagement is that it's a lack of agency on the government's part so a government person say a program manager will start talking to a private company they'll start tech scouting and that private company then gets a signal that this person has the agency to make the decisions that need to be made for the deal to get done you know or for some contract to be put in place and they're going through the process and this company's told okay you're going to have a contract in place at this time for this type of work and that time comes and then it slips 12 months or it slips 15 months and I say this because it's actually happened to us now several times and I've seen it happen at many other companies and the most frustrating aspect of that to a private company is that what it shows on the government's behalf is that they didn't do their homework correctly they didn't account for the realities of their own system oftentimes program managers or whoever's doing this tech scouting doesn't understand how money moves within their own organizations and that is probably one of the most frustrating aspects of this of this interaction it's something that I think DIUX gets right very well and what Raj has talked about and how they plan all this out before they go and actually start engaging with this private company and so it's not necessary it's not about okay we need to go do more policy it's about how are we changing and empowering the people within the system to adopt this non-stove piping mentality when I was an engineer in the Air Force I got very good at contracts and people in my company think that I'm some kind of contracting expert and that I must have been a contracts officer when I was in the military but I just understood contracts because I had to to get things through the system and I think we need to be encouraging people inside the government to adopt more of this mentality great so Alana you used to be a DASD for MIB or sorry Deputy Assistant Secretary of Defense for Manufacturing and Industrial Based Policy there's saying in the military that the enemy gets a vote well so does the industrial base and the building itself from your perspective on a policy level you know how to how do folks like Steve Welby who's here former ASD R&E how to folks like him and you and your previous job how do you guys create and craft good policy how should the current administration craft good policy that enables both the defense reform initiatives you know enterprises like Rajas as well as these prototypes throughout their life cycle and the hopefully the really good ones in the program of record or in MDAP yeah thank you for that question and a quick shout out because Steve Welby was indeed an amazing colleague and Raj is doing some great things and it's indeed piloting the way DOD should do its work but he has not invented anything that wasn't there before there are an enormous amount of exceptions there are two different systems for purchasing there are there's even been a piloted authority to keep a little bit of your the extra savings that you might have so things have as Raj had said there have been a ton of studies and laws on the books and the last couple of NDAA's have put more laws on the books that are now making themselves felt into in a new commercial purchasing handbook which will be done sometime this summer which is the way you can the the Pentagon can purchase using the commercial rules instead of the military rules which are much simpler all that is good i continue to be concerned though that that won't be enough and i'll put it down to three simple things which is leadership culture and incentives because all of that has been there for such a long time but what i've seen when i've seen at work i've seen the three those three assets being brought to bear so special operations command has a reputation for being able to do procurement really really well i think that that's partly because they've got great leadership i think that's partly because they have a great reputation for warfighting that lets them take risks like other people don't take risks so we need to help our procurement core take risks we need to put in the right incentives right now when you're a contracting officer and you're somewhere around a GS 13 or a GS 10 which is sort of a early to medium stage career person everything is a disincentive you make one mistake the i g's on you you didn't save a penny that's that's a problem in your personnel file but when you create savings when you get something out to the field quickly that doesn't show itself in your personnel file as easily and it's something Eric Schmidt started to allude to a little bit in his earlier conversation so we need to rethink some of those incentives to give people that cover i mean people have to do a good job they can't be corrupt they have to be smart but they need the incentives to do the right things and to be risk takers diux was set up to be a risk taper risk taker but we can't just do this around the edges as much good as Raj has done it is pennies compared to the budget that do d spends all the time and that's not at all a hit on you it's just you're you're you've got a universe to play with that is showing the way but guess what the whole building and all the buildings around the country can can take that way as well so the right incentives and the incentives are not just around personnel it's also about the flow of money again i think erica was talking about it earlier that you know savings disappear well congress actually provided a tiny little pilot authority a couple years ago that that would allow you to maintain some of the savings and it's a long story as to why that didn't continue to evolve but there's a precedent and so that's that's sort of on congress to think about extending authority for some of the the funds that people may have when they're able to save or be able to transfer between programs in an easier methodology than you have now when you have to go through all of these offices and finally get your whole package together make sure that somebody didn't trade away your pack part of the package and then it goes to congress and then there's another set of briefings and debates i'm not saying no to oversight at all as a former congressional staffer i am saying sometimes it that means that it's a 12 month or a 24 month program so and then and then the other thing about you know i don't know if you want to put it in i guess in culture you have a lot of really really dedicated people and a lot of really smart people but a lot of them have spent their entire life in government and that means you're not necessarily as nimble as the private sector when you're trying to negotiate especially with the non-div the commercial base and especially the silicon valley the brooklyn navy are to like harken to new york or even which talk cancels with you know some innovative planes that the textron had put out at no tax no no taxpayer expense so i saw when i was there a pretty modest sized fellowship with that i think was created in the mid 90s and it's a secretary of defense's fellows and they get to go spend i believe it's just a year or two out of the private sector cyber security at a bank or contracting for one of the defense contractors they come back and they they you know we all know what it's like to experience as opposed to here in a lecture they've experienced what it's like to do business a little differently and then their follow-on assignments have nothing to do at least that was my experience have nothing to do with what they just did and so all that knowledge dissipates we should be doing this for a lot more people i would love to see it for the entire procurement course somewhere in the course of their career and i'd love to see them have to go back it's sort of like a payback when you get a scholarship to college you have to stay with the company for a couple years i'd love to see them stay and then and then um use that investment in their fellowship to to bring about a different culture in the procurement core so that would be my prescription so so let me turn this on its head uh raj from your perspective i mean in my opinion i think i think you guys have had pretty solid early success um since last year and i guess my question to you following on alana's comments uh what what can you attribute that early success to um and then more importantly how do you scale or or grow those early early wins into uh you know alana's domain right where you start to you know you start to chalk up programs of record and then they they grow on the you know 30-year sustainment cycles yep no great questions and and i agree with a lot of what lana said and particularly around you know so you asked what made us successful i think it's those things that you highlighted right we leadership we had senior level leaders in the pentagon that supported us and and candidly protected us from from some of the folks more resistant to change we were allowed to to fail and experiment and try things differently um but more importantly uh how do we scale right because we are a rounding error of rounding errors if you look at the defense budget um i i kind of view you know dx's is mission uh is to do agile culture change right so if you're going to write a big piece of software right so if i use a software analogy you don't go and write all of uh a massive program over five years and then release it you build little pieces of it and you take it out and you test it and some things work some things don't work you fix bugs and you get this continual feedback that few years in you have a great leading software package i feel that's our mission at diox to do culture change right we are never going to be the acquisition arm of the department we're not the r and d arm of the department uh we have existing structures to do that i think our job is to try new things and on a low cost basis the things that work we can scale the things that don't work it was a great experiment it didn't work we can at a very you know without wasting a lot of time money remove it so you know what kind of changes uh do i mean by that so incentives right uh and uh incentives for agility right so we the way we structure our team we track how long these things take and we incentivize folks that are able to move it through the through the system faster you know what of those can now be be scaled the entire ot system uh is again not exclusive to us so we wrote a a guide on the website anybody can go and download it on how do you do a commercial style ot and there are now other established groups procurement officers and groups within the department that are starting to do this which is great and then you know there's finally you know if i think about the the technologies that will be very important on the battlefield of the future things like autonomy things like artificial intelligence some of the best engineers and best minds working on those are working in commercial companies that uh have their own markets right they can better recommend a product on their website and they get a one percent growth there will be hugely rewarded but those same technologies have applicability to the defense world and you know we the department has traditionally been a monopsony buyer right yeah but now these we're a competitive buyer and so we have to change our our culture and process to to achieve it so you know so long term we experiment with things the things that work scale you know if in an ideal world i'm nervous to put a timeline on it but you don't need a d i u x anymore right because those things have been adopted and and then we move on so jackie hacking for defense and the the enterprise around that and the the people who support it it's also appeared to me to be more of a movement than a particular you know program or a class you go through at georgetown or stanford or whatever piggyback you know of some of the previous comments how do you take that that culture and the i guess the the people who kind of self-select for a program like that you know for especially from the military side how do you take that and grow that whether virally or through mandate or fiat how do you grow that across the pentagon or is it something that should grow across pentagon and instead of just being in a petri dish yeah i mean there's two versions of hacking for defense the for people that don't know what it is there is a hacking for defense course that we taught last year at stanford university where we basically took national security problems brought them through stanford and applied the lean startup methodology and how basically the science of innovation back to those problems just to see what would happen and it turned out it was a successful course there was a a group that came out of it that got venture capital funding a number of the teams went on to get continued government funding and now there's been other universities that have adopted this course including georgetown and part of the the rationale behind this is that the culture thing goes both ways right it's inside the government yes there needs to be a culture change and we all recognize that but there's such a divide between government now and this new generation the millennials in terms of there's just no connection there unless they actually go and enlist in the military or they go and become a government civilian and so this is one way that we saw to provide students an opportunity to participate in national security issues using their classroom knowledge of entrepreneurship on the other side of it through my company we provide hacking for defense for our government clients and the our rationale behind this is that there's a lot of talk about innovation and being in silicon valley what we've seen as the most the most interesting aspect to this is that nobody goes out and says we're going to be innovative they do it by what they do right whatever it is they're building but but through that process there's definitely a framework a science behind innovation and so we have wanted to take those same principles and basically bring them back inside the system that we think we're that needs it most and so through that we've taken the lean start-up methodology which basically encourages people when they're looking to develop something just like the entrepreneur to be able to identify very early on the the value proposition your key partners come up with hypotheses around what it is that you want to build do iterative testing and and really understand the feasibility and and the desirability of what it is that you're building for your users and this is coming off the problem before that we coming from acquisitions myself we built a lot of things for a lot of money that didn't solve anybody's problems and so we're hoping that through this process this will impact culture and how we look to putting our dollars our taxpayer dollars to the things that we're building great so Alana quickly and then we'll go to Q&A a lot of innovation in fact increasingly so at least to the extent we're aware of it happens outside the United States MIBP certainly has a role to play in the CFIUS or foreign investment process review process what are your thoughts in terms of engaging compelling foreign technology but at the same time protecting our own IP and our own kind of supply chains as we try to be more innovative yeah no that's an excellent question I would I would actually add to it and it's the CFIUS that's the internal investment from overseas but also export controls and how do we protect what we've invented so my predecessor used to say you know not we everything all the best stuff used to be invented here but that is just no longer true and so that creates a dilemma for DOD one is that we want to probably buy some things that are not invented here and we can but not we anymore but DOD can but it just has to go through a process where it has to get through the defense security service particularly if it needs clearances for facilities clearances for personnel which can be more cumbersome depending on the the nation from which this company is coming we also have the issue that there are some amazing things being done by US companies but with two problems one is DOD is no longer the largest buyer so they if they're going to sell to DOD they run the risk of being prevented because they're in a classified setting for example from then exporting that and there's also the broader risk of export controls stymying their growth and often even some fairly modest sized companies are using engineers from abroad or they have their their manufacturing abroad and so that creates all kinds of classification and export control issues and that is something DOD I think has to really grapple with it's not a DOD only issue the state department has a huge role to play the commerce department the Obama administration actually did a great deal in terms of updating the export control rules but things move so quickly that certainly around software and hardware and certain types of you know like infrared or certain other types of very specified technologies I think those export control rules have to be constantly updated because there there is so much happening overseas that either we're going to want to buy or we're going to not want to prevent our companies from selling abroad because they're no longer so far ahead of the curve that it makes sense to stymie their commercial development but back to Syfias really quickly it's just it's also very interesting as an issue because you know I wasn't those Syfias conversations and case decisions which I can't talk about here but what I will say is they were becoming more and more complex people think about it in terms of oh you're buying that wind farm next to a particular installation in Nevada which is all public knowledge it's all you know in the front pages don't forget about Smithfield now that's a big deal or Smithfield you know but but what's but they're more and more complex kinds of cases where there are all kinds of relationships between the various stakeholders and investors and how much say do they really have and how much insect will they really have and what are they buying will that technology that they might be buying impact so many different systems on that are in our critical infrastructure and possibly a dog needed hardware etc and it's very very hard to draw all those pieces and you can't just shut off all investment so it's becoming a much more complicated set of problems that don't envy the people around that table now boy wouldn't it be great if we had a graphic graphic user interface that depicted our entire industrial based supply chain imagine that so we've got a couple minutes left for for questions so if you have a question please raise your hand wait for the mic to come to you tell us who you are and please actually ask a question right there in the back Liam Collins from the Modern War Institute at West Point Alana I'd agree with you that special operations forces there have been uniquely successful and primarily because of leadership and culture but do they also have authorities I mean this isn't a question it's leading into it but don't do they have authorities that allow them to you know short circuit the process and and if that's the case you know can the military the larger military make the changes to the innovation or the acquisition process that it needs to or does it require some kind of major legislative change if you look probably in the last 75 years probably the most significant changes from the military weren't some kind of new technology but more of the you know legislative changes to the National Security Act in 1947 and Goldwater Knuckles in 1986 so do we need some kind of major legislative change or is it something the military can do on its own in order to kind of reform this acquisition process and move beyond the 1960s PPBS system you know I'm a big believer in understanding what you need to fill once you start doing it and there's so many authorities that are already on the books I'd say there's a lot that can happen before you go to Congress and then you can look for much more modest slivers of changes then you could explain better to Congress when you actually show specifically what you need I know Frage if you have a doubt yeah there's tons of authorities recent from the 16 NDA from the 17 NDA that apply tons of flexibility and not just outside the far in the OT world but also within the far having now spent a lot of time actually reading it there there's actually quite a lot of flexibility it's just it's getting the cultures and the incentives right to empower contracting officers to feel like they can do that can I say one thing if you're if you're sort of looking at this and researching this you might want to take a look at what cybercom is doing because cybercom had only recently gained some acquisition authority and they were going to SOCOM to get their sort of advice around how to do things so it's an interesting experiment right now okay right right there up front Colin Clark breaking defense Alana and this is directed all three of you I think raised the issue of incentives but you also have the training and the length of time that an acquisitions officer actually spends in a program the services run those for the most part how do you get across the board change for that workforce when each service has the authority to keep fiddling as it wishes now Colin you're you're totally correct if I understand you correctly because for the long programs of record for example clearly requirements continue to be changed and that's partly what drives the cost and what makes it so difficult for companies who have huge gaps in their production so you know that is that is partly what AT&L I think was meant to fix obviously didn't get all the way there and now there's a question of is there an AT&L in the future but that's definitely you know internal reform that that needs to happen as well and perhaps we we can look at incentives for that kind of internal reform and whether a disincentive could be created when you continue to fiddle and you have to wait for the 95% solution but you wait for another five years or you take the 80% solution you get it out to men and women you know in the field today one other thing the labs are an amazing resource the labs know more or less what's going on they're a great way to just put a check on hey does something already exist you know do we really need to reinvent this or can we take something that exists and maybe slap on some classified equipment or something that hasn't existed yet but not invent the entire thing I one of the things I've noticed working across contracts organizations is that the ones that are reimbursable meaning that and I think it's I think GSA does this there's other organizations that do this but basically they they are their salaries are actually tied to how many contracts were let through their shop versus a case where you know if you did one or if you did 50 you're going to get paid the same the same exact amount of money and I I do think that there is a model that for support organizations in the government there can be some of that I think incentives put in place in terms of your salary is actually tied to how much you put through the system I guess in the last 30 seconds the thing I would say is we it's difficult to take a one-size-fits-all approach to this and incentives so we have processes that I would say arguably work well for buying large programs so the way we buy an aircraft carrier should use one set of processes that are more deliberative and requirements based if we're going to try to build a UI system for data you know that changes so rapidly that we need an alternative set of incentives and approaches and try not to use cost plus yeah small programs yeah and by the way the next session will be how to do all this in a classified setting so don't hold your breath thank you very much please join me in thanking our panelists today