 Welcome everyone and thank you for joining us for today's Defense Innovation Board public meeting. My name is Dr. Marie-Nathia Dottu and I'm the Executive Director and Designated Federal Officer for the Defense Innovation Board. Today's meeting is being live streamed and recorded to allow members of the public to attend and the meeting virtually now or view it later. Thank you to the Defense Media Agency for providing their expert support to support this event and to the Defense Innovation Board staff team and everybody else involved making this event happen today. The board would now convene in its public session. Please allow me to share a few procedural comments. The board is discretionary independent advisory board operating under the advisory, the Federal Advisory Committee Act and the Government Sashine Act. Today's meeting was announced in the Federal Register Notice posted on February 21st, 2024. There have been no significant changes to the meeting's agenda as posted in the Federal Register Notice. The public was invited to submit written comments for the board and we received several written comments in advance of today's meeting. Those have been collected and called and they have been posted on the Defense Innovation Board website. As a reminder, these are comments to the board and not a question and answer session. And now I would like to turn it over to the Defense Innovation Board Chair, Mr. Mike Blueberg. Mike, over to you. Well, thanks Marina and thanks to everyone tuning in and watching online. In case anyone in the audience isn't familiar with the board's mission, our job is to provide independent recommendations to the Secretary of Defense and other senior leaders across the department. It's a job we're all honored to do and by doing it well, we can help empower even more of our men and women in uniform as their jobs become even more dangerous in the face of escalating challenges around the world. Now this public meeting will proceed a little differently from our previous meeting at the Pentagon in January. This time, we're inviting the public to hear directly from some of the experts who will help inform the board's two current studies, which Secretary Lloyd Austin and Under Secretary Heidi Xu have asked us to undertake. To help set the table, let me summarize the two studies quickly. The first focuses on how the United States is innovating in partnership with allied nations. Board Member Charles Phillips is coordinating this study. We'll identify not just the strengths and technology strategies we have in common with our allies. We will also look at some of the barriers that exist across multilateral agreements and how we might work to remove those barriers and improve how we collaborate. The second study focuses on how we might make the department's adoption of new technology both faster and less painful. Retired Admiral Mike Mullen, who knows a thing or two about navigating the halls of the Pentagon, is coordinating this study. There's no innovation without taking risks and right now it's clear that we need to incentivize more individuals and teams across the department to take more risks. So we'll collect insights from department research labs, contracting officers, service members from each branch, and external partners across industry too. That way we can better understand the different timelines and incentives of each of the parties involved. Then we can better align those timelines and incentives and speed up the adoption of promising new tech. Nothing to it, right? But seriously, I think it's fair to say that we have plenty of work to do. We'll provide a detailed update on the studies at the public meeting next month. But first today, we're glad to welcome several members of the Nonprofit Defense Entrepreneurs Forum and to hear their first-hand perspectives on the challenges we're tackling. To ensure we hear from everyone in the 45 minutes that we have asked, I've asked Marina to moderate a Q&A with a group. Board members should feel free to chime in with their questions and comments throughout. Marina, over to you. Thank you, Mike. We're delighted today to welcome the Defense Entrepreneurs Forum members and hear from them, as Mike mentioned, and from your insights and expertise. The Defense Entrepreneurs Forum connects, inspires, and empowers innovators across the national security ecosystem. And I'm personally very fond of the Defense Entrepreneurs Forum because actually I was involved in the early stages probably five or six years ago. And delighted to welcome today Jesse Levin, Megan Metzger, Michael Madrid, Ivana Hu, and Ian Aishen. And I'd be remiss if I didn't mention that almost everyone has been a TEDxDAU speaker here. So if you're not familiar with TEDxDAU, look it up and try to attend. So let's zero in today's conversation because we do have only 30 minutes. I'd like for us to split the time in 15 minutes for each of the studies. So we're going to start with optimizing how we innovate with allies and partners. And I'm going to ask Ivana to lead the first to kick us off with answering the first question. And so as a reminder, the answers need to be about three minutes. So no matter how many people respond, let's try to keep it around three minutes. And then we'll have about six minutes total for members to ask questions. So Ivana, how do we optimize the way we innovate with our allies and partners? It's a very broad question. Over to you. Thank you so much for having me. And I'll do my little spiel in about a minute and a half. And so I would like to actually set the scene by talking a little bit more about why we need to work with our allies, right? And so for DEF, we see as the best way to strengthen our mutual defense posture and to prevent further democracy backsliding against authoritarianism, especially against the backdrop of great power competition. Partnerships are seen as a linchpin in the U.S. Department of Defense's strategy deterrence as outlined in the 2022 national security strategy. Yet little capacity and few initiatives exist in terms of fostering cross-national camaraderie via cultural intelligence and atmospherics, and these socioeconomic human factors have taken a backseat. To remain relevant and competitive, our younger junior officer and enlisted corps must garner in depth cultural awareness and empathy for the nuanced operational and social realities inherent to these likely feeders of operation. Hola Society informal working groups are incredibly well resourced, supporting kinetic operations and are vastly disrupting the acquisition, supply, logistics, and tech development and fielding cycles we would normally associate with DOD. Very few cultural exchanges intended to foster true rapport and working relationships with ally partner forces exist for the junior officer corps and the enlisted DOD personnel. DEF aims to fulfill that gap through informal exchanges and convenings with our international partners, specifically those in the Asian theater, including Singapore, Taiwan, and the Philippines. With that, I'm going to turn it over to Jesse to discuss the solution. Thank you, Vana, and thank you to Dip for having us back. As Vana mentioned, innovation, it's almost been indoctrinated in the western DOD now. We have a number of programs of record. The dip is an incredible example of how we've evolved to ensure and expedite technological advances in the private sector and get into the hands of the warfighter very quickly. However, what's happening around the world in various AOs, with these proxy environments, the game is very different, how things are transpiring on the ground. And as Vana mentioned, there aren't a lot of opportunities for our junior officers and our enlisted DOD to get exposure to the socio-cultural elements of how operations, logistics, acquisitions, things are transpiring in environments like Ukraine, for example. And a lot of what we hear as an example using Ukraine because that's a very current situation, FPV drones and other technologies have changed the face of war as we know it. What's also very interesting and important to study is how the entire whole of society has been engaged and how hundreds of millions of dollars have flown through groups like YPO and social impact entrepreneurs who have partnered up with local businesses and adopted direct action units. And the way innovation is being done on the ground in these environments is radically different than what we've built systems to support. It doesn't make it right, wrong or otherwise, but it's imperative that our junior officer core and our enlisted another DOD personnel gain visceral exposure and establish rapport with individuals operating with our allies. So when we have to support, whether through the advice assist the company or from a deterrence perspective, we are really well versed and understand culturally what is transpiring on the ground. So we're asking, we'd love for the DIB and what DEF is going to try to do is within six months, we're going to work to stand up a fellowship. And this is going to really largely rely on business relationships in these international environments that Yvonne mentioned, you include Singapore, Taiwan and Philippines. And there's going to be a number of challenges from security clearance from any number of structuring issues that we're going to face. And we'd love for the DIB to hold DEF accountable to this project. And we'd love to ask the DIB and the individual DIB members for support as we structure and architect this cultural exchange. Thank you. Thank you, Jesse. Yes. So you're building a fellowship. And so how do you see the fellowship program addressing the biggest challenges and opportunities? What is the fellowship program looking to accomplish? Yvonne, you want to take that or do you like me to take it? We're looking to accomplish this right now. There's a tremendous amount of focus on tech and additive manufacturing, AI. These things are of course imperative. How we do this in the private sector and how we channel these improvements and these innovations into DOD of course is imperative. But what's not being studied enough in our estimation is the human element, the human terrain. And the fellowships are not new. It's not a new approach. There's nothing revolutionary about this. However, there are not a lot of opportunities for the junior officers and for the enlisted to gain this type of exposure. And to have systemic change, we feel it's imperative for the younger officers and people getting involved in the national security community earlier on to gain exposure to how our allies are operating in their environments because we cannot expect them to do things the way that we do things. And if we want to be the best position to support and again to advise us as a company, we have to have a visceral understanding for how society is operating in their respective countries and how we can best plug into the existing cultural nuances of how their respective ecosystems function. Thank you for that. So let's pause here and open the floor for our board members to ask any questions. Adria Mullen. So Jesse, as I listen to this, I mean historically the cultural barrier for the program you're talking about is allies and partners come here. If we've had fellows typically, I mean it's not exclusive, but they actually come here in numbers to our war colleges, to our graduate schools, to our training commands. And to do this, I think what you're talking about is, and I don't disagree with this, it's just that getting the services to cut loose with numbers, not one-ups, because typically we do one-ups in Singapore or one-ups in Taiwan or the PI. There's a volume requirement here across DOD. I'm not just talking military. We are so acculturated to people coming here that we don't even think about being in their shoes, in their country, which is absolutely critical. So it's a big bar that you've raised in terms of trying to get over, at least as I understand it, in all the aspects of it. I couldn't agree more. I think one of the things that we don't do well that we need to focus on is how do we see it through others' eyes? How do we see it from their perspective? And we're not very good at that. Absolutely, sir. I think you hit the nail on the head. And we understand there's going to be tremendous cultural barriers in the DOD to get requisite permissions. And it's a massive cultural shift. However, from what we've seen on the ground operationally, we have a tremendous amount to learn. And it goes both ways. And it's very difficult for us to ascertain those things from a distance. And I don't know that we need to have large numbers off the bat. And I think if you find the right people, you start with one. Our goals actually have six billets filled. You know, when we launch this program, hopefully in six months, and you've got to start somewhere, but I could not agree more. Yeah. And we also want to pull them out of the defense context, right? And so that they also understand where the civilians are coming from. So we almost see it as like a mesh, a mesh up between the Aspen executive seminar, where, you know, you have certain amount of philosophy and history that you have to read about other cultures, plus that hands-on experience with their defense counterpart, but also being exposed to what the ordinary civilian is going through. Yeah, I don't disagree with all that. It's just the difference between what you just described and living there, but literally living in a country, living with a family is almost night and day. And to me, that's where you want to get to, that kind of understanding and all the pieces that go into that. The other note I took just as I was listening to you is how the NYPO triggered me because the relationship piece there is huge, it's global, but it goes, it really sort of goes through the business school. I mean, that's kind of central. That's where those relationships get started. And is there a way to do this in that regard, which could make us attractive from a career standpoint for our junior officers in particular, to go to school, create that kind of connection in those relationships and then bring them back. So, but you got to go, we got to go to other countries. The normal cultural adaptation we get in other countries, sadly, is typically when we're in a fight and we don't have it before we go. I mean, that's, you know, so I admire the challenge, but it's a big challenge. Let me ask a question. Can somebody give us examples of things we tried to address but failed and why they might fail in the current system? Chris? I can give you an example of how things have actually been happening in Ukraine versus what we might require through our system. I had mentioned YPO. You know, we've seen literally and touched and helped facilitate literally hundreds of millions of dollars of money coming through social impact groups, private entrepreneurial venture groups, groups that have absolutely nothing to do with government contracting or SIPRs or primes, but simply wanted to be supportive. And we've seen, you know, millions and millions of dollars of procurement acquisitions and last mile distribution done with zero money before operational budgets even hit based on goodwill, handshakes and established rapport. So I think to answer your question, the best of my ability, our process even for innovation is very systematized and very regimented and it's clearly a place and a need for that. Absolutely. We're not saying that that's not required. However, we would be remiss and we would I think be setting ourselves up for failure if we were not to identify the fact that the way that money flows and support is being provided in other environments is drastically different than what we're used to. And we are seeing tech cycles and fielding of tech and iterative, you know, evolutions of, you know, drone technology and, you know, software backdoors being, you know, fielded, tested, changed and brought back up to the field in 24 hours for pennies on the dollar because civilian and entrepreneurial groups are just involved and have adopted front-line units, something that we don't see within our system. I hope that answers your question. Part of a necessity because of how Ukraine is structured. I've been there and been with the units on the front lines and it's very much a Confederacy and doesn't feel like the U.S. where combined arms maneuver is possible. And so I think the parodies and other entrepreneurial efforts that have sprung up because there's been a lack of a central procurement mandate from the government. But I do think that you've got Ukraine as an example that you can hold up to the U.S. and all of our allies and partners, especially with respect to junior officers and enlisted. And that's in Ukraine. It is often the junior officers and enlisted who are being looked to as the technical experts. They're the ones that are working with the companies or sometimes doing things inside the units because they've got more, you know, more currency and technology. And so I've seen that across all different echelons. The junior officers are leading the ideas on tech. And I think that's something that could become part of training and scaled across most militaries where senior officers lean on junior officers for greater insight on technical matters. And if conflict occurs and we pray it doesn't, the same kind of development and operations don't happen yet. And now you've already trained for that. You've prepared for that. You've educated for that. So I think there's a great opportunity to make having tech currency being brought into the units a focus of your first training efforts. And I think that's something that every nation that we work with would want to partner with and try to standardize that so that our teams and our units would be interoperable in a conflict. So I think there's a good idea there that you can focus through a lens based on what's making innovation go so quickly in Ukraine. And that's one imperative to the dev ops, developers and operators working together. And three, there are typically a few tech savvy people in the units who can speak tech speak and help on the rapid iteration cycles. And that sounds like something we could codify into a process that could be scaled. Well, let me answer something. The craziness is going on with Congress where they won't even consider authorizing money to go to stop, to help the Ukraine stop Russia. Our war fighters, our people on the ground, what do they think about the domestic political process and how little we support or do support their efforts? Well, our units aren't there on the ground, but I've spoken with many of the Ukrainian units. Of course, they're very worried about the West losing support for funding the war. I think the lesson we should learn from what's working in Ukraine that should be in place right now in the Pentagon, as well as our allies and partners, is Ukraine's in development mode always. And we're in procurement mode. We're trying to help them win the war with yesterday's technology, which Russia has already responded to and countered. And I would argue there's likely never been a war in history that's been one with the equipment that was possessed on day one. Some of the great innovations of World War II, Barnes Wallace building the bouncing bombs and the bunker buses, but these were developed during the war. And I think that's the lesson is that as opposed to trying to squeeze supply chains until they're dry, we should be going back to the drawing board with our innovators that we have quite a few of in the US and figuring out what new systems can help them win. That's an old lesson that the Pentagon should really relearn because that is what Ukraine is doing day to day. What they don't have the ability to do is to think further than like a month cycle. In some cases, more than a week cycle, but we do. And I think that's an area that we could really help and maybe find some more cost effective solutions that would make supplying the war more politically palatable. But I think it's critical that Ukraine prevail in this war. What Russia is doing is a template for other nations to follow in future. Thank you so much, Will. I think this has been great so far. Hey Marina, just one quick question. It's one thing Ukraine brings a sense of urgency despite the fact our troops aren't there. How do you take this and Jesse, this is a question for you, how do you take this and create deterrence, which is much more difficult so that we don't go to war in Taiwan? I don't need to answer that. I just have that question out there. Thank you, sir. That's a great way to wrap up. Very, very short discussion on allies and partners. Let's shift gears now to incentives and aligning talent incentives to drive tech adoption. So I'll turn it over to Megan to talk to us about the low hanging fruit when it comes to incentives for talent. What are we missing? What should we be focusing on? Megan, over to you. Fantastic. Thanks so much for having us. So I'm going to talk about this in three groups. I'm going to talk about the innovative workforce that we do have, the incentives for them, the remaining workforce that surrounds them, and then the workforce that we need to pull in. So starting with what I would call the folks that are already in there taking risk, they're doing more innovative things. First recommendation would be around promotion pathways. So Will Roper talked about codifying these tech savvy roles, helping create a tech currency, but currently the promotion path for those types of skill sets is not immediately obvious. So when you look at incentives, a big motivation for individuals is going to be autonomy and mastery. And right now, some of these roles where they gain skills, become innovative, learn tech, the technology they need are then rotated into positions where they cannot use that skill at all. So I joke that someone could go to a software factory, learn to code, and the next position might be collecting your analysis samples. It doesn't line up. So we need a better promotion path to help them retain that mastery that they're gaining. The other is around creating promotion, aligning it to the outcomes. And so you can drive accountability into these roles. So when we look at the promotion pathways today, it's a lot on time on schedule, on budget, but not a lot of the outcomes that we want to drive by driving innovation. So keeping the warfighter safer. So really understanding the outcomes and aligning how promotion paths are done against the outcomes for that. The second, I'll talk about the workforce that has to surround them. A big challenge right now is you might have a skill set like these tech roles or these innovative individuals that are more forward leaning and the folks around them are not sure how to enable them or weaponize them for what we need them to do. So it's a really common theme to say, hey, everyone's just risk adverse and we point our fingers and I like to say, with all due respect, we built them this way. So I think a recommendation would be look at the training and the upskilling of all of these careers that are in the critical path and really audit it to where are we building in such a risk aversion along the way. How can we upskill them on the technology to a level where they know enough to know how to do their jobs differently, like contracting, for example. The second one also be focusing on our senior leaders. Senior leaders mostly did not come up through the ranks thinking they're going to manage major IT programs and now almost everything is an IT program and how you have to lead that and the questions you need to ask has wildly changed. So assessing how we upskill the workforce that we have that's required to enable those risk takers as innovators and those technological skills will be very necessary. The third will be the talent that we need to attract and we need to retain. So some recommendations I would have two main ones. One is bridge the skill get the pay gap so that we can, you know, be on par, attract talent that might not have come to the organization otherwise, especially in those technical pathways like data science and software development. And then two, allow for these types of roles to more easily and fluidly go back out into industry and come back in so that they can keep their knowledge fresh, learn and understand the commercial sector and things like DevOps that continue to evolve and change every day and bring that knowledge back in without such a big bureaucratic red tape barrier in front of them. So those are my three, I would say they're low hanging fruit, but it doesn't mean that they are easy, but there's a lot of opportunity there. Thank you Megan. That's a great way to help us think how to align and focus on the different areas or categories if you will of talent that you outlined and I love how you highlighted potential recommendations for each. So as we dig deeper, we know that we have to incentivize risk taking to ensure that the leaders across the organization are rewarded for different approaches to traditional problems and you alluded to that earlier, but how do we how do we dig deeper into ensuring that we train our talent to take calculated risk? And in my mind that requires leaders that provide top cover and leaders that are also trained to be able to have the right skills to be able to cultivate innovation and foster a culture of change and provide that necessary top cover. Let's hear your thoughts of that as well. Yeah, absolutely. I do think there is a great opportunity at the leadership level to one, help them understand how they're going to lead differently in a tech forward environment to understand what skill sets and what it looks like to take calculated risk so they can provide a left lane and a right lane, allow them to innovate within, but not go so far that we are now in a risky situation. Now with that, what on earth does risk mean? I think there's a lot of retooling to help understand how to reframe risk appropriately in a tech driven world. So how it translates down to staff might be risk of a protest in a contract, but all you did was shift the risk to the people with the most to lose, which is the people on the front lines. So when we look at how we're evaluating leaders and what risk they are taking, helping leaders learn how to recalculate, reframe their risk so that they are taking appropriate levels of risk, but it's not having an impact on the mission that we need. A lot of times that connectivity between the risk aversion and the mission outcome that they need is not tied together. So I know that that's a very nebulous thing, but I think there is a lot of training, intentional training around calculated risk, reframing risk and how to lead more innovative forward leaning talent differently so that we aren't being risk adverse but in the wrong places. And Michael Madrid, I know you have some thoughts around that. For example, one notion is to transform the frozen middle to these force multipliers that are actually the ones that are unleashing innovation because they now have the skills to be able to foster innovative risk taking. Michael, what are your thoughts on that? Well, Megan and I actually didn't have a lot of pre-coordination, but you'll see a lot of themes that are coming between our thoughts. And there's probably three things I would highlight. But first, let me just go back to something you mentioned, Marina. You mentioned top cover. And in my experience, top cover is necessary but insufficient. It's actually quite necessary and quite helpful to have a senior leader like a flag officer embrace an innovation program or recommend a junior officer or an enlisted person who has a bright idea, but it has actually historically been insufficient for that person to actually go and make change because the flag officer doesn't have the time or it's not their responsibility to kind of handle that solution through. And so there are a lot of other people, Marina you're referring to as a frozen middle that play a role in whether or not those kinds of innovative solutions see the light of day. And so I'm thinking about sort of incentives for people throughout the organization, including the middle. There's three things that came to my mind. The first is to disestablish the golden career path. Megan and others have highlighted this already and allow those who do innovation assignments or who kind of strayed from the very typical military career allow them to continue progressing in their field, if that's what they want to do. And I think part of this is looking at the metrics by which we measure our personnel and how we evaluate them and think about what those evaluation metrics incentivize or disincentivize. And there's both positive and negative ways to incentivize people. And I would actually put forth an argument that people who are naturally innovative or innovators don't need a lot of positive reinforcement. We've seen programs throughout the years that offer cash bonuses for innovative ideas or try to reward people for doing innovative things. But I think if you have that bug you are dying to go do it yourself and you actually just need things to get out of your way. And so really I think more about the negative incentives we need to avoid unintentionally punishing those people who go do innovative things or limiting their career or limiting their opportunities because that's when either they innovate and then they decide to get out or they have the impulse to innovate, but they restrain that impulse because they don't want to be outside the norm. So that's the first thing is thinking about career paths. The second thing would be around celebrating stories of smart risk taking, even when it doesn't work out well. I think this is one of those things that aligns really well with what Megan was talking about, defining where feeling is acceptable and in fact necessary for learning and for moving with speed. And in those arenas where failure is okay, actually celebrate that because people need to see examples of taking a risk and not losing your job or having your career ended because you took a risk and it didn't work out. We have to combat the notion that quote unquote nobody ever got fired for picking north through Grumman. And so the third thing I would say is also exposing people to what good risk taking environments look like. So whether it's the frozen middle or anywhere in the stack of the organization show them what good looks like. We have experimented in the DoD with various tours with industry, tours with startups, embedding people with venture capitalists who do smart risks taking all the time as their day job. You may be familiar with the defense ventures program and it's an incredible shame that that program ended recently and Aforx didn't renew the contract when there's an incredible body of people who have gone through that program, returned to service. In the data they say they're more likely to stay in the DoD longer having had that experience. It doesn't accelerate their exit from the DoD but rather they go see what good looks like at organizations from Amazon to Andrew Roll and all sorts of organizations in between and then investors who do risk taking every day. So show people what good looks like, celebrate the failures when they're acceptable and think about what the career path looks like. Thanks Michael. That actually highlights you talked about rewarding and recognizing innovators whether they're winning or whether they're failing and making sure that we've established metrics to do that and that brings us to our final question that I'd like to ask Ian to address which is how do we know how do we know we're being successful? What are some of the metrics that we can incorporate to make sure that in the short term and the longer term we are measuring success and continue to iterate so that we can continue improving and scaling. Over to you Ian. No I really appreciate the question and I appreciate the Defense Innovation Board's time today. I think one thing we have to do when we talk about incentives and we have to separate the organization from the innovators themselves and so what is the measure of success for the organization? Well I think the fact that a fellowship program that kind of offered really fast dynamic fellowships to people to bring them back into service as opposed to something like skill bridge where which is a great program by the way where you do this work you learn this industry and then you go out and use that during your transition it's much different to bring that knowledge and network back to your unit and so we have this this major offering that got canceled and I think that's a view of the metric of success of the organization to take something that was working and is part of the new NDAA language part of everything that's coming out of the National Defense Strategy and the National Defense Authorization Act and yet it died and it died due to organizational plaque probably. There's lots of different stories about why but I think we have to make sure we separate is the organization as a whole and that can be at the OSD level you know something like this could turn into a micro sector fellows program but you need to house something at a level that has the right authority and budget to make sure that it continues. We separate that from the metrics of success for the innovator we've got a lot of founders in this room we have a lot of innovative people you know on the call talking about this today and if we think about it the metrics of success for an innovator is not did your idea succeed because you you have to understand that 95% of your ideas fail now as long as you learn from those ideas that's one metric and the velocity of ideas and the velocity of which you're able to bring those ideas and learn from them I mean it's a huge metric of success it's also a metric of success not just for the innovator but back to the organization because if I'm able to continually bring multiple ideas up in this organization succeed fail good ideas bad ideas it doesn't matter as long as I'm able to bring them forward it means that I've got a culture and I've got an organizational structure that allows that to happen in the night reframe everything that you know my team has said today the fact that we don't money is great as long as you have enough to not worry about your family after that it's just being able to do great things and use the thing that you're good at to help the overall organization that's why our people serve whether that's active duty reserves guard or civilian force but they're here to serve and so let's let them do what they're good at. Thank you so much Ian so this in this section we heard about we heard from Megan talking about upskilling and re-skilling our talent and focusing on talent we heard from Michael Madrid that top cover is necessary but not sufficient and then we also heard from Ian on how we should be thinking about separating the metrics of success for the organization and make them outcomes driven for the warfighter and for the individual so we'll post here and open it up for questions Mike, Mr. Chair over to you to kick the Q&A here for this section. I think we've heard a lot today that will help inform our current studies. Sue Gordon do you have anything you want to say? Yeah just one quick one I love so many of these ideas I love the upskilling including senior leaders to be able to understand technology because generally what I experience is they view new technologies additive risk and sometimes it's not additive risk and that's hard to discern so it just kind of stops there but I would encourage you to I love the topic of risk I think it is a complex one and I think it's complex for innovators because someone who builds something thinks that risk is their thing not working and an operator often views risk as losing the opportunity to do something again so you might want to think about this risk in a bi-directional way maybe there's something that goes to the innovators to have more exposure to a different kind of risk equation that sometimes is what seizes the system from putting it in but I love where you're going y'all. Sue I'd also be remiss if I didn't say our thoughts are with you and Sue lost her husband about 30 days ago and you're nice to come and participate and thank you for everything you've done for this country and everything that he did for this country so we're a better country because because of this. Thanks Mike Jim was truly one of the great innovators in our community and it's because he never believed that anything should stop good out from come happening so well you're certainly keeping that going good deal for everyone watching online mark your calendars Wednesday April 17th is when the board will hold our next public meeting and we're looking forward to it until then all the best Marina over to you. Thank you Mike great conversation so we have a few minutes left and I wanted to highlight that we did receive a few comments they have been posted in in our on our web page and I just wanted to highlight that the comments we received actually resonate with the discussion today a lot around talent management and how can we maintain the competitive edge in not only the talent we have today but also the talent we need to bring in so that alludes to some of the comments that Megan was making we have comments about the adoption of new technologies and that pace and how do we accelerate that pace which is very much at the cracks of what we're looking at in the incentive study and also an idea about borrowed from the Australian Defence Forces about doing a 12 month enlistment and during the gap year for graduates high school graduates before going to college so some interesting ideas we encourage are the citizens and everyone within within the DOD national security network to continue sharing your thoughts and insights we the div is listening and so we look forward to to receiving those and and next we're almost at the very end of of our session here today Mike did you have any additional thoughts you wanted to share looking ahead only that Mike Mullen can add a lot but I just wanted to say that we're very lucky to be able to serve and help the country and help our military our security our future is in their hands and they are ill appreciated unfortunately but those of us that have watched them understand the sacrifices they and their families make so that we every day can go about our business Mike Mullen do you want to add anything no I don't think I couldn't I couldn't have said any better Mike I appreciate the ideas the the difficulty years how do you translate great ideas into you know operational effect you know tied to that mission and it is a huge challenge but but that's the that's the motivation for all of us here is to try to figure out how to support those that are making making our country more safe. Yes I was wondering if Charles had any additional thoughts you wanted to share Charles. Yeah I was just going to add that I was down last week at the Pentagon with Dr. Hsu and Dr. Hiem as well we are making progress on working with our allies and partners and they gave me many examples over 100 programs of where we've discovered technology at an ally and brought it to the US and manufactured it here and vice versa what we're missing is scale though because that's been driven by them personally and so it needs to be more programmatic but I was surprised at how much it happens there's some you know kind of embedded barriers as you might expect around ITAR and other requirements that we have and they have similar requirements regulations in their countries maybe those two things could come together but there's a lot going on and but it all boils down to trust so where we started the discussion around people on the ground and the ally being comfortable turning technology over and discussing it if you have people on the ground there which is how this happened and therefore a lot they build those relationships people start to share their ideas and what they're working on it and we've done it with our close allies obviously that's where you start with the UK and Canada and Australia but if we can have more people on the ground to our earlier discussions seems like things like that would happen beyond just the relationships in the culture you actually get to all sorts of technology and data sharing as well great thank you Charles and with that we are actually coming to a close for today's meeting we greatly appreciate the Defense Entrepreneurs Forum for joining us today we appreciate your time and comments and insights we hope that these are going to be valuable to inform our board members insights on the current studies and with that I'll turn it back over to you Mike for your closing comments had my closing comments and I think it's time for us to get back to work all right thanks everyone and we'll see you soon take care