 All right. Good afternoon everybody. Thanks for coming. My name is Adam Furtado. I'm the Chief of Product for Kessle Run, a U.S. Air Force initiative to revolutionize the way that the Air Force builds and delivers software. I'll stand over here. All right. So all your opinions are mine alone and not of the U.S. government, DOD, or the U.S. Air Force. Speaking at a cloud of foundry summit after Keith Sreeny is a little intimidating, I want to make it clear that I'm not a technologist. If you have questions about our pipeline or our release engineering strategies, I can put you in touch with the right people. I'm a career intelligence professional, dabble in organizational management, sprinkle a little IT on top. So in some ways, I'm a microcosm of what cloud foundry can provide, technology, complexity, abstraction. So in the same ways that our developers are able to focus on their products, I'm able to focus on the things that are important to me, our larger digital transformation and our people. People are the most important thing to me and people are why Kessle Run has been as successful as it has been. So while we have been focusing on our products, we've also been focusing on other things that have enabled our larger digital transformation, things like our security and testing and acquisition and things that Brian and Tori talked about a little bit earlier today. We think that we've built a recipe for other DOD organizations to utilize in our work with working with Pivotal Labs and utilizing Cloud Foundry and thinking about our people and our organization a little bit differently. We are in the business of transforming the way that air warfare is fought and we started very modestly with one single application with Pivotal in trying to transform the way that we optimized tanker planning or air refueling planning. Brian gave a really good rundown of this application earlier today. But he also mentioned that modest efforts can have major impacts and we found that with this tanker planner application we turned it into an optimization software that helped us save the Air Force about a million dollars a week, helped us stop deploying about two airmen into harm's way away from their families just from the efficiencies that we found. So as we move forward we're going into more complex areas to truly try to optimize the way that we plan, execute the war moving forward. So as the chief of product for Kessel Run I certainly do care about our products and I really want them to succeed. Now I'm concerned about that but I don't think that products are the most important thing. The most important things to me are I think that great products are an outcome of the culture by which they're developed within. So I think that it's my job to develop a culture that allows for people to have growth mindsets and are able to learn and grow and I think that the great products will be, you'll inherently come out of that if we do that in the right way. So some of our successes to date are talked about a little bit earlier this morning. The first DOD team to achieve continuous delivery to Sipernet, our operational network. This is a huge step for us, for all of us who worked within the DOD. Getting our actual capability to the warfighter has been a real challenge. From an infrastructure perspective we're in the midst of building an 8 node private cloud to modernize what the Air Operations Center is utilizing and planning the air war on. Again we're transforming policy testing security. We're disrupting the accreditation process with the help of some folks in higher-ups in the Air Force. We're focusing on changing the way that we think about products as well with the help of Pivotal. They've helped us really adapt the way that we think about the capabilities that we're providing. Focusing more on products and not projects and using those lean startup principles to focus on products in a different way. One of the major things I want to talk about today is more of our cultural transformation. It's been one of the more important things that we've had to focus on. We've also tried to focus on our people and our people transformation. So far we've turned 70 airmen into product managers, product designers and software engineers. Utilizing things like lean startup principles, user-centered design and extreme programming. When I say we've turned them into that I mean that fairly literally. We've kind of had to start from scratch. The Air Force doesn't have those type of skill sets and roles inherently on its own. So we're going to focus on different things to try to identify folks who we can grow into these roles. So we focus on identifying things like people who have user empathy or a growth mindset or the ability to get out of their comfort zone and try something new. We've had some pretty decent success with that so far. Isaac Taylor said that we're turning buzzwords into reality and PowerPoint theory into operational software. I think a lot of times when we get into forums like this with commercial industry and DOD talks about DevOps being innovative a lot of the industry probably scoffs at that. But when you take innovation as the definition of the introduction of something new what we're doing really is innovative and it's a massive change from the way that we were working before. It hasn't always been this way though, right? Like the DOD hasn't always been lagging in the innovation department from the internet to the microchip to duct tape. The DOD was kind of out in front from an innovation perspective. One of the things that's had us fall behind a bit was the lack of R&D spending moving forward. As you can see here, in 1980 the Western world spent about $240 billion on scientific R&D. The Department of Defense was one-sixth of that. As we move forward 20 years, the Western world spending in that department went up by 50% but our spending in the DOD went down. And also a full disclosure, this is my first time speaking at a conference like this and I really want to have pie charts to build my credibility. Boom, bar graph. Cool, so what this is showing here is to follow up on what we just talked about, the commercial software industry left us in its dust in R&D spending in a lot of ways. Apple in 2016, since they spent the same amount of money in R&D as the entire defense industrial base, all of our defense contractors combined. If you combine Apple, Google, and Amazon, they outspent us four to one. But the R&D spending isn't the only place we've kind of struggled with our innovation. It's also been really hard to even get things out into the field. Tori earlier today talked a lot about what it is from our acquisition system and the bureaucratic nature of things to get our systems out into the field. She showed a graph like this. We're on average right now of about eight years, it takes about eight years for IT systems to get delivered within the government. That's not even taking into account the requirements process on the front end. So we're taking literally a decade to design a system without any real meaningful user input and providing systems to warfighters that they no longer need So ACC commander General Holmes said years of institutional risk aversion has led to the strategic dilemma we have today replacing a 30-year fleet on a 30-year timeline. And so I know this to be true because I was one of those warfighters who really failed to get the capabilities to the warfighters. I spent 10 years utilizing some of these systems or really developing workarounds to do my job in spite of our systems instead of being enabled by them just to show you that I'm not kidding. This is a picture of me in 2011 using our systems. So not only was I a warfighter before but then I got myself into the acquisitions community where I became one of those guys who was giving the worthless systems to the warfighters. So I kind of know both sides of these things. Sometimes when you're inside the machine it's hard to understand the damage that you're causing outside of it. But there was an optimism in General Holmes quote as it continued this thing. He said, this dilemma also provides us an opportunity. And we think Kessel Run is that opportunity. Kessel Run was a team of people who were tired of working in this way and were trying to find avenues to work within the constraints that we're in to get the capability out to the warfighter a lot faster. I think a lot of people this week are talking about how Cloud Foundry has enabled their business outcomes. What I want to focus on is how Cloud Foundry has enabled our people outcomes. One of the things we struggled with as a leadership team in growing this team we really were concerned about the lack of inherent organic engineering and software development talent within the Air Force. The Air Force actually has our own software development enlisted AFSE or job code. It used to be in the thousands. It's dwindled down to under 500 software developers whose primary duty is to actually write code for the Air Force. So this left us with a challenge to build technologies and build cloud-native applications. We were really concerned about how we were going to grow this from this core group of people with the engineering talent that we needed. But then we realized as we did our research that commercial industry has already dealt with this as well. We've had people from the banking industry, insurance, highly-regulated bureaucratic entities have focused on how to grow their talent in the same way. We learned a few things from them. We need to, in the same way that the banking industry had to get more software developers than bank tellers, we need to transform the Air Force into a software company that delivers air power. We've done that in a few ways. We have a pretty good recipe for it so far we think with Pivotal Labs with Cloud Foundry and thinking about our organization in a different manner. So for Pivotal, Pivotal has been huge for us in enabling our team and growing our workforce. Our model of pairing with our product managers, developers and designers have allowed us to identify people who have that growth mindset and the willingness to want to learn something new and pair them with industry experts to really grow that expertise. So they're instilling a lot of lean principles, we're thinking about problems in a different way, trying to maximize value, eliminate waste, think of things in a lean manner from a minimum viable product perspective. We're building a product foundation with them and a bunch of teams all over the country right now at various Pivotal Labs locations. We're working on different problem sets and then eventually as we grow this core cadre of folks that we can organically have to move this forward in a very real way. And they're also growing our organizational leadership so they're not only focusing on our product teams and growing those people, but they're also helping us out in learning how to actually lead an organization like this moving forward in the right way. So from a Cloud Foundry perspective it's really lowered the barrier of entry for us, abstracting a lot of that technical complexity. We are very much understanding that we don't have the inherent technical acumen that exists elsewhere in industry within the Air Force so Cloud Foundry has helped us concentrate on growing our people in other ways. It's also increased the available partners that we can use really no longer do you have to be a defense industrial based company that has heavy domain knowledge and understanding of the third party systems. It's kind of opened the door to other smaller more innovative companies in some ways to also come work with us and work with the DoD but it's provided the underlying foundation of what we're doing. But the third thing I wanted to talk about is how we're thinking about our organization in a different way. Google with rework did this study on finding the best way to identify the traits that made good managers and employees. And if you notice the top seven things here are all soft skills and the STEM skills came in at number eight all the way at the bottom of the list. In our work we kind of found we've kind of landed in the same kind of area that was more important for us to as we looked for folks to work in this way to find people with that learning mindset and the ability to grow more than it was people who had the inherent the technical skills already so we can grow off of them. So as we move forward Google also did a study based on team effectiveness and these are the five things that came out as the most important thing for an effective team. An interesting thing here is these things match up very well with military culture inherently and the things that we know as military people and as veterans. So we'll go through those. Dependability so dependability is inherent to military culture. There isn't a time where you're not relying on somebody else in the battlefield. It's the same way in our organization when we do modern software development we have this balanced team model where our product managers are relying on our developers or relying on our designers and vice versa. We have this culture where people are trusting of the people that they work with and help us move product forwards from a really substantial level. Structure so instilled in us is this kind of need and desire for structure and discipline for our veterans and for our Air Force developers. A lot of times when we get senior visitors who come to our lab and come to Pivotal they get distracted by like the t-shirts and the sticky notes and the whiteboards and it gives the impression that we're working in this wild wild west type of environment. When in reality all the practices that we are actually doing are very much disciplined from our test driven development which we're very dogmatic about to our schedules 906 standups 1230 hard lunches 6 p.m. hard stops. All those things are very disciplined very structured and it's fit like amazingly well with our Air Force folks who are used to working in some kind of a structured environment that we've become used to. The third thing was meaning. So there's been a ton of studies that have gone around about this generation they care more about meaning than money. A lot of the questions we get are based around retention. So you give these airmen a bunch of skills that they're more valuable in industry now how do you keep them or more so how do you get people from industry with the government if we can't pay them enough. We have a hypothesis that's yet to be proven wrong that if you provide somebody with the right mission and the right meeting in their work combine that with a great culture and work environment. Most people will be willing to take a little bit less money in that environment not everybody but in a lot of cases that ends up being the case. Recently I was in San Francisco and I saw this billboard and there was this app about that helps you optimize the route you walked your dog on so they could be on grass and not on pavement so I thought that was a really good idea but those guys can probably pay a lot more than the government can to revolutionize air warfare. But at the end of the day when you think about what you accomplished that day we think that there's a big difference there when you go home and think about what you did in working with us so that's been something that we've been really concentrated on. From an impact perspective what we're doing with Kessler Run is inherently good regardless of political leanings or your general thoughts on wars we are being way more efficient saving a bunch of taxpayer dollars. We're being much safer by being more precise and providing the right data to make better decisions off of. We're actually preparing and growing our airmen to have skills to better transition out of the military whenever that may be. So doing a lot of good things and providing impact at a very high level. So if you look at those top four things there those four are inherent to the military and to airmen and warfighters. So we've found that in trying to develop this workforce to work in a different way from a technical perspective the inherent things we need the inherent skills that will help us grow into an effective team are already within us with one fairly major difference. Psychological safety. So psychological safety is based around the feeling of acceptance and safety in your ideas and feeling to give your ideas without fear of backlash and feel comfortable. This is something that the hierarchical nature of the military doesn't usually promote. But it's something that we've tried to promote at Kessel Run. So there are some ways that we've tried to do that. The first is we've gotten rid of all the uniforms and made everybody talk to each other on a first name basis. This seems fairly trivial but it's kind of a big deal. The hierarchical nature of the military and the top-down leadership is imperative to warfare imperative to be successful in the battlefield but it thwarts innovation largely. So even the concept of uniforms is to strip you away of your individualism where the individualism is where we actually get the creativity that we need to solve really complex problems. So we've removed the uniforms on our operational floor everybody's on a first name basis and level set the playing field so everybody feels free to ideate, share their ideas and be creative. At least to our balanced team model that we worked with Pivotal on. So we have product managers, product designers and engineers all working together with shared ownership over all of our products and shared ownership over all of the ideas and the decisions that get made. So from this perspective we have we've had E4 product managers working with majors and captains who are developing code. All of them are on a first name basis and the level set is not a fear of group think or the highest paid person in the room or the highest raking person's idea winning out. And it's not something that's new to the Air Force or new to the DoD. We have this air crew model it's based on a meritocracy largely so the best person for the job wins out and then whoever's doing that job has the power to do so. So this isn't something new it's something that our airmen have really like adapted to and have really enjoyed they've enjoyed working in this way and feeling more free to be creative and have a positive impact on the idea. One thing that we are working on is developing a feedback culture. Those of us who have been in the military or currently in the military we know that feedback is almost inherently negative most of the time in the military the way that we describe it so we've had to really work hard to make it so that feedback can be positive can be constructive and want people to seek out feedback to figure out where their blind spots are to make feedback no longer look like that and be a really productive thing and we have some work to do here but we're kind of on the right path. So I say all this to say that we are on the right path to creating environment that we will be able to grow our engineers and grow our developers and Brian mentioned earlier our airmen have hope again that's not just from our developers that's also our war fighters as we work with our users they're starting to feel more a part of the process they're starting to feel like they've played a part in where things are going we've gotten very good feedback so this is just one of those things where traditionally war fighters have felt ignored I felt that way when I was one so now that they're a part of the process we have really short feedback loops they can really drive where these capabilities are going which has been really exciting for us to see So three weeks from now we are opening our own software lab in downtown Boston to basically have a place that mimics Pivotal Labs and other tech industry companies where our product teams can really thrive we're calling it the Kessel Run Experimentation Lab or CREL and we're taking with it all the things that we've learned so we're taking you know leaving the cubicles behind and taking the open workspace concept we're leaving the we're taking the sticky notes and the white boards leaving behind the staff meetings leaving behind the uniforms and taking our t-shirts and hoodies right? The most important thing we're taking with us is what we've learned about culture what we've learned about running an organization in this manner what we've learned about having people with growth mindsets and trying to grow our people we called it an experimentation lab not only because we're experimenting with our products and with our technical implementations we're also experimenting with the idea of culture and organizational management So earlier this week Eric Schmidt was testifying to Congress on the state of DOD innovation he said that the DOD violates every rule in product design if you read the rest of it he wasn't really just talking about thinking of things in a minimum viable product perspective or build the next most valuable thing it's also just kind of the entire ecosystem of how we are building capability and growing our people from an innovation perspective so with that, that is all I got are there any questions? Thank you