 Good morning to everyone here and online and on Twitter and wherever else you may be following us. I'm Anne-Marie Slaughter I'm the president and CEO of New America and this is our panel on the technology deficit Attracting tech talent into government and civil society So I first thing I just have to say is this panel is the reason I came to New America. I mean in Microcosm that is really true. I partly to be able to be with this great group But really when I looked at New America and I was on the board for a long time before I left Princeton in The last summer and came on board what I saw when I looked at this place was the place in Washington that is I think best positioned to bridge technology and policy And here's how I see it and then I'm going to lead into this report and this Conversation but you know, I'm from the policy world from the foreign policy world But it really that that that Domestic international difference is not nearly as important as the digital natives non digital natives difference And in my policy world when I went back to Princeton the fact that I could use Twitter made me a guru Right made me a tech expert. I'm by colleagues would anytime technology would come up My colleagues would look over and say oh Anne-Marie understands that that's because I could use Twitter That was the level of technological Sophistication required to bump you into a different place in the policy world so and then I you know went and hung out at PDF for the personal democracy forum for two days and saw all this energy and all this excitement and all this Possibility mostly young people digital natives thinking about how to change the world using technology and putting those two worlds together Is absolutely essential The people who are in a position to make change who understand the traditional way policy works and I'm going to come back to that in a second And the people who really understand an entirely new horizon of possibility through technology so that intersection is What we're about today and what we're about today is talking about how do we get more technology or more importantly more? Technologists into government and civil society That's what the the Ford MacArthur report is about that's what we're going to be talking about and you know There's a number of points that right the current pipeline is insufficient that is clear I mean trying to hire people again at state who had real technological savvy Part of it was they're looking at the bureaucracy and fit thinking how on earth do I ever make a difference? Part of it of course is that they have lots and lots of options in the private sector and neither Government nor civil society can match those salaries or anything like it So we have to be able to offer something else and that's something else is the possibility of really meaningful work But that gets you back to the bureaucracy so, you know the Problems of salary of kind of creating that pipeline the barriers to recruitment and then to retention We did get some fabulous people in and some of them did some great work Jared Cohen's at Google ideas right and he was he worked with me He had all sorts of exciting ideas and he did implement them. It was impossible to keep him It really was now partly. Yes Google ideas is like a one in a million job I get that but just in general he felt like he'd done as much as he could do within the government But that's exactly the kind of person you need to keep or at least you need to keep coming in and out The other so thinking about this and one of the ways that I think this report is so helpful Is well, let's look at models from other fields, right? Let's look at places that work that do actually succeed in integrating technology With other areas of work and let's see how we can we can borrow from those And then let's look at how we do education and training and Critically and I know we're gonna hear from this culture change right because the biggest Difference that I found and again if you go back to my example of working in government and then Running a public policy school and being back in a public policy school and PDF the culture is radically different Right, we always say well, it's the difference between hierarchy and and networks the vertical and the and the horizontal Yes, that's true, but it's so much bigger than that It's it's really a whole it's it's what I would call the difference between the culture of the presumptive No, and the presumptive. Yes, the presumptive. No, you come to me You say, you know, I've got this idea and I say why would we want to do that? And I grill you and I ask you all sorts of questions You're not ready to answer and at the end of it I say well maybe and you walk out think of we're never gonna do it the presumptive. Yes is the culture of Silicon Valley It's the culture of technology. It's that's a cool idea Let's see whether that would work So you don't always do it because sometimes it's not a good idea, but it's a presumptive Yes, so that culture change the presumptive. Yes is not the culture of Washington. I think you can That's maybe the understatement of the century so That so all of those things have to be tackled But before we we turn to our our panel and we're going to talk about this report And we're going to talk about the work that that various of us are doing both in government and now out of government I want to leave you with one final set of thoughts, which is We do need to get lots more technologists into government and into civil society And we've got ten coders upstairs and a whole group of people around them. They are You know hugely helpful for all of our different areas of policy whether it's education policy or social policy or economic policy or foreign policy They're valuable not only because they know technology, but also because they are often skeptical about technology Which is also important right the non technologists often think great. We'll get an app All right, we'll get a program the technologists are often the ones who also understand the limits of what technology can do as well As what what it can do so you need both of those things But you also and here's where I would leave you and this may be the next report You also need to teach technologists much more about Policy politics and process I travel between here and the California and in DC Nobody really gets technology except for some of the people on this panel and Tom Khalil will be joining us and you know There's a there's a small group of us, but most people don't get technology then I go to California I do do a bit of fundraising in my job and in California people are just completely either blind to or impatient with or dismissive of Policy politics and process. It's messy. It's complicated. You have to compromise It's slower than they want, but I want to say to them guys I'm sorry right you cannot fix all these problems just with technology in the end. It's human behavior It's the clash of interests. Don't tell me they're no politics in you know I'm watching Silicon Valley the sitcom like everybody else. You're really going to pretend there's no politics. That's ridiculous There's the clash of human interest and there will always be the clash of human interests So we need to do both we need to get more technologists into government for sure But we also need to teach the mindset and the understanding of why politics process policy are in fact as Essential to making true and lasting change as a wonderful technological solution. So with those preliminary thoughts I'm going to turn it over to Alan Davidson, who's the moderator of our panel Alan is the new I love saying this that the the director of the he's the director of the Open Technology Institute and the vice president for Technology policy and strategy here at New America And he is not only focused on what OTI does but how we link what OTI does to every other part Area of our policy and he is going to introduce our panel and I get to sit and listen. So thank you very much Well, thank you very much for that that excellent introduction and overview of the issue and Welcome to everybody. Thanks for coming out early on a very non-techy time in Washington But we're glad to have you all here and have everybody out on the web watching and I'm Alan Davidson I'm the director of the Open Technology Institute here at the New America Foundation I'm also a co-founder of the MIT information policy project, which is co-sponsoring this event And in fact, this is the first event in Washington that the new information policy project in MIT has Co-hosted so we're very excited to be arriving in that way. So Thank you. And I will say it's it's this is a topic near and dear to my own heart Emory has laid out a lot of the the big issues, but we're very we're lucky to have this report now at this time This is an issue that a lot of us have been pondering for a while, but recent events have really underscored the challenges that the government has in dealing with technology and producing technology for the citizenry and We have at this moment an excellent report that gives us a Framework to think about these issues the future of failure report sponsored by the Ford and MacArthur Foundation Produced by the Friedman consultant group and that's going to be our topic for discussion today It finds after talking with many of the experts in the field some of whom are here today That in fact there is there is a gap There is a talent gap that we have to address in attracting As it says a technologically oriented human capital to government and civil society and that there are really serious issues that we need to deal with that Are ramifications of that talent gap at the same time It does give us a glimmer of hope That there are models for attracting this kind of talent and for having the kind of conversation that emory talked about That it's a two-way conversation between the technological community as it were and those of us who work in civil society and in government I Would say that the report is a little bit heavy on talking about information technology Although I think a lot of the lessons that it Underlines are broadly applicable and we can talk about other kinds of technology and government I'd say there's actually a good reason for that and that is probably because There's probably no other area right now where the pace of change is so high and where the the issues are so squarely presented to to all of us and If you look at the pace of change in technological adoption, it's incredible how it is accelerating I mean if you look at the The it took it took 55 years For the automobile to reach a quarter of the US population Telephone took 35 years television took 26 years The PC took only 16 years and the web took less than six years to reach a quarter of the US population If you think about that trend it actually seems to be continuing to think about how quickly So many of people are adopting social networking Wi-Fi You know wearable computing You know what's next right? I know got the Fitbit So and and we are that pace of change the reasons for it the underlying technology trends behind it Seem to point to a continuing if anything increase in this pace of change and that raises huge issues For how we react to it how we as a society make decisions about the technology That's changing our world so With all that as a backdrop I would say we have an all-star lineup today to help us think about these issues And let me introduce them. I'll start to my left Susan Crawford is a visiting professor at Harvard Law School She's a professor of law at the Benjamin Cardozo School of Law And among her many claims to fame In terms of being here. She was President Obama's First special assistant to the president for science technology and innovation policy, maybe the only one I don't know there have been others Broke the mold But she has seen this from inside government and outside. She's most recently the author of a book Captive audience on the telecom industry To her left is Dan tag Tangirlini who is currently the administrator of the general US General Services Administration GSA as we especially know it. Thank you so much for being here. Dan has a Long and distinguished career in the federal government and also in city government He did some senior posts had some senior posts at the Treasury Department But was also the Washington DC city administrator and deputy mayor So we bring both of those perspectives which we appreciate and his left is Ashkan Sultani an independent researcher and consultant who has been working on privacy and security issues in In the public eye quite a bit lately He's Was the main consultant for the Wall Street journals What they know series on internet privacy which was highly regarded and he's done a lot of work on the Washington Post coverage of the The Snowden affair so With this line of and I should say we're going to be joined by Tom Kaliel who is currently the Deputy director for policy at the White House Office of Science and Technology Policy a senior advisor for science technology and innovation in the National Economic Council did a similar tour of duty in the Clinton administration and has just been one of the really thoughtful long-time People in government who've been watching this whole issue of and working on this issue of how we get technologists and into government So we've got this all-star cast so let me start by just posing a general question to the group which is Well this report says that we've got a problem that in fact there is This lack this talent gap in government Let's just start by kind of testing the assumption Do you all agree that we have this gap? And is it a big problem Susan? Why don't I start with you? Well, we're here this morning, right? So there's a Converted We're probably speaking to the wrong people because everybody here already understands that there's a giant gap between The affordances of technology what's possible using it in policy and in the operations of governance and in Making civic life richer and the people inside their heroes inside people like Dan and Ashken who are to understand this and who are Trying to move Tech from a ministerial position. Oh, those are the machines that someone else runs into a seat at the table in policymaking and in making people's lives better And there are lots of ways that this report suggests we could build a pipeline of talent But it's going to require lots of stages. I've got another book advertisement I just finished with Steve Goldsmith a book called the responsive city which talks about the need to with strong leadership especially in cities and Bringing the millennials in and empowering government employees and using technology to carry forward and implement Technological and policy change. We're at this inflection point a very interesting point in the history of governance using Technology and being more responsive to citizenry that may actually Enrich and thicken democracy. So it's a great opportunity and we're at risk of losing it because of the internal structures and obstacles and perception Barriers where people think it couldn't possibly be engaging to work inside So that's our problem and I've got lots of ideas about how we might fix this But I really want to hear from Dan and Ashken. So I'm not sure how to follow that Other than to say amen. Yes, exactly where I testify But I think it's interesting if I could pick up on the idea of cities I think you're going to see a lot of innovation coming out of cities because there is this A Direct political effect of not getting it right and you talked about a an inflection point We're an inflection point in the way services are delivered in general and people's expectations At how services are delivered and how people go to market receive goods and so government service is There's going to be a demand for people to receive government service in a way They can receive almost any other service the always on always available Possibility of receiving service. I think in the federal government. It's a little harder You you're a little more removed in terms of your operations your service delivery from that political imperative And so it's there's going to be some Need that's going to come from the local through the state government up to the federal government where the expectation of Government services delivered through technology is going to really put pressure on us in the meantime You know, we're struggling with the ability to deliver the the basic services that we that we're required to deliver Demand is not going down but resource constraint is is coming way up the possibility of being more efficient Effective and delivering the services by leveraging smart technology is very clear. It's very there But how do you get it? How do you go out and actually make the connection between your desire and the result and that's through Smart people who know actually how to do it. It's through technologists The promise who do you ask in an organization that hasn't invested in in those folks and has difficulty attracting them? You know Ash kind of That's a great Discussion of the kind of government services angle But there's another piece to this which is also the question of how we as a deliberative democracy Have real conversations about technology now You've been deep in the weeds on the national conversation right now about the NSA and Government surveillance generally how do you see the issues around the technology pipeline playing into our ability to have a good debate about that Or do good oversight? absolutely, so I think you know what one of the things to think about is And I think Anne Marie hit it on the head is culturally We're starting to see this kind of rapid development towards technical infrastructure and the use of technology to deliver services but There's you know on many regards we saw that with phones and with roads and with other kind of infrastructure technical infrastructure But we didn't call them like roadologists or you know We didn't call you know we there were we just observed that there was a technical development and that It was important to know how infrastructure works in that context, but not to kind of separate it And I will say there's definitely a skill divide, but part of I think the value of Technical-minded people is the ability to demystify what the actual technology is and bring it back to key kind of societal Democratic or or whatever your political Bend is Kind of key societal Conversations and processes about the core factors some of which might be technical if you're talking about infrastructure and scale But some of it might just be around Other values that we have in the ecosystem that you can just openly debate and I think so to frame it as technologists and non-technologists I think does it disservice to understanding that it's just developing technical literacy and there's people in the audience You know lawyers and others that have technical understandings that might be Need to be supported and nurtured and developed as well So at the I worked at the FTC as well, and we had a handful of lawyers that were Silicon Valley background And they were fantastic. They weren't referred to as Technologists, but you would ping them and they would be able to they knew enough to how like the web works or how you know I worked on the privacy issues So like how cookies worked or how browsers worked to really engage and then you could have a technical person Solve that 20% you know kind of in the weeds part, but you could still kind of foster a good debate by just not Making technologists or technology like this religious thing where you have to be a priest to understand it But really just like it's a bunch of you know, it's a bunch of technical issues and you here's what you need to know Here's some important bits. Let me know if you have questions, you know kind of framing it that way. I think helps Why is it hard? I mean, I guess you know to find people who can do this Well, I want to announce the era of the hyphenate so there will be many more people in the future who are cross-trained as Having this literacy and being aware of policy and process and politics would and we talked about right now It's hard because the funnels going in don't reward hyphenates You know, you're either an economist or you're the IT guy No one seems to reckon or a few people there are enlightened people throughout government We're not dumping on them But it's rare to find an HR process or an intake process That recognizes that hyphenates are everywhere Especially among Millennials and need to be encouraged and felt that they they will have an impact I think Dan's mention of being able to see the impact of your work on life is extraordinarily important everybody's looking for a meaningful life and technologists hyphenate policy people want to be seen as Having our seat at the table, but we have to be able to create the processes that allow that seat to be created but I think those systems and processes have been developed and iterated over many many years and they've gained a certain Protection around it. Yes, you you described it right in your opening remarks. We're talking about the web So adoption took six years and it was about six years ago You know the PC's adoption took 16 years and that was maybe you know 26 years ago And so we have systems that frankly don't reflect the pace of change and it's very hard They can't keep up. It's almost as if they can't keep up It's almost like the answer to this is a technology, but that's too meta A group like this to swallow. It's very appealing for the silicon But how do you actually then get people who are willing to adopt it in a smart way? Think about tools that will actually help us solve the problem when you have this entire system and and the various different equities It's designed to protect that you have to protect as well I'll actually push back a bit and say that So with the exception of and we should kind of differentiate say developers people who are like building healthcare duck of them that are Doing database administration and kind of really you know again building roads or building You know building electrical grids and the people that are making policy decisions and kind of trying to engage in public policy issues that technology impacts we should kind of Notice that in fact there you could support the people doing the work already right instead of kind of pushing back and saying these infrastructures are in place to promote a particular agenda or a particular pipeline and it's not supportive of that the the technical growth you could just say that in fact, you will just Allow the people that are in place to to engage on the technical issues make some mistakes kind of try their best at kind of deciding on the issues and Get them basically put them put them in the deep water quickly As the technology is evolving kind of have them dive right in rather than kind of make it this very Thing that people are reluctant to engage on right so members of Congress are a good example They oftentimes you rarely see and now on a house floor a technical conversation because no one wants to look You know no one wants to be the internet is tubes guy, right? Nobody wants to just and he's actually he was accurate, right? It's a bunch of tubes in some respects, but no one wants to be that guy and so no one's willing to Kind of engage in debate Which is kind of silly. I think we're all trying to figure out what this new thing is We all are trying to like figure out how our iPhones work even the technologists I think the difference between technical-minded people is they'll butt their head against the wall on a new thing that they don't understand until they figure out and they have that confidence to poke and hack and Manipulate the technology and I think that's a different culture than people that are in government that are they need to have experience and process and understand Sorry, go ahead. I was just gonna say is in part of the solution to that problem actually having coders and and other highly technical people Working with those folks so that they can in a very safe way go down to the you know that smart Woman down and you know and then the other on the other floor and say okay So how does this thing work before I talk about it? So I don't sound like a We're still thinking of technology as a tool that implements policy And what I'm hoping for is that we get way beyond that that these things are deeply integrated and You we can't know policy. We want to adopt education climate change anything else can be divorced from Deep integration with with technology So you need the high-level people at the policy table who already are unafraid and already create leadership for integrating technology into has a better I mean the old story was you know the member of Congress who had never seen email before who never used the web before Didn't have a computer on their desk and now you know the president's got a blackberry and members of Congress are tweeting or at least Somebody's tweeting on their behalf I'm not sure it's gotten that much better actually. There's still a perception that there's a screen in front of me So the internet is broadcast things are being pushed at me and a deep Fundamental lack of understanding that this is the permissionless world that in fact you ask for things and they are created by other people Without permission that very fundamental point is not known by members of Congress that the internet is actually different from either broadcast or a Phone system, so we still have a big education. We have a lot to do and a lot of this has education as Generational change. Let's call it gently as just Demographic tied shift if this will happen, but we can accelerate the shift. That's that's what I'm pushing for especially in educational programs and I really want to talk about what's going on at Harvard and And and also bringing people secunding people in from the private sector who can infuse the territory with this awareness every once in a while And then leave they can leave after a couple years, but they go in I'm getting a deep. Don't do that How much of this is cultural I mean cp snow and Ann Marie Slaughter have both talked about the the divide this cultural divide and Ashken you said You know you use the words let people make some mistakes, right? We don't seem to have a real culture of risk-taking or mistake-making tolerance in government and that's very much a part of the Technological approach that we see from a lot of people How big an issue is that beyond that even how big an issue is culture and all of this in terms of attracting people Yeah, I think culture is the kind of the biggest issue in fact I think I mentioned it I had mentioned that 9 a.m. You know or the fact that this sounds silly to most folks But like you know wearing jeans. I know in America. You can't wear jeans, right? Like I Was told like you've heard it now Except when we're moderating panels sure sure that is like the fact of just culturally, you know like yeah in some respects It's sad, but technologists are kind of these these are not technologists with developers in this case We're talking about developers, but they're kind of these artist types that work on their own hours and kind of dive into things deeply and don't like the kind of bureaucratic Processing and some of it is you know little things like time and and You know clothing and this kind of stuff, but some of it is also around generally had some notes here by like kind of language for example the use of language the Kind of efficiencies for example like when I worked at the FTC I had to you know I guess it's been long enough like I stole my machine I stole a machine that wasn't being used formatted it like installed my tools Tethered it to the and that's how I did my research because it took six months for me to get my research lab set up And like there's just no and so I would just work around the system And so there's also the culture of just basically efficiency and like You know, why would you do it that way? Because you could do it this way and so bridging the culture of like understanding bureaucracies understanding That DC works at a different pace Understanding that the language for things are different. We're talking about the same thing I think those issues and and one final point which I think for me in DC At least I've observed is both in government and civil society The lack of kind of community right so each NGO has one geek in house, right? And I organize this if they're lucky right if they're lucky and I organize a happy hour Which is kind of like bringing together a bunch of the politics like the tech policy crowd And people are starved for kind of interaction and just water cooler talk and just you know Sharing notes and swapping ideas because in fact, you know when I was at the FTC I know a bunch of other technologists here. They're the one person at their shop That's the go-to kind of dictionary slash internet interface for the shop But they have no growth and no culture and no support internally as well to grow and to learn more things and Other than the policy stuff, but in their own technical community with their own vocabulary So I think that's another cultural piece too. How much of this is also You know, we've sort of jumped to the sort of what it's like to actually be in these spaces for the people who are either hyphenated or actual developers or technologists How much of this is even just about how we Get this group of these people who have this combination of talents I mean, is there a is there a training issue here as well just in terms of people's exposure to these issues You know, we're talking about the people who've shown up to be that one That one technologist in the media or the set of people who are working within GSA to think of innovative solutions How do we even you know? Does that pool exist out there? Do you feel like it's growing? So I think at some level we have to recognize a lot of this stuff is still so new that there's going to be an Inherent scarcity to the people who are really good at this stuff particularly the deeply technical the coders the the engineers The unhyphenated the the will or even you know heavily weighted towards one half of the hyphenation So the question then is how do you attract those folks? You can begin to develop a critical mass within your organization You're right if everyone's walking in and going into their you know designated office in a suit and everyone has to play the the role according to the You know the bureaucratic cultural established norms It's kind of hard to get someone past the first visit You know right never mind then a process a hiring process that may take three four or five six months They moved on and the alternative is like foosball tables bicycles and free life And I don't know if foosball tables if someone's like I'm going there because they have a foosball right right you know And if if that's the quality of the decision-making then maybe there is a big gap But I want to go there because there's a really meaningful mission There's something important they're doing is some way I can make a contribution and it might be worth You know waiting my way through whatever hip-deep swamp of bureaucracy and challenge, you know That might be worth it So there's a couple of big policy problems one of them is civil service reform another is a procurement reform These are huge problems for cities and the federal government to make sure that other than that we're done But but these all these things have to be worked on in parallel You can't have these these people coming in attracted to a culture you do your best and then they get Slammed by the various processes that keep them in their place these wonderful systems were put in place and here's comes the hero Right on cue We're just talking about fixing everything Well, but we've moved to the we've moved to your sweet spot, which is the solution space So but so maybe that's a good segue But okay, I want finished off fixing on policies and right then so I want to tell you what's going on at Harvard With the cities I really believe with Dan that cities are places where civic has meaning where people can see the impact of their activities And I've had been very privileged over the last few years to set up a very good relationship with City Hall And they're delighted to have meaningful projects being done by design school students Kennedy School students Harvard Law School students These are real projects not make work because they don't have the resources to right and mechanics Well, that's a start and they're wonderful people the urban mechanics that they're working on it But it's the strong leadership of the mayor then drives these these projects into reality Here's the next thing we need to do these students are all excited about working in local government We need to create ways for them to do that in a kind of a Peace Corps two-year way I want to call them the last mile fellows because they're going to come in and actually implement the projects that they started working on as students and Get it done and do the cross agency work. That's required this and this can come We need money for this has to come from the philanthropic sector also from the private sector get these people in so that They already know that serving in government is an honorable thing They've now learned that in graduate school get them the time to experience it and then go back out and spin in another time As we're sort of turning to this thinking about the solution space Dan You've got this 18 f group that is working in GSA. Can you tell us a little about that and whether it's we'll think about all those challenges You just described about a bureaucracy as code Right, why not get some smart people who know how to code to help you hack that and Understand how you can maybe take the thing that you already have and Find better and smarter ways to get it to produce different results while we're waiting for civil service reform and procurement reform And I've been working in government in some form for 23 years, and I'm waiting I think the trick is it's really the responsibility of leaders and managers and all of the employees to say well Wait a second is there some other way to get to the same outcome? And so that's what we're trying with with 18 f we were trying to develop capacity build some Build some critical mass so that we can have a better understanding of how we go to market and buy this stuff we have a place where people can actually try things build a minimal viable product and give it a shot and Frankly part of the the really exciting part is figure out how you build something how you build a startup Within the context of something as big and complicated and well established as the federal government Tom do you have Ideas that you can share with us about what's going on right now to address this you can you've been working in this for Well through to Democratic administrations for the last right 20 years almost yeah, so a couple things one is that One program that we started a couple years ago is called the presidential innovation fellows There's something that our CTO Todd Park started And lest you think There is any lack of interest in this For 18 positions we got 1200 applications, so this is more competitive than the IVs and So you're doing with the leftovers There is I mean we actually do ask them You know if you can't become a presidential innovation fellow would you also be Interest in being considered for other positions so so and the other exciting thing is that some of them show up Intending to do a short tour of duty and then stay So a good example of this is one of one of the PIFs is now the chief technology officer of the Veterans Administration And is going to be recruiting another class of PIFs to help round out her team So I think a couple things what is that we have a tendency to have this sort of binary view Which is that either you are a political appointee and they you know Presidentially appointed Senate confirmed Someone like the honorable administrator of the you know GSA a big deal Or you are a civil servant who's going to work in the federal government For 30 years and that is not everyone's cup of tea And what we find that some of the most innovative organization within the federal government Explicitly recruit people for a short period of time. So if you are at DARPA Your tour of duty is typically four years and you have an expiration date on your badge And that gives you a certain amount that gives you a sense of urgency because you are you know You are not going to have the ability to be at this amazing organization Indefinitely you have a short period of time. You're only fresh until 2016, right? Exactly, so I think that if And we the federal government actually has a lot of tools to bring in people for what we call term appointments, right? a year or three years or four years and so I think that we could do a fair amount a By using those authorities more expansively and be Doing a better job of publicizing all the amazing change the world opportunities that Exist within the federal government, so I think those two things would make a big difference And you know Dan you you in addition to all the innovative stuff that's going on at GSA and in the federal government You actually seen this from the city government point of view I mean how well did some of these things scale at that level is this going to be something that cities can really do as well or Again, I still think that what we're doing is we're chasing cities to some extent and in understanding the possibility that You know finding ways to really recruit people in either for a day through, you know, civic hack right hack the best festivals Through challenges And then through programs like the one you're talking about what sounds exactly like the PIFs actually so that sounds like Some chance to come to back down to the city government I really think that The big question is the one that Tom was raising is what are the tools that we actually have available to build enough critical mass Within the federal government that other people can begin to build the the community that they can begin to teach each other How you actually begin to overcome some of these things that people think are very tall hurdles Because it's mainly oral tradition. It's not actually it's all tradition and girl a warfare a lot of the time We want to find a way to systematize and just make it part of the water that this this is how things get done in ways Um, yeah, so I think that as we think about this another key area is how do we build up Interdisciplinary individuals and teams that are are facile both with the technology and to have a deep domain Expertise in in a particular application and so an example is What what are the opportunities to? Transform the way teachers teach and students learn using technology. I think there are huge opportunities but there are also Major barriers as well. So if we look at the market for educational software, for example, we have 15,000 different school districts We have lengthy adoption cycles So we're not getting the sort of private sector investment that we would like so we need people who can say all right what will it take to Create markets for learning software that is as compelling as the best video game That improves the more students use it and is as effective as a one-on-one tutor So those are the types of aspirations that we should have for technology to help address major societal challenges and in some cases You know changes in policy Investments in research are going to be required and we need the the people who can help drive those initiatives at a National regional and local level. I mean even just listening though even articulating those aspirations Even thinking up those aspirations requires people who have a good intuition about the possible, right, right? I want to support the role of the leadership for this that having someone who can speak for the agency for the program and say this These are the aspirations. It's also extraordinarily important that all our band of fellows and workers and hyphenates Needs people at the top who can Support what they're up to. Yeah, I was just going to echo that if you if you go to Tech crunch disrupt and just say that I suspect, you know next in six months. You'll have a bunch of Potential startup ideas around it and potentially innovative models that try to experiment on those topics I think a lot of it is Communicating what the needs are and what the narrow path is like everyone knows how to you know Or not everyone knows but it's commonly known how to go from say a startup to a company in California, but it's not known how to go from a prototype to You know implementation a small city on educational software And so just kind of someone that knows how to navigate and there's a pitch of what the needs are and communicates those needs I think is also a huge opportunity This this does come back to then some of those fundamental challenges you raised about these business processes systems and structures That inhibit the ability for us to connect, you know great ideas that are frankly possible with the technology we have and And then the you know so the grant-making process your German process and how do we Actually find enough interest to wade back into that that part of the value chain where not a lot of people want to go and spend a lot of time and Figure out ways that we can improve those systems and processes so that there can be that the connection between that great idea and And the policy implementation I've seen this work not in government per se, but I've seen this work around government in two areas one is around Kind of zero-day vulnerability development and development of say what that is really Cyber security software Both for attacking and for protection around a conference that happens every year called black hat and def con and In Las Vegas in Las Vegas But there was a program by the government to essentially fast-track Grant-making for research projects and cyber security and it was you know This is the cyber fast-track initiative Yeah, and so it allowed a developer or a researcher like myself to quickly get something like 50 grand for a year to Look into Some critical software vulnerabilities and it worked very well I think it got put in it it would fast-track both the government process as well as the clearance process or Mudge did this much did yeah, so He was you know from this community that became a one-star general randomly, which was kind of awesome So that's one the other one that I've seen that's working really well is this Scholarship for service basically a cyber corpus program where you can essentially Go to schools that teach a very specific type of again cyber security kind of tuition and A curriculum and you can essentially Have your tuition covered for the time that you attend school if you go to a federal agency Or you know federal related agency like MITRE or you know research group And do that kind of work now. This is for offensive cyber research So this is like developing weapons and developing, you know the companies that are recruiting here are the NSA's and the miters and the CIA's Trying to develop cyber capability, but the model is is sound which is like you will teach you some cool things That you should learn and will pay for your tuition if you come to government and apply those things And so I think that's also a good model to think about We've talked a lot about government one of the things that the report also talks about is civil society and then the public interest Community and how it fills its needs You know Ash Khan or Susan you work very close all of you guys probably work closely on a time You work closely with a lot of civil society groups. What do we do about building capacity there? How do we attract the kind of talent? Whether it's hyphenated or otherwise into that community. Well, it doesn't seem as if there's an enormous appetite for this I mean the students that I run to they don't trust government They would rather work for civil society and they have these skills They just want to be confident that they'll be paid if they have student debt They want to be able to pay it back They want to be confident that they won't run into a sort of disultery life because the nonprofits aren't actually having an impact So but if we can identify and advertise This is the advertising point that Dan's making make sure that everybody understands these jobs are out there and how much Fun they are and that they're well paid. I think actually there's an enormous number of people But good missions And that's true in government and insults Yeah, I mean so the challenge is so it's Usually not well paid usually kind of not a lot of career growth on the technical side Yeah, and not a lot of community as we said so then what's the similar challenges to what we're facing right? And so the question is what's the draw and so the draw is potentially things like Convincing people of impact Kind of educational different respects really you're gonna find the people that go to these roles are First and foremost interested in whatever the kind of NGO or civil society goal outcome is or will be and And and the technology side suck it So they're just coming and so you have to connect with those people because it's really hard to have another I don't I don't see for It's very hard to draw that talent to these orgs to say like come and come to our 20-person shop and be our tech guy And on a really important issue, but like do you know rather than all these other opportunities, right? Sounds like the challenge is the same in the government You need to convince folks that they're not just a bolt-on that they're not gonna be a mechanic They're not gonna be sitting downstairs with the boiler. They're actually gonna be part of the process Yeah, I'm sorry and they can make a difference if I didn't if I didn't clarify. Yeah, that's absolutely One of the key so again, we've talked about I've seen a number of even Kind of centers at various universities kind of speak of yes We want more technologists and what they mean is we want developers to implement some great great idea The lawyer had we wouldn't and they're like we want to do the next verdict And so we have a great idea and you should come and build it for us And that's a very different thing than saying like come participate. Help us shape what we do help us identify the important issues That remains a problem as opposed to make sure my email works. Yeah, right Different question which is um Diversity is a huge issue already within the within the technical community that stem community You know, how do we work to improve diversity? even as we build this capacity in government and in civil society I Recently gave a talk to the computer science department at Harvard and there was one woman in the room I don't know how that works I it seems to me we have these deep problems with gender diversity that we have to take on from the earliest age starting in Junior high school high school get just get everybody seeing themselves as potential future engineers Um racial diversity is also an enormous problem. These problems seem so baffling that I'm going back to net neutrality Just kidding We can talk about that too, but We'll pick an easy problem. How much this is self-reinforcing are the good examples and good role models And we haven't done a great job on our panel. Yeah My 14 year old daughter though has been able to you know, gain some interest in this through going online and using code Academy Yeah, and I'm wondering if we can begin to break down the barriers and make it something that can happen in the safe zone and say hey I'm actually good at this or I like this. This is fun and find a way that it is less kind of Maybe initially institutional bait institutionally based and then they can pursue. It's curiosity. Yeah, I would I would say it's We've kind of touched on earlier. It's the same problem of getting kind of lawyer types and government engaged, right? It's it's taking it out of the Kind of technical weeds and saying this is like I can frame a Technical issue in a way where all of your eyes will glaze over and no one will have an opinion or want to engage Or or have any thoughts on it or I can take the same technical issue and provide it in a way that you know kind of appeals to you you might have an opinion on you might actually have some expertise on and and Being able to do that. I think we'll we'll engage people that would normally feel reluctance to dip their toe in they would say It's this scary thing or that there's no reinforcement for me to try it or experiment And I think both from non expertise as well as kind of diversity issues You just want to bring people in and make them realize that this is like we're not talking about Super technical things here. We are in some sense But we're also talking about kind of the fundamental issues that you care about And you have language for and you have great expertise on you should just engage and not be afraid to engage. I Want to open it up for questions in the audience But maybe I'll just ask one last blanket question for the panel Which is is there anything else in the solution space that we should be thinking about that we haven't touched on yet that? There's a lot that's in the report if there's anything else that's Leaping out here that we've we've missed in terms of low-hanging fruit or otherwise So one of the ways in which the government creates a specialized workforce is that it invests in research? particularly university research, so You know what happens when a professor gets a grant is that they recruit Graduate students and postdocs to do the vast majority of the work And so, you know, that's why for example, we have a huge Workforce in the area of biomedical research And so one of the things I think we have to start thinking about is that we have some agencies that have vast Capacity to do that, which is why we have a specialized workforce in those areas and we we have other agencies You know particularly those that work on Domestic issues and issues related to social justice and poverty alleviation that have almost zero capacity to do that And so I think that's what one of the reasons why We don't have some of these communities Well, I should just say as somebody who works part-time in a center that's trying to build Capacity in this area. We would welcome more federal research dollars And actually that the meta point here. Yes universities are platforms for exactly this kind of development And universities are incumbents that are threatened by the advent of technology just like everybody else and they've got silos just like the government agencies do and Finding ways to help universities to see themselves at the nexus of public and private interests Developing this capacity seems genuinely fruitful and it's very rarely happening these days There are some policy schools that have deep interactions with the public sector not a lot Some also try to bring in a few random token technologists not a lot This could be much more intentional and spread across the country in ways that would be extraordinarily helpful and foundations could play an important role Yeah, I mean Rockafell the Rockefeller Foundation created the discipline of molecular biology Whitaker played a critical role in creating the discipline of biomedical engineering And you know what one could imagine foundations playing a similar leadership role in this area All right, we've got hands in the audience many hands. Why don't we start in the back? Well in the middle back David Robinson hi Such an important conversation and so glad that we're having it So just by way of Introduction or I want to point out a kind of a pattern that I've seen that I think is important that cuts across OTI and 18f and and what I do which is a small public interest focused tech expertise Consulting from which is that not only is there an important sort of skill set for Technologists as Anne Marie said learning how policy works. I'm learning how to engage But the flip side of that coin which I think is also very important is the capacity of policy organizations to use Technologists and to know what kinds of value technologists can deliver and I think that's been there been some real success Cases as Alan said when they've been senior people who kind of have the stature to come in and say Here's how technology can help you change the way you think about your policy problem You know like for example at the FTC there have been a series of chief Technologists who've had that role which I think has been extremely productive But I think it's it's hard when you have somebody coming in at a more junior level into an organization not accustomed To this kind of innovative activity where that person may not have the stature within the perception of of the organization that they are joining to Realign how that organization thinks thinks about these problems, and I actually think that's one reason why we've seen sort of centers of Tech expertise that work across a number of different issues or different organizational units and I I would say that I'm That I think that's a that's a model that's that scales well because those organizations like for example OTI is developing this cross disciplinary cross Vertical expertise in bringing technology into a policy area and then doing that in education and health and economics and And so on I think that's a big part of the solution Comments yeah, I think I think you actually within that you point out a very interesting point that for most Most big organizations technology is is operations and operations is risk Right and and you go there if you have to but remember there be dragons Right. Yeah, you don't know what's going to be on the other side Of the horizon line and so by having you know part of what we're trying to do is through 18f is actually Create an insurance policy that allows the organization our Organization and the organizations we work with to say okay. We can go and try it We have people who are inside who are on our side who can help us figure out these problems Build minimum viable products teach us how to to do this You know approach problems in a different way before we go out in a big way and and potentially, you know have a big problem Steven Levy from wire there's been some oblique references to the people of the Existing culture and there's a gorilla warfare or the contracting infrastructure Can you be a little more explicit about those who may not actually want to change things and and seem to be In a pretty good position so far in actively resisting change or deflecting it As the current system works sure when doing research for the responsive city I talked to lots of heroes in different city halls around the world and Basically, you can see this as tribal There are different tribes inside city hall who have their interests to protect and they have job security to hang onto And bureaucratic need to keep an issue inside their shop and not share And so in order to break all that up you have to have leadership at the top who understands that world The world has to change and the technology is part of policymaking not divorce from it not ministerial And that the people who understand tech Should be raised in the organization should be given dignity and respect So it's a big it's just tribal warfare and you have to have a chieftain who can beat heads and make sure that everybody else cooperates I think the other issue is is What is the incentive structure right? So if you're in a situation where if you try something new and it goes poorly that leads to GAO IG congressional hearing Washington Post and if things go accurately well necessarily I And if things goes go goes go well there is not you know the equivalent of equity or stock options And so I I think so I think another important thing is To actively encourage Responsible risk-taking and to promote and celebrate Innovation so one of the things that HHS does for example is that the the secretary has a program called HHS innovates Where they celebrate some of the most successful projects that civil servants have been working on so I think that's That's one of the things that's important. Do you think we'll be able to get to a place where we? Embrace or at least tolerate failure a little bit more or That's a little harder I think the trick is to make it to make the failures not the epic fail. You know figure out a way You know that you could you could build it small and fail small and come back and see if you can move towards succeeding big but To Tom's point our budget director at GSA coined this great phrase about federal tech the philosophy too often is if it ain't broke Don't make it better Right if they result if it's giving you the result the problem is you miss Generation after generation of upgrade and iteration so what happens is the ability to then go back and and fix it Ultimately gets harder and harder because it becomes more and more ingrained into your business process It becomes more and more expensive to pull all the data out and so what we need to do is figure out a way that we can more aggressively and continuously iterate and evolve Particularly in an environment where we're really reaping the kind of the midpoint of Moore's law here where things are changing so dramatically So quickly and culturally like in terms of drawing talent or retaining talent Nobody wants to be the guy working on you know Windows XP and 20-year-old technology But these are a culture of people that are drawn to innovation to new technologies to figure out new things And so if you're supporting some you know ancient architecture or ancient infrastructure And your job is on a day-to-day basis to be laughed at because you're like, you know, you're using punch cards like Or using a floppy disk to do nuclear launch codes, I don't know if you guys saw that Don't make it better Hey, I think that there is one community that has been left out of this conversation and I think it speaks to some of the problems with the ACA and that is the community of people who are Have difficulty actually accessing digital services. So, you know, we can talk about the digital divide But it seems like in that conversation those folks were something of an afterthought Like oh wait a minute people might not actually be able to get online and register for health care I mean, I think it's important for two reasons one is leadership and the other is diversity So, you know in terms of leadership What can we find in local communities in terms of people who understand the issues of those communities? And how can these folks we're talking about in government and in civil society work better? With local leaders and folks from the community, especially from underrepresented And underserved communities and the other in terms of diversity Unless we have put those perspectives in these conversations Like sort of the atmosphere is never going to be welcoming for those folks So I'd love it if some of you could speak to those issues So I think that's where the the local government relationship is really so strong So when I was working as a city administrator, we're very interested in that We had a we had a population the city that actually have very high literacy illiteracy rates So it wasn't even just a digital divide. It was really a service divide The trick is the question you have to ask yourself Is it possible through smart use of this technology that you can overcome some of those divides? And can you free up resources so that more resources can be pushed into closing that gap? So in that sense what I what I worry about is it becomes a Fight between people who are actually advocating for the same outcome And about whether it should be digital or not digital rather than how do we deliver the full? Spectrum of services to people who actually need them and I think in the ACA actually you saw You know there were people actually physically out there signing up folks We we needed to rely on them more frankly than we wanted to Because we we didn't have the full capacity available initially through the digital side And so the real trick is how do we how do we? Maybe prevent ourselves from arguing with each other over issues that we are all committed To delivering services and figure out ways that we can provide the full spectrum of services to people who need it Hi Frank Torris with Microsoft you talked about the role of foundations in the government I was wondering about what you think about the role of the private sector in some of this I mean we can certainly encourage and companies have encouraged You know education to promote science and technology and math in the schools to help build up You know a crop of engineers emerging can certainly support fellows and NGOs and other places, but it seems like there's a gap between You know fostering you know more participation in the government perhaps you know coming from the private sector and in some of this Well seemed to me that someone who has reached his or her mid-career point They're feeling relatively confident in their job in private sector should be given the opportunity to take a sabbatical of some Kind and go in and spend a couple of years and then come back out This Tom can make it happen. He can make anything happen in the federal government You heard it here, right and cities can make it happen. It's just will and interest and suspension of fear to Encourage people that's part of your career. That's part of your civic duty and fabric is to go in and come out With a newfound respect. It also helps people then be reminded They really need to upgrade those systems because it's embarrassing when this woman or man shows up And it feels so uncomfortable what they're faced with so I I think they're much more could be intentionally done along those lines Oh, hello back there. Hi, I'm Gwen Costa and I work for Dan Full disclosure so he may not want to answer this question the I'll later today The question I have oh Started you all mentioned in different types of ways, which is the ongoing challenge that we have in the big dogs of Procurement and HR. Yeah, and we talk about that as being a fundamental change And I've only been in government for 10 years. So Dan lapsed me in that space by a long shot But I was in the private sector. He doesn't let me an age And so my question is how do we make those fundamental changes? I've been government for 10 years I've been frustrated for 10 years. I don't see it getting any better than 10 years Cuz it's not going to be an easy fix It's not going to be something we're going to be able to bolt on or make this quick change So what are and I don't know maybe it's touched on in the report But what are we actually doing and what can we do to make an impact in this space because as far as I'm concerned in 10 years, it has not gotten an inch better So I say frustration is actually a really great emotion because it means you can see a better future You just haven't figured out how to get there yet. So as long as you're frustrated and not desperate. We're still good It's a desperation call me before that But I think the trick is to recognize that we need to continue to have this kind of dialogue and point out What those limitations are but then not stop and say well I've got these limitations I I can't evolve and I frankly think that that's where you know to get back to this last question That's where the partnership with business can come in Can business actually see the procurement system less as a protection of incumbency and more as an impediment to innovation? And work with us to get the you know political support necessary to go and maybe fight some of the bigger Fights associated with that. We also then need to find ways that we can break down the the oral tradition barriers You know, oh, you can't do that because it's against the law and actually, you know Partner up with the lawyers and say why is it against the law? Help me understand. I mean we found through 18f and having that group and Allowing them to go and learn from the best practices of other agencies and develop some of their own that Some of these barriers that we we took as insurmountable are actually more mythical than actual Yeah, can I just build on that so? There's a book on procurement from a lieutenant colonel by the name of Dan Ward called fire Which stands for fast inexpensive restrained and elegant and he does everything he does he's doing under You know the federal procurement laws as they exist Not as we wish they exist And so a lot of it is a is about mindset and he is the only lieutenant colonel that I know who is who's done comic books To explain his approach so we need to lift people like that up who right are You know figuring out how to make it work in in the in the current system and and spread some of those approaches Got about five minutes left. You want to go to the back there, and then we'll move we'll try and get through everybody real quick so hi, this is Megan Gray and I just think it's fantastic that folks are taking this issue head-on it is is Just going to be fantastic to see everybody address this directly and try to find solutions on that point having read the report which I also thought was very very well done and talked to some Very insightful folks to collect the information and insights But there were two points that I thought were missing in the report One was that there was nothing discussing the clearance process so to go into government Especially for the positions that require technology. It requires top secret secret all sorts of different levels and even just the the the Most basic level to get into government is a Trust level and that is still very invasive. I think for most people to Fill out those forms and know that it's going to be more than just checking to see if they have a criminal record I mean it is really Intimidating so I was surprised that the Solutions haven't addressed that Hurdle and then the second point is this idea of conflict of interest and I think the idea of having Suconments is fantastic that Experience with how things work on the ground with companies is incredibly helpful in government but for a lot of these positions you Once you've gone into government you are conflicted out you have a lifetime ban of Going back to private sector and working on or working for a particular company For example, I know that the FTC has had some problems getting technologists in because of the technologists are working on an investigation of you know a privacy Issue for you know name any company then that technologist who may be working on the Investigation would then have a lifetime ban of going to work for that company and that's a career Altering change that most folks aren't going to want to take on but so I wonder if the succumbents Can't can actually work in practice and just one last Comment I'd love to get your feedback on is this idea of the revolving door Because I think revolving door is good, but we all know in DC that it has a very negative stigma So how do you address that yeah comment on any of these? Procurement and civil service reform clearance processes next and and I think these are all subspecies of the same problem These things were set up for very good reasons and they become overgrown with kudzu and vegetation And they're just you know you can't hack your way through them and they're getting in the way of effective and efficient government processes So we have a lot to do and some people will work through the process as Tom's suggesting and some will try to fix it I you know I am hoping that this a condiment idea can be done in Context of a waiver from some of the conflict of interest rules. I understand that's really difficult I just want to give you a counter story. I talked to some Harvard Business School professor who told me if I don't have a conflict I'm not interested I That's the private sector view. No conflict no interest, right Okay, how about we have one back there and two in front and maybe we can Do that and then these two and then we'll wrap I can be quick. I'm drudge a blend from OTI So one of the things I just we've been talking about tech a lot And actually one of my pet peeves as a designer in technology is that we talk about it as if tech can solve the problems When in reality, we're actually talking about an ecosystem of people who work in technology So I'm wondering if you guys can talk about that in your talked about a little bit in the policy technology space But we're not necessarily talking about what makes up the fields of technologists And that it's if you guys could talk about that a little bit and how your teams work maybe within 18f and OSTP I think that would be really helpful for people to hear about Well, I personally don't think that technology per se is a solution to any problem I really needed to have some idea of what the outcome is and even just an idea I think smart technology then tests that idea against you know people's reaction and and and and whether you succeed or not so it really comes back to a another area where we have a huge weakness actually in our in our Employee base, which is really an analytical ability an analytical capability So between the technologists the coder who can write the code the the lawyer type Who can come up with the policy? There has to be you know the the analytical capability to understand whether you're actually having an impact or not so part of what we're doing is Less less trumpeted than 18f is building a data analytical capability within our office of government-wide policy And we believe that between agency problem identification The ability to go in and develop tools and then the ability to evaluate the relative performance of those tools and solving that problem We might actually have a virtuous cycle. They'll begin to better apply technology to our problems So a good example of this ecosystem that you're talking about is the government Making more information available in both downloadable machine readable format holding Data jams that encourage technologists and domain expertise to identify potential applications that could take advantage of that and solve real problems in areas like health Foundations like the Robert Wood Johnson foundation supporting the health data initiative and then Companies and nonprofits actually developing these applications that have been identified So you're absolutely right that it really does take this Ecosystem of different players with different skills To really create value and just a really short answer So the kind of both your question and part of Megan's question Comes down to just sophisticated like sophistication in early days, right? So that there's this general Word technology or technologists that's kind of blanketed. So you if you work on technology for Google you're forever Conflicted from working on technology for Google, for example, or if you're a technologist you do all of this technology magic stuff I think as we get more sophisticated will identify that there it's siloed and there's specializations and there's design and there's user interface design And there's data analytics and there's a bunch of factors with regards to what it means to work with technology And as the vocabulary develops, we'll you know We'll develop both conflict rules and we'll build titles and and resource calls that are that are more accurate and more Kind of true to the nature of the work and to respond to the pet peeve This is all about humans these stories are all about people and so focusing on design and the human computer Interface and just our relationship to this is also part of the ecosystem and I'm very Attuned to that and very interested in broadening the conversation along those lines Okay, why don't we lightening round if you guys can both ask your question and then we'll wrap Excuse me. Hi, Joe Hall chief technologist at CDT. I Have had a lot of success in teaching people about technology and sort of Bringing people to the tech bringing policymakers to the tech the thing I've had continue to be frustrated by is Teaching technologists about policy and trying to get them to care and I remember taking a cyber law class in Berkeley in 2001 from Pam Samuelson. There were three computer science students in that class We aced it and those are the guys It was three guys in this case that I know would be specifically interested in doing more like that So if anyone has tacit or things you've run into where you see that spark Let me know because I've been living and eating and breathing this and the Freeman report was awesome I've been thrown at everyone and so if you see anything like that, let me know Graham lampa from State Department in public diplomacy Where we're the sort of the folks at the State Department on the cutting edge of technology But I'm the only technology person in my strategic planning office so I understand that that the Barriers there my question was about contractors and contracting generally and sort of the body shop companies as well and the way the degree to which the availability of personnel Contracting which we also saw in the case of Edward Snowden where people aren't necessarily as vested in the organization This also sues into your point where I was shaking my head about people who sort of parachute in from the outside Whether they're at the State Department Franklin fellows We have triple a s fellows or even piffs for instance Not being seen as being invested in the organization and therefore being kept at arms length Yeah, and we generally especially in public diplomacy are hiring contractors with technical expertise And there's no way to bridge between the technical expertise and civil service Because of just the division in your status, right? So then no one who has technology background has opportunity for advancement to get into positions where they can actually have that that that Sort of managerial oversight even if leadership is on board that big middle of the organizations Aren't being penetrated by people with with technical knowledge So I wanted to have one thing one question on the degree to which the availability of contracting and smaller government perspectives and contracting that way has actually reduced Interest in civil service reform Throw something on top of that Goes to the HR issue as well Which is you know for me to work at the FTC through regular channels took forever as a contractor takes like You know a tenth of the time and I can get in quickly and do a small work So it's it's kind of a double-edged sword To have contractors that get in quickly but then don't have this so I want to give a him here to the long-term view Teddy Roosevelt made his career on civil service reform if we are always going to the very short-term answers to these questions Like find a way around it get sneak some guy in we'll never fix this problem. We have to confront these giant policy issues but I think actually the The bigger issue is this this issue of focusing on these FTE limits as if that's very important rather than asking What's the cost of government right and I think one of the things we're working on through the president's? Management Council is actually developing benchmarks of what is the cost of technology in each agency and that's the all-in cost Part of it is to expose the bigger than a bread box question. They're like why are you spending this much? What are you getting for but more importantly so we can ask ourselves? How can we best utilize those resources? My view is look if we can get someone smart in as a federal employee That's ideal if I can get someone smart in as a contractor that'll work to the real trick Is to get the smart people in working on our difficult problems as quickly as possible so that we can begin to resolve them On that hopeful note, I'll just say obviously huge challenges But also some really interesting ideas about what the solution space can look like we've heard a lot of good thoughts about 18f presidential fellows Innovation fellows all the things that are happening within government and civil society and I will say on a on also on a hopeful note I think if we were having this conversation 10 or 15 years ago Well, we wouldn't have been able to have this conversation the level of sophistication within government and civil society has increased a lot and it's demonstrated by the sophistication and Real commitment to these issues that we've seen in the panelists that we have here today, so thank you all for being here Please join me in thanking our group