 I'm Joe Castle, and I'm going to talk about the U.S. government's effort, Ospo of Code.gov, the U.S. federal source code. And my colleague, Sarah Kopu, was going to do this with me, couldn't make it, unfortunately. But I did actually speak with her this morning, and I said, tell me what you would say if you were here. So I'll kind of filter that in throughout the talk as well. This was definitely, for us, it was a labor of love, which is interesting for government. I feel like people in government are definitely mission-oriented and are passionate about what they do for citizens every day. But there's some programs that kind of resonate more with you than others through a long career. I'm sure it's the same for a lot of you folks. And this was one where we don't even do it anymore and we still talk about it. That's how much it meant to us as we were doing it. So I'll talk about it from an Ospo perspective of the lessons learned and sort of this arc that we went on with the Ospo. Code.gov is still there today. It's somewhat on life support. So it's interesting. I didn't put this in the description because it almost sounded too negative. But it's like, what does it look like to start a program, basically get it in flight, like you're doing well. It's maintenance and getting community and more code bases and things like that. Getting agencies to actually publish and understand what code is, getting executive leadership to understand code. And then you get to a point where priority is shift. You lose money. You lose executive sponsors. And typically in the government, it's not financially motivated as much. I mean, yeah, we try and be good stewards of taxpayer dollars. But the reality is that Code.gov started with an administration priority. It started in the Obama administration in 2016. And it was one of the major aspects of digital government and open government. And administration changed. Priority shifted. I'll talk about it a little bit more. And then basically it was less money, less emphasis. Agencies still have to follow the policy, but they don't necessarily have to publish. And there's a bazillion priorities that they have to deal with so they don't do it as much anymore. So it's still there. I say it's on life support. There's not a lot of interaction with the platform and publishing code. There's still a lot of open source code use. And I'll talk about that. But yeah, so pretty much went through this whole arc. And then now we're all gone too. Like I'm in industry now. Sarah's in industry. Everyone else that worked on it that was in industry, the contract's gone. So there might be one or two people there that kind of other duties as assigned and they'll update the site or something. But no one's really championed the open source for government these days, unfortunately. So let's dive in. I have a couple slides. I won't do too much about this because you can just read it online. So probably the highlights of this is I wrote my PhD about government open source software. So my research is on it as well. There wasn't a lot about open source government or open source software in the federal space. So I feel like I at least contributed in that aspect. And then the other part is I had worked at the White House in 2013 and 14 and I was working on open data initiatives and then later on in 2016 I was at General Services Administration which is one of the major agencies for the US government. And then the White House asked me again to run code.gov, which was pretty cool. I was doing it for my agency and then they reached out to me and said, hey, will you do this? We actually did a panel together at, if anyone remembers the, let's see if anyone remembers this, the O'Reilly, Oskan. The last one that I went to was in Austin, Texas. And then the last time I think I spoke about this was in San Diego for Linux Foundation. If you all remember, if anyone went to San Diego, and then I was going to go to Portland but that all, a member of the code.gov team that I did not know was here. That's awesome. Yeah. All right. So let's dive in. So real fast, like I said, I went to industry, I was with GSA for 18 and a half years, I was in the US military for three and a half years, so I have 21 years of federal service, public service. Now I'm with a company called SAS, SAS, it doesn't stand for anything anymore as far as the name goes. It used to stand for statistical analytics software but now it's a SAS. It's based out of Cary, North Carolina. We focus on analytics and AI and like predictive modeling and things like that. We are in most of the federal agencies, US federal agencies, biggest customers being US Census and Social Security Administration, I have to think about that. A lot of state and local, a lot of education, different universities and definitely industry, all the big banks and things like that. If you apply, if you ever applied for, I'm sure you've done it, you applied for a credit card or a loan and you find out within minutes or seconds that it's approved, that's our software doing the, you know, through predictive models to give you that return. Been around for 46 years, quite a long time. So with SAS real fast, and this is kind of interesting because I was talking to Sarah this morning and I was like, I spend most of my time, so I'm a huge fan of open source. I write code and open source Python, did it for my dissertation and still do it today for some analytics work and it's interesting, I actually spend most of my time telling agency chief information officers why they don't want to use open source. So it's kind of an interesting, it's like a debacle, it pains me in some cases, but there's reasons to do it and reasons not to do it. There's reasons for proprietary software, there's reasons for open source. And that's all I'm trying to do is just educate them, not say don't ever do this. Like yeah, do that and then integrate with a good proprietary software product like SAS and then you can have the best of both worlds. And then we also have, we do open source with SAS as well and we're moving more in that direction, so feel free to check out our repositories and all that good stuff. We're on GitHub. So let's dive into the hospital, I'll stop talking about SAS, we'll get into federal government source code. Let's talk about federal source code for a minute and I trace it back to, this is the part that I said I actually wrote like in my dissertation, but I traced this back to actually World War II. We put a lot of funding into laboratories, a lot of military based laboratories and Grace Hopper is in this picture here, as you probably know who that is, but she did some work on cobalt compilers at a Navy laboratory and basically would go, it's almost like an education conference, like would go to a conference, have her code and she would distribute the code, I guess it's all in paper at this point, right? You distribute the code and then other people chime in, do some things on their computers, put it on paper or whatever it is and then they share or they send it to her later and then she would put it into her code. So there's a book about this and I can't remember the name of the book, but it's Keith, I want to say Bayer or something like that and he basically said it's the precursor to open source, right? We all know by the name open source was around 1980s, so obviously it's pretty before that for sure, but the idea is that you could get others to come and help you with your code and that's what folks would do and they were doing it from different labs, like you had England and it wasn't even just in the U.S. Then fast forward to the 80s, I was just talking about this last night, the Homebrew Computer Club and remember the Altair? Someone asked me last night, can you still buy an Altair that actually works? Does anyone know if you can buy an Altair that works? You can buy fake ones like you're doing your Zoom virtual meeting and you put it behind you just to see if anyone even knows what it is. But anyway, there were hackers in the good sense of a hacker, right? Just developing code, working with hardware, all that good stuff. And then for the federal government, like most organizations and industry, we would consume a lot of open source in development and software, but we weren't necessarily publishing what we were building with that open source. We weren't giving back to the community, which me personally, I take issue with that, right? Like you're using taxpayer-funded money to build something, but you're not giving it back, but anyway. So we do that kind of like everyone else. So in 2016, I'll do this really fast. This is sort of the nerdy government stuff that gets really boring quickly. But in 2016, the federal government issued a policy. So if you don't know the U.S. federal government, the way to get agencies to do things is to issue policy. And it typically comes from the White House and then the major executive agencies have to follow it. And it basically told them to do three things. It was right to run source code policy, basically how you're going to work with your code, identify the code, right? So create an inventory and then update your acquisition language to capture custom code as it's being developed. Because for the U.S. government, when we do contracts, it's typically written into the contract that the vendor holds the code, not the U.S. government. So what was happening is they would in turn resell the same code to the government over and over again. So we're in essence buying the same software over and over again. There was a provision within the policy that said release at least 20% of the code as opens source software. I don't know how they came up with 20%. That was always the big question. I don't know how they did it. I always wanted to do open by default. Nobody would ever go with that. It's too far out there. That policy lasted for three years. It was part of the pilot that they called it. And then the policy went away, which was also kind of the lessons learned of like, you know, the main policy is still there, but that portion, that pilot of open source went away. And I worked with OMB later, and I actually rewrote it to put it into another policy and they would just never do it for whatever priorities and reasons that they had. Which was unfortunate because once the open source portion went away, then there's nothing telling agencies that they have to actually publish any open source, which was too bad. But the policy's still there. They're supposed to have inventories and update the acquisition language and all that stuff. And that was kind of done. Some did it better than others, basically. So code.gov is still there. I think this is what it still looks like. This is an older image. It used to look different. It changes over time. We had some pretty good projects on there from the different agencies. Some highlights are Lawrence Livermore Lab had a program called SPAC, which was a package manager for high performance computing. Had a lot of downloads as well. That was one of our shining stars. Another one was the US web design system, which basically gives you code and templates for websites for the US government. And I had seen a number one time that for every time someone uses US web design system, they save like $100,000 in building a website. So I don't know how they got that. I just saw it in a document somewhere and I was like, okay, I guess that works. Oh, the other thing I was gonna say that I missed on the previous slide was at the time, this was 2016, 2017, there was some research that the US government spends $6 billion, that's with a B, $6 billion on software every year. So I don't know how that breaks down, but that's a heck of a lot of money that we could be using open source and probably saving some money on that and at least giving some of that back. At the time, the US IT budget for all the major agencies, like Department of Justice, State, GSA, et cetera, at the time it was about 80 billion, eight zero might have been a little higher, like 90 billion, half of that is for Department of Defense. So our Department of Defense and then the other half was the civilian agency. So there's a lot of money in IT in the federal space, really big organizations. Like I don't know it offhand, but like our Veterans Administration, their IT office alone is in the billions, like 10 billion, I don't know the exact, but it's huge, right? Like you can imagine, our whole company at SAS is like our annual revenue is three billion. So the IT department at like VA is like three times, four times the size of our company. So just to give you some scale. So for lessons learned, I kind of broke this down into three, basically three main areas based on our strategic plan. So we went through, we had changed over time, we had changed the people and leadership over time and what have you. And at one point we had gotten, the US federal government uses a lot of contractors and at one point we'd gotten to more feds and less contractors, just because we didn't have as much money for the contractors. And so we sat down like the feds and the couple contractors we had and we said, okay, kind of new day, let's day one, right? Like let's set up a strategic plan. What are we trying to achieve as a program? There were like five of us at this point. So at our height we were like 12, 13 maybe. So we were less than a half at that point. And it's like, what are we trying to achieve? How do we make ourselves relevant? How do we get agencies to play along to do what's in the policy and then publish more open source software? And we basically focused on three areas. So it was community, platform, and of course policy. And so I'll talk about each one of these, from a lessons learned perspective, and then sort of the overarching theme as well. By the way, we developed this in like 2020. I use this like all the time, this little slide. And usually when I show people they're like, can you send that to me? I wanna replicate that for my program. So if you ever wanna hit me up on LinkedIn, I'll send you this PDF. Okay, so let's talk about community for a minute. Now some of these will pertain. So some of them are high level like program perspective. Even though we'll focus on the program, they're pretty high level. And then some of them are more like at the repo or the engineering level, right? So it kind of varies. For the community it was really about how do we get agencies to understand the policy? I think there's one more. Yeah, understand the policy and really engage with the public. So what I noticed over my 18 and a half years is that a lot of people are making generalizations. So this isn't everywhere. But a lot of times folks that are in government don't necessarily wanna work with folks that are outside of government, right? There's almost this sort of, how to explain that? There's sort of this like gotcha factor. There's a risk adverse, it's like, my job is to focus on this program. I don't wanna be a people person. Those sorts of things. And so for this program it is very outward going, right? Like we had to go to agencies and work with their leadership and get a sponsor at each agency to basically like take the charge, understand what we were trying to do from a policy perspective, understand what we were trying to do from a code perspective, which in the federal government a lot of the federal employees aren't as technical as their contractors. They tend to focus more on project management and contracts, right? Contract management, project management. They tend to not be technical. So I'll give you an example. I was talking to an executive. He used to actually run GSA and I had met up with him for lunch just like after he wasn't doing that anymore. So he ran GSA for George W. Bush, the president at the time. And I said to him, he said, what are you doing now? And I said, well, I'm running code.gov. And we're trying to get people to publish it. I say all these words and he's like, I have no idea what you're talking about right now. And he goes, so code.gov, he goes, what is that? Like law, like code that's law? So we have codes that are a law for us in the U.S. And I was like, no, no, it's like software. It's like, you know, you're writing software. And he's like, oh, why wouldn't anybody want that? It's like, you're not necessarily my audience but this is the whole point, right? Like, yeah. So anyway, so a lot of the technical ability comes from the contractor staff instead of getting people on board to understand. You know, it was a very niche thing. We had a sister program at the time that had started in 2013 called data.gov. You may have heard of that. And that's basically where we put our open data sets, right? And that actually started as a policy is the M1313, so it came out in 2013 and became law, the data act, right? But open source, now what's interesting is that you do see some bills being suggested and recommended and what have you but open source never got that attention, right? Just kind of a niche, it takes a certain audience to understand the value, even just what it is. So anyway, so make it easy, right? Work with the community, whether you're working inside with your agencies, your organizations, or you're working outside with individuals that you wanna interact with the code. Obviously be responsive, build relationships over time, some pretty high-level stuff here, be inclusive and all that stuff. So some of that pertains to the repos but then some of it pertains to the actual program itself, right? Like trying to get more people involved, garner interest and all that good stuff. So I just, you know, a couple screenshots of the repos here. We kind of went crazy when we built this and we had like repos upon repos for like various components of the, we actually went to MP, I don't know, I've never seen this in my engineering career at all but we used NPM and I'll show a screenshot of it later. We made our whole website and everything that made the website and that API that supported it, that whole thing was like a package in various packages. I was like, this is crazy. So I would spend all my time keeping all these things updated. Now I was like, why did we ever engineer it like that? That's like the most ridiculous thing I've seen. So don't ever do that, we're to the wise. There you go, there's a lesson learned. Don't ever do package upon package and don't do like 10, 15, 20 repos for one website that, so, you know, the maintenance headache. And then we had automated, I'll just say real fast, we had a lot of DevOps automated testing and stuff. Oh my gosh, so we had different environments like staging and testing and production, et cetera. We would, it was crazy, like trying to keep all the packages updated, trying to do all the testing and every time you would push something it would kick off the same thing. So we were in essence doing everything like five times or something, right? And it could be a simple like, I changed a word on the webpage, now I've just kicked off every single process. And every package that's out of date, now I gotta go and update all, it was crazy, crazy ever engineered. So don't do that, don't be that silly about testing and engineering. Okay, so let's jump into the next one is the platform itself. So like I was saying, we had an API, so we had a harvester, this was pretty neat. So we had the website and we weren't hosting the code, right? It was a pointer in essence, right? So people would give us their JSON file. We would, it was just an inventory, right? And we would take that and then we would display that on our site. And the goal of that was to get individuals to come, like if you wanna find government source code, you go to code.gov. That was the whole intention of it, right? And with that, then you could see what projects are out there by agency and then you could click into that and it'll take you to GitHub. In most cases, it was GitHub. Some cases, it would be like an agency website or a different place. In some cases, it didn't go anywhere at all. But we built this harvester in this website. And I was talking about the DevOps. I said modern and then I told you the story about what not to do. I said, yeah, don't be that modern. Obviously documentation, code of conduct, contributing, license, all the standard things. And we always had to run, I'm sure most people do, we had to run everything through our legal office, especially code of conduct. And we were like, what happens if someone comes and they don't abide by it? Like can we just totally, and we're the government, so a lot of times like, no, you can't really do that. We have to be open to everyone, right? Like you can't pick and choose who you're open to. And he had said, this is our lawyer at the time, Dan. And he had said, well, has anyone done that? You know, has anyone done anything that's broken this code of conduct? We're like, no, no, not yet. I mean, everyone seems to be fine. And he's like, we're like, what would you do? And he's like, well, if you ever get something like that, send it to me. And then we'll figure out, because we can send them a nice letter from the legal office at GSA and tell them to calm down or something. So I was like, ooh, okay. Luckily, we never had to do it. But yeah, here's the package. Nothing really too crazy here. Just the API documentation and the package information as well. One thing I don't think I put in here is to talk about analytics. So we should talk about analytics here in a minute. So I guess one of the big factors, and this is, yeah, pretty good. I'll stop at decent times so we can do a couple of questions. But one of the important factors to Sarah and I, when we were talking today, I was like, it really came down at the end to executive leadership. So having the leadership buy-in. And everyone says this for everybody. You could say this for any program, right? Like any product you're building, any program. And it's so true. It's like, huh, that is true, right? Like it's leadership buy-in and resources. And for us, it was money so we could get people so that we could continue to champion this thing. So if you don't have those two things, the other thing that I noticed is that, and I'll say this, and this might be kind of controversial. So don't hate me when I say it. But in some cases, because like, at least where we were, our Osmo, our code.gov was not always well understood. Like I give you that example with that one executive. And it's an easy thing to kind of push to the side almost like an overhead function, right? And so you have to find a way to make the program important to people that make decisions. Because if you don't do that, then at some point you're just another program that gets swept aside, right? So you have to find a way to like embed, value prop, right? And do that with the leadership that then show value. One of the things I didn't include here, I still have a couple more bullets, but the metrics. One of the things that we can never figure out how to do, and I'd love to hear from others on this too, is the return on investment, right? And even as a government organization, we would look at that and we'd be like, how do we measure this, right? Like we can tell the stories, we have so much code reuse. At least when people would tell us, right? We could see the number of downloads, we could see the clones, we could see the forks, right? But that doesn't mean anyone's actually doing anything with it of value. So it was usually people had to tell us. It was all anecdotal, we had to get the stories. So I talked to GitHub one day, I was working with their policy office and I'm actually a GitHub star as well. They have this program for like super users and we sign NDAs and they give us like sneak preview of like new stuff that's coming. It's pretty cool actually. And then they use this as like a focus group on the stuff. So if anyone's interested in that, check that out, it's GitHub stars. And they hold their big event. I think the, Sarah just said the CFP for GitHub universe just came out. So anyone that wants to speak at universe, it's always in San Francisco. So it's close to the HQ and all that stuff. But anyway, one of the things we couldn't figure out was the analytics. So I said to Mike one time, I was talking to Mike who runs their policy office and I was like, you know, is there any way, he had the same issue, right? We talked about the same thing. Like how do we know who's using this stuff and how much does it save them? And then, you know, that's a good thing for us as well. Just to say we're saving the rest of the government $3 billion because now they're all using these, you know, open source code and packages and things. And I was like, is there any way we could do like MPM or like GitHub now, I forget what they call it, but there's like a package like tracker. Like you can see like who, you know, who's using it because when you do MPM it puts the package within someone else's code. And then, you know, it tracks that number. And he's like not really because then you're gonna be you know, you would in essence, like you'd put something in your code that would send data back to a different, so it's almost like you're, yeah, you're overstepping bounds of like now you're in other people's code kind of thing. So, of course not. But we always wanted to know and like at one point I did like the napkin a couple times and I was like, okay, if you have this many downloads and this repo and you know, but it was all just like made up stuff. You know, I was just trying to figure it out. But part of the reason why I was trying to do that is because I was trying to say because we have a program that any costs a million dollars a year, we're actually saving the federal government, you know, one billion dollars, right? So like, you should invest more in us and we'll make it even bigger and we'll keep this thing going for a long time and all that good stuff. But I can never make a good like strong enough case for that as well. I just didn't have the metrics. It was hard to get the metrics. It was all just stories and that only goes so far when people want quantitative data. So unfortunately the arc, you know, started off with the policy, it was great. Had a big kickoff, the US CIO who was appointed by President Obama, Tony Scott, went out to a conference, main stage, I think it was GitHub Universe actually, main stage, touting was awesome, right? Good launch, held maintenance for a couple of years, had a platform, you know, went out and did a bunch of events, had agencies playing along, we had up to like 7,000 repos at one point, which we'd always tracked that number. We were like, well, that's pretty cool. Like, at least we have some growth, right? And then that was going well. Administration changes, priority shift, which happens every four to eight years for us. And then the plane starts going, you know, we're losing our people, losing interest, open source policy goes away, the pilot part of it goes away, which rewrote it, they wouldn't use it for whatever reason, couldn't get good metrics, right? Like all these things compounded, and then, you know, we started losing people, and then finally we were just like, okay, if we're gone, no one cares about us anymore. You know, why are we still here? We loved it, we still would love to see it come back. I have doubts it ever will, at least from the open source perspective, for Code.gov itself, although if we could get some of these laws, or some of these bills passed, there is some open source bills, like software bill and materials, has become a big one for us. There's something else I saw the other day. It's his GAC, which is the committee, I forget what it's like, Homeland Security, and et cetera, et cetera, but they have a couple of things that have passed, actually passed the Senate, but it hasn't passed the House, and it has to pass both, and then the President has to sign it. So we'll see if we get there or not. But the changes didn't help, it was kind of another nail in the coffin. There were some, all the different program things, but yeah, we were busy going out, trying to do outreach and drum up interest and really get it off the ground, and I think we did a fairly decent job with that. So hopefully I gave you enough lessons learned. I still have swag, by the way, I don't have one out here, but one guy on our team, T would do a lot of our communications, and we got swag, we would send people t-shirts if they contributed to our repos and things that, you know, fun stuff, right? And we got these little cord keepers, which were awesome, and they sent code, and I still have, I took the last bag, and I have them on all my wires and stuff, but anyway. So always a reminder, I think I have one of the last stickers on my laptop, so it's sort of the end of it, but that's pretty much what I had. I'm happy to do questions. We're at 11.30, I think we get to 11.40, so 10 minutes, and if you don't want to ask publicly, that's fine too, I'll be around after, as well, and I have microphones. Did these work up here as well? Yeah, let me just grab one. Just switch it on. Oh, gotcha. Any questions? I could repeat it, but that's always hard to do. Yeah, thanks for the talk. What's your relationship between this and the US Digital Service? Is there like a strong overlap, or is it really different? So it's different real fast. The US Digital Service started when healthcare.gov didn't go so well on launch. The website, not the policy. It always gets connected like, oh, the policy failed, still there, websites there, it works fine now. So they brought in basically healthcare.gov didn't go so well for the deployment of the site, right? A lot of people headed at once, technology wasn't there. We brought in the administration, not me, brought in a whole bunch of folks from various tech organizations, Google, Twitter, et cetera, right, to come in and just help out. They knew a lot of people in Silicon Valley and other places and said, come in and help out, right? Got it working, and then they went away, right? And then OMB was like, OMB is the Office of Management and Budget within the White House. There's like, I don't know, 12 different offices. OMB says, why don't we actually, that was a really good thing for government, where we got talent to come in and do service for a period of time and then become term appointments, right? They come in for two to four years, do something. So they started the digital service as a team of almost consultants, basically, that would come in on term appointments from various places and then help government on sort of the thorny issues, like veterans, healthcare, benefits claims, like the big things that we're struggling with as government. That organization ended up staying around still in existence today, obviously. They would embed different teams at different organizations. I actually stood up the digital service group at GSA as well and they would embed and then work on various projects at the highest level with the agency administrator or the federal agency. So they are proponents most times of open source software. So what they would do is consult agencies about you should be doing these things. You should be publishing software and kind of helping us as well, but it was totally different, like separate program offices as well. Sorry. Yeah, that's one last question. So when you are looking at open source tools versus like vendor selection, just an example, like in the data engineering space, a lot of open source tools are not super great and everything else is like the better tools are in vendor selection. Does the government take that into account like for the preference for the open source tools or how does that analysis them? So yeah, good question. Yeah, it depends. It's a big government. There's a lot of different offices. So it's like typically what happens when we buy software is well in preference goes to the people who are buying the software on like type of software and how they're gonna do what it is, how they're gonna use all that stuff. And then there's also the scissors out right the security office where they have to review things and scan things and do. So a lot of times for CIO like centralization, they wanna like centralize on so many products, right? And they want the visibility, the control, to see what's happening with the open source. So it just depends on like the way the acquisition's done, what you're actually looking for as a product. What I saw in government just kind of puristically is like a lot of people that were using open source were obviously technically capable because they could do it. But it tended to be more for like small applications, Python scripts, write things like that, some packages, websites. It was pretty basic development with open source. And then the other place, if you look at like enterprise software, it'd be like, are there any open source products that could, but some of that they didn't have, there was no acquisition process. So it's kind of, and you don't really know what's there. It's kind of like you don't know what you don't know. So you can't really do much with it. Does that help? Any other questions? We have seven minutes. Hi there. So that was a project that was done years ago. But if you were given the same problem to do today, how would you, what would be your approach to that doing that today? Doing code.gov and federal source code and yeah. So I think the trajectory would be similar, start with the policy, make the policy where things don't end. So that was kind of a weird thing, right? Like the pilot program of the open source portion of the policy, to my knowledge, that's never been done before. It's usually you write the policy and then that's it, right? Until the policy goes away, they can retire policies. And what they'll do is they'll write a new memo, it comes from OMB and then says all these previous policies are now gone. Here's the new ones kind of thing. So that would be step one, right? Like have a solid policy where things don't go away because that didn't help. Find, if we could, I don't know how we'd do this. Of course if it goes into law then you're fine anyway. So yeah, definitely like try and get a policy then law if you can and help that process more. From an acquisition standpoint, agencies had to update their acquisition language and we didn't really have visibility into that. So we have this thing called the FAR, the Federal Acquisition Regulation. And it supersedes agency policy or regulation for acquisition as well. And so they'll take clauses from the FAR, put it in their immediate contracts and then buy things, services and products and what have you. And so I had talked to the FAR team about like can we just make it, like if every agency has to do the same thing, why are we relying on them to write their own acquisition language when we could just put it in one place and then every agency just has to follow it. But for whatever reason, we couldn't seem to get that through either. So that would definitely be a big, so that's sort of the program at like get the policy, get the law, get the FAR, get all the things that government has to adhere to. Website was fine, we worked on that. We actually would curate specific projects that we would go out to events like hackathons. We'd hold hackathons at different universities and speaking things, things like this and we would highlight projects that we thought were easier to start for people, for various people like new time contributor and things like that. So that worked, took a lot of time so we definitely had to put more effort into something like that. But that helped a lot as well. And really just kind of champion, like you have to be out there and really push it. So it needs to be funded, it needs to be thought of as an administrative priority in some sense, which is hard, because there's so much out there over time. I don't know what else I do. I have to think about that. But yeah, and I think it would be easier today. Like I said in the beginning where I spend most of my time, like I'll say educating leadership at agencies about using open source versus, and a lot of them are saying, oh, we want to do more open source, we want to be locked in, we don't want to pay for your stuff and all this stuff. And I'm like, well, that's great, but like what's, do you really know what that means? Like what does that look like to you over time? So, I'm not good at this, just three minutes. So you just started to touch on something I think is super important and that's that busy executives above you get an impression of what they think open source is. And most of them don't really understand or know what open source is. And how much of an impediment was that and has things changed in your new position? Right, so the impediment, no, just hang on to that for a second if he's the one to do it. Yeah, that's, I mean, it's like the one executive I talked to who wasn't in IT, so that's not necessarily fair to him. That's what, yeah, I noticed a lot where folks would say, oh, we do open source and we do open it and we do. And it's like your office, it's not really doing that. Like even in my own office at GSA, it was like pulling teeth to get people to do the source code or do data sets for open data. Which happens everywhere, right? Like someone once told me that you have to explain something like three times for them to finally do it or something like the open data. You have to explain what you're trying to do and why three times and then usually you can get a data set or something. And so, yeah, hopefully with the law and then just kind of more use of open source and more general knowledge of it, that it makes it a little easier. I definitely hear it more, which I was really surprised of any minutes asked for 10 months and they're like, oh, we just heard it again. And I'm like, who are these people? Because when I was in government, nobody was saying that. They were all like, oh, we don't want anything to do with open source. Forget that. So it's like, ha, like what changed? And it's just maturity over time, probably your education over time or understanding there. I think it was said yesterday in the keynote real fast was like that open source enthusiasm and use is on the rise. And I, yeah, they're in that same boat as well. Go ahead, real fast. Yeah, so I am a service designer for the federal government. I'm based in D.C. And so working with federal clients, what you said about, they don't have the technical talent in house. It really resonates because that, and I, in my opinion, I think that the federal government is way too reliant on contractors to fill that technical gap. What's your outlook on senior technical leadership in federal government roles in terms of getting people that have that technical acumen, not just to be consultants, but to actually be in these roles making decisions? I know that a lot of people come in from the private sector and they just find the bureaucracy too much. So what's your outlook on getting those really technical people into the government to stay and break through the bureaucracy that they need to drive substantial change? Right. Wow, that's loaded in like 10 seconds. Yeah, I'm always, I'm an optimist. So I'm always optimistic about the fact that we can get the talent we need. We have started to do things like digital service and 18F, if you've heard of that, and technology, and USCIS has done some great stuff from an IT standpoint. Mark Schwartz was there for a while. He was an outsider. He's at Amazon now, or AWS. Yeah, and so there are people that are willing to come in mission oriented, like for periods of time that do have the capabilities. And we've had, we've expanded the seats at the table, like we have CDOs now in the CDO council. We have, I always had CIS, but there's new roles popping up, analytics, officers, AI officers. And I think that helps a little bit because then you start to expand and term appointments help because people don't necessarily, if you're coming from industry, you may not want to stay for very long time. You want to come in and focus on one problem area with a certain amount of budget, a certain amount of people and just get the job done and then move on. That's good and bad. I saw, you know, I was around for a long time. I would see people come in and kind of say, oh, we did all these great things. And then they leave and they never actually really saw it through. So they started something, but they didn't. So that comes problematic long term. It'll always be a challenge. Big bureaucratic organizations are hard to work with. Some do it better than others based on culture. And, you know, so I'm hopeful that we can keep moving the ball forward and that additional seats at the table. And, you know, there's definitely talent out there that we can get a hold of. So, well, thanks everyone. We're in time. Thanks everyone for coming, for sure.