 Hi, and welcome. My name is Cecilia Munoz. I'm a vice president at New America for Public Interest Technology and Local Initiatives. And it is my great pleasure and honor today to welcome all of you. Thank you so much for participating. We are very excited about this session on building and reusing open source tools. This particular body of work and this event today and the report that comes along with it is special to me for a couple of reasons. The first is that it was produced by two teams that I have the great pleasure and honor of working with here at New America. One is the Digital Impact and Governance Initiative. And the second is our Public Interest Technology team. The two teams concluded more than a year ago that there was a need for a comprehensive framework for open source approaches for governments. And so they have been researching, interviewing people, digging deep into determining what it would take to create a framework that could be used to really support using open source resources across governments. Part of the reason for this is because we've heard a lot of questions which you all might be familiar with. We hear the kinds of things like, well, if we're using the software, is it free? And maybe who else is using this code? So after months of research interviews and some debate, we released a report just yesterday which provides a high level framework for delivering digital solutions using open source software. And also very importantly, outlining some practices to maximize its use. So the report which we hope you take a look at and read, it was really designed for policymakers. And it feels we have some sense of urgency about this because if we've learned anything from certainly the events of this summer, that it is that we need government to be working at its very best and that there are times, and in some ways I would say really all the time, people's very lives depend on it. And one important tool in the arsenal of making sure that governments are effective or acting as effectively as possible is the ways in which they modernize and the ways in which they're able to deploy and use open source approaches. So the report is intended for policymakers but I'll just give you the punchline. Our big takeaway from this work is that software to benefit the public should be open, period, by default. So the panels today that you're going to hear are gonna dive deeper into this but I'm gonna set the scene just a little bit. So as you and I are sitting here not at New America necessarily, but this is, we invite you to New America so you should think of this what's on your screen as being New America in action. You know, it's possible that you or I could also get on our phones and order a pair of shoes while we're talking. That would be easily done. You might be ordering your groceries while we're talking. But if you and I, if someone on this panel were to say the other thing we need you to do is order up from the government a replacement for your social security card. You might break out into a cold sweat because your first impression would be that's not such an easy thing to do. It's gonna be easier for me to order the pair of shoes or the groceries than it would be to do what is a relatively simple process that's run by the United States government. That shouldn't be true. I've given you a relatively benign example although having a copy of your social security card can be pretty important for certain times in processes. But obviously in this moment of pandemic which is on, of course, on everybody's minds. The reasons for a government to be more effective and more efficient and to be able to use data and to be able to use major platforms to advance their work are just more apparent than ever. And this relates to the second big kind of scene setting point which is that government being as effective as it can possibly be in addressing these kinds of challenges is essential to our being able to maintain trust in the institutions of our democracy. Again, something which was always true but which is feeling more true in this moment of crisis. So you're gonna hear more about this from really an extraordinary group of panelists. I hope you'll read the report. I hope you'll ask questions today. You've heard the voice of God which is really our colleague Angela Spitaletti describing how you might do that. And I hope that you'll engage with this topic as we go forward based on the really insightful things you're gonna hear from our panelists. So before I hand it off to the first panel, I just wanna thank the big center at Georgetown, our friends at Code for America, the Linux Foundation, California Government Operations Agency, MOSIP, U.S. Digital Response, and of course, New America's own Public Interest Technology and Digi teams. And a special thanks to our events team who since this moment of lockdown began have hosted and facilitated more than 170 events. We are now reaching a global audience including at this event, which we are very excited about. And I just wanna tell you, you are in very, very expert hands in the way this webinar will be run. So with that, I thank you. I thank our panelists and let me hand off to the first panel. Thanks to Cecilia for that wonderful introduction and thanks also to Angela and the New America events team for making this event possible. I am incredibly fortunate today. I'm to Micah Tilliman and I lead our Digi program at New America to be joined by an extraordinary group of panelists. I'm gonna introduce them in just a moment as they turn their cameras on. Before I do that, I want to first say a few words about the participants in this event because I have the advantage of having seen the list. You come from a wide array of the world's most important institutions, the United Nations, the World Bank, many government agencies. You also represent some of the largest funders and foundations in this space. So you are uniquely positioned to take the insights that we hope we'll be able to deliver in the course of our discussion and translate them into action. We need you involved in this conversation. Please use the chat. When you use the chat, please make sure that it's pointing at the people that you want it pointed at. Make sure you have it set to either all of the participants or the moderators. We will be incorporating your questions throughout our discussion today and are very excited to have you with us. Our panelists, Robin Carnahan, who is a fellow at Georgetown University's Beck Center for Social Impact and Innovation. She is also the former Secretary of State of Missouri and the longtime director of the state and local practice at 18F, which is a U.S. government agency that plays a key role in digital transformation. Laura Kotler, the director of engineering at Code for America. Sri Joni Sen, who is the counsel at MOSIP, which is the modular open source identity platform. And we'll be hearing more about MOSIP over the course of our conversation. And Brian Bellendorf, who is the executive director of Hyperledger, which is part of the Linux Foundation. Brian also created Apache, which is the most popular web client on the internet and open source. He's the former CTO of the World Economic Forum. And I hold Brian somewhat responsible for getting me hooked on open source software. And the genesis of this dates back almost exactly four years ago when the two of us were sitting by a lake in Canada, back when Americans could still visit Canada, those Halcyon days. And we were having a discussion around how you can use technology tools to solve big public problems. A bit of personal privilege and context here. I had for years before that what was probably the best job in Washington. I led a team reporting to the Secretary of State that was responsible for developing new solutions that could improve governance, democracy, and civil society around the world. And we worked across about 55 different countries. And what we saw is that almost all of these governments had the same set of challenges. And almost all of them were building their own very expensive, very inefficient, cumbersome solutions in attempting to solve those challenges. And the results, frankly, in all but a handful of cases were not that inspiring because they weren't learning from each other. They weren't innovating based on the work that others have done before them, but they were really expensive. And so Brian and I were chatting about this and Brian is one of the OGs of the open source world. So he already knew the power of this set of tools. And Brian, I want you, if you're willing, to take us back and give me a bit of a description of what open source is and why public problem solvers should care about open source. Sure, well, not enough time for the full history lesson, but I think it's important to start with an understanding that the internet came about through open source software. Even long before we called it open source, that was a term that really came out in 1998. When I joined the internet in 1991, we already had the domain name system. We had email, then SMTP standards, a whole bunch of other standards that were collaboratively built by researchers. And then those researchers into finding those standards also wrote software and shared that software widely to be able to talk those standards and show that those standards worked. And those, that software and those standards together went and were commercialized by some microsystems and Cisco and eventually everybody else who plugged into the net, but it was the underlying openness of the IP and the collaborative process that really brought about this foundational infrastructure that we've been building on top of ever since. And it's really an example of collective action, right? It's an example of literally a technology barn raising kind of event, right? Where parties get together, they realize they have a common interest and a lot of that is enlightened self-interest perhaps, but also a sense that, hey, what we can do together is so much bigger than what any of us could do separately. And in 98, the term was defined, it was associated with a certain set of copyright licenses that favored that degree of sharing and remix and that sort of thing. Most importantly, it was geared around providing for vendor neutrality, which has been really important, especially to government. But there was always early on this sense that it must have been some sort of anarchist takeover of the technology industry or something like that. And it wasn't until 2004 when the DOD published a memo that basically said open source should be treated the same as COTS, commercial off the shelf software, that folks in government felt safe to use this technology to work with vendors who specialized in this technology. And it started to get more in the mainstream. But the real leap then was to start helping government agencies realize that they had a possibility to play a role in determining some of the path for this technology. That open-source software gave them the legal tools as well as operationally the ability to remix, to extend, to add features, to push the direction of these technologies. And that's really been the arc for a lot of this. When I worked for the Obama administration in 2009 and 2010, we actually, I was over at HHS in the Office of Science, the group focusing on standards for the interoperability. Brian, I just lost you. I don't know if others have lost Brian, but I'm gonna turn to Robin and ask Robin to continue that thought because you've also spent a lot of time in government looking at these issues. And as Secretary of State, when things go wrong, it's really bad. When an election goes wrong, it's really bad. Tell us about how you think about open source and to Brian's point earlier, how have you gotten comfortable with the idea of using this basket of tools in your work? Well, thanks to Micah. I'm sorry to miss out on Brian. I can't fill this. We'll bring him back, he'll return. Yeah, I will say, I confess to being not really a technologist in this very techie conversation. I'm somebody who has been in government and been tasked with delivering for people who need some function out of government. And so I came upon this notion of open source in a different way, which was, I had this problem when I was in office and it turned out there were 50 other people who had the exact same problem who were Secretaries of State or ran elections. And it made no sense to me that we were all out there solving these problems on our own and reinventing the wheel and going through that risk and stress and trauma. And I often say there was nothing I did in public office that caused me to lose more sleep than the roll out of some tech project because it almost always didn't work quite right. Later I found, and there's a thing called the Standish Report if you look that up, that shows that 13% of technology projects in government that are over $6 million succeed. So let that soak in for one second, like 13% is a crazy number. And yet we continue to do the same thing. And so that's what brought me to the notion of open source, that we can with the modern way technology is developed have tools that are shareable and reusable. And so that there's no reason anymore that folks have to reinvent the wheel, whether you're in a county or a city or a state or frankly a country because so many of the functions of government are the same. And so I'm excited to be able to talk more about that in detail to Micah. I could go on forever, but what's your question? Well, let's go back to Brian quickly. And Brian, we are glad to see you with us. We love Zoom, but it is not open source technology. So there's still a few bugs to be ironed out, but please pick up where you left off. And I'll give them credit. I'm sorry, even if it were open source, it might still crash. Of course, it's important to recognize open source is not infallible. And I sincerely don't know where I dropped off. So I'll just say, open source software has represented the best I think of folks in the technology industry working together in a common kind of barn raising mentality to build common infrastructure and common technologies. And I think the role for government in there is not just as a consumer that technology to be able to do more and be more efficient, have true vendor choice and neutrality, but also to help guide those technologies to a better place than they might otherwise be left in. And to use them as a tool of policy as a way to see costs come down and resource sharing happen for not just one government, but for governments broadly stated. So Laura, help us out here. We've heard a couple of really powerful reasons why governments should look hard at open source as a tool to help them in their work. But we also know that a lot of governments really struggle when it comes to the use of open source technology. The private sector is finally moving pretty quickly on this. If you look at the folks that Brian works with at the Linux Foundation, it's kind of a who's who of the world's largest corporations. And if you look at the code that's actually out there in circulation in the private sector, the overwhelming preponderance of that code incorporates open source elements. Some of the audits go up to 99% of code bases now incorporate open source elements. So we've seen the concept take hold really rapidly in industry. It's been slower in government for a variety of reasons. And you live this story every day. Help us think through what's going on here. Yeah, I think that there are a number of reasons for this and it kind of depends upon what type of open source technologies we're talking about. One example that I can call out is, so we worked with some partners a few years ago to build a tool called client com, which was an open source tool that we built that allows probation officers to communicate with clients who are on probation or parole and to try and ensure that they basically don't end up going back to jail for a technical violation that could have been solved through communication. So this tool was like really popular with our partners. They found it hugely valuable, this ability to use text messaging in their work. But when we ended the project, we gave them the option to take over and adopt the project because it was open source. So several of our partners are really interested in that and almost all of our partners actually did go on to find some way to incorporate text messaging because it was so valuable to them. But only one of the partners actually ended up adopting and using the open source product that we built. And so we got feedback from the different partners about why that was. And there was a variety of reasons. So one reason was that many of the departments we were talking to, like so a lot of government agencies, right? They're sort of small and local. So criminal justice in the United States, for example, is generally administered at the county level. And so these aren't departments that necessarily have a ton of resources or a ton of technical staff. So in this example, like many of the departments didn't have technical staff who had the ability to like run and maintain a software application because with open source, even though the code is free, there's still costs, like you have to have a server to run it on and you have to have a team of people who knows how to operate it and who knows how to make sure that it's secure and do security updates and maybe continue developing it to add new features. So that was actually a barrier for a lot of our partners. Some of our partners didn't have technical staff, but those technical staff weren't familiar with the technologies that we built it on, which in this case was Ruby on Rails. Some departments already had a bunch of existing sort of technology infrastructure and they wanted a custom solution that could integrate with that. So they built something on their own. Others ended up just going sort of different routes like procuring cell phones for all of their probation officers, which was kind of a thin that avoided the issue of having to have technical staff or maintain a website to begin with. So there were a lot of different options that our partners chose and only a small number of them actually ended up adopting the product as it stood. And so yeah, so I think that those are some real barriers that people face. I did another one that I would call out is just fear. There's oftentimes a fear of transparency, if you're working in the open up here that people will find something wrong with it and that you'll get sued or that people will be mad or these types of things that have definitely come up in a lot of conversations I've been in. And I think there's sort of a cultural shift that has to happen where people see the risks of not being open source as equivalent to the risks of being open source. I think there's a lot of benefits that you can get in terms of the increased transparency and trust and community engagement that you can have through open source. But it's, I would say that we're not there culturally like in most of our government agencies where that's seen as a benefit rather than a risk. I know in a past life I spent a lot of time trying to convince governments around the world oftentimes governments that weren't particularly democratic to engage with civil society and work with outsiders. And the paradigm that I usually tried to advance with them is like this is having a coach. You need a coach who's gonna help you do your job better and they can point out the things you need to change. And if you look in the great report that's been produced by our teams at New America there, it touches on that. You have a lot of folks who are gonna actually be able to help government do their job more effectively if they can see what's going on inside this code. And that's a very powerful opportunity. So Johnny, I wanna go to you with a question that's come in from the audience and it's a really important question. And I think your work and Mosif go to the core of this. The question is who's going to help identify the infrastructure that we need to build for government technology. And when we look at this issue, which we do a lot in the DG program at New America the base layer, the foundation layer of infrastructure that you need to have better working public sector technology is always digital identity. So I want you to tell us if you're willing a little bit about Mosif but also and Mosif for the uninitiated who will be initiated momentarily is a digital identity platform. But talk about why you decided to make it open source and what was the thinking that went along with that? Sure. So digital identity as a way of providing basic services is a new concept in itself. It's about just about 10 years old. Before that, if you thought about why government provided national IDs there were as many concerns of security or international travel. But as more and more services started being provided online by government, the ability to accurately and easily a person to be able to assert their identity accurately and easily and for government to then accurately transfer benefits to that citizen was something that governments around the world started recognizing as a crucial part of the foundational layer of all of government service. And so the idea of digital identity was progressing but the reason why the open source nature of our project came to be attractive to governments as well is because across the regions where we work so that's primarily in Africa and in Southeast Asia the number one problem that governments unanimously agreed on when trying to implement these systems was issues of vendor lock in. And so the relationships that they had with vendors was actually getting in the way of creating this national critical digital infrastructure. And so what was really unique about the Mosep project is in all of the countries that we've worked with and we've got MOUs now with four countries, Morocco, Philippines, the Republic of Guinea and Ethiopia and we've conversed with many more. The political will for this is at the very highest level so it will be the council of ministers or the prime minister of the country who believes that their country needs to have digital ID. And while the more community building aspects or scrutiny aspects of open source may not be something that is high on their list of priorities they do like the idea of owning and controlling and understanding the underlying technology and the code themselves. And they recognize that open source is a way of also building that capacity up. So Mosep is actually a university project. We run out of the International Institute for Information Technology. And how we work is that the open source platform itself is there on GitHub for anyone to use. We also parallelly enter into MOUs with these countries. The countries then run their independent procurement processes. So because of the large scale infrastructural nature of these projects they will go through the process of selecting their vendors and usually a large number of vendors coming together to build the system while mandating that they use Mosep or maybe in the future any open source underlying core platform. And then we support the country on a pro bono basis as they work with their vendors create those contracts through to implementation and the commercial support. So there is a commercial ecosystem that is there and that needs to also understand open source. And I do think you also need more government focused support which could be in the nature of philanthropic initiatives or more kind of civil society oriented support to government as well as a flourishing commercial ecosystem all of those things have to come together. That's a great overview and I will note in the interest of full disclosure I'm on the international advisory group for Mosep and it's a fascinating project. I do Brian want to come back to this point that I raised with Shujoni as well which is how do you at Linux Foundation decide what to build? That's one of the big issues with open source that's one of the real challenges that governments encounter when they approach these opportunities. What's your thought process look like? Yeah, we're so used to traditional technology companies that have a very top down approach to product management and have a very kind of marketing driven kind of view of the world and how to channel consumer interest into revenue open source projects are much more organic much more bottoms up when they're done right when they're done successfully they're driven by the end user community who realized that they have that lead user role that they can play in establishing where the technology goes with the vendor community the red hats of the world, the cloud infrastructure providers for that technology but also the support organizations and so that tends to be a very successful way to drive a large amount of innovation into a platform into a product by kind of fanning that out and decentralizing it but well run open source projects do still have a governance model driven by those developers that say here's what we're going to fix in this next point release here's what we're going to target for the next major release here's how we're going to take this thing that's a big chunk of code and avoid any one person or company having to write it we're going to split the load and kind of fan out across it and then usually they have a separate amount of kind of governance model for looking over the legal kinds of things everything from trademarks to kind of the IP management making sure everything came from a known good source and so at the Linux foundation all the projects that we set up everything from the kernel itself to hyper ledger the project I lead to over the 150 different initiatives that we have at the Linux foundation really has that kind of bicameral model if you want to think of it that way right the ideation and the collaboration around where the software goes has to come bottoms up and be very driven by you know the phrase history is made by those who show up right that's definitely true in an open source project and then having the companies that are helping fund what is really just a small kind of program office to kind of you know think of it like air traffic control make sure all those processes run smoothly what they oversee is the budget and kind of the you know the trade marketing and all these legal issues but you keep these things separate and that's the way to really balance all those interests most effectively we think Robin I want to bring you in on this a little bit as well and we had another great question from red one and the audience and I encourage all of our participants to keep those questions coming you have thought a lot about the structural questions around creating open source solutions for government as have we because the institutional architecture that exists in the context of a Linux foundation which does beautiful work around its projects or Mozilla which does you know a lot of important work on Firefox and related solutions isn't really there in the same way for governments what are the institutional solutions that will have to take shape in order for governments to realize the potential of this tool and I know that's a long and complicated question but try to distill down some answers that we can apply you know I guess I just think of these things in very practical terms right as someone who was in a position of having to deliver some service and I assume there are people on this call who are in that position and like you know from my perspective I don't have a lot of strong feelings about how that gets done but you know you have to do it you have to cover everybody you have constraints and money and time and you have to have something that's sustainable so one of the key things I think when we're talking about open source tools is you just have to know who to call and something breaks right if you're the person responsible for it and so that often is a troubling question that people need to have an answer to straight up so they want things that are gonna be responsive to their actual needs so they wanna have some control over the governance and the prioritization of what gets done right to reflect something that me or my users actually need and then they wanna know who to call when it breaks so that means there needs to be a robust ecosystem to be able to respond to that and you know again I think of this as the value that you bring to taxpayers or the public is to be able to provide a good service at the best cost and how do you do that? You do that by having competition right built into the system and one of the problems that we see in government technology now is that there's not a lot of competition right once you select a vendor that tends to be locked in for a long time there's a proprietary tool that you are now dealing with and either by contractual agreement or by technical functionality it's hard to get your data out right so that means you're really stuck and you can be stuck for 30 years with some bad situation and I refer to it as like an abusive vendor relationship that's too expensive and politically difficult to get out of so the goal is to not get in those relationships how do you not get in those relationships? You have portability of data you have standard APIs you have open source tools that lets you have a competitive marketplace and environment so like that's how I think about the magic of all of this and why it has real value for governments and for the public Well you are bringing up a series of really important questions Robin and we have an amazing panel right after this that's going to focus on pandemic response but I wanna highlight one little anecdote which I think illustrates this so vividly many of us yesterday got on alert on our phones from the New York Times talking about the county health office fax machines that were being overwhelmed by testing results with the coronavirus pandemic and the reason that that is happening the reason that we have one of the most important public health crises of our lifetimes the most important public health crises of our lifetimes being bogged down with a fax machine is because the testing providers don't wanna pay $5,000 for a package of software that would enable them to send the data electronically Now there is already an open source solution that does this, the X-Road system that was pioneered by Estonia which is one of the countries that we benchmark very extensively when we're looking at how to build better solutions is already being used on an open source basis in Germany to share health records back and forth but the United States doesn't unfortunately have that same infrastructure in place and we're paying a really profound price for this right now I wanna turn to you Laura because you are one of the people that gets called when things go wrong and when we've talked about open source you've highlighted the benefits and certainly the financial benefits but you've also said to me that open source is free like free puppy as opposed to free like free beer and it takes a lot of care and feeding to continue nurturing it over time and it's easier to do frankly if you have a whole community that's willing to help take care of whatever the solution is what are some of these suggestions you would offer to leaders and government and institutions as we come through this really tumultuous period for public sector technology to think about how we should do this differently how we should build the communities that will help us solve these challenges on an ongoing basis going forward Okay, there was a lot in that question. Yeah, I just wanted to I guess go back to the facts, machines and the data question because I think that that is actually a really, really big deal and it's another one of the kind of like barriers to adoption that we've seen in our work in some of our open for some of our open source projects. So I do think that like one you know, one thing that I just would strongly recommend to people is making sure that you're thinking about the data that's in your systems and how that can be accessed so making sure that you include APIs in any projects that you're doing. So APIs are basically an interface where a computer program can talk to another computer program and get data in between them and then data standards like if there are data standards in your industry like build projects that like conform to those so that the data can be interoperable because for open source if you're trying to talk about reusing things between different jurisdictions in different areas or even different fields like one of the big problems with that is that the data, the underlying data if it's all in different formats or it's not standardized then you can't just take that same solution and repeat it in multiple places. We see lots and lots of examples of this in our work at Code for America. In terms of building a community there's a lot that goes into that so Code for America has a network of volunteer network that we operate called the Code for America Brigades and we have basically volunteer groups in cities around the United States that work on issues that are relevant to their local communities and there's actually like a lot of work that goes into building a community like that so we do things like try and provide tools for organizers to be able to use to build their own local groups. We provide opportunities for connection so providing like forums for communication for a community is really important cultivating leadership within that community empowering people in the community to be able to actually work on the things that matter to them. I think there's like a bunch of other things that I just spaced my mind, but... That's a pretty good list. Yeah, there's a lot of work that goes into it but the benefits can be really profound when you actually have more engagement with your constituents or with the people in a community that are actually gonna benefit from the services. So I think it is work but it is work that is worth doing. So, Srirajani, I wanna follow up on another question that has emerged from the panelists. Sorry, did you have something, Robin? Yeah, I just, if you'll mind, I wanted to say something about data standards for a second, which like 10 years ago, Robin would never have said those words. But... Wunks and geeks collide. Yeah. That's what we're witnessing on this panel. I know, it's incredible. So here's the deal. Data standards are not gonna just naturally happen, right? They're only gonna happen if funders in particular, so if we have anybody on the phone that funds any of these sorts of projects, and we do. And it advises those, right? So what do I mean by that? I mean, like it's in nobody's interest to spend time thinking about data standards. So your only interest when you're in government is to take care of your own little narrow silo of the thing that you're dealing with. So it's the funder's job to think about how you can create efficiencies. And data standards are about creating efficiencies. And these APIs, for anybody who doesn't know exactly what that means, like this is my non-technical brain, it's like Legos. If you think about Legos, they have those little standard connectors, right? That you can create this whole universe of modular pieces that can swap out things. And it is a magical thing in technology today that we didn't have 10 or 20 years ago in the same way that changes the game. So we need to, I just wanna double down on any funders on the phone, put data standards and data portability as this very important holy grail in anything you give money to. And if you attach strings about that, you can change the marketplace and how all of this works. So Joni, I want you to help us understand those questions in a little bit more depth because one of the fascinating things about the MOSIP model is that it has been seeded by philanthropy. You have the Gates Foundation, the Omidyar Network, the Tata Trust, all coming in to de-risk the development of the technology for governments. Now, you still have a very formidable hill to climb when it comes to convincing governments to use these tools, but tell us about the model that has grown up around MOSIP and talk to me about whether there's a role for philanthropic capital in de-risking the development of some of these tools and creating the communities and ecosystems around these solutions. Absolutely. So I want to just start with an anecdote which stays with all of us as we do our work and this came from a bureaucrat in one of the countries that we work in in Africa who was talking to his peers, other African government officials in a meeting about why they chose to work with MOSIPs. And his clear stance to everybody else was that you shouldn't adopt MOSIP because it's easy. The outset, it may be actually easier to go with the proprietary software where the vendor will come and tell you that you don't have to worry about everything, anything. You don't have to look under the hood. We will take care of everything for you. So you don't do it because it's easy, you do it because it's the right thing to do. So the amount of de-risking that a philanthropist or even the MOSIP project is doing is actually very minimal compared to the gigantic battle that governments still have to fight. Open Source nearly provides a framework in which they are hopefully better able to do so. But side by side with the provision of the code is an easy part. Having experts talking to government on this is also relatively easy part. But for government too, that mental transition of saying that the government will build this and vendors have to come in according to how the government wants to build this. And to see that through, that is really the biggest, the most difficult part of this puzzle. And that is also where I think a lot of philanthropic capital and a lot of support, initiatives and reports such as yourselves, that's the really tricky part I think. Having the code out there is essential, but it's a very, very small part of the problem. Brian, I want you to build on that and help with two questions that have come in, which are really intriguing. One comes from someone who worked at Google for a while and he said that when he was there, the glue between all of the projects and teams was the description of the data exchange that was expressed using protocol buffers. And that's a wonkish way of saying we need to have a clear vision for what's moving back and forth across these systems. There are a handful of countries that have built out some really powerful solutions in this realm. And certainly this is an area where we spend a lot of our time and energy thinking about what a comprehensive open source modular government stack should involve. But what's your take on that? And then the other question when you finish on that one is about the value of open source solutions and what the total cost of ownership is on open source versus non-open source. Yeah, APIs and standards and protocols are essential to this and essential in particular to developing a well-orchestrated stack where identity and payment rails and all these kind of common functions across, especially citizen government interactions can really come together. But I think standards are not enough and the fax machine debacle that you mentioned is particularly painful to me in 2009 and 10 I worked in DC for the Office of Science and Tech Policy seconded over to HHS to help them lead an initiative around adopting standards for the exchange of medical data between institutions starting with the VA and the DoD and Kaiser and the explicit goal was replace the fax machine as the primary method of health information exchange, right? And clearly 10 years later maybe we didn't have the impact we wanted and it was not for lack of technology. In fact what we led was an open source driven process that said let's come up with the standards for this and we actually based it on SMTP on email with an extra layer of security over that. And we worked with the public sector, Microsoft, Google a whole lot of startups were part of this and the processes were all very public. What wasn't driven though was the right governance model and the right incentives to actually drive that adoption and get people out of their bubbles of thinking if data is the new oil then I have to hoard it and I have to price it high and I have to own it, right? And it's a shame particularly in healthcare in the federal sector because there is such a great tradition in the Veterans Administration of actually an internal open source project around the development of their health record system called VISTA and the tragedy there was that about four years ago VISTA was kind of thrown over the side of the boat out of the premise that it was antiquated and archaic because it's been around for a while in favor of centralizing the future path for both the DOD and the VA on a single proprietary electronic health record system vendor. And when you do that you remove the incentive to interoperate, you remove the mandate. You know, and we had things like meaningful use and some of these other criteria that attempted to drive that but it's really through channeling that investment into software development as an enabling technology and getting it widely adopted out there. In industry and with government as a substantial user of that technology that you get really the incentives aligned enough so that the standards matter. And part of that finally is just finding common cause with other organizations. You know, it was hard trying to find common cause between the VA and DOD when you think that the longitudinal history of a warrior should be enough to pull that together but finding that with the private sector was really hard and that's where better governance, better leadership could have mattered. And finally I'll end on the TCO question. It's incredibly hard, you know, open source or not to manage costs around a technology initiative. We don't have an easy way of plugging in some variables and getting a hard and fast number on the outside of that process to say how much are things going to cost. It's why you have to talk about factors and variables and influences and optionality. And as has been mentioned, when you are locked into one vendor when you're locked into a roadmap defined by somebody else you are much more subject to having to, you know be on the receiving end of a lot of that rather than being a driver of bringing those costs down and flexibility. So I don't think there's any organization who having made a substantial investment into open source comes away from that going this was a lot more expensive than, you know compared to what we got out of it than the alternatives but it's really hard to get scientific about oh it's going to cost 47% less to build this you know as open source than as cost. It varies tremendously by project by project. We have just a few moments left in this panel and I want to thank all of the participants who have sent amazing questions. Keep them coming because the next panel will get to some of the questions that we were not able to get to I hope and they're eager to address additional points that arise. I want us to conclude on the issue of trust and I want us to end there for a couple of different reasons. One is that we are at a moment of historically low trust in government institutions and this is something that unfortunately has become deeply detrimental to our ability to solve some of the biggest challenges facing humankind, whether it's the pandemic whether it's climate change it's very, very difficult to build common ground and get people to believe in the institutions that are responsible for safeguarding their interests. We have with open source something that in the words of our CEO, Ann Marie Slaughter is pre-political and hasn't really been dragged into the mud by either side yet and by virtue of that it seems like there's an opportunity in my mind. So I'd like if we can to close with a quick lightning round and each of you could just offer a word or two or an anecdote on how you believe we can harness open source technology to build and bolster trust in our institutions. Robin, I'm gonna start with you if I can. Bottom line, it's not about the technology it's about delivering for people and if open source and these other kinds of tools help you deliver better, faster and cheaper for people then it's gonna build trust and so that's the magic and whether it's tool and I will also say that there are a lot of these open source shareable solutions that already exist. There are some internationally that exist. One great example is a thing called Notify that was developed by the digital service team in the UK that basically is a way to push out text messages and emails and mail and that's been reused in Canada and Australia. There are other ones who's a great one in the state of Georgia where there's an appraisal system and almost all the counties use the same thing and it's totally open source. They didn't even know what that term meant when they did it but they had a common need and they collectively built something to respond to that. There are lots of examples around the country. We're seeing it, I know we have someone from the US digital response team on the next panel. We're seeing it over and over because you have governments with the same problems at the same time and when that happens, it makes total sense to have shared solutions that they can deploy quickly. So best way to get trust, deliver quickly. Fantastic, Laura, 30 seconds please. I'll try and be quick. I think one of the places that I feel this most acutely has been in our work in the criminal justice system. So there's I would say a trend in the criminal justice world of having more and more sort of algorithmic justice things. We work in this a little bit in our work in record clearance where we're now writing computer programs that may be making decisions. We believe in automating as much as possible the criminal record clearance process because we think it removes the burden from people of having to apply. But it does mean that now sometimes computers are making decisions about justice outcomes for people and I think that that's an area where proprietary closed source software can really undermine trust because there's no, like I think one really, really important part of people perceiving something as fair is being able to understand how a decision is made and what goes into it and the actual process and in closed source systems, that's not possible. And I think open source provides a really, really important opportunity to actually build trust in a system and have people understand what's going on and see it as fair. I do wanna just say, I think open sourcing code for criminal justice algorithms is a very, very important part of that but it's not sufficient. I think in addition to that, justice agencies need to be thinking about also accompanying that with solid documentation, plain language explanations of what an algorithm is doing and what it's using and why. And so it's open source is a piece of the puzzle but I think it needs to be also just accompanied by a broader attitude of transparency around those issues. Fantastic. Srijani, one bullet point, please. I would just say that I completely echo Laura's point when it comes to digital identity, again, something that will always invoke deep distrust. Open source does not by itself address all of those concerns but it moves the conversation a little bit further than understanding what the system is doing. And Brian, take us home. So to paraphrase Reagan, trust and verify is kind of a key thing here. And just the one place I think this is gonna play out most over the next few months is, we have a landscape of different solutions that different public health authorities are deploying for exposure notification woven into their contact tracing processes. Many of those are not proving to be successful out there and it's largely because people don't know whether to trust the app because they don't trust their government necessarily and they certainly don't trust Google and Apple and the technology companies to deliver these things. We think those being open source and being well-governed and coordinated and reused and shared and thus scrutinized and verifiable is really key to building adoption of those. So follow this space. Well, our thanks to you, Brian, to Robin, Laurence, to Joni for the extraordinary discussion today. I know that as we continue our work to develop a modular open source stack for the public sector, your insights are going to be invaluable. We're looking forward to hearing more from our partners in the second panel. I'll leave with one note because we have California participating. California, we work with a lot of different states that develop open source solutions. California came out with one earlier this week, which was a pandemic response tool and they signed the code base in GitHub with love from the people of California because they knew that this was something that would be valuable to many others. And hopefully we get to the point where all public code is gonna be developed in that same spirit. And I think we're nearer after this conversation than we were at the outset. With that, our gratitude to all of you and Hama Shank, over to you. Thanks to Micah. That was a really great conversation. So let's turn it now. We'll move over to open source and crisis response and then we'll have a chance to talk a little bit more about how we've been seeing people use open source in response to COVID, among other crises. So, so excited to introduce the panel this afternoon. We have Raylene Young, who is the co-founder and CEO of US Digital Response, which we were just talking about. She was a 2020 fellow at the Aspen Tech Policy Hub, focusing on the intersection of policy and product. Prior to that, she was an engineering executive at Stripe where she led payments, product and global expansion in Asia Pacific and prior to Stripe, she was an engineering director at Facebook and led efforts on privacy, content creation and newsfeed. And Halaka Kehate, who is also known as Angie, is the assistant secretary for digital engagement at the California government operations agency. So hoping to dig in to a lot of the work that California has been doing and product owner at alpha.ca.gov. In her role, she develops and implements state level governance structures and policies for digital innovation, including web standards, web accessibility, open data and open source. She was a member of the GovOps DMV Stripe team that Governor Newsom announced in January, 2019 and champions initiatives such as Code California and the state's open data program. Most recently, she led the team that created California's response website to COVID-19. And we have Mark Lerner, who is a fellow in New America's digital impact and governance initiative and public interest technology programs. And one of the authors of our open source report, Lerner is an engineer, strategist and design advocate with expertise in digital transformation. Prior to starting his fellowship at New America, he served as the deputy executive director of the US digital service team at the Department of Homeland Security, where he empowered a team of 30 engineers, product managers and designers to improve critical services, serving immigrants, asylum seekers, disaster survivors, schools throughout. So let's start with, I think one of the key challenges that we identified in the report. So at New America, we are privileged to have this bird's eye view of what people across public interest technology are doing. But one of the things that we saw that we discussed in the report is that when you build something open source with the aim of having others use it is getting other people to know that it exists. And to apply it to their own needs. And in a crisis situation, this feels like something that is especially critical and probably especially difficult. Critical because there's added pressure to get the word out, that thing that you have exists, but difficult because the people who might want to know about it are probably completely overwhelmed in managing their own, so starting with Angie, I would love to hear what you found to be most effective in getting the word out. Any stories that you have to share thoughts you have on how to tackle this. Sure. Happy to thank you. I would say at the end of last year, I was honored to be asked to lead a small team, which is now known as the Alpha team. We had 12 to 14 weeks to completely rethink the experience of how Californians try to access information online. And so we came up with about 20 different prototypes. And the important thing about the approach is that we shared everything. Everything that we had for this particular effort was baseline. We tested for accessibility, for performance, for readability, and everything was out there on GitHub, and everything was tweeted and shared on a weekly basis to make sure that we were capturing the lessons learned of what was working and what was not working within our different prototypes. I think this particular approach is really important when it comes to open source and sharing because we know that we're not the first ones that have tried something and failed, but also we're not the first ones that are trying things to try to make sure that we're capturing and delivering on the actual needs that people have. And I think one of the biggest outcomes from being able to share all these lessons learned is that when COVID happened, our team was tasked to stand up the COVID-19.ca.gov site. And one of the prototypes that we worked on in Alpha was actually used for a different team that helped stand up the pandemic unemployment insurance application. And so I think it really shows that by you being able to share your experiences and your approach, you're building a community to be able to have a collective response to a crisis. And you never know how your lessons learned will be able to have an impact on something as as big as the COVID-19 response. So it's interesting that, so that is how to share even just within California government is not trivial, which I think is interesting, like that you created something and then just like to let everybody in the same state know is not an easy, is not an easy task. Mark or Raleigh, do you have anything to add? Stories you wanna share? Yeah, happy to jump in. So I think that the idea of discoverability of projects in general is something that, I mean, even when we were researching for this report, we found it to be such a huge problem, right? That people as you were creating your tools or as you were trying to find tools that you could reuse, discoverability really only goes as far as your network or as you can search, there's not really easy ways to try and find open source tools because by and large, a lot of teams that are developing open source don't tend to have marketing teams that are trying, or sales teams that are trying to sell you their tools. Being able to share within your own community, within your own government, I think is a really excellent way of being able to do that, the fact that I had that your team was able to inspire others to pick up that work was really incredible. By and large, a lot of what we've seen was that these projects can be created and they really solve key problems, but they have difficulty being expanded out. One of the things that we actually did as part of my work here at New America was to create this thing called the Pandemic Response Repository, which is a website that actually lists projects both from California and from the US digital response teams that basically collects all of the open source tools and projects that are trying to respond to the coronavirus pandemic and hosts them all in one place that you can more easily find them. We found that discoverability was probably one of the things that we wanted to help with the most and great people like Raylene and Helico were working on projects that we thought everyone needed to be able to see. I still don't think that there's a silver bullet for how you solve discoverability though. I would love to hear if there is one from either of you. Just to kind of add my perspective, because it's a little bit different. So what's interesting about US digital response is we're a brand new organization that actually started just as, I think the pandemic was really kind of hitting a lot of the country and states were internal shelter-in-place restrictions. So we're brand new and you mentioned in the beginning that sometimes people maybe think of open source as is it gonna be slower as a not actually the most expedient way to build something, but we deliberately chose open source because from day one in our DNA was how do we get something working as fast as possible and enable kind of a decentralized group of volunteers to help pitch in on projects that might start off in one useful for one jurisdiction but that we wanted to be able to quickly expand to other cities and states as we saw them facing similar problems. So we've used it as a huge tool and I think kind of as Mark mentioned, we've tried to just publish it all in the open. So we don't know if that necessarily on its own increases discoverability, but we've found that in conversations with new government partners, we can kind of bring up the work we've done with previous partners and say, hey, look at this tool we actually built for DC. Like maybe it will work for you in Seattle and we've actually made a lot of those kind of connections just using the power of the tools that we already have well documented and all built in open source. So picking up on this, you mentioned being a volunteer organization and one thing that is sort of factored into when you make the decision to do something open source you are by definition relying on a volunteer community to be involved. So it's not that it's fully a volunteer organization but generally if you're doing something in the open you're involving people who are probably not on your payroll. So particularly during a crisis that probably seems scary anyway for government especially during a crisis when the stakes are so high that feels especially scary. So Braylene, can you talk about are there things that you have found that work or don't work when it comes to dealing with a volunteer community or any risks or work arounds? Yeah, I think maybe there's also a bit of a myth to just fell around open source technology and kind of like access to internal systems or data or security concerns and I think I want to separate that a bit. So US Digital Response is empowered by volunteers so all of our projects are done by volunteers and we've done over 100 projects to date with government partners and I think the last count was 24 states. So we have a huge reach all of it is done by unpaid volunteers really lending their time and one thing that we try to make really clear is when we open source it's really to kind of enable collaboration it's to use like off the shelf tools it's kind of to open kind of the playbook to see what are the applications we're choosing how are we tying them together what's the functionality we provide but that's actually different and separate from the actual like data or kind of backend systems that a government team might use. So I'd say we kind of provide a product or service that you can layer on top of what you're already using internally and that way we can restrict and look and kind of preserve all the access to the government teams that already work in that data day in and day out and we're just more of a supplement and we kind of partner with them to get more like user facing tools up and running. I would say the other thing that is actually a benefit is the idea with volunteers is people may personal circumstances may change and so they might need to come in and out of specific projects and by using open source we kind of enabling them like even volunteers to collaborate with like past groups of volunteers and so it gives that's another thing that I think helps us to share our government partners is it's not that a volunteer will come in and a week later they'll cycle out and the project will kind of be stalled instead of we've now had projects that have been going on basically since the start of March, multiple months that have been built by multiple groups of volunteers that have been able to cycle in and out and to the government partner it's kind of uninterrupted support but on a technical level we might actually have different people contributing each month. Angie? Yeah, just to add to that I think it is an important distinction between open data and open source. I think what I've learned over the years having worked at the State of California is regardless whether we're sharing the data or the code what I've worked on is really to try to build a community of practice which is where the volunteer network comes in as well is being able to have a space where you're sharing the best practices providing guidelines. We have a Code California we launched in December of 2018 which had a playbook and a new open source policy that the state adopted but for open source to truly be successful you need to really rely on that community of practice and volunteers and folks that are dedicated to try to build in those common solutions. And for California we've been able to build that community of practice and expand on that and I think it really benefited us in the response to COVID because we had some foundational pieces in place for us to be able to know who to go to and how to approach and being able to even open up CalCAT which is one of the modeling tools that was referenced in the previous panel. I think we've been working very hard in ensuring and building the trust within our government partners and the administration and ensuring that we're able to create the space where we continue to share not just the data but also the code. Volunteers pop up time and again in your community. I think I missed part of the question. What was the question? Can you repeat that? So are you seeing the same volunteers pop up like time and again like, oh this person is like they're pretty dedicated? Yes, like we've had several instances where people are really tracking the work that we're doing and it's so inspiring for the team and folks that are working on this to have someone raise their hand and say like, hey, I'm willing to contribute time and effort, like I really want to be able to participate. I think that's been one of the silver linings of this whole thing is really we were able to all come together to really respond to the crisis and having others interested and invested and contributing their skill sets and to that cause has been really, really inspiring. Yeah, it's interesting to see how throughout the pandemic we've got a lot of people of various different kinds of backgrounds showing up to volunteer to help with the solutions that we're building out right now. I like Epidemiologists coming in to provide pro bono services, plenty of other legal services they're coming in and by and large engineers also want to come in and do this. Designers, product managers also want to come in and this being able to produce and use open source tooling is one of the best ways that you can actually bring that talent in your house or allow those volunteers to come in and help you out. I think that the ideas of people being worried about, well, does this mean that other people are going to have access to my internal data or am I opening up too much or can we accept this sort of stuff? I've seen that a lot of those worries tend to be a little bit less present in the face of crisis. People just want to tend to respond to the crisis very quickly, not to say that they aren't warranted, right? You do need to make sure that your data is secured and you have no appropriate access in that sense, but it shows that there is willingness and the ability for people from the outside to come in. It's just a matter of how do you actually go about building that community? And I think that both from the California side and again, USDR has done an incredible job of building out that community, but basically opening themselves up and saying, hey, we're here, we're building things that are getting used and we would love to see your help. Well, it's interesting because in a way, the way that you're talking about it, I think of open source as and building these communities as kind of being a throwback and actually Brian was talking about this in the previous panel, it's sort of a throwback to the early days of the internet when it looks like everything's free and everything's open and we're gonna make this great society, it's gonna be a great equalizer. And, you know, tech has obviously moved away from that in recent years. So it's interesting to see that there is still, you know, that open source is kind of coming, I don't wanna say back, but it's something that's more forfeit of mind and it's still being championed. So I'm curious if there are, so taking into account the fact that this is in sort of an old idea in a new time and especially given where we are with the pandemic, I am curious if there are ideas or practices that you've seen that are specific to the moment that we're in right now, ways that the practice has evolved or things that people are doing that maybe wasn't really a thing that was thought of a year ago or five years ago. And Mark, let me start with you. Sure. So I do think that by and large, a lot of the underlying foundational elements are the same. You mentioned a lot of the philosophy around open source from earlier days in the internet where it's just, you know, we believe in free and open technology and that you should be able to modify your own source code, that you should be able to own the software. And generally speaking, you know, industry shifted away from that into proprietary software as a business model, right? So you've stopped being able to see your own software, you started using more SaaS products, more products that were just binaries installed on your computer. But in particular when it comes to this concept of digital public goods, this concept of public software that's gonna be used for public good, I do think that we're seeing a larger and larger shift even from like a top-down policy perspective. You know, we have a federal policy here in the United States that says that all federal code should be open by default unless there's a good reason to or that there should be targets of how much open source code should be default. Different agencies have their own open source policy. And I think that we are seeing a shift in terms of understanding that there's a responsibility, you know, again, open source is just sort of like the technical term for this thing is transparent and accountable and part of the communal ownership. And I think that we're seeing that as a public institution, we have more responsibility to be open and transparent with the systems and software that we run, particularly as it relates to the services that people rely on the most. And I think that's why we need to have this sort of stuff open and transparent and we need to have this sort of stuff open and people are realizing that and putting it into perspective and into policy and into practice. So that's being a little bit more solidified on that front. There's one more like, I totally agree with Mark and kind of more like technical answer, I guess or just a more specific thing I thought of and how maybe open source has changed a bit and we see in our work is I think early on, you know, there was there are fewer also like off the shelf and it's easy to use tools to kind of deploy a front end website or like visual components and a lot of open source may have been focused on internal libraries or code that you would run like as scripts, they're kind of like back in libraries rather than standalone products. And I think something we're trying to make a lot of use of is we use a collection of low code or no code tools along with open source to actually build interactable demos that can also be open source. And I think that idea of like the demo itself being obviously of value, not just the source code, we've seen it is hugely valuable in the time of the pandemic because a lot of times you can explain how something might work. You can say, oh, if you build a testing site this way you can allow people to register online but that's far less powerful than being able to show people a demo of how a user might come along to this like free website and sign up for a COVID test. And just one example that came to mind is unemployment insurance and pandemic and unemployment assistance are huge topics right now. I would say affecting people in every state all across the country. And there was a lot of confusion around what are the requirements to be eligible for PUA versus traditional UI, how does a state implement it and so forth. And we started working on it. Today we've worked with around six different states and I would say we came in maybe thinking we could build something and then provide software. But what we actually found is one of the most impactful things we did was we built a demo, a front end like website demo that people can interact with and like pretend they were going through the application themselves. And in the end, as far as I know, none of the states we worked with actually used the underlying code to implement their PUA applications but they played with the demo a lot and had the UI and UX inspire their own invitations. And in some cases they gave the open source demo to their vendors and then their vendors were able to kind of mimic or learn from our or interact with the demo. So I just think there's actually a lot more power in open source now around the user experience being something you can share freely much more easily than before. Yeah, I mean, sorry, Angie, go ahead. Sure, a huge change exactly on that front is that there's just more open source software that is easier for standard folk to use, right? WordPress is open source, Firefox is open source. Those are two examples that we referenced in the report quite heavily because there are things that a lot of people use and they are referred to as frequently. I mean, you can take a look at Android, the entire mobile operating system, Chrome, along with Chromium is open source. There's tons of stuff that people are using right now that is top notch quality software that you can just put your hands on and use. And that is very different than where it started or where it has been, I should say, over the last five years. Yeah, Justin, in my experience, what I've observed is that open source has been more accepted and adopted at the government level at different layers. And that the principle of open source is being applied in different areas, right? Like it's not just about the code anymore. It's really is about how do you build that community of practice? How do you ensure that you're creating a better user experience? Like everything is interconnected, right? Like you can't just operate in a silo and the spirit of open source is being able to continuously share what you're learning and connect the dots of the different end to end services that you're creating for the people that you're serving. So that just, I'm curious, are there networks in particular that you have found to be really useful in creating that community or in sharing the work or even in like getting user feedback or it sounded like you had some experience you were thinking about, a specific experience, I don't know. Well, I would think it's this community, it's AT&F and USDS, it's all the different organizations that have collectively come together to share those lessons learned. Like it's all the work that the UK digital teams have been doing as well. I think I've been able to have the privilege to be part of these type of conversations with other great leaders working on amazing work and building better government services. And a lot of the work that they do really depends on sharing again those lessons learned and the code and the experience and the failures and the successes. So it's a network of networks in some sense. I don't like Mark or really wanna add to that. Yeah, I mean, it's definitely that network. It will say that one of the people that we interviewed in terms of, again, for the report, a colleague, Jeff Maher, he was saying that when he needs to find an open source tool, he'll literally just go to Twitter and ask the Twitterverse for recommendations. And that, again, goes back to the issue of discoverability, right? Like the community, there are certain places where the community centralizes, but it's still fairly disbursed and it can be very hard to understand and see what are the... I mean, we do have the Linux Foundation on the call. They're a really great and central organization that does quite a lot of that work. But even so, it can be tough to sort of find that aspect. It reminds me of, once I was doing a redesign for a large university website and we found out that the way people found information on, like the students would go to Google in order to get to find pages within the site. So I think that just, or just asking on Twitter is sort of the present day equivalent of I just can't find this someone telling where it is. So I wanna get into, we have danced around it a little bit, but the intersection of open source and data privacy and security. So once we start talking about using open sourcing government, obviously this is a thing that comes up. Once we start adding things like contact tracing into the mix, it becomes an even bigger issue. Can you talk about how, so and judging from the way it's come up already in this conversation, my sense is that the real issue is explaining or lack of risks to maybe leadership or getting buy-in. Can you just talk a little bit about how you have handled any data privacy or security concerns? What are real concerns? What are imaginary concerns? How to talk about that work? Angie, let's start with you. I think on our end, what we've done is coming to an agreement on knowing that the data that we need to protect must remain protected, I like. And in essence, what we're sharing is the code and anything else that can actually be shared. But there is a way for us to be able to explain the level of risks with the different stakeholders that often really are protective of the information that they hold and sharing the benefits of being able to share some of that information in a protective manner with other key people that may also benefit from that same information. I think one of the great things that California has recently done as part of the CalCAT rollout was also, there was a letter from the cabinet member to the rest of the administration saying that's any COVID-19 data that can be shared shall be shared unless obviously you can't share it for HIPAA or whatever reasons, right? And so in some sense that that's a commitment to ensuring that we're sharing the information that we know can share as part of the COVID-19 response while also holding the principle of data protection when it needs to be protected very close to us as well. Braylene, what's your experience been? Yeah, I think we're, part of what we do is we try to be pretty responsive to the government partner's needs and we're very flexible. So I'll give a few different examples. I think in some cases, totally agree with AMG. If you can, if you're thinking about aggregate data, dashboarding, data that you sort of might need to make use of across agencies for different purposes, there's a lot of power in kind of figuring out how do you build the right APIs? How do you build this right dashboard to visualize and digest the raw data? We've certainly done projects like that. I would say we also have had other ones where the data, let's say, is quite sensitive and actually should be managed by the government team themselves. So one example is we have this tool called Neighbor Express which we set up pretty early on in our existence. And what it does is it helps collect local residents who sign up to volunteer to deliver food and packages to homebound citizens. So originally it was started to deliver food to homebound senior citizens. That's an example where we're working very closely with the city and we don't necessarily want to be or need to be the ones who are collecting this information from local residents who are signing up to help or from the senior citizens who are homebound. So that's an example where we were able to build a tool for them in an open source way and show them how to use it. But in the end, we kind of handed the keys to the city office themselves. So from our side, we never really look at that data. We don't have any access to it, but we've helped them kind of manage their own list of people and that's another way that we can kind of accommodate both what the partner needs and what we think is right for the data that we're actually handling. Mark, anything to add? Well, I think I might take one step back to the general topic of talking about risk and controlling and managing risk in open source. I think that in general, a lot of the folks that are in government as government leaders have not had to manage open source projects as part of their career, which is totally fine. It's not their job. Their job is to manage government. And by and large, because of the ways that government has gone about procuring software, they're used to the idea of proprietary software or even building it internally and then keeping it internal. I do think that a lot of what Angie and Rayalene have been doing with their teams is showing the proof is in the pudding, showing that you can do these things and manage your risk extremely well. You can do these things and still maintain your data. And the more examples that we have, like these excellent projects, the more that we're going to see people accepting that actually the risk isn't about it being open source, the bigger risk is about already delivering things that actually work. That is ultimately the biggest risk that we have. This is something that Robin talked about in the last panel and Cecilia mentioned, public trust in government as a big theme in terms of service delivery and open source is one of the ways that we have to be able to do that to lower the risk of failure and deliver services because we get to build on top of tools that other people have already used. We get to build on top of tools that have been proven to work and we can short circuit the trial period of trying to figure out something that does work and just jump straight to work that other people have done. I think that any risks that people feel, things like, oh, is it secure? Is it something that we can rely on? We have seen through various, various, various different examples that, yes, we can rely on these things as they are secure and, yes, we can deliver to them. I'm curious, Angie and Raleen, if you have seen that bear out, like, are people more open to, open or, more willing to take the leap or, you know, is there, like, this cultural shift needs to happen, is that starting to happen? I think in some cases, yes. Obviously, as a public servant, it takes time for you to build a relationship with the key players that may be able to make some of these decisions. But there have been instances where, particularly with big, big IT procurements, something that we've talked about is the benefits and really weighing the risk of just remaining in the status quo. I think, at least in California, we've learned over the past that some of the big initiatives or projects that the state has taken on haven't been very successful. And that's not a mystery to anyone. And because of that, because of those pain points, I think we've also been able to build relationships and introduce some of those key decision makers into a new way of doing things. They're open, essentially, that's at least having that conversation. And knowing that it hasn't this approach of the past hasn't worked, let's talk about maybe building it in a different way. Being able to have an open conversation and kind of show them the way and, again, having the network of networks and being able to point to these type of examples is really a benefit to be able to kind of get people on board and get them willing to jump on a new approach with you. I would say we're very results and utility-driven. So I think one thing we heard is like, when we talk to a government partner and we say, hey, we've seen this problem, we built this tool, it's open source, we can customize it for you and get it running within days. Everyone's just like, yes, please, you know, the faster the better. And I think seeing open source as a means to an end is great, it just helps us speed things up. And so I think it's been more like proof that we can deliver things quickly versus a block or even something people are necessarily excited about on their own. All right, well, so we are at time. That was wonderful. I do want to open it up. If there is anything that we didn't cover or things that you think people should know. Are there, yeah, please add. Read the report, it's all in there. I think I'll also say I'm a relative newcomer and first started working with government technology as part of this response, but I would just say I'm kind of blown away by the moment that we're in. Obviously it's a very challenging time for government teams all across the country and people, but I also think there's a lot of happening in just terms of how quickly people are moving. You sort of see it even in the course of conversation sometimes, how quickly people kind of get it and are like, yes, this is gonna make a difference for people faster. They're sort of embracing the new technology and changes. So I'm pretty optimistic about that. And I think we can kind of use this time to try to kind of push a lot of these teams and systems into something that's better. I share that sentiment. This has been very inspiring and I don't think we could have done that without being able to tap into the principle of openness and sharing. Looking forward to learning more with you all. All right. Well, thank you, three of you so much for being in this conversation. This is such a critical topic, especially right now. And thank you for the wonderful work that you are doing and for sharing it with our audience today.