 Hi everybody. Good afternoon or morning depending on wherever it is you are. My name is Cecilia Munoz and I am honored to be in this company and very very excited to be helping host this event and this conversation today. So while the whole focus of the event is to lift up amazing work that's going on, that's focusing on helping people access benefits, we are also here celebrating a new book which is called Power to the Public. It's a blueprint for how governments and nonprofits can harness the power of digital technology to solve public problems. It was described by no less than President Obama as a good read for anybody who cares about making change happen and I can vouch that that is true. It is a great work and it's also inspiring because it shows how we can solve problems even problems that we think of as intractable problems and we have the authors of the book here today so I'm going to introduce them first. Hannah Shank was one of the early members of the U.S. Digital Service and she's a founding member of the Public Interest Technology Program here at New America. Tara McGinnis was a senior advisor in the Obama White House. She recently ran the domestic policy team for the Biden transition and she founded the new practice lab also here at New America and the way we're going to proceed here is that the authors are going to interview the state leaders and digital collaborators that they wrote about in the book who were part of a really important and inspiring transformation in the state of Vermont. So one of those leaders, Genevieve Gaudet, is with us the director of design at Nava, building a more people-centered and equitable government. Prior to Nava, she co-founded the New York City Service Design Studio in the office of Mayor Bill de Blasio and because these kinds of partnerships need awesome government officials, we have Cassandra Madison, an experienced public servant whose expertise sits at the intersection of technology, operations, and policy. She's currently VP of partnerships at the Tech Talent Project and she spent the last 15 years helping to ensure that the big ideas get implemented in a way that drives innovation, improves people's lives, build a positive culture in the workplace, and Cass served as an appointee of Vermont Governor Phil Scott which is how she enters this story. But I want to start our conversation before I kind of turn the microphone as it were over to Tara and Hannah. To start, I have a short conversation with Shaw Wong. He is the COO and co-founder of Nava, which is a consultancy and a public benefit corporation, which is all about making government services simple, effective, and accessible to all. Nava was kind of born out of the experience of rebuilding healthcare.gov after its troubled launch. And now they work with federal and state programs like Medicare, Veterans Appeals, Medicaid, and SNAP to improve their digital services. So we are very grateful Shaw that you've joined us. Thank you so much for being with us. And I'd like to start the conversation with you just by asking Tara and Hannah have thrown themselves into sort of telling these stories about how delivery of government benefits changes. I'd love you to just tell us why is it important to tell these stories? Why is this necessary for folks to hear? Thank you so much, Cecilia. And before we get into that and answer your question, I do want to spotlight your experience, Cecilia. Cecilia Munoz served for eight years as President Obama's senior staff, became the nation's longest-serving director of the Domestic Policy Council. Before the White House, there are some longtime fans of yours at Nava. She served for 20 years at the National Council of La Raza, now Unidos, U.S. Her name is Carther Fellowship for her work on immigration and civil rights. In 2020, Cecilia published an award-winning book More Than Ready, a book for lessons for women of color on the rise. And now Cecilia is a senior advisor at New America, living in Maryland. I wanted to pass on really some gratitude and admiration. I'm actually one of the executive sponsors of Juntos, which is our Latinx employee resource group at Nava. And so our leads, Mario and Vanessa, wanted to share a really deep appreciation and admiration for your time and work through the decades. So again, you have a lot of fans at Nava. That's so nice. Thank you so much. Yeah. And I think to go to your question, why is it important to spotlight and share some of these stories that are both challenging and rewarding? I think the power of the narrative and the case studies that Tara and Hannah are bringing here are showing really the difficulty and the patience required, but also the impact and the practices and the shared lessons that can be taken forward. I'm reminded of a statement that Sarah Schulman made who is a kind of historian and author who interviewed hundreds of folks involved in ACT UP, the kind of AIDS awareness and activist group that was active during the 80s and 90s and still today. She put together this collection of narratives and oral history of the group and what she said, which resonates here too with the work that Hannah and Tara are trying to spotlight. She says that focusing on heroic individuals or these kind of shiny, glossy, heroic narratives, aside from being inaccurate, can mislead contemporary activists away from the fact that in America, political progress is won by coalitions. And that resonates very deeply with the roles that I think Nava feels like we have to play in this space, the roles that organizations like New America have to play and the varied, I think, perspectives and stories being shared through power to the public. So it's incredibly important, I think, to shine the spotlight on both the challenges and the rewards here. Thank you so much. It's a wonderful way to get us started and I should just say that Nava has big fans at New America. I think that's part of the reason that you all are a character in the book, in a sense, because there's a good story to tell here about how we can do a much better job of reaching people and accomplishing what the government sets out to accomplish. So thank you for what you do. Thank you for being here with us. And with that, I am going to turn the microphone over to Tara and Hannah to lead this next part of the conversation. And I hope that folks who have joined us will start thinking about your questions because we're going to have a conversation with all of you as well. So Tara and Hannah, over to you. Thank you so much, Sha. Thank you so much, Cecilia and Sha. I think in particular, I'm going to jump in right away and make sure that for folks who are grounded in who we're serving, we start right there. But I hope people can see that the mission of the book and the event is to show how many types of people. And just as Sha, you know, I think underscored that this takes a kind of village of different folks who lead companies and folks who lead domestic policy councils and design directors and storytellers and important, important public servants of all different levels of leadership. And we're hoping that you see a number of us here that if you're listening at home, you see a place for yourself. There's many different ways to put your hands on making change. But I want to start with the people we're serving and maybe just we're going to go into a deep story about Vermont. But before we go narrow, just take us a step back from your broader work on government benefits and why does it matter? Yeah, well, I think it's important to start that when we say government benefits, certainly at Nava, we're really talking about services that support vulnerable populations. And this can mean health care beneficiaries, veterans, low income families, you know, the un or underemployed folks with disabilities or experiencing homelessness. And it's pretty difficult to understate how broad of a population that is, I believe, Medicare last year served 44 million. Folks snap around 40 million Medicaid, I think is in the 75 million. And, you know, just in the first few months of the pandemic, I think about 45 million Americans ended up claiming unemployment insurance. So we're talking huge swaths of the US here. And that's certainly, you know, why Nava's focused there because these services have the size and the scale and complexity that when we improve them, we're able to have a really big direct impact. And we're also be able to tell, you know, really credible story about how government is capable of meaningful change in serving people's needs on a reasonable time scale. So, Cass, why don't we start with, let's get in, let's dig into the story here. So can you tell us a little bit about how the state of Vermont got involved in this integrated benefits project in this idea to streamline or simplify how people apply for benefits? Sure. And first, let me say, too, it was such an honor to be a part of this book and to work on this project with Genevieve. I feel like I've been inducted into this family of public interest tech. And it's a really wonderful community, and it's an honor to be a part of this story as it unfolds across the country. So in Vermont, setting the stage, so when we got involved in the integrated benefits product, the state was really on its third attempt to implement this integrated eligibility system. And I'll put system in quotes because it's this unicorn we had been chasing for 15 years. The idea was, if you are a Vermonter, of which nearly a third of the population is receiving some sort of Medicaid benefits, a huge swath of the population, but if you want to apply for a benefit program, whether it's SNAP, or fuel assistance, or Medicaid, or even health insurance coverage through the health insurance exchange, you should be able to visit one website, call one call center, fill out one application. And the previous iterations really relied on this big bang approach to delivery. The intention had been to sign one contract with one big vendor and to do all the planning and implementing in a really linear way. And because of a variety of reasons, which we can get into, but you think about project complexity, budget size, the fact that we were continually prioritizing documentation over working code, the project could just never get off the ground. And a bunch of dollars would be spent on planning, and then it would be canceled. So we were on attempt number three, and we were really determined to do something differently. Didn't really know what differently meant at that point in time. And really decided to partner with some of the organizations across the country like Nava, Code for America, 18F, Center on Budget and Policy Priorities, who were really helping lead the charge and helping governments think about delivering tech differently. We knew what we were doing before it wasn't working, and this became really an opportunity, you know, a doorway to change and to partner with some organizations that had some tools and principles and practices that we thought could be really promising. And that turned into this document uploader project. Can I ask you for one second? Like, how did you, how did you come upon these, these friends you were in Vermont? What was your role? What were you doing before we jump into the experiences of the way the Affordable Care Act maybe informed the Vermont experience? Just like, tell us about the kind of really the state, the state perspective. Sure. So I guess the helpful history here is that I, you know, I had worked in nonprofits for a long time. I joined government as a policy person right at the start of, in 2012, when the state was implementing its health insurance exchange. And I was bright-eyed and bushy-tailed and optimistic about all the great ideas we had about, you know, we've even talked about the health insurance exchange sort of being the gateway to single payer in Vermont. We were going to give everybody health care and just had this front row seat to what happens. No offense to policy people because I am a policy person at heart, but what happens when you have policy people in charge of implementation? And, and it really all fell apart. And so the history of integrated benefits initiative starts in a really difficult failed project. And, you know, for as bad as it was on the outside, people's perception of it, it was way worse on the inside. You know, customers, the stories of the customers who couldn't sign up for benefits or who couldn't pay their bills or who couldn't change their information in their accounts were gut-wrenching. And not only that, but staff were stressed and they were at their limit and they were in tears every day because they were getting yelled at and there were endless backlogs and they felt helpless. And then they went home and saw their employer on the news. So it was just a bad situation all around. And quite frankly, I just started jumping in where I was needed. And it turned out I really love operations. I really love tech implementation. But so, so my searching for something different came rooted out of the sheer pain of that experience. And, you know, it took us years to get out from under that and to stabilize things and to gain the public trust. And to this day, I don't, I think when you say Vermont Health Connect, it's still, it's it still has a negative connotation. And I don't think we'll ever get out from under that. And so, you know, I think I think somehow I stumbled upon like the Code for America conference and that just like started started me down this path. And I realized and I think I said this in the book that a lot of the things that I felt intuitively like, why are we doing such big projects? And why aren't we thinking about our users? I realized that like, oh, there's a methodology out there and there's a language and there are people who are thinking about these things. So, you know, we really said everything that we did, we're not going to do that again. We're going to do the exact opposite with with integrated benefits initiative. And that's where we really started to say like, we're going to take, we're going to prioritize our users, we're going to take incremental approaches. We're going to implement modern tools and practices. And we're going to evolve constantly as we take in new information and gather new experiences. And so, you know, this document uploader project with Nava really became our first small step with a user centered like agile project to really help us turn our IT delivery model on its head. So you fell down this rabbit hole of this CFA 18F public industry technology rabbit hole. And you sort of still in it, right, fair, still in it. And in your rabbit hole, in the rabbit hole, you gave me Nava. And as you mentioned, so the plan was to start small with this document uploader. Genevieve, can you talk a little bit about what was the thinking on the Nava side around starting small? Yeah. And I want to shout out that we wouldn't have been able to do that if CAS wasn't open to it in the first place. So certainly that that willingness and the partnership from Vermont was really critical to doing anything. And also that, you know, on the ground when we were first looking for problems that we could solve, you know, on a reasonable time frame and Code for America and the Center on Budget were partners in that work to identify what were possibilities, you know, we could bring to CAS and say, like, here are a few problems that we think we could solve. And the uploader came out of, you know, a very rapid sort of site visit discovery period where we interviewed folks along the delivery path in Vermont. And, you know, everyone from CAS to the folks at a field office who are meeting, you know, Vermonters every day and helping them process applications. And it came forward that getting documents, so like maybe a pay stub or proof of where you live or proof of who you are to the programs so that they could process your eligibility was a very manual process, both for the staff who are actually kind of determining eligibility and for the Vermonters who needed to submit them. So you could mail them in, you could bring them in person. And all of those are expensive and are problems that we can solve pretty quickly with technology. So we knew that we started with this, we could provide real value to the state, to Vermonters, and we could do it very quickly to get some momentum behind, you know, CAS's vision that this could be very different. Can I ask you, I think this question of momentum that today raises is real for folks in the work. And can I ask you to talk about buy-in? You've been to the Code for American Conference, there's a story about really taking a demo to the floor of the Vermont Statehouse, I don't know if that was the first ever demo day in the Vermont Statehouse, but can you talk a little bit about what it takes to bring a whole state or team along? Absolutely. And this is this is one of the reasons why I love the story in the book, because it's not a fairy tale, right? Like getting buy-in is really difficult and some things went really well and some things are really hard. And I think there's lessons to be learned in it. When I think about buy-in, so first things first, like the great starting point that we had was there's literally no one, whether it was the legislature or the frontline staff or the executives that thought the way we had been delivering technology was working. So if you can agree there is a problem, that is an excellent first step. So that was an easy box to check. Now, agreeing on what the cause of that problem is or what the solution is to that problem is much harder. And so for us there were really, I would say, three key stakeholders. One was my leadership within the Agency of Human Services. That was actually pretty easy because they had seen me in the trenches and our team in the trenches on Vermont Health Connect and had lived through some of that pain too. And so I think because of that leading that turnaround, I was lucky enough to have their trust and support that had to be built over time. The second key stakeholder was the legislature. And that was actually, it was challenging, but not unmanageable. Because again, they also had all of their constituents calling them when Vermont Health Connect failed. And so they were ripe for change. And talking to them about tackling projects in smaller increments, aka reducing budgets, and focusing on delivering working code that would actually make people's lives better instead of endless planning, resonated with them immediately. And even with the incremental approach, for example, like right away, we cut our budget from $36 million to $13 million. So it was an easier sell. I think the biggest challenge we had was within within our own Agency of Digital Services, which is essentially our centralized IT. And I think right there, there was some new leadership that hadn't experienced the pain of Vermont Health Connect. And like what happens in a lot of government is you there tends to be some infighting at times about like who's in charge and who's going to make decisions. And that can really you can get in your own way. And so I think one of the big lessons learned is like the need for that executive alignment and buy in, which I know we'll get to later. But speaking of the demo, I think looking back on it, the single most effective thing that we did was to turn the tide with stakeholders was to show them a product that works. And you're right, we had never done that before. And in some cases, we had never produced working code before. So in the case of the legislature, like they were so used to us promising to deliver something and that day never coming that with the document uploader, we brought together the four committees in the Senate and the House that were critical to approving the funding for the project. And we took time to explain the overall approach to them. And then we demoed the product. So we gave them a fake pay stub and the link to the uploader and we had them actually use it themselves. And even the legislature legislators who had some challenges with technology were able to use the uploader and they were blown away. Not only because the technology itself was simple and it was not fancy. Like this was minimum viable product situation. But it was simple. They could use it themselves. And then we were able, because we were already piloting it, to immediately show them stats to say, this is how it's improving people's lives. So, you know, we've piloted in this place. We see, you know, X percentage of people are uploading documents outside of regular business hours, the average time to eligibility in the pilot went from nine days to four days. Like the impact was immediate. And it was really a game changer across the board and became the model for how we wanted to do everything going forward. Like no more big launch days, we're going to pilot everything and we're going to show people code that works. I love that story. I just also love the visual, the image that that conjures up like going into the state house and doing this demo. And it's, it's just, it's great. So, can let's back up for a moment and talk a little bit about the pilot, which she said was at this point was already running by the time you demoed it. So, Genevieve, can you talk a little bit about, so you started this pilot, it was in the Bairie office with 50 people. Why the, why that office, why 50 people? Yeah. Well, in the book, as we mentioned, that office had Jimmy, who was a supervisor there, who was not only willing to let us, you know, sort of do this in the first place, but was also willing to, to sit with us to collect metrics, right? That's the whole point of running a pilot, is to see, does this even work? And because of the way some of the Vermont systems were set up, we knew we needed to collect those metrics manually. And Jimmy was excited enough and invested enough that he was willing to sit down with us and go through every single case that had used the uploader to collect the metrics that Cass just mentioned then. And then the 50 comes out of, you know, it's, I think anyone who works in government knows that launching anything is terrifying. It is scary to put working technology out into the world. It's why we pilot and we wanted a number that was big enough that we could get metrics that people would believe, you know, if we did five people, it was, it would be like, oh, that's not really enough. But we didn't want so much that we were going to terrify our stakeholders, you know, we didn't want to put Cass in the news again. But we wanted to be able to prove that it worked, that we could reasonably measure it, you know, and get a big enough number that we could build the case to actually rolling this out to other offices if it went well. And it turned out it did go well. Can you bring us up to speed? Like what, what is, where is it today? I know book publication takes a while. So tell us about the state of things today. Yeah. So at the end of that pilot, what we were able to figure out, in addition to some of the things Cass already mentioned, we saw that over half of people were sharing their documents with the state within 24 hours. And only about 10% of folks have been able to do that before. So this means that about 10% of people who had gotten a request for documents where either were probably driving physically, taking their documents by hand into an office. And then we expanded that with the launch of the uploader to 55% of folks could get their documents within a day. Because of that, as Cass mentioned, we're able to drive down the time it took to get a decision. So it was about like 45% faster, I think. And then we saw some other interesting ones were about half were coming from phones, which is great. Folks aren't having to like go into their computer to do this. And about a third were outside of business hours, which just wouldn't have been possible at all. And so using those stats, we were able to then roll it out to a second office. And in that second office, we worked really closely with a product owner, Thani, who has also mentioned in the book, to kind of work with her staff on that second rollout so that any future rollouts could actually be handled by the Vermont team. So they could do the training, they could talk to the caseworkers. And after that, it just sort of kept rolling. And now I think it's rolled out to all of the regional offices in Vermont. And so this story has a coda, though, on the end of it, which is that, and I don't want to speak for you, but obviously, when you run a pilot, the hope is it's going, it's the first step in a very, very long process, long modernization process of moving forward. Can you talk a little bit about where the project is as of today? Sure, it was that question for me. Sure. So after the uploader project, Nava also began to work with the state of Vermont on combining the applications for the integrated benefits program, which included the health care programs like Medicaid and economic services programs like SNAP. We started with Medicaid, but because of some procurement issues and kind of some of the issues that Cass had mentioned, we weren't able to continue that though I believe, like all integrated benefits and long term modernization initiatives, I believe the state is restarting some of that work now, which is very exciting. Cass, did I get all of that? I already captured that right. So this is such an interesting conversation, and it is also it's really so affirming. It's like lovely to see how the story turns out. I want to remind folks that there's some folks who have put questions either in the Q and the Q and A or the chat. We're going to start to ask them, so please keep the questions coming. I want to start though, so this is a story specifically about Vermont. Does do states, I mean at some level, every state grapples with its own versions of the same thing. Are there ways that these kinds of lessons move from state to state? Like how do you, how do we get to a place where things are going more smoothly everywhere like Cass, you un-knewed it, so let me turn to you. I think pausing on lessons learned is really important here. I think the power of the Vermont examples that we're able to show that working differently produces better results. So that's a really exciting baseline. I think the two other biggest takeaways, I think for me, that I think are helpful for other states. One is just, and this comes out in the book, like the need for executive alignment is critical because you need permission to be able to do things differently and know that at all levels of leadership that folks have your back. The other big thing I think is that could be helpful for states is trying to share lessons on how to work with vendors. Appropriately, I think a lot of states are struggling with entrenched vendors and vendor lock-in. And when you start to shake things up a little bit, it's hard to get your entrenched vendors to play nice in the sandbox with new faces. And there's really no leverage to be able to do that. And so I think one of the shared lessons is for states is how to work with vendors more effectively. And I know AT&F is doing a ton of work with procurement, which is really helpful there. And I think organizations like New America, like Code for America, can be conduits of information and bring states together to be able to share what's working and what's not working, which can help move everybody forward. And I think we're doing it pretty well at certain levels, but I think it needs to happen from the executive level all the way down to sort of the frontline staff and really create those communities and share those best practices. Thank you. And I'd love to open this up also to Sean, Genevieve, right? Because you work with a variety of states and you work on a variety of programs. I mean, there's a logic to the idea that if you build something in Vermont, you should be able to use the same thing in Montana or somewhere else. Like, does that hold true in your experience? Like, how do these things move around or do they? I think what we've seen is that the needs of constituents or beneficiaries and even the needs of state staff don't change so much from state to state, but sometimes the implementation of the technology does because you need to make it work with whatever system is already up and running. And Namba makes a specialty of that. But yes, in general, folks need to upload their documents in Montana, maybe even more so than in Vermont because your Montana is really big. And so there are a lot of lessons to be learned there. And I think that's really exciting because it means that we don't need to do the same research all the time in every state, but we do need to be able to translate it to the technical and operational needs within each state. Thank you. Sha, anything you want to add? No, just to, I mean, just to plus one, everything that Cass and Genevieve said, I think one of the founders of the UK's government digital services, Ben Tarrett, used to say things like bad services are nouns and good services are verbs. We can get caught up in the nouns of these programs which acronym does which things, which way, how does Medicaid define a household, how does TANF or SNAP. And there are so many nuances, many times for good intentions and good reasons, adapting to the needs of these specific programs. But when you pull it away and when you step away from the acronyms or from the nouns of these things, what you come back to are some really shared verbs like uploading documents, verifying identities, kind of confirming, validating household information or submitting income information. And when we're kind of, for us at NAVA, I think we're trying to kind of carve a way back to those kind of common verbs and being able to kind of validate or invalidate the things that are common across different programs because this document uploading is not just relevant for health or economic programs, it's also relevant in many other situations. So I think it's hugely valuable and I think that internal tension that Cass pointed to of kind of some of the silos or kind of entrenching of ways of working, I think can often be kind of productively upset when coming back to these lived experiences and these kind of common verbs. That's a really, really lovely and interesting way of thinking about it. Thank you. All right. So we have quite a number of questions from the audience. So I'm going to start with one from Kara. From a civic tech perspective, how do you balance disrupting the status quo by offering tools that operate outside the system, which may feel like fast solutions, versus working with state agencies to transform the system, which is feels more like sustainable solutions. That's Genevieve, you look like you're like you're pondering the question. I guess I don't think of them as all that different. Offering tools that operate outside the system versus working with state agencies. Oh, are we saying I wonder if this question is sort of looking at the difference between tools like maybe Code for America's get CalFresh, which has been hugely successful in getting Californians, food assistance versus maybe working internally with an agency like we're talking about in Vermont. I think our community balances us in that way. We need a diversity of solutions. And I think there are some things you can do outside of state systems to really speed up or draw attention to certain issues. But personally, and I think at Nava, the approach we're taking, which is not better than it's sort of counterpart is that long-term sustainable change needs to happen within agencies because that's where we have the lever to impact operations and culture and these supporting technologies that can really drive outcomes that I think government is accountable for. Cass, I see you nodding. Anything else you want to add? Just plus one to Genevieve. I think there's you learn very quickly that there's no one-size-fits-all in government. And that some of the fantasies we have about just like build one thing here and it will magically work everywhere else or it's just harder in practice. And so a diversity of approaches allows you to try things and then amplify what works and move away from what doesn't and that's harder when you try to do things in a just pick one approach and go with it. I'm just scanning to see if anybody else wants to dive in. In which case we will go to a question from Paul. Agency CIOs and CIO offices and the effective delivery to modern government digital services helping or hindering like what role should agency CIOs CIO offices play especially in states? Any thoughts on this? Cass? I definitely have thoughts. I think the role is monumentally important. You know I think you need the technical leadership and the business side leadership to be working together to be able to do things like change strategy, prioritize users, etc. And you really have to be aligned on business goals. You have to be aligned on how much room you're going to give folks to experiment and try things differently. And when that alignment doesn't occur you can really you just end up tripping over yourselves or swimming upstream and it makes things harder harder than it has to be. And I think there are some great examples across the country well at the state level and the federal level of CIOs really leading this charge and building cultures within their agencies that are modern and are willing to take risks and I think we need we need much more of that. Thank you. Connor, Tara, do you want to add anything to that just from your experience within government? You were both alumni at the federal level. I'll add that I think the way agencies are structured right now the CIO office it's a make-or-break scenario depending on who's in there for how services get delivered. Whether things need to be structured that way I think is a good question but I agree with I think you know with Cass's assessment and it it's very very dependent right now on who is in that in that seat. Fair enough. Okay a question from Brendan. How do you set up for success across new political leadership and administrations? Alaska was part of the co-america integrated benefits program but things stalled with a new governor and a new administration. Tara do you want to start on this? I'm happy to I mean I think this is this is the challenge transformational change often takes a while and so I think you can see with the panoply of different folks here. If your project resides exclusively in the political architecture and it's unlikely to sustain itself this was a big challenge for us in the Obama administration. I spent a lot of time with Cecilia in the final year it's really making sure that the table in which ideas can be surfaced includes both folks like the CIO who might not be there where the kind of problem is put at the center of the table but also long-standing career leaders and frontline workers and you know often if the project is branded with one political leader it can lead to challenges but if the practice is shared by folks in the you know in the Bari office where that you change a bit of practice change you think you have the opportunity to keep things going. So I think individual initiatives Brendan can be challenging but I also think if you are doing something that's mission critical and you want your name it city county state agency to do for a decade and you're and you look around and can see only one of these categories political leaders career leaders and outside technologists a single vendor then you're not building the web for change. Sometimes you can have all of those things and still get stuck it's part of what we eliminate but I do think really being broad about where innovation comes from and being you know having big ears and listening for the idea that you may be coming to the table with someone else may have tried and failed a decade ago and have lessons learned from the front lines to be shared. Thank you. Anybody else want to dive in? One thing I would add to that is that I know this isn't true everywhere in the country but probably true in Vermont that political leaders can come to technology begrudgingly it's not you know or something that they fear because they don't understand and if you can get to a place where you're delivering technology regularly in small increments and it's working you're much more likely to be left alone to just keep things going right if you can keep keep the stories out of the news and keep services improving it's a good way to just be able to it becomes a part of your day-to-day operations and that's the goal instead of a big project that we're all going to like take time out on and want to know the status of that this is just your incrementally improving technology it's a part of your day-to-day business and how you run your operations and it's just a part of running good operations and it can allow you to weather the political changes more easily. Can I also add I think this is why we saw in the integrated benefits initiative that it was so important to start small we saw that a lot of the water that would be carried across administrations was carried by you know frontline folks or people who weren't political appointees and so you know having something like the uploader live and running means that's not going to go away whereas like maybe the big unicorn master person management project is just not the priority anymore and that one sort of fades off into the to the you know the legacy of the agency um so it's it's not so much a setup uh question but it is it is one way to see see these projects kind of continue because you've already delivered the value and because they are as Cass said kind of in the DNA of those operations um so there's another question they're now like flowing in so I'm struggling to keep up but there's a question from Natalia that I think relates to a conversation that we started here which is how beneficial is it when states like California contract out to private companies to provide better services is it sustainable or effective compared to change within government agencies themselves right this is attention that we that we deal with thoughts about this I'm happy to touch on it a little bit I think the um I think in addition to what Hana you said around the CIO's office or seat being make it or break it um frankly what we've seen in our experience is that too too often the agencies themselves do not have um the funding or the resources or the in-house um kind of talent pipeline to uh to do this themselves um and that's that's a incredibly uh that that's a real problem right where not only is uh the then then you're not only kind of outsourcing to vendors some of your work but you're also beginning to outsource the literacy and the oversight and frankly the accountability about when things don't deliver the outcomes when you don't end up like Cass mentioned with something to demo um that that's actually something uh that that's pretty important to Nava is that the government this is why we do work for the government we believe that these agencies and programs should be able to keep the promises that legislation is making to the public and that's why we actually um actually endorsed a bill that Senator Wyden and Murray introduced a few weeks ago uh around funding state uh state and local digital service teams so I think that's a you know there are decades I think of under-investment or things like office of technology assessment or congressional research service that have kind of uh dwindled over the last couple decades but I I am encouraged and excited by uh you know California's office of digital innovation Colorado's digital service uh Connecticut I think has a digital service team as well so this type of renewed investment I feel like creates pulls a lot more agency to use a overloaded word back into the agency right which I think is where it where it should be and then if there are folks like Nava who can help agencies accomplish or make accelerated progress against their mission or pull from other experiences from elsewhere and other programs or other levels of government then great um but but the mission does need to live in the agencies and not be outsourced I see vigorous nodding thank you um so I'm gonna read a comment that was sent to the panelists which relates to a question which got sent in by somebody else so Jennifer just wrote I love most that Nava and Vermont focused on the most vulnerable those who are historically marginalized too many approaches do whatever was done in the private sector and backlog what they refer to as the nice to have which is so flawed this shows what's possible when we prioritize those most in need thank you for this approach so I thought I would say that out loud and it connects to a question from Sonal how do we incentivize vendors and government to be more explicit about bringing an equity lens into government and IT projects how are outcomes being measured by race and ethnicity so thoughts on this question well I would say I think one of the things that we try to make visible in the book is that if you do the things that we are showing that other people other teams have done in the book like engaging the community and looking at real-time data and talking to users that part of what is wonderful about those pieces is not just that you get a better product that actually serves people but that you get a product that serves everybody's needs um in whatever form those needs might take so if you really are digging into actually who are the people that we want to serve and how do we best serve them you can't help but have an equity lens we do also talk about one of the things we discussed in the book is that teams who prioritize bringing people with the right lived it with the applicable lived experience onto their teams also benefit tremendously from that point of view and having that perspective on the team anybody else have anything they want to add Tara I think you know this is really pronounced in a bunch of our work I might just lift up unemployment insurance where you know black and latinx workers face a greater challenge receiving unemployment insurance and their white counterparts while they're overrepresented in the unemployment data 78% of white workers make up 50% of the unemployed workers with 78% of UI recipients and black and latinx workers make up 40% of the unemployed workers but less than 20% of the recipients they're there um it is often really difficult to sell how well we're serving people when we can't disaggregate data to see how well we're serving people and so I think um I lift up the unemployment insurance example because it's truly pronounced and some of it is very old as to who was cut out of the policy from its origin in our labor laws but some of it is baked into very technical things work that we've been doing with marine and it's in California there are a bunch of American names that cannot be recognized by government unemployment you know insurance system Cecilia I think you have one of these two character last names we may have a few books and so if you if you test to this critical question based on the John Doe user and not on a four you know four names to which are the same then you invisibly lock out a bunch of people so I think the the kind of threads of which are structural and the threads of which are technical get intertwined at some point in time and this is a space where we have tremendous opportunity but I do think that the kind of data lens that has generally raised like basic collection and see how we're doing for these 50 people because often reveal um we can't see the barriers we're putting up for some populations and so can't underscore how important this is in our work at the lab and really you know the importance of diverse teams at the state and local level reveal the kind of invisible challenges you wouldn't see if it's a team of John Doe's no one might think about two character last name yeah I'll just say that this this person with two first names and two last names which is a very Latin American thing and I has never found a government form that gets along with my name is very grateful for that perspective but Cass I stepped in front of you sorry no thank you um I would say in Vermont uh we needed to do better on this so I will fully admit that um a couple other things that come to mind for me one is that working with advocacy organizations in the community that really know and have trusting relationships with vulnerable populations is was really critical and we needed to do even more than we did the other lesson learned was um you know the importance of pairing the business process work with the technology because technology for a lot of groups isn't going to be the answer so when I think about the document uploader we were able to see that the pre-document uploader 50 percent of the traffic in our district offices was just people dropping off paperwork and if if half of that 50 percent really wanted to be able to do it on their mobile device that is great and that would reduce traffic in the district offices so that folks who had trouble um navigating the electronic systems whether English was the second language for them or they had a disability barrier then they could go in person and they would have much more space to be able to have their needs met so I think when we it's hard not to get enamored with technology but it's important to remember that that's not going to solve everybody's problem that is a very convenient channel for a certain part of the population and you still need to pay attention to how you're going to serve other people for whom that is not uh the the primary channel that they you know trust or can interact with amen amen to that so I have like a bunch of other questions like teed up and ready to go but we're also running out of time and I want to give each of you a chance to answer a closing question and this is right the proverbial you know if you had the magic wand or a few minutes with the senior government official that wants to address the challenges of benefits delivery like what is the most important thing that that that that this person should focus on like what's your best nugget of advice or the thing that you most want those government officials grappling with this stuff to know and I'm just going to go across my screen but so let me start with you Cass I would just like to play back for everybody Shaw's last answer because that's my answer but it's it's worth repeating um that talent to me is the biggest issue probably not surprising because then I left the state of Vermont to decide to join the tech talent project which is all about inspiring people to serve in government um but I really feel that delivering better technology is impossible without the right people in the room and we need government leaders and teams at all levels with um empathy curiosity commitment to mission and modern technical expertise and I think to Shaw's point like time and time again government seeds not only the delivery of the development of the technology but the strategy and the oversight and the decision making to vendors but as a government official you cannot outsource risk so when things go wrong I promise it's still going to be you in front of the tv cameras it's still going to be you in front of the legislature and so I would love to see um government leaders internalize that and really invest in talent and I think it's not just in bringing in new humans although we need to do that but it's also in um investing in the civil service and upskilling people so that we can meet the needs of 21st century technology and service delivery thank you so much Tara I will plus one the talent plus two the talent and also add I think um I think if the project you're aiming at is about is like the goal is a uploader the goal is a website then um then we've missed the opportunity to center on people and I think um and you know really making sure that clarifying is incumbent upon those of us who are in the tech translation or tech work that if you get too far afield on the on the pathway to increasing access um that you often can get really lost and so I'd say like yeah on the checklist manifesto of government benefit delivery if your headline goal doesn't involve clarifying very clearly how you help humans go back and work on the headline goal thank you Shah how about you what's your negative advice I'm I'm feeling uh a little nerdy and lame getting stuck on something but I can't get it out of my head so I'll share it of um that that agencies should understand that they own the systems uh that that have been either built for them or built by them that that there are contractual ways that people can exercise accountability but too often feel disempower it's too often frankly an abusive relationship between vendors that have all of the technical expertise and control uh when and and then cast like cast mentioned the the agency is actually the one that holds all the risk um so I that that from my perspective is such a um it's such a degraded dynamic uh that I would really love to be inverted and kind of reset amen Genevieve yeah plus four to everything everyone has already mentioned I think one phrase that cast said um to not get enamored with the technology really stands out to me I think I would pair that with Tara's coming around orienting around the outcomes like pick one outcome that you really want to impact and then use the simplest tools with the lightest connection to just put something into the world to try to solve that you know and it's often just this the the simplest least sexy technical you know approach that you can take is probably going to get you the outcomes that you need and and you can build from there thank you kind of the last word goes to you um I would um ask them to come with me to the benefits office and apply there you go nice this great great and thoughtful way to end the conversation which is a really really terrific conversation participants you will have seen in the chat both links where you can purchase your copy of power to the public but also links to a survey if you want to hear your your thoughts about the conversation today um thank you to our participants thank you thank you thank you to our panelists not just for a great conversation but for the work you do in the world which could not be more important um we really appreciate you thanks for being here and we will close out