 Hello, everyone, and welcome to our event, Routing Out Vaccine Rollout Failure. We know your time is at a premium, so we'll just jump right in. Before we start, I'd like to just give you a quick overview of why we're doing this event and introduce our speakers. So for those of you who are not familiar with New America and our public interest technology program, New America is a bipartisan think and action tank. We're a civic platform that connects a research institute, technology lab, solutions network, media hub, and public forum. New America's programs cover topics across the board from education to labor to political reform and our public interest technology program, which we call PIT for short, focuses on strengthening nonprofits and governments by providing them with the technological support and tools they need to deliver outcomes that better serve the public. We also adopt best practices in human-centered design, product development, process re-engineering, and data science to solve public problems in inclusive, iterative manner. So this event came together very quickly, and it came together because our director of strategy, Hannah Shank, was fielding questions from reporters pretty much daily about the vaccine launch and why it was so unsuccessful. With the issue getting so much coverage right now and so much conflicting reporting, we thought that providing some foundational knowledge would help people deepen the collective reporting about the topic as well as provide a good background of understanding of why what we're seeing today with people struggling so hard to get appointments and this huge lack of public clarity was really completely predictable. So just a note today's event is on the record, and if you'd like to follow up with any of our speakers, feel free to reach out to me at bannonknewamerica.org and I'll drop it into the chat. So with that, let's introduce our speakers. Oh, and before we do that, I'd also like to give a nod to New America CEO, Anne-Marie Slaughter, who's joining us today as well. So our first speaker is Brendan Bab. He's the State of Alaska's Anchors Chief of Innovation Officer and their I-Team Director. We have Sasha Hasselmeyer. He's a public interest technology fellow here at New America and as a social entrepreneur using public procurement as a tool for change, Sasha has led government innovation projects with more than 135 governments in more than 40 countries leading to better outcomes in areas like transport, energy, health, education, economic development and social care. We have Rafael Lee. He's on the project's team at the U.S. Digital Response and the administrator of the U.S. U.S. ER Health Data Program. Zina Nee is a designer, organizer and artist who believes in public institutions that work for everyone. She's leading New America's sprints around the vaccine rollout. Zina actually has another event at 1225. So another fellow from her team, Alberto Rodriguez Alvarez, will be handling the Q&A. Alberto is a policy innovation analyst and strategist here at New America. Hannah Schenck is the one that started it all and she's the Director of Strategy for our PIT program. And she's also the Acting Co-Director. Hannah, along with another New America colleague, Tara McGinnis, has a book about public interest technology coming out this spring called Power to the Public. It's actually a fabulous primer for anyone who wants to learn more about PIT. We have finally Latanya Sweeney, who I know many of you already know. She's a professor of the practice of government and technology at Harvard University and the Director of Data Privacy Lab in the Institute of Quantitative Social Science at Harvard. She's also the faculty dean in Courier House at Harvard. And with that, let's get started. So I'm going to turn the presentation over to Hannah so she can give us a quick overview of the problems we're facing. And then we'll ask each of our speakers to elaborate on a specific question. Everyone will speak for about three minutes and then we'll take your questions so we can let everyone get back to their day. Hannah? Thanks, Karen, and thank you so much, everybody, for joining us. So the other day, I tried to sign my 77-year-old mom up for a vaccine appointment and that didn't go well, not surprisingly. So this is an issue that we saw coming as soon as we knew that the federal government was only handling the logistics portion and that the last mile delivery would be handled by the states. We know from our work that the states aren't really not equipped to handle that kind of work. They're not equipped from a staffing perspective and they're not equipped from a technology perspective. So we have been watching this unfold in almost exactly the way that we expected. Okay, so let's get started with our first question. And that's actually a good segue into the first question. So Hannah, why can't the government build tech that works? So I will try to keep this to three minutes because we want everybody to talk, but I could talk for like 10 hours on this. So at the root, there are two issues. One is that government is really old. Government has been doing government's work for a really long time. The world has changed like where that work happens has changed in let's say the last five to 10 years with most of the delivery piece of government being online. So the way that people interact with government is by a website or by a text. Government is set up for people to interact in person or on the phone or even by fax or by mail. So one piece is just that this is kind of new even though it feels like we've been ordering things from Zappos for a really long time in government years. It's actually a pretty new competency. And along with that, government lacks tech savvy. So the people who are doing the policy work may not be steeped in technology, they are steeped in policy. The people who are doing the delivery are steeped in how to do delivery in a different age. So part of the issue is that government doesn't have the know-how and nor the staff to make these kinds of projects work. Okay, you were good on that that three minutes. So let's move to Sasha. So Sasha, why does government keep hiring the same big tech vendors when they have failed in the past? So why can't the private sector just do it? Yeah, I think this is a great question. So the first thing, of course, I want to talk about public procurement. This is how government awards these contracts. And I think what we're seeing right now is that we've known for many, many months that one day vaccine will arrive and we'll need to distribute it, we need to schedule it. As early as April last year, I heard from companies like ZocDoc that offered government to help with setting up scheduling for first protesting than for vaccination. And really, when we're resorting to the same old tech vendors that have failed in the past, it's usually because the teams and government haven't really had any foresight, and they didn't use what we call commercial thinking, which is to say, how do we get the business outcome we want? And in this case, right now, we're facing an uncertain environment. We don't know exactly when vaccines would be there, what protocol we need to follow. Is it one or two? How many doses would we have? And you see, you have the ability to just like vaccines are developed with many candidates, governments can actually have many candidates for these scheduling and booking solutions and technologies, and prepare the process to be ready with a number of different options. And clearly what we're seeing right now is that this hasn't happened. And instead, governments have resorted to what we all talk and see, which is just we need the biggest possible vendor with the biggest insurance and the biggest contingencies. And in the end, it undermines the quality of the product we're getting. And it's not prepared for all this uncertainty we're walking into. Thanks so much. So, Brendan, so far, we've talked a lot about the problems, but can you tell us, and I know you've got a good story here, or any states doing a good job with rollout? Yeah, thanks. Thanks for being on here. West Virginia is doing a strong job. And one thing they've done differently is CVS and Walgreens were supposed to roll out vaccines to assisted living locations. And West Virginia chose to do it themselves, which really, when there were problems with the national contract, that really put them ahead. Alaska, it has been bumpy. We opened it up to seniors earlier than most places. And there are like seven screens you have to fill out. And it's kind of a perfect storm of a very difficult web app. And people like my mom, who's 83, incredibly smart, but not as tech savvy. I helped sign up people over the phone about 15 seniors. And every senior I had asked them if they were pregnant, if they were breastfeeding, if they were going to become pregnant in eight weeks. So there have been challenges there. Alaska has a tribal health organization, and we have a large indigenous people population. And they separately got doses of the vaccine. So one thing that's helping Alaska is we have about 20% of our population has a dose in state right now. So we'll be doing second doses. So we have a much higher rate. I think Hawaii is next at 13%. But it has definitely been bumpy. And people are complaining. I think people are focusing on software that helps report to the state and federal government and not on user interface for signing up. And our innovation team works on human-centered design. You really have to just try it. And signing up 15 seniors in two hours really made me learn where some of the flaws were. And I agree with other people. We kind of saw this. We were hoping flu shots this year would be a dry run for figuring out how to do vaccines. And my team tried to put in input, but the state is controlling it. They're doing a great job with what they can. But it was like watching a slow-moving car crash in the sense of this is going to be challenging. We are using prep mod. It's used in California, Maryland, Rhode Island, a couple other states. We're trying to share as much as we've learned as quickly as possible. We have a GitHub repo. We wrote something in R that just basically scrapes the prep mod site. Makes it easier to see what appointments are available because we couldn't get an API. But I would just suggest people, just any means necessary, come up with easier ways for people to sign up. And short of that, make it easy for people on phones that are handling voicemail or talking to people. Make tools that make it easier for people who are helping. And I think that's most of it. I mean, it's definitely been bumpy. But on the plus side, we have the highest doses per hundred right now. We will not maintain that in Alaska because we're hugely remote. But the tribal health organizations have been huge, getting it to far flung parts of the state on dog sled, even on planes. But we also have one last thing I'll say is the 1918 pandemic really hit the Alaska Native community hard. So there are generations that remember grandparents that died during that. The Iditarod is a famous sled dog race we have here. And the whole race is based on taking serum to gnome a thousand miles away. So there's definitely a more historical and cultural reference to the impact of pandemics. But we could do a lot better job. I think there states that are using these large software, as Sasha mentioned, and there's chances for them to talk to each other and form cohorts. And at this point, it's just any hack, any MacGyver thing you can do to make things easier at this point. Thanks. Thank you. That's a great. Those are great stories. So Raphael, I think I think I saw all of us laughing when Brendan told the story about asking seniors about getting pregnant or being pregnant. But it sort of begs the question. So why aren't all the states doing a better job of standing up scheduling software? It's got to exist. And someone in government has to know how to make it work. Yeah, it's a really good question. My program, the health program at USDR has been investigating the vendor landscape for a few months. So we are developing some insight into this question. Let's start with the people first. Like let's say that you're the CIO at your state's Department of Health or the Department of Technology that's been tasked with supporting your DOH. Your team is very small. It was never that big to begin with. And it's hard to scale it to the size of the problems that we have here because of budgetary reasons and also because it's just hard to hire lots of good people in a short amount of time in government. This past year, their parties have shifted many times. They've been setting up testing sites and getting exposure notification apps up and running, setting up and updating informational websites and adjusting test result reports from labs and individual testers that are often coming in through FECS. And everyone's been working long hours, 60 to 80 hour weeks since March. There's a very human side to this that I think is not visible to the public. And they're operating within a system that has not set them up to be nimble and with constraint that produce a ton of work for them. So back in October, all the states and territories were required by the CDC to submit a vaccination plan, part of which entailed figuring out how to use technology for scaling vaccine ops. But at the time, relatively few solutions actually existed. And the ones that did exist were originally designed for something else outside the context of COVID-19. So when you look at scheduling software, you can kind of bucket them into two large groups. There's the big enterprise ones that are end to end and encompass a variety of different needs, including scheduling, but also eligibility and SMS reminders and integration with your state's immunization information system, and so on and so forth. And in theory, these are one stop shop. They're like Costco. You go there, you go to one place, you get your pizza, you get your tires, you get your houseplants, like you can get everything in one place. To Sasha's point, it makes procurement a lot easier because you're only dealing with one place. But actually, every state has a different rollout plan. They have different existing technologies. They have different IISs. They have different budgetary constraints, and they have different populations that they serve, right? So these one stop shops, which did not exist very long ago, are hard to change on a dime. They're being built for a general purpose, and they're hard to adapt for the specific needs of a given state and all the local health jurisdictions in that state. The other bucket is individual solutions that work together. So here, I'm thinking of non-health-related scheduling solutions, like Eventbrite, or health-specific solutions like Solve, which originally developed for scheduling COVID-19 testing sites and being repurposed now for vaccines. Again, everyone is scrambling, including the vendors, to update their software to meet the new needs. So if the one stop shop is Costco, then the individual solutions working together are like going to that one neighborhood that's full of boutique stores that each only do one thing, but they do it really well. But now, although each individual solution may be better, you now have the burden of knowing what you need to buy, selecting the best vendor in each class, and getting all the procurement and contracts done, which can take a really long time, especially at the state level, and you better hope that all the data is interoperable between everything that you buy, because you still have to get all the reporting together back to the CDC and to the public. So no matter which route you go, enterprise or individual solutions, it's going to be a lot of work, and you're only as good as the vendors that you go with. I want to echo what Brendan said. States should talk to each other. There's no reason why, like, every state that's using one solution is learning about its strengths and limitations very quickly, and there's no reason why any two states should have to learn that the same painful lessons twice. So the more connective tissue that CIOs and CTOs and state health departments develop, the faster we will get past this. Great. Thanks. Thanks so much, Rafael. So Latanya, so it seems like it's not just for the vaccines that, you know, we're having a problem. Why does the government struggle so often with delivering services such as healthcare and vaccines? Well, thank you for letting me participate. My co-panelists have covered a lot of the same issues that keep coming up over and over again. It has to do with the difficulty of the government to be nimble and agile in these situations and to be innovative, and it really is particularly striking because our society is used to things in the minute, being able to get things on demand. We're used to being able to use text or phone as easy ways to sign up. So all of a sudden, when we see these sort of antiquated, not user-friendly systems, it just exasperates the gap and makes it less accessible. It's not just to the elderly, but it's to tons of people who it becomes less accessible because it's not even working the way they expect technology to be working. When you add to it healthcare, you get also the questions around identity management. And if the vaccine is also delivered in healthcare, there's a place where public health crosses healthcare. And one thing we haven't talked about is after I get the vaccine, you know, to what time, how do I show that I've had the vaccine? How do I offer credentials and authorizations? Or if I lose whatever you gave me, how do I recover from that? And so if you start to think about all the ways this is going to stack up, the fact that we're still back at how do I schedule it? And we can't even get to these other questions is incredibly limiting and really frustrating. And, you know, whereas with technology, we're used to very nimble, agile programming, and we're just not getting that. Well, our last question is for Xena. So, you know, we've heard a lot about the process sides, but I know you have something to sort of explain to us about the language gaps and how they're affecting the overall problem and why it's so important to provide accessible language. Yes, absolutely. So like many of my other co panelists have been talking about the last mile problem in getting vaccines into arms. Our new America team, which is really focused on getting vaccine information to immigrant communities, is seeing a similar last mile problem in getting accurate information about the vaccine directly in front of people who are really looking for it, so that they even know, you know, that the vaccine is safe, and they should try to make an appointment to get it. You know, some of my co panelists spoke really, really compellingly about watching people older people, especially try to navigate vaccine appointment websites. It's something that we're seeing when we are testing vaccine content with immigrant folks. We're seeing the same kind of problems, but imagine you're trying to understand if you'll get serious side effects for a vaccine and the website is hard to get to. When you get to it, you see a wall of English text and you have to somehow navigate to find the Spanish translation. When you click on that link, you see a giant PDF of just even more text. So our team is really focused on ways of getting existing accurate and good information that governments are working very hard to put out into formats and with trusted messengers that work really well for immigrant communities. Something that unfortunately we're seeing is that misleading information about the vaccine travels faster because it checks all the boxes that people are looking for. I think we're all really feeling like this COVID Vandek is a interlinked series of crises, and especially for some parts of the immigrant community. People are experiencing loss. They're experiencing a financial crisis. They're experiencing a health crisis at the same time. They're trying to navigate these complicated websites just to find some information that makes them feel safer or that they can trust. And like I said about misleading information, it often comes in short, plain language. It is very visual. It might be circulating as short videos with people who look like they're part of the immigrant community who look like they are health experts. Often misleading information is framed with strong emotion or touches on existing threads of emotion that people might have about the government or healthcare. And when you compare that to accurate government information, it's not checking those boxes. It is those long walls of text. It's not using a bunch of expertise like we have so many immigrant community leaders, immigrant healthcare workers, immigrant faith leaders who are ready and willing to sort of translate vaccine information into ways that their community members will find more accessible. So again, our team is really focused on how do we get this accurate information to check boxes of being more concise, more plain language spread by people who are trusted within the community. And I think there is some good news like Raf and Brennan were talking about connective tissue between state officials. I really think that state governments are not alone. There are so many community leaders, community groups, grassroots organizations, small influencers on social media that are again ready and willing to help spread more accessible information to immigrant community members. We just need to give them the resources and give them the spotlight so that they are more able to spread existing information. And that's something that our team has been working on for the last few weeks.