 Thanks for everyone for coming today. I'm Allison Price. I'm with new Americans digital impact and governance initiative. Our team also goes by Digi. So good afternoon we have I feel like every time zone covered with this great group of panelists so good morning. Thankfully I don't think we have any good nights but thank you for joining us today and I want to make sure we say thank you to all of our panelists and the audience for taking the time out to join us for this event. So, I'm with the Digi team I'm a bit under the weather I'm a little embarrassed but, and we work to advance systems and solutions powering the field of digital public infrastructure digital public goods there's so many terms he states through cross sector collaboration, which really brings us to the year long process that has been this report and then today with the event. We're excited to host this panel discussion today with our friends with the open contracting partnership OCP. The open contracting partnership, rightly recognizes that the world runs on public contracts, and that those contracts should be open fair and efficient, and we agree with them wholeheartedly. They've joined us today to host this event for the launch of our paper that came out today called reconceptualizing public procurement to strengthen state benefits delivery and improve outcomes with an improved outcome excuse me it's the longest event title ever. We tried to get a lot done the scope is massive. Our report landscapes, the connections between public procurement digital systems and the delivery of safety net benefits and offers recommendations and five core areas on how to improve people centered outcomes. We did this through interviews, research and a collection of dynamic essays from experts in the field to improve state benefits access and delivery through public procurement reform. Many of the report contributors have joined us today for this event, and they're in the audience as well and we encourage all of you to share your ideas and questions through the the Slido app during the event to make it interactive. So this past year, as I said, we worked with several new America programs on taking a closer look at public procurement practices, and how they are found a foundational approach for how state governments by and manage state digital systems. There are a few programs in particular, working on the broad area of responsible tech at new America that we'd especially like to thank. The practice lab, a design and policy team focused on family economic security and well being. Please take a look at their playbook for improving unemployment insurance delivery. Also the public interest technology program, which expanded its work around the pit university network to prepare and train more graduates for careers and work in the public interest technology field, as well as the open technology API, which continues to advance safeguards that policymakers can adopt to mitigate accessibility and privacy concerns and past comprehensive privacy legislation with effective enforcement mechanism. This paper and our event today have been unique for the amount of perspective and voices at the table. In addition to our friends with OCP, our co host today. We have the code for America, the Beck Center at Georgetown, New Jersey's Office of Innovation, and many others. I'd like to think we broke numerous acknowledgement parameters for report, but please see our report and all the people who've been involved in this work. So where did all this collaboration leave us. Our first conclusion was that this is a super nuanced and complicated topic layered with urgency and status quo, which makes it very odd. But our second is that the people working on it are inspirationally passionate about this field because there's so much room for improvement and change by putting people at the center of digital solutions. So this is an interesting technology as a tool to improve equity, inclusion, justice, and access. This is what our panels will be talking about today. So this event is a two part panel, the panels, excuse me, the panelists in the first panel will landscape the broader procurement culture and identify leadership priorities needed to improve state digital solutions and people centered outcomes. The second panel will offer insights into specific actionable recommendations for tackling common procurement hurdles and capitalizing on opportunities for growth for practitioners and teams. And with all of that, I'd like to introduce Marina Nitsa for this. She's going to be the moderator for the event she has a much stronger voice which is great for today and lots of very insightful ideas on this front. She is a new America fellow with the new practice lab. She's currently a partner with layer a left a crisis response firm that specializes in restoring complex software systems to service and recently co authored the new book I'm already for it. Hack your bureaucracy Oh no it's blurred but it's, you should get it it's available wherever you get your books. To fellow at new America's practice lab, Marina works on improving America's foster care system through the resource family working group and the child welfare welfare playbook. Prior to this work Marina was the chief technology officer at the US Department of Veterans Affairs under President Obama. After serving as the senior advisor on technology in the White House and as the first entrepreneur in residence with the US Department of Education. The advisory boards of a number of incredible organizations and previously authored the book business efficiency for dummies. We'd like to thank Marina for moderating today's panel and for serving as one of our experts who contributed to our report with an essay and recommendations. Marina please take it away. Thank you Allison we hope your voice comes back today. Thank you. So in my time at the VA working on federal procurement and now in foster care working on state procurement I mean nothing underlines to me more how it is the key to everything right it's the secret to the universe. And it's exciting to be here joined on two panels by other people who share that view. Done right procurement is something that can unlock benefits and access for millions of Americans and potential Americans when we talk about immigration and make sure that we uphold compassion justice and inclusion at every step of the way. I got to write an essay here about Harry Potter actually which I did not expect to be in my professional portfolio but here we are. And a few things that I really touch on is that at the end of the day procurement is humans, as Allison mentioned earlier, the lawyers the budget analysts the procurement officials like they are there to do a job they are operating in a framework of risks and incentives. And if we understand those, if we find small ways that we can help them, and we can shift those incentives so that doing the right thing is easiest and best thing. And I think that's how as a country we're going to move, move forward. So I am excited to introduce our first panelists. First off we got Bruce helped the, who's currently a director with the public sector and education practice of Alvarez and Marshall performance improvement turnaround, my favorite, and innovation work. He's currently leading a procurement transformation for a K-12 school system. Bruce is the former director of budget and performance in the Office of Management and Budget in Harris County, Texas, and director for performance improvement and innovation for the city of Houston, where he empowered government staff and departments to improve outcomes by reducing violent crime, transforming the jail, revitalizing countywide fleet operations, all while exploring how to harness data and procurement to drive innovation. I work with Code for America, Bloomberg Philanthropies, and the World Bank. We also have Afua Bruce with us. Afua used to be my boss at Pitt here, so love to have the family technology policy and society. She's the principal of ANB advisory group, a consulting firm that supports organizations in developing implementing or funding responsible data and technology. Her background in software engineering, data science and artificial intelligence, combined with the experience developing and deploying technology in and with communities. Afua incorporates an equity based framework into her engagements, and Afua also has a new book that everybody should get the tech that comes next, describing how technology can advance equity so welcome Afua. Our third panelist is going to be Catherine Froucher. She will be the executive director at the Open Contract and Partnership, a DC based organization that works at home and around the world to transform local governments and federal agencies improve equitable service delivery, economic mobility, and environmental sustainability through new procurement approaches and digitization. And then last but absolutely not least we have Jennifer Wagner from the Center on Budget and Policy Priorities. Jennifer joined the center in 2015 and is a member of the health team. She focuses primarily on Medicaid eligibility and enrollment issues including the policy operations and technology that affect the enrollment experience for clients and staff. She's also coordinated SNAP and TANF staff to analyze opportunities to improve access and advance coordination with Medicaid. Before joining the center, Jennifer served for five years as associate director with the Illinois Department of Human Services, where she's oversaw SNAP and cash assistance policy as well as medical assistance. And she assisted Illinois with Medicaid expansion under ACA and improved customer service through business process re-engineering in the local offices. So welcome to our panelists. Great to have you. So Bruce, I'm going to ask the first question to you. So we've seen from the report there are countless diverse ways to improve the public procurement landscape. What are one or two core principles that you believe should influence public procurement in order to improve outcomes for the millions of people who are relying on our government's digital solutions to access benefits and services? To your question though in terms of core principles, I think it really comes back to the basics, which I think we can often forget about. It's really focusing on the why around procurement. So why are we launching into a specific type of a project procurement? Why are we actually focusing on procurement transformation? But I think it's easy to say to focus on the why. I mean that's something that's kind of obvious to a lot of us. So then I think it really comes down to the how of how do you actually start to really get to the why that is most important. I think that that's the hard work of engagement. Engaging very different stakeholders than we've necessarily engaged in the past. I don't think you can put all this on a purchasing team. Even on the internal workings of a governor's office or a mayor to actually manage this. You have to start bringing new people into the conversation. And so for instance, when we're looking at this school district transformation, one of the things that I'm really looking to do is, so it's not just engaging the purchasing team in this effort and the superintendent's office and the school leaders in this effort, but also how do we think about broadening the base in terms of the supplier ecosystem and having Sasha Hasselmeyers, he's a contributor to the report. I remember him talking about how do you just generate and open up possibilities in terms of creative problem solving within procurement. So how do you get people more open to these ideas? How do you bring more people to the table suppliers you've never seen before to look at what what is the more general problem we're trying to solve when we're actually trying to procure or build something. And then also in the schools context. I mean to what extent do we actually involve our teachers potentially our students can you actually have like almost like a school based hackathon with student teams and community members really looking at some of these problems. So again, I think a lot of this type of engagement it takes a lot of time it's a lot of work but that's how you really help to keep yourself centered on the outcomes that are important. That is so important. Can you say maybe a minute more about how you got like the teachers engaged what worked what brought them to the table. So this is definitely the plan with the school district situation so we haven't actually executed that yet. But I can say that, at least in a couple of other contexts within the city views and one of the big things that we actually did was it was building out our lane six Sigma program. So this is actually something probably relates to some of the other questions we'll touch on but how can we actually get more people across the organization involved and provide them the type of training and experiences for them to actually get into both the why what and also to get the tools that they need to be successful and then also another experience I worked on the, I lived overseas in Oman for a time when I was responsible for the National Transformation Program. The project manager for setting up the National Transformation Program there. And one of the things we did we set up this six week long full time set of labs where we had 250 people from across the country in the public sector but then also from the public sector civil society actually engaged and really diving into what are the problems that we actually had, and then moving from problems to generating solutions to then coming up with okay what are the milestones in terms of actually achieving success on the 121 projects that we actually generated from that project. What are the KPIs where the people that are going to own this what is the budget that's going to be necessary to support these things. So, again, there's, I think there's a lot of ways you can do this whether it's internal engagement or external. And I'm hoping to leverage a lot of these approaches, at least with this school district transformation. Awesome. I'm going to pass one over to you. I think we share a pet peeve about the word modernization, right. Just because something is modern doesn't mean it always results in improved processes are out of a current that in your experience, do lead to improvements. Thanks, Marina one for having me today and for that question, you're correct. Word choice matters when we were talking about procurement systems when we're talking about the work that we're doing and modernization, as you said, process of modernizing something using a more modern technology doesn't necessarily mean anything has inherently improved because we haven't really considered how we are intentional about centering people in the processes that we're doing and how we're thinking about the systems and processes that support the work that needs to get done. So then I think as it relates to procurement it comes back to as Bruce said, thinking about how are we centering on people what is the problem thinking about how people need to be engaged. It's really in light of many efforts at federal, state and local levels to diversify the government to ensure equitable support of services and equitable distribution of services. I wanted to think about procurement as a way to strengthen and advance these efforts. So how can we think about using the procurement process to make sure that we are thinking about who we're engaging in those contracts what supports can be done to evaluate different proposals and to create different proposals and to bring that through the process, and then it can ultimately lead to how do we actually think about doing the work or have we written a new request for procurement and how to extend what has already been done or have we taken time to think about how we do things differently. How could we solve problems differently, how could we be more inclusive in allowing for people of different backgrounds to think about different solutions to think about different ways of proposing the problems, and to really even sometimes be seen as the right customers for the systems that are being rolled out. I love that not to put you on the spot but is there a chance you have a specific example of doing that that you've seen that was successful. So I think one example I've seen of this being successful is working with a city in Kentucky actually, as they were looking to bring on some new staff and so looking then at what are some different processes they could use for writing the specific RFPs for their work and who could be done. So in this case with procurement actually led to some different hiring practices and so what could be done there and so expanding it to allow university students to be able to work in that process and to work on a particular solution. Awesome. Priority of public procurement away from strict compliance and risk management toward delivering actual people centered outcomes. What are ways that departments and jurisdictions can lead this shift. Marina you, you cut off in the beginning was that a question for me or for Jennifer. That was a question for you Catherine I'm so sorry. Yes. So what are ways that departments and jurisdictions can lead a shift from compliance and being afraid of risks towards people centered outcomes. Great, thanks. That was loud and clear now. I think there are some sound issues because for you also came in for a moment I don't know if there's anything we can do about it for the audience I hope it's better now. So first I also want to say it's delightful to be here. It seems like there are many of us here and many of the report writers you know how real procurement nerds and have found procurement as a way you know to create better social and economic and environmental outcomes and I think that there are several of us here right in the world there's the few of us including in the United States so it's just a great joy to be here and to read the report I think the report was excellent and to go you know from a focus on social safety nets to focus so much on the issue of procurement. I thought that was not a big picture view that many take and that you were able to brought that out and I mean who doesn't want to read report that mentions Harry Potter and procurement in one sentence so I really enjoyed that. And yes, I think that is a great question I think what we really need is a system shifts and a culture shift and that is exactly what we work on on at the open contract partnership. You mentioned so some of those numbers in the report but just to throw them out here again right global procurement is a $13 trillion market in the United States alone federal procurement is nearly $640 billion. And there is a lot of opportunity here to do things better. The World Bank estimates that if you change procurement and make it more open more human centered more effective you could use $1 trillion for having a more resilient or more green and a more fair economy so we have our work cut out for us as people who care about this. I think some things that are needed to make this happen is the main recommendation of the report which is really about making procurement more accessible. And I think that is really important. Someone already mentioned Sasha who is one of the writers but he has those great statistics how in some cities or states you know you might be on the city portal and you will never ever get to an RFP, no matter how often you click. And where has six different procurement portals to find your opportunities versus in other places it's much more streamlined and you can get there with just three clicks, and that really makes a difference. If I can highlight three additional points I think that are needed and then come out a little less in the report is we do need better procurement technology. We talk a lot about better safety net technology, but the procurement technology is one of the problems by it impacts how procurement officers see information and often they have no information they don't know when they're buying from what they're buying from how much it costs. It also impacts how as a small business I might be able to submit my RFP and how painless or painful that experiences or how as a citizen I might be able to see which breaches being built and how much it costs. So we need better technology. I love what Alice said in the beginning about the status quo and the status quo of vendors is an issue for safety net technology and procurement technology. I think we need to grow the market of vendors that can provide those services and bring new vendors in and that's also a system shifts that needed you know in terms of working with financial institutions institutions with business incubators and who owns these businesses so there's a lot to do there. And then the third thing and that was mentioned in the report as well is who works on procurement right I mean none of us grew up and said I want to be a procurement officer when I'm a grown up right. So it doesn't always attract the most creative innovative people and often procurement still works in silence so I love the reports in the book that talk about you know we need to bring in the creative lawyers we need to bring in the user researchers we need to bring in people who you know work maybe on safety nets with the clients and know what the end product should look like and if you work as a team, you can really change you know how overall procurement runs for state for city for country, but also the specific procurement to buy that specific technology so I would say those four things you know making procurement more accessible, having better procurement technology, having better vendors and a better team would really make a difference. Awesome, those are super actionable and specific I love it. When you're talking about getting better technology like what, what could that look like or what could we put in our procurement for the technology to drive to drive what we need. That that is a great question. So, you know, we work globally and around the world we have seen a lot of innovations. And that's not always possible but you know, buying the off shelf solution doesn't always serve you well. We have seen that the best solution come when again this diverse team works together and either sometimes build something that they really need, or, you know, buy things in pieces and build the piece of the puzzle so that it serves them well, which is complicated and links back to the staffing issue right that some cities or states don't have the capacity to do that right they need one big vendor who can do everything from delivering the technology to servicing the clients to managing the sub contracts and that just doesn't always lead to the best results. But I think it is being very smart and outcome oriented about which technology you need and then I do think some of the vendors need more help right it is hard to compete against the status quo, we all know that it's hard to win against some of those established numbers and what can we do to give some of these more progressive companies that exist a fair chance so that they can win some of that business as well. Yeah, all great opportunities. So Jennifer I'm going to go to you next. So leaders can make or break any team but with so much riding on this benefits delivery at the local level that leadership is really more important than ever. So from your experience and the culture surrounding it and and what can they do better. So thanks first of all for having me here. I'm so glad to be with you folks, I am definitely a policy walk but totally attack wannabe and am way too excited about these procurement issues and excited to be on a call with so many people who know so much more about it than I do. Being from the policy and I really kind of focus on the outcomes and what the system can and can't do as far as delivering benefits. The policy space is messy. It is complicated. They're very different rules for all the programs and it's really hard to coordinate them. But there are opportunities that are out there. There are progressive things like automating Medicaid renewals, like connecting programs that if you're enrolled in one or eligible for one, you're automatically or streamline eligibility for another. So what we find is that the system often can do those things, even when there is the policy available that streamlines eligibility for for enrollees. The system can actually do it. So I think we really need to find where there's flexibility, or they say agile or iterative. There's so many different kind of terms for it. But how do we make sure that the system can be responsive to people. And those people are both clients, but also case workers. You know, sometimes the system may meet the requirement that's written out in the procurement that it actually can do something, but there's not a lot of attention to how, how long it takes and what it requires. So it might take a case worker 10 or 15 clicks or eight screens to do something very simple. And what we have on in this space is a lot of states still operate in legacy systems, 40 plus year old green screen cobalt based systems, but they can do things faster than in the new systems. And there are these worker rounds available there that are more flexible than the new systems. And so how do we take some of those lessons from there and kind of limit our reliance on these vendors that have kind of a black box around the rules and how the system works on the end and allow workers to have more ability to do what they need to do and have the system be actually responsive to the worker, because if it takes them four times as long as it should process a case, people aren't going to get benefits. They're going to get delays, they're going to be delays and they're going to be problems. So when it comes to leadership, I mean I think it's a lot about challenging the status quo. When I worked for state government I had a great boss that I said oh this is how it is and she says is it ask, ask questions. And that was such an important lesson because there is actual written policy, whether it's about eligibility or procurement or contracts. But then there is lore, and what people believe and what is generally understood. So you need that leader who really asked questions challenges things and fights for the outcome that they need. But we need a lot of support. I mean, look at what leaders have gone through for many years but especially since COVID, standing up new programs, putting their staff to be remote, serve in a huge increase in population as the need has changed during COVID. And we're also asking them to challenge things and come up with innovative ways to lead and be creative. And so how can the federal government support folks and do some of the legwork for these leaders. And bring lessons from other states that have overcome some similar challenges and hand it to them whenever possible. So they don't have to find that 30 seconds of spare time in their day to kind of fight through all these issues on their own. Yeah, those are great tips and I just want to remind the audience you've got questions for our panelists please submit them now and happy to ask them. So when you're talking about states learning from one another. That's something I work on in foster care all the time and find to be very effective what are you finding is an effective way for states to find out what one another doing or sharing a procurement or other things that are really working. I think it's a kind of newer field that we need to do more on. And so I think it is somewhat low hanging fruit. The attention in my world has been so much on the policy on that enrollment numbers on the legislative angle of things. And so to shift the attention to the people that are working to have a repository of procurement documents to have things around best practices. There are conferences or opportunities to talk to people. You know when I was a state leader just hearing about other people's problems and knowing you're not alone was very helpful. And so I think it is somewhat low hanging fruit. So we're talking about the policy on the enrollment numbers and the legislative angle of things. And so to shift the attention within this kind of policy and program space to really looking at the procurement and just bringing that information together in an accessible way, and create in a forum for people to share experiences. In my world we've got only a couple of vendors who are doing most of the eligibility systems. So can we have, you know, who's got vendor x let's have you on a call monthly. are similar across states. Every state is gonna have to unwind the Medicaid continuous coverage provision and renew Medicaid for millions of people who haven't been touched for like three years. We shouldn't, each state should have to go to their vendor and say, okay, let's figure this out in a box how we're gonna make the system work and change things. There should be more shared experience, more collaboration so that there's one solution and we put all that brain power together and work with these couple of vendors and come up with ideas. I love that. I was on a call earlier today where somebody was referring to procurement as like the JV sports team where the policy is like the varsity sports team. And I was like, man, I think that might get to the heart of what we need to change, right? That implementation, execution, procurement, these are the varsity sports like. So anyhow, I love that for me to help lead this work and lead this transformation towards people focused outcomes and not box checking exercises. Sorry, who is that directed to? We lost you. Yeah, you can't be cut out of the beginning. Anybody, that was open. That was an open question to anybody that wasn't Jennifer because you just answered that. So Afua, Catherine, or Bruce. You wanna repeat the question one more time? I'm happy to do that. Yeah, what leadership, what advice do you have for leaders to leading the transformation from being a compliance exercise towards being focused on people outcomes? Yeah, I'm happy to talk about that. So we thought about a lot about that at the open conflict partnership and it kind of matches our evolution of how we work with countries around the world to systemically transform their procurement. I love the phrase that comes from our colleagues at the GPL and they call it the extreme procurement makeover. And I think that is really what it is, right? Because you have to think often reforms in procurement happen in silos that people either think about the data or the process and the policy or the technology or the people. What we have learned that if you really want a systemic change, you have to bring these things together and have a clear goal behind it. But then one thing also that many of these reform teams lack is the reform design and change management support. And that was something that really, really surprised us because we always in the beginning also thought, right? Oh, you need more assistance with your data. You need more assistance on how to go and change that regulation, right? Or find that right person to be your director. However, what it really often means is for this team to come together and have to dedicate a time and coaching to see a reform through from the beginning to the end. Which is not so new for us who work on some of those things but in the public sector, that is not what happens, right? This is not the time that you get to design and implement your reform and think about how do you get by invite? How do you get your governor, you mayor to be on your side? How do you overcome a position that could come from a vendor who might be losing out? And how do you have a public campaign around it so that your citizens, residents, you know, all are on your side as well? So I think paying more attention to the change management side of it could be really effective and powerful. If I could add something really quickly. Please. So I think something that's really a critical error that a lot of people, not a lot of people, but I mean, I've seen too many times, it's just, I think that too often we put down a lot of the people that are currently in the system, especially procurement specialists and such, they don't necessarily have the skills or the desire to make things happen. I've seen this in consulting world, which I've just joined. I've certainly seen it inside of government itself. And I think that there can be just a lot of negativity around the people who are in some of these roles, which I think is really and truly not warranted. I think that there's also, it's not surprising at all how much cynicism you might even see from some of these people just based on the revolving door that they've seen over time in terms of leadership and priorities and what we're going to change about procurement. And so I think that it's something that's really important for us in this field and when we're doing this is a lot of the people that are there inside of these systems, the procurement specialists, a lot of them really are quite talented and competent, but it's also, it really takes an effort in terms of how do you bring them on board to where you build the trust between them and yourself and the initiative that you're doing to be able to make a change. So this idea of change management that Catherine was just talking about, it's critically important. And I think that there's just a huge untapped resource in terms of the people that are already managing these procurement systems. Yeah, and actually if I could build on that, I think your point is incredibly valid, Bruce. I also think that when we think about changing procurement processes, when we think about what leadership means, we should recognize that systems innovation is innovation as well. I think oftentimes we put an emphasis on the particular technology once we already have the right teams in place as we would just determine whatever teams those are. We've determined what the right technology should be, but the actual process of doing the work and how we innovate in that process should be valued and respected as well. So I think that for leaders, leaders should recognize that we should encourage innovation in the process and innovation and what that looks like in the procurement process. And so working with the talent that's already on board to say your innovation here, as long as it's legal and within the guidelines of the law, which this former FBI person would strongly encourage people to do, is to say, how can you be innovators too? How can we lift up that innovation? How can I encourage that innovation and really help push back against some of the cynicism, help push back against some of the systems that are so entrenched to really make sure people understand the level of empowerment they have to do some of this creative problem solving and to think about how we can really use procurement to change health systems that are delivered to change. Thank you so much, Afua. Do you wanna talk a little bit about the essay that you wrote for this project? Absolutely, thanks so much. And my essay in this project and just so honored to be amongst so many other people who wrote so many great pieces, my particular essay focused on the power of procurement in expanding diversity, equity and inclusion. And so really using procurement to think again about how can we expand who has access to participate in government systems and who is there and affected by government systems. So really using the procurement process to think about how are we writing our RFPs? How are we putting out the notice itself, not just putting it on a website somewhere, but how are we really and truly sharing the opportunity to work with the government with a wide variety of people and using that to help diversify the space. Then also thinking about the evaluation process and really encouraging evaluators in that process to really have space and back to the creative problem solving, think about what the ultimate outcome needs to be and not the processes that we have historically used necessarily. And so that then allows for, if I want to make sure the new website we are developing or whatever new technology we're rolling out is going to have the most number of people, the greatest number of people, how do I in the evaluation process make sure that I put things in place that reward people for inputting into their work, the time to actually talk with community members and the time to actually do some long-term training and support. And what does that look like? How do we embed that in our evaluation process as well? I think that can get to supporting some of the diversity, equity and inclusion goals that we have. Yeah, I love the idea of articulating those so that we actually have, we see it happen. And that's how a lot of change is driven. So thanks for sharing that. Could I add to that for one second? I just want to say, I think it is really one of the superpowers of procurement and it is to give more business on a repeated basis to minority-owned companies. And we have seen that over and over what are the difference that can make? And I mean, in the US, we have some pretty shocking numbers around that, that women own 38% of small businesses in the United States and get less than 5% of federal contracts. A lot of people of Poland, the United States are 39% of the population own 29% of businesses and only get 12% of contracts. So just imagine what you could change and how much more income and economic opportunity there would be, but especially then if you focus on businesses that are based in communities that are often under service and then they could also service their communities better. And we haven't seen that in the United States, but in other places like Columbia, we have really seen that when a government starts to invest in better data policies and processes, they can really increase the share of women and minority-owned businesses and how that then can increase, income and livelihoods across the board. So it is a different way of improving economic mobility than social safety nets, but it is one of the most direct ways of how procurement can do that. So thank you for lifting that up. I think it is a really important power that procurement has. Yeah, that's a great point. So I'd love to go around to each of you for one last question, which is can you tell us about a procurement win that you are a part of or that you've seen that you are the most proud of? And maybe Bruce will go first with you. Most proud of, I'm not sure if this is the one that most proud of, but it's certainly one that comes to mind where we had been pushing at least in the city of Houston for a long time on how we could actually change over street lights from incandescent over to LED. And it was through a long process working with procurement where we were able to then engaging a broader audience, engaging the mayor's office. I mean, it was a lot of different conversations around how you would actually do this street lights project, but with enough attention put onto that and some additional assistance, that's something that now saves tens of millions of dollars per year for the organization. And so that's, it was, again, huge impact financially. So I'm not sure if it's my favorite one from a people perspective, but I guess it's just revealing there's huge opportunities inside procurement. I love that. How about you, Jennifer? I have a limited number of experiences to choose from, but really proud of my role in working for the state of Illinois as we procured a new eligibility system that serve as SNAP, Medicaid and TANF. Many bumps along the road, many of the kind of worst practices that are highlighted were employed during that, but we ultimately did deploy a new system, expanded Medicaid and changed our budgeting for Medicaid all on the same day, back in 2013, which was quite a visible day. And it really facilitated us handling the volume of cases that were coming our way. And the legacy system, we had office leaders literally meeting folks on the side of the road to exchange, you know, paper files in order to try to kind of share the workload and make sure that things got processed. But this really allowed us to have a call center to exchange work among different folks and ultimately sort of people with Illinois. I feel like making all those changes in one day should be like a state holiday, like celebrated for years to come. Afua, how about you? Sure, I'll go back to something you led with Marina, which is procurement is people. And so I think one of the procurement projects I worked on that I'm most proud of is working with the people who were making those decisions at the FBI at the time, working with the procurement officers on just some of the training as to how can we think about doing this differently? Our process before does not necessarily need to be our process coming up. We are working on sensitive things in this case, it was to support and explosives, analytic center. But how do we think differently about what the actual outcomes are that we want and not just recreating the wheel and just pushing for the status quo? So that's that. Awesome, thanks for sharing. And Catherine, how about you close this out? Yes, wonderful. I will go with people. As you said in the beginning, we work globally, right? And it is really hard work, right? And some of these people have to do work in dangerous situations to fight for procurement and transparency and hold the governments accountable. So I'm so proud of the procurement champions that we get to work with around the world. And these are teachers and moms in Bogota that were able to reform school meals through procurement and get better school meals for their kids to HIV patients in Moldova. Who were able to get life-saving medicines and lower the prices to now a shout out to our champions and colleagues in Ukraine, you know, who have built an incredible procurement system and they're still tagging each procurement in a war situation with war so that they know where the money is going and coming from and can form the rice coalition for our accountable and effective rebuild of the country. So I think it's someone said that before, right? It's the people in procurement and the allies we have around us to really make this change happen, both in safety nets in the US and around the world. Awesome, that's a powerful note to end. Panel one on thanks so much to our four panelists. We heard so much about how we have to put people at the center of our work and a lot of really concrete ideas for what leaders can do to move from a compliance sort of environment and culture to outcomes oriented ones. So thank you all so much for joining. We're gonna move into panel number two. And as a reminder, if you are in the audience, you are welcome to submit questions at any point and I'd love to ask them of our panelists. And so I'd like to introduce our panel number two panelists. First, we've got Kristin Lara. Kristin is the behavioral analyst, sorry, the behavioral health administration's health information technology director. She guides Colorado's department of human services on how to use technology to enable and accelerate behavioral health system reform. Prior to joining that agency, she spent over a decade at the Colorado department of public health and led the development implementation and support of multiple public health information systems. And we have Corey Atehauser. He serves as the deputy director for the department of innovation and economic opportunity for the city of Birmingham. In this role, he's responsible for operationalizing economic activity to the mayor's vision for an inclusive and resilient economy by directly managing the following operational lanes, business recruitment, retention, expansion, workforce and talent development, business diversity and opportunity, real estate management and sports and entertainment. So it sounds like you've got a really small portfolio. In addition to the department's strategy and vision casting, he served as principal architect informing the city's recovery team, managing the $140 million disparity study and inclusive procurement programs, which is a topic we heard about in the panel one. So we're headed the city's new markets tax credit applications and serves as a strategic collaborator to mastercard in solidarity, $500 million pledge to black businesses. And then we have Ayushi Roy. Ayushi has just joined us at New America. So we're excited to have you here, Ayushi. She is the deputy director of New America's new practice lab here. And she returns to the lab after supporting as a benefits delivery fellow where we got to work together on the unemployment claims delivery sprint in 2021. Previously, Ayushi was the director of state and local technology at 18F and served on the leadership team for the U.S. Technology Modernization Fund, which is a little billion dollar fund to modernize technology and improve consumer experience across the federal government. So welcome to our panelists. So Koreate, I think I'll start with you. As we know, change in procurement doesn't always need to happen through huge movements and often small thoughtful steps can get us there more effectively. What's one small step that every department or jurisdiction can take right now to improve their public procurement processes for the short and long term? And I think I'd actually love to ask all three panelists this. So Koreate, if you could go first, that'd be great. Absolutely. Well, first, thanks for having me. But I say there's a couple of things, but I think there's two that are very important. I think one will be staff training and sort of the behavioral training, right? That's something that most leaders can do immediately through staff retreats or sort of weekly meetings to just make sure that people understand what inclusivity looks like and why it's important. Oftentimes policies are driven and they're sort of changed, but no one really understands how that affects their sort of local and day-to-day work. And so I think some of those short-term changes are really just sitting down and talking to your staff and making sure they understand, one, the importance of it, but two, you sort of get a sense of where they fit in that landscape of inclusive procurement. And so I think that's something everybody can do every day, every week, month by month that will ultimately produce some change and move the needle a little bit. I think another one is partnerships, right? And ensuring that you have the right partners at the table in order to expand your capacity, because as for me at least, for government, we have very little dollars in a sense to do the work that's needed, but we also try to make sure we bring on the right partners in order to amplify our capacity in what we're trying to do. So I think those two things are very short-term and somewhat obtainable for any leader in government. Yeah, that's great advice. Christian, how about you? What do you think? It's great listening to the first panel and I have a million and one different ideas in my mind for where we started or how to respond to this, but I think something that is very, very tangible that people can start with or do is really seek to understand and to be understood. So I think a lot of our procurement challenges, especially in technology because you have procurement, which is complex and nuanced and then you have technology which is complex and nuanced and then you have the two of them together which is very, very challenging or can feel very challenging or right for miscommunication. And so I think really bringing procurement and technology teams together to understand what your goals are across both of those spaces, having those feel closer and closer to being part of the same team, having procurement strategists as part of your technology or digital product teams are ways of really changing the game for how we think about procurements as part of the way that we do or implement or execute on technology. And so I would say really understanding the why behind procurement and the why behind whatever our technology goals are across teams can be a really valuable, tangible way of moving the ball forward. That is super concrete. I love getting out of the office and getting teams together. Ayushi, how about you? What do you think is one small step everyone can do? Yeah, talk to people and I'll tell you what I mean. I think that oftentimes procurement specialists to Kristen's point are sort of separated out from a technology team. They don't have the opportunity in the same way to meet with the constituents who might be directly benefiting from the service of the technology that is being procured. That's bananas, that's bananas. Like procurement specialists are part of the technology team in my opinion. And I think one small and very easy step is to have them be in the room when you're conducting your user research, when you're conducting any kind of testing that you might be doing with constituents, just have them in the room, have them be a part of those conversations, help them talk to the people who are gonna benefit from what they're helping you buy. Yeah, so speaking of technology, so government digital solutions are often not interoperable. They don't have the features that the constituents and the end users actually need. They're not necessarily focused on getting that end user, that benefit that they need and deserve. And then there's a lot of things, like portals that will do all things for everybody and sometimes kind of do everything poorly for a few people. What technology innovation tools would you like to see more incorporated into the procurement of digital services? And maybe Kristen, I'll start with you. Okay, so I think that, I don't know, this is maybe like a very big answer, but a lot of the work that we're doing in Colorado right now is thinking about building technology systems that will align with our human systems. So like you can only make systems change at the systems level, right? You can't work in individual siloed pockets and expect that the whole of the work will work somehow magically. That doesn't work. Systems change happens at the system level. And so I think that technology changes or innovations that make procurement more efficient, make eligibility systems and processes more efficient really have to do with the core components of that system that make the system work together. So core information services, like how do we put people at the center of making decisions from a consent management, e-consent management perspective, identity resolution services to fundamentally shift the way the system looks and shapes and the way that we interact with it to make the person at the center instead of the systems they interact with at the center can be very valuable. That can be overwhelming to think through. And at the same time, I think the more we work together in partnership on how to build out and collaborate on the creation of those systems, the more we are able to capitalize on what's going on within fellow agencies, fellow programs, fellow communities to build on that together. Yeah, Ayushi, I bet you have a perspective on what technology tools you'd like to be seeing more in part of procurement. What do you think? You know, this is a question filled with weeds, right? I think it really depends. I think what I wanna convey actually in this answer is that there isn't a single right tool. I think what's far more important is how you do it and who's in the room than what you buy. I think that sometimes technology is seen as this like fix all and I hate to break it to folks, but it's not, it's really, really not a solve. It's actually much more the people, the culture around the technology that will make or break the way that the public is ultimately served. And so I actually don't wanna provide a prescriptive answer here about what technology tools I'd rather see coming out of this like schedule 70, right? Like I think it's actually much more, let's be mindful of, you know, do you have a dedicated product owner who's working very closely with your procurement experts? Do you have the procurement experts and the product owners speaking regularly to the people in your community who you're trying to ultimately help? And that to me is going to allow almost any tool that you might end up getting, whether it's a portal or whether it's a sort of specialized verification tool to be effective in the way that you might hope for it to be. I love that answer because it's not a surprise that I agree with you. Often people think there's like, if you just buy this one thing and that thing doesn't exist. So that leads nicely into my next question, which is gonna be about, we need, as you're saying, these cross-functional collaborative teams that are deeply embedded with our end users and understanding that. And without them, I think teams are just naturally keep gravitating towards the traditional vendors, the traditional way of doing things, the way we've always done it. So what should we include in the procurement process going forward internally or externally to incorporate more points of view, more expertise, more diversity in every sense of the meaning? And Cory, I'll ask you that. Yeah, thanks. So I think there's a couple of different intersections there. I think obviously the vendor needs to be in the room. I think coaching providers should be in the room as well. And just general providers from other sectors should be in the room because I think in order to really reach some type of parity in our inclusive procurement here, we have to start engaging more than just one set of people and also outside of the industry that we typically serve in because I think there are learning opportunities in all of that. And so if we start bringing more intentionality to bringing more diverse groups together at the top when decisions are being made or when RFPs are being written, I think that would also trickle down into the process and then the awardee as well. And so we cannot talk about inclusive procurement and sort of being diverse without showing that at the top of the chain there. And so if there's a group of men sitting and writing at RFP, we're at a loss already because we only have one perspective, right? So we have to start at the top there to make sure we're inclusive. And I think that involves not only people that are in your industry, people outside of your industry. And I would also say inviting some past awardees if that's the case to the table to say, hey, give us feedback. What was the process like for you so that we can continually make those improvements? Because as everyone else on the panel has said, a lot of this work is so human-centric. If we forget that and we don't go back to that advice, then we build something that's not useful for anybody. And so making sure as hard as it is to make sure that we're always including people there, I think it's very important and from diverse backgrounds, both gender, social and economic status, you know, industry that you work in, I think that makes all the difference in what the end product looks like. Yeah, that actually touches on a question later on, but I'm gonna hop to it. So it's really about having diverse vendors. So among other disparities on every level, the National Equity Atlas published statistics that people of color are 39% of the population own 29% of businesses, and yet they're getting less than 12% of federal contracts. So maybe I'll start with you since we're talking about this, but I'll hop to the others as well. When looking at, you know, the RFP process, what steps can we take to make the process more open and they know, you know, how to find the RFPs, how to apply for them? Yeah, I think a couple of things, right? Forecasting your budget, I think is a necessary step. I know it's sometimes daunting to do, but taking some sort of forecasting into the equation so that you can have your vendor sort of plan for that engagement, that shows a level of responsibility, I think on the provider itself, to say, here, I know we're gonna spend a million dollars here, here are the base criteria for this, and so vendors are able to add capacity to get ready to sort of bid for those. So I think that's a part of it, is making that a part of your RFP process. I think also adding in partners who can help, again, coach, there are some small businesses and sort of the diverse vendors that we often see and talk about have very limited capacity, whether they're able to be bonded and insured and their employees. And so they don't always have the ability to sit in multiple zooms or meetings and talk about the RFP. And so how do we make sure we add capacity through partners and ecosystem partners that we have in order to amplify their capacity to build on to that RFP? And I think the sort of last thing I'll say is allowing sort of bridge funding, right? Like being innovative around how the awardee is funded, right? Like if you know that there's flexibility in that RFP to award dollars upfront without it being paid and at 30, 60 or 90, go ahead and put that in RFP so that then those vendors are saying, okay, at least I know I'll get 10% of the total be it upfront and that'll help them sort of operationalize and be able to mobilize on that RFP. So I think those are a few things I think we should include in the RFP process that helps break the barrier for some of the diverse vendors that we often work with. And we know what the issues are, but sometimes we have to work around policies and legalize and all of those things, which we all have, we're centered in Alabama. I would say the most restrictive but a pretty restrictive big law that sits at the state. And so we have to be innovative in how we approach making sure we add capacity and not burden to those diverse vendors that we're trying to serve. Those are really great examples. I know I've had some contracts that pay six months out and it's really unrealistic to expect a company to live on that. So that's a really powerful point. And saying. Yeah, Kristen, how about you? What is something that you could change about the RFP process to expand the pool of possible vendors? So I thought of a couple of things that might be noteworthy for this group in particular. One is being very intentional about what we're asking for people to respond to. So I feel like that is very like, oftentimes the way that we structure RFPs or the information we collect, we think about everything under the sun that could potentially be of value or use to us without some very specific methodical understanding of exactly what pieces of information are making or break it. And instead we just throw everything in there and that makes the application process itself much more daunting and time intensive. And as a result, if you are struggling to complete or like participate in multiple RFPs or application processes, we're creating actively a barrier to entry by not being thoughtful ourselves on the front end. And so I think that while that takes more time from us to do that, that is something that we can do that fundamentally makes I think the application process more open, more competitive, more equitable if we do that work on the front end. So that one feels very tangible. It's just says do the hard work on the front end. Don't make other people do the hard work for you on the back end. The other recognition or thought that I have is around somewhat similar to how and when vendors get paid is also thinking about what work you're having them do before you, in order to get to the point of winning a bid or a solicitation. And so something that we've been successful in in general in Colorado is instead of requiring, and we do this both ways, but instead of requiring on the front end that communities or vendors complete assessments to then bring all of that information to the table to say why they would be competitive for something, creating opportunities for that assessment process because it is something that we value so much. Have that be something that people or vendors or communities or organizations receive funding for to participate in that. So that we are actively creating a space where the information that we value and we care about and we want available to us and available to our communities, we're paying for them to spend the time to do that. Instead of making that in and of itself another barrier to entry or a barrier to having a competitive application. Yeah, those are really, really strong points. I usually anything else to add on this topic, any other tips and tricks? Lots of like me plus wanting as both of them have spoken. The one thing that I'll add that I don't think has been said is just that there are different vehicles of being able to engage with and build the relationship with sort of smaller businesses or women minority owned businesses. I think so much of procurement is relational, right? It's inviting folks in. And so there are other ways to invite folks and that may not be through a formal RFP. So challenges, startup and residence programs, entrepreneurship and residence programs have all been tested and tried in different cities and states and at the federal level that have been really effective two examples that come to mind that I'll just share quickly is the startup and residence program that was started in San Francisco back in 2014. It allowed the city of San Francisco to work with a very small startup company named Binti to do foster care instead of working with a much larger provider at the time. Another example is with NASA, the small satellite technology program now runs something called CubeSat that you can look up and it's basically a challenge with universities that chunks up money into 100K pieces to test out small satellite flight programs. And this was a program that initially came to us with like a huge, huge budget that we said, why don't you hold that? Let's find a better way for you to de-risk and spend your dollars. So I think there are other kind of methods that can be used to build those relationships before the RP even goes out. Awesome. Now we're gonna move into the Q&A portion. So again, audience members, feel free to submit your questions. Don't submit your answers, I guess. Submit your questions and I will ask them. There's a really fun one that I like here in the Q&A. What's a popular procurement practice that's a waste of time? What should we stop doing? So Ayushi, maybe you go first. Oh, I'm ready. Stop writing requirements. Stop with the requirements. I mean, there are many things we can stop, but I'll just start with that one. I think it's really important to, I think Kristin, you were saying this earlier. I think Crayata, you mentioned this earlier too, is bring your people in upfront and agree upon the problem. Like that is already going to be immensely helpful. Establish a problem and a vision statement. You do not need to be prescript about the requirements that will solve said problem by the end of your engagement with this vendor because chances are there are actually many other ways to better improve service design and delivery. So stop with the requirements. Love it, Ayushi for president. Crayata, how about you? What would you stop doing? Yeah, so there's a couple of different things. I think that makes sense, right? If you're in government, there's certain things that you have to put in there, right? Like you just can't get around it, but I think making it very clear on what your needs are, especially in the space for government procurement, it's like just make it very clear. Take whatever the legalese out that's not needed and make it very clear for what you need. We had personally a project that we were doing on the real estate, some sitting on real estate property. And so we knew internally, we wanted to see three different options, but we did not bake that into RFP. And I thought, well, if we already know what we're looking for, let's go ahead and make that very clear that here are three options that we're looking to do and enhance this property. So let's make it into the RFP. And so it was just kind of a light bulb, like, oh, well, it's like, you don't have people shooting in the dark, like just make it very clear. So we're not wasting, you know, vendors times. And so I think, as she said, taking out the requirements, I think is one. And I'll sort of flip over to the vendor side. It doesn't have to be all colorful. You can write it in a one-page response. That is fine. As long as the numbers are there and what we're asking you to put in there is fine. A lot of times for us, we get very illustrative sort of documents and they all pictures and charts and graphs and they make no sense at all. And they're not responding to the RFP at all. And so it could easily be done in two pages of just paragraphs, right? I think the ultimate goal for us is to make sure that we have a good quality vendor. How that comes through in written document, it really doesn't matter if we've done the work on the top and to make sure that we're gonna commit to a diverse pool of vendors. We'll take it however it comes, but usually the colorful ones, usually, not all the time. I'll preference that not all the time, but usually those are the ones that if they are chosen, we have to dig and dig and dig into to make sure that the work is complete versus the ones who read the RFP and said, I'm responding to exactly what you put in there and they were able to mobilize that project pretty quickly. So some caveats in there, but I think that's sort of my answer there. Awesome, thank you. Those were great ideas. How about you, Kristin? What would you stop doing? I love the answers to this so far, by the way. I think that I would suggest or note that I feel like sometimes people think that they can procure out or contract out their flushing out their vision unintentionally. So if you're executing in an agile manner, like you have to meet your vendors halfway. You have to meet your partners in the middle. You're in a very, very, very, very important part of that equation. And I think we can underestimate how important we are in that equation and think we could draw money at that part of the problem in replacement of our time, figuring out our why and our vision and how we need to organize and where we need to support and guide the vision in partnership with our vendors in order to get our own needs met. So some things you can't procure for or don't contract out your vision. That is very powerful. So the next question in the Q and A and Coriace, I think you touched on this a little bit. So maybe I'll toss it to you and then Kristin or Art, you should feel free to chime in as well. How can jurisdictions best improve transparency in the procurement process? I think it being top-led, right? One of the things I think for us that we have fortunately for our mayor is transparency is one of his core principles. And so it drives all of our processes internally. And so making sure that folks understand not only was written in the procurement process, but those unwritten things as well. And we all know from being in this space that there are written things and there's unwritten things. And so being able to really mitigate the two so that the vendor knows exactly what to expect. And I'll give a real example and then sort of pass it on from here. For us, we have a mayor council act that we have to sort of abide by. And so that includes the council approving certain funding that goes in and out of the city. And so on top of the RFP process which we detailed pretty well what that looks like, what the payment process looks like at the end, what we don't have in the middle is the approval process from the council, right? There are certain subcommittees that you go to and they have to be in by a certain time. And all of that has to be right. And so I think the transparency of detailing everything that happens with that RFP not just externally, but internally as well will help the transparency there because there's oftentimes where vendors are like, I don't understand why I'm, you told me I won the RFP and now I have to wait 30 days. Well, there's some internal processes going on right from probably your city attorney and the mayor signing it or whoever needs to sign it. Why not go ahead and detail that out so that they can understand what mobilization looks like, right? Like we may say, well, it'll be 60 days, but why? Well, this is why because they make it do some other jobs in between so that they're not just twiddling their thumbs thinking, oh, it's gonna happen in the next day or two or week. And so I think making sure that people understand not only the external process, but the internal process as much as you can, right? There are certain things that you can't share because there's confidentiality there, but where you can share it, make sure that those expectations are shared up front so that vendors understand what that transparency looks like. Yeah, that's so important because that's probably another reason why I think we're going back to the same old vendors or the same ones are applying because once you've done it 50 times, you know how it works. And if it's your first time, it's a pretty scary black hole sometimes. Absolutely. Christina, are you usually anything to add on that? For how to increase transparency? Okay. Then I would, I think this, what procurement wins have you been a part of or seen that you are most proud of? And Kristen, maybe I'll have you go first. Can not get the unmute button to turn. So one of the things that I was part of in Colorado was our COVID response. So I was at the Department of Public Health at the start of the COVID pandemic and was deployed for, I think, nine out of the first 12 months of our COVID response. And a lot of the focus of my work was rapid building and deployment of technology to support the state's COVID response. And if you think back to like the start of COVID that summer, that first summer in 2020, we were very focused on what is contact tracing, what is case investigation? How do we do this at scale very, very rapidly? And so we had some emergency, clearly truly an emergency where we had, I think more flexibility from a certain standpoint than we typically do. And so we got to be more, I would say, creative or nimble than typically we are able to because we were in a very, very true real emergency. And we rapidly procured, and I'm trying to remember what it was, we rapidly procured for case investigation contact tracing technology for COVID. We did, I think it was like two to three week demonstration by multiple vendors. We invited multiple vendors to demo, like provide test environments of their products so that we could have, we did usability tests with local public health agencies during this two to three week window. Selected one of those vendors, procured with it or like executed a contract in a week and a month later, four weeks later, had local public health agencies online on that system. And so we went through a very, very rapid process, but also did it in a, I think really innovative way in a way that was very truly people centered in the moment, even though it was a very high intensity environment. And so that was a very exciting win, I think for the state of Colorado, and I was really happy to be a part of it. Awesome, love that. Ayushie, what's your success story, your favorite one? Oh yeah, I'm very fortunate to have been part of a few different experiences that I've enjoyed. I think the one I'll reference is actually slightly adjacent. So in the state of Michigan, in my 18F days, we did a lot of procurement workshops where we would go around to different states and cities and federal governments and kind of do these rapid RP writing process with the procurement teams and really show them how you could take a process that might otherwise be sort of six months or eight months and actually do it in like two weeks, believe it or not. It's possible. And one of the things that came out of that experience was that the state of Michigan actually adopted the language that we would teach about how to de-risk procurements into law. It was huge. And so now it wasn't just relevant to that one procurement way to walk through support, but it's now sort of baked into all of their various processes. And it's one experience I'm really, really proud of. And I'm happy to sort of link out or share out the de-risking report that they were pulling from. I think it's valuable broadly for a lot of states and cities who are working on this. Yeah, I think definitely. Let's find a way to get that out to everybody. And then Coretta, bring us home. What's your, what's project for you most proud of? Yeah, so there's two in one. I'll say first we completed our disparity study for the city, which, you know, us that are in this space know how important that is to legally stand up race and gender conscious remedy programs around disparities. And so it was quite a long time. We started it at the height of COVID and we worked through all of this electronically. And so we were able to complete that and now have recommendations in hand on how to remedy the disparity that has been identified. And so that's gonna change the landscape for us to be able to legally make sure that we have goals in place and that we hit those and when we don't there's remedy for that. So that's one side of it. I think the other part of that which includes the disparity study is our program called vital and that's in valuing inclusion to accelerating lift. And so the mayor and I we put out a corporate challenge to all of our top 10 corporations in Birmingham, regions, Blue Cross, Blue Shield, University of Alabama, Birmingham, Alabama Power to say, hey, we know nobody has it right. Everybody needs to start somewhere. So after the George Floyd murder we put out this challenge to say we need to do better and includes a procurement. So how do we do this? We need our corporate partners to buy in to make sure that we understand what is our baseline for spend? And so we were able to produce a spending report with all of these organizations on board to say, listen, it's not right. We got a long way to go but at least this is our baseline and then we're going to build on this year by year. And so getting that commitment from those CEOs to open their books really and to show their hand that we don't have it all together despite what we may have produced in the past. We're now being held accountable to the people that we say we're serving and our customers and things. And so we were able to produce that report in 2021. And so it has continued our work in making sure that community is still at the center of inclusive procurement because we understand the generational change that happens when inclusive procurement is done right. Because typically those are the ones who are supporting your PTAs and neighborhood associations and things of that nature is those small businesses. And so in order for them to function and keep turning and scale and have legacy we need to make sure we get it right. And so we're very, very thankful and grateful that those CEOs understood that challenge issued by the mayor and signed on and put people in place intentionally who could drive that focus. And so those were some big wins for us that I'm excited about and to continue building on. That is super inspiring. Thank you for sharing. So thank you so much to our second panel. We heard a lot about really concrete ways to increase diversity. We heard some things we should stop doing. And I think we just heard three really great stories of successes. So with that, I'm going to close out panel number two and pass things back to Alice and to wrap us up. Thanks everybody for coming today. And Marina, thank you so much for giving such energy and enthusiasm and Harry Potter references. We will never let that die. And for hosting this incredible panel. Thank you to our first panelists, Bruce, Afua, Catherine and Jennifer for identifying and discussing with us where the current state of public procurement is and encouraging us to think about the why of procurement and shifting the culture towards people and away from just compliance. I also want to thank the second panel, Kristin Coyote and Ayushi for sharing their on the ground experiences and thinking through what is truly bananas and teaching us how to make the procurement process more people-centered through both big and small interventions. Again, we're so excited to share our report with all of you. It's been a really long process, but we're proud of how it's turned out and what we've written. And we're thankful for all the written contributions from the speakers you've seen today and countless others. Thank you for coming to the event. Please continue to be procurement champions wherever you are and have a good rest of your afternoon. Thank you everyone.