 So, welcome everybody to our, our event on tackling design operations. I will just start by introducing myself and then I don't want to take up too much time, because we have just an incredible group of panelists today from the design world and this event has been a long time in the planning process because public sector designers are very very busy people. So, I want to introduce myself and our panelists and then we will get going. So, I am Hannah shank I was one of the early members of the US digital service and a founding member of the public interest technology program at New America. I co authored the book over my shoulder with my colleague Tara McGinnis, who was a part of the healthcare doc of team in the White House. I'm the domestic policy team for the Biden transition, and she is the founder of the new practice lab at New America. So, this event today is really two fold. On the one hand, we are lifting up the work around design and government and kind of digging into some of the more unpleasant pieces of it probably. We are also here celebrating coming up on the one year anniversary of this book, Power to the public, which is a blueprint for how governments and nonprofits can harness the power of digital technology to solve public problems. It has been described by President Obama as a good read for anyone who cares about making change happen. Finally, my background is as a designer, I was a private sector designer for a long time. So I am especially excited about this event, where we'll get to dig into some of the extremely difficult work of bringing design and specifically standing up design operations at all levels of government. So I'm so pleased to introduce our panelists, all of whom are people who get up every day and take on public design challenges. We have Mike Land, who is a design leader dedicated to leading and building exemplary user interactions across a range of digital channels, mostly in a civic tech space. He is a senior UX designer at the United States Digital Service. He is currently focused on scaling design at the Department of Homeland Security. And Mike is featured in Power to the Public's first chapter on the disastrous launch and then retooling of the nation's immigration system. And now, Mariana Kano is the design director of the Service Design Studio, which is housed within the New York City Mayor's Office for Economic Opportunity. And we profile some of the incredible work that the Service Design Studio has done in New York in Power to the Public. And finally, we're joined by Sid Harrell, who is a UX researcher and product manager who has helped US city, county, federal and state agencies unlock the power of technology to serve constituents. She's worked independently with the Center for Civic Design, Code for America, 18F, and has been a mentor to many people in the field. Sid's work isn't specifically featured in the book, but she's been involved in the background of many of the efforts we write about in Power to the Public. And she is also the author of her own book about Public Interest Technology, A Civic Technologist's Practice Guide. So I thought today a good way to get started is to talk about titles. I think I'm going to talk about the range of titles that we've all had while basically doing the same job. In the private sector, I have been an information architect, a user experience designer, a producer, a UX researcher. In government, I was a digital service expert, services expert. And now I just say I'm a tech person because it's easier. So in government, our titles are complicated, complicated by the fact that there aren't real titles for designers. So you kind of have to make them up. So why don't we do a round robin? What have your titles been? Sid, you kick us off. Okay, when I joined Code for America, it was as a UX evangelist for that nonprofit. And then I went to 18F where my official title was innovation specialist, though I acted as a strategy lead. And even when I was chief of staff, I was still an innovation specialist officially. And then in my recent work at the California courts, I have been a senior business systems analyst, always on all those things a designer. How about you, Mari? So in this job, I was first titled as a computer systems manager to the point where like when I did see an offer to come in for the interview for that I thought that they had emailed the wrong person and I almost didn't go for the interview because I was really confused. Yeah, and then I was also called I'm now titled the research projects coordinator. And I wanted to help share something funny when I was like commit computer systems manager I actually initially also failed the computer systems manager civil service test. Because I basically didn't use like the keywords that were really needing to be very, very specific about my responses so I ended up having to write a five and a half page appeal doc explaining why I believed I was qualified to pass the test, even though I had been in the role for So that's a little bit of my story and just really glad that a colleague of ours here in the city Mike Hickey, he recently wrote a great blog post explainer about like the NYC civil service testing process so definitely go check that out if you want some plain language guidance on how to pass the test here. Mike, how about you? I've been in government, just over 19 years now, federal government, and I've been everything from an information technology specialist which is the 2210 series that everything gets lumped under across government from design to developers to whatever. I'm a public affairs specialist though I will note that I did no public affairs work at all. That was just my role that they put me underneath as a designer and leading product and things like that. Currently a digital services expert, which is what all the US DS folks get kind of pulled in underneath. I'm still not sure what that quite means but happy to be there and doing the work. And, you know, all the while like like said mentioned, just been kind of leading design efforts. So whatever the title doesn't matter right. It's the work you do. So to summarize. No one knows what our titles are what our skills are. Maybe we do computer stuff. I don't know. Yes, I think that's a good. It's a good jumping off point for the rest of this conversation so all three of you have experience not just doing design working government but standing up what are essentially brand new design operations at different different levels of government. One of the things that makes this work so hard is that, yes, you can incorporate design thinking and focus and focus on the user into one project, but then how do you embed that capacity into government so that it just happens as a part of the process. So we often talk about the way to solve this issue with the phrase design ops, and that is the free the title of this of this webinar, but design ops can mean a lot of different things to a lot of different people. So we would love to invite the audience to think about what does design ops mean to you and put it into the chat. And we can have a conversation about what actually is design ops. And then while you're telling us your thoughts about design ops for our panelists to react to. I would love to do something just a little bit different for this book talks and normally we share stories from the book. But for this event, I would really love to flip the script a little bit and ask each of you to tell a story. Can you each share a story about a specific design related challenge you've faced and what you did to overcome it. So, Mario we have talked about the service design studio in New York, as being sort of the Cadillac of government design ops. In a way you've been around for a long time you've had a lot of time to, you know, try and fail and learn. Can you tell us a little bit just a story about your experience running design in the Big Apple. Yeah, sure. Thank you so much I mean it's it's really flattering to be called a Cadillac. I mean, so I lead a municipal level team of about like six, really six amazing individuals who are not all designers by trade and we're part of like a 50 60 ish member team at the mayor's office for economic opportunity. You know, we support a city with a population about 8.6 million people, 300,000 civil servants across about 135 agencies and offices and well I know that's really large. You know in itself I think the scale of my work compared to what Sid and Michael are doing at the state and federal levels is probably this is probably much more modest than what you all have to take care of and there's a lot less cats to her. I think here in a city, but when we officially launched our studio in 2017. You know, one of the first things we had to set out to do is like just getting on people's radar explaining like what the heck service design is, you know just like the 101 and really start providing resources and easy to use tools and tips. You know, I think like, you know, our work is considered kind of wonky because we're not like many of the sort of digital service teams who are able to put out more of these sort of like tangible products that people can like really understand and grasp and like see and understand everything when we're really out there promoting a methodology and really trying to show how the service design practice could be utilized for just about anything from designing a productive meeting to even like something crazy like co designing an RFP process or program with community based orgs. And so we kind of knew that we couldn't just like jump into service design projects with a snap of a finger and we really needed to spend a lot of time socializing and communicating what what we do who the heck we are and like how we're here to support. So over the first year, plus or so, we really took the time to set up an ecosystem of service design offerings and those offerings were like for people to come learn about service design to try things out with us and to also just like come for really kind of therapeutic event sessions about the silos and challenges everyone was facing in their own agencies. So we set up office hour in office hour program which is talked about in the book, where we would be able to like bring in folks from any agency or office to come to us one on one for an hour. We set up, we put together what was what's called civic design forums which is a community of practice so really trying to de silo government and inviting people to the table to come learn about service design and digital design practices. And we also have put out this field guide and publish this field guide called the tools and tactics. So that was like a little primer that like really people could have like on their desks take around with them in their bags and just really start to get to know service design in a lightweight like plain language way that was written for government folks to be able to understand. And with that we were also offering workshops, and just like doing many projects with folks alongside doing larger projects so we're doing these sort of small medium large touch points in the beginning. And a lot of times I like really question like, man, we are spending so much time like socializing all of this and like talking about what we do and you know are we getting, but I want to get to work, you know, but I think, as I really think about it and take that step back, it was totally worth it to spend our first years pushing communication pushing a brand, you know, a visual brand. And because at the end of the day, all of this explaining helped us build ally ships and camaraderie like comrades across multiple agencies so that helped us really get to know the different types of projects that were going on across agencies and then we could kind of turn into matchmakers where we're like we heard DHS is doing this nature is doing this, you all need to talk to each other to share to share resources and share tips and stuff like that. I would say within these first few years I mean number wise like we've offered about like 300 office hours now talking to like 600 public servants at this point, run over a dozen of these service civics design forums with over 800 folks, and then they have supported around 50 agencies and offices in the last couple years so I think those impact numbers were really proud of and will continue to keep doing so in this new administration. So much of what you said, I just resonate with so much of that and I am seeing a lot of nodding Mike and said do you have, especially the like having to do the talking in the selling first before you can do the work is thoughts. Well I really do think it's an interesting contrast to the private sector, and even some civic tech rhetoric where this idea that we need to act we need to buy us toward action. We need to build we need to get things out there. And when I tell my story in a minute about early stage, you know there's, there's a lot of talking that comes first because we don't have any idea what the thing is when we showed up. Yes, I'm on my end. I use the word diplomacy a lot, because because a lot of its diplomacy getting out and, you know, connecting with folks across the, you know what's an agency or whatever it is. And, you know, having those conversations to build relationships and I think that's an important first step. So, so Mike we in our earlier conversation, jokes that you're driving the Honda in this scenario. Do you want to. So, so you've been engaged in the very difficult work of bringing design to USCIS, which is the entity that handles immigration for the for our nation. Do you have a story that you can share about what some of that work has been like. Yeah, so I'd like to think it's a Honda right. I'm not sure it's so dependable every day. So maybe there's some other cars who might think out there but my story really picks up where at the end of chapter one of your book. And that is, you know, we've been engaged to USCIS for since 2014 or so we meaning us digital service and trying to modernize their process, you know whether it's integrating agile or just trying to, you know, think about how human in the pie there. Um, one of the things we started to figure out after a while was we were having some success enough success where you were seeing pockets of design across the enterprise, you know, across the agency, and the reality became at some point that whether pockets, they weren't connected in any way. And so there was this decentralized model of innovation happening where individual teams were working on problem spaces that were very similar across the enterprise, but not being connected any way and not learning from each other and not, you know, building knowledge. So one day I kind of sat down and wrote a position description for a director role job. You know just, hey let's do this right I mean kind of the US DS mentality of just kind of throw something out there and see what happens. And I just kind of carried up the chain and eventually, and that was about 2018 about the end of 2019 they came back to us management came back to say hey let's do this thing. But we don't want to you involved, we want to have one of you guys run it and so it just so happens that I end up stepping into the role. And you know the role is, you know, very design operations focused right a lot of things we think about an industry tooling and research, trying to figure out how to align research across an enterprise. The fact that they're the lack of culture or coordination that I mentioned you know the we had design teams that didn't even know the other teams existed, and yet they were working on a very similar things and so the first thing I did was go out and listen to all these different all these found all the teams that did kind of a designer safari out there had to find these teams across the enterprise it was interesting. I kept finding teams and thinking like, have I found them all and I'd find one more and be like, Oh, there's one more team. But the good thing was the very these folks are very eager to be connected. And so it was a matter of just trying to create community and start build a community of practice, started to, you know, create a slack channel for example a team channel and started to, you know, connect folks and have many practice meetings, where they could, you know, interface with other and, you know, we had people finding other people like, Oh, I didn't know you worked at USIS, you know, as a designer and, you know, because they work together in the past. So it was really, you know, the design ops portion of it was pretty tried and true for design ops does the challenges I think in the government space are, you know, around things like risk aversion for example, you know, getting designers footing at the table to be able to actually do, you know, do research for example and apply that research across, you know, and those types of problems are the ones that I think you see more typical government tools for example, you know, we can't just go out and grab any tool. There's a very long, really awful, you know, sausage making type process to get tools applied, you know, to get tools procured. And then contracting for example, trying to make sure that design is a part of contracting out there and making sure that your design is part of the evaluation and even an indicator whether companies actually decent or not. You know, if they have design, you know, there's a lot of folks that know development out there but not fewer folks know design your companies do. So there are some differences I think between traditional design operations though a lot of what we did focus on was, you know, I focused on was trying to build a, you know, design program with all of those different elements and ultimately in the end it was about making designer jobs easier. And that's that was, you know, my entire goal. And with that I just want to remind people so there's an open question please drop into the chat your definition of design ops we are not grading it there's no right or wrong it's purely just for the conversation. I think that was a great summation of what what may or may not be design ops. Oh, thank you and I. I also a lot of nodding, my answer do you give things that that sparked. Oh, I think just a lot of empathy and recognition I think, especially on the procurement part, you know, tooling is hard. Getting ethno was like a year and a half long process at the courts. Yeah, and we also talked about like try getting more mural approved. We're gonna, let's come back to that. Because there's definitely I don't know if it's just a therapy session on how difficult it is to get these tools or not but I want to be sure we get that if we get to your story. So, in this terrible vehicular metaphor, we said like maybe you're the bicycle here or you're on a scooter I don't know. But you are kind of, you're often sort of the first, if not the first designer I would think like the first foot in the door for a lot and a lot of the work that you do. So, would you share a story with us. Absolutely and yeah that resonates I like to be first in the door and I like later stages to but it's fun to go in and sort of discover that there are people in almost every government organization who are facing design decisions, and often making really good ones but they may not identify as designers and be able to kind of bring them this practice in this community. So, when I arrived with a partner named Jack Madden's at the California courts, four years ago as embedded consultants and I, we have just wrapped that in the last month. We were asked to work on something called the self represented litigant portal, which is a lot of syllables. But short version, if you are in a civil case, you are not guaranteed a lawyer so if you have an eviction or an abusive roommate you need to evict or you need a restraining order or you need to deal with child custody or divorce or change your name because you're transitioning. The system doesn't provide a lawyer for you. So about 80% of California's court cases have one side or the other that's not represented. And this is obviously really difficult for the people involved in them it's also difficult for the court system because court is complicated and it's designed from the beginning over centuries of practice for the idea that everybody has a knowledgeable advocate. And so it doesn't do a very good job of accommodating or supporting people without them. So a committee of judges and lawyers had identified that one thing the system really needed was portal which some of you will know the quote about how that's a great word for an expensive website. Four people in this situation. And, you know, one of our first questions was okay well what should it do. We don't know. They have, they had websites they still have them. They have support centers in every court in California that will help self presented litigants for free. If you can get to the courthouse in a pre pandemic state or if you can find them in a pandemic world. There are legal services corporations that will help people that are means tested, and also oversubscribed so often really hard to access to so there's a clearly space for a digital component and helping people figure out how to do court or actually do court. And so, coming in and looking at what's here. I'm looking into a situation in which there's not a modern CMS. There's not a design system. There's not a content practice. In fact, one of the more interesting things about the court system if you've ever dealt with it is there are these forms that are used for almost every action in the court system. The judicial council of California manages about 1200 standard forms, and they are part of the rules of court capitalized so they have the force of law, and they don't employ any information designers. So, often those things come out difficult for regular members of the public to use and the web is seen as one way that maybe we can soften that help make it easier for people. But the big question for us was where do we start building this multi syllabic thing. When all of these components of what you think of as a modern design practice aren't in place. In California, as good civic tech practice we found a tiny little bit of legal service that nobody really owned, and where we weren't going to run into arguments between county courts about what the procedure should be. We thought. And that is the act of serving papers. And so we started out just building some web pages that would be mobile capable and have a design that was clear and step by step and shortening the content enormously from the 3000 words about that that existed on the old site. As much as we could do involve people in this we did a really, we're really lucky to be given permission to do an extensive research practice, and visit 20 of the 58 county courts and spend, you know, a day or more at each of them understanding, what's the experience of going to court without a lawyer in California so that's, and even to get there you know had to write a lot of memos on letterhead and have the kinds of diplomatic conversations that Mike was talking about. We thought that we were able to build something nice, and it worked well, and people were really interested in why this was better and what we could do about it and gave us permission to move on and start expanding to probably the most common court case type that people do alone which is divorce. And then getting into the work on that the thing that became clearer and clearer as we started to build some components of a design system. We started to work with Drupal and instead of the older proprietary CMS that they had was that, like in so many spaces content was critical, but we did not have the standing to talk about content. Design was not the right place to be talking about that it had to be talked about by an attorney. The way attorneys think about content is not very aligned with great web practice, especially for a broad public. So, I think, you know, the way Mike talked about establishing a role and my also talked about establishing a role. The role that was so important for us to establish early for design ops to continue existing is called web content attorney. It's a permanent role for a judicial counsel staffer who is a person with a law degree and also with web experience. And that made an enormous difference in the kinds of processes that we could have so now as I and my partner are rolling off and things move on to a new stage. The web content attorney does a lot of the user research as new case types roll on to this portal. She works with the design lead in the web group to make sure studies happen and make sure the right attorneys who have subject matter expertise see those. Make sure that testing happens watch results so she's almost like a product manager, but the title was chosen to really carefully fit in to the system and so I'm just this little act of serving papers, we built divorce and then small claims and then eviction restraining orders parentage or paternity as you might know it. And with each of these we iterated practice and kind of looped in new sets of people to see how design worked. So it's still not an enterprise design practice but it has multiple people who regularly do user research and a lot of people, even attorneys and a couple of judges who regularly see user research. And some of those key components of tooling although not at the highest level in terms of design systems and a more nimble CMS. And other things that go with that. So that's, that's four years of work for a tiny team to kind of get to that stage. So there were two things in there that I want to pull out for you guys to react to. So one was that you started small. And that was I think a pretty critical first step and something we talked about in the book, but I'm also curious if Mario and Mike that resonates with like, you're like, yeah, that makes complete sense I did essentially that or in my own scenario or whatever and then the other thing that I think is really interesting is the role of legal like as a designer you have to in as a as a public sector designer. I think it's fascinating that you I had not heard that story that you created this sort of hybrid role. I'm wondering, Mari and Mike if you have had similar experiences or so whoever would like to take either of those questions. That's an amazing role, Sid. So we don't have like a case like that I would say but there's something really interesting about this like starting small kind of thing. I mean, partly, there's no way we could advocate for like let's just have a big service design team on city staff lines like that that just wasn't going to fly so you know in the beginning of the service design studio, we really leaned a lot on like academia actually so person's new school folks. We leveraged a lot of their expertise and skill sets to support some of the design pieces when we didn't have enough folks, you know capacity inside of government to really show what it could what design could do, and partnered with agencies to like really observe how that process could go down. And a lot of our designers for quite a while we're on grant funded lines. And so we had to really spend a lot of time building up the case for like why these these folks should move into staff city tax levy dollar lines and now now thankfully all our designers are on staff lines. It used to be like two or three that were on grant and then they're all moved into staff line so that's like a big win for us. I think also something that Sid said to it's like, you know, it's not about always like just looking for other people who have this sort of same pedigree coming in, you know, we definitely are always looking for each other for sure. It's also about like training up folks who are already in government who are curious and hungry to like tack on a lot of these methodologies and tactics into their work, and finding ways to work alongside them so that they can really train up and learn it and do it on their own. So, in some of our projects, part of our scope of work would be sometimes with an agency to say hey, we'll go through a service design process together with you all. But we want to bake in learning and training components so your team can do journey mapping and do interviews by yourself. So we've seen that happen with like for example the administration of children services. The community based strategies team who we work with for a year. And now they're running a lot of things on their own and just kind of calling us every so often for like a couple tips and tricks kind of thing. But we also see them contracting with other design vendors without much help or guidance from us at all these days. And the other thing is I think with our office hours and civic design forum programs we captured all the names, all the titles, all the frequent flyers that come through these programs, and really reached out back to them and said hey, you know, you are an exemplary like design should have them design champs, and we would write blog posts about them or really support uplifting the work that they were doing. A lot of these are staff members who are like mid level, you know, and we recognize them a lot like in in our writing and everything that we're doing and now some of them have either moved on to like other jobs in design, or have like moved up in their agency so it's really great to see that too. I'd like to resonate with me. By the way, I love the idea of a of a web content attorney I mean I'm stealing that idea 100%. So thank you. Yeah I mean, you know largely what government modernization about right now is digitizing forms right I mean that's 90% of the work at this point is trying to figure out how we can digitize digital forms and large part of that process. And the large part of it's really dealing with on the government side, and the federal government side it's legal counsel. It is privacy. And then the other piece of it is everyone's favorite paper or production act. Which we're making some headway on, you know in government right now there's been some memos that have gone out from Oira which is, you know, kind of making usability kind of more part of the picture and things like that but you know they're all of those things are, you know, I don't want to call them barriers but they're part of the process, you know, and, you know, as you said legal counsel is not really legal counsel and playing language don't all you know, not in the same sentence quite often. And so, that's a challenge on the on the simplicity side or starting small. I think one of the secret tactics tactics and it maybe it's not so secret, or maybe I'm spelling the beans today is to start with a pilot project. And we all talk about that it's like, okay, you know, if you come with an idea or something like that and, and folks are reluctant, you know, the risk aversion kind of creeps in. Suddenly it becomes magical in some nature and it's like, oh, yeah, we can do a pilot. Yeah, sure. And so it's starting small with that kind of pilot project and building upon it showing value for one and then building on its critical piece. Can I build on something Mari said. I wanted to respond to your talking about your colleagues now hiring design agencies with little support from you, I think that's an underrated component of design competency in these public sector organizations is the ability to really craft a strong design brief and manage an agency and hold a contractor to it. It's really, really valuable in these organizations because it's just unusual for them to be able to have the size of design team that might support a private sector company with similar responsibilities. Yeah, I think that goes back to getting more in federal side it goes back to getting more designers at federal service, because then they're able to be there to help interface with your procurement folks to, you know, make sure that language is right, they're able to go and be a part of procurement, you know, vetting processes so you can, you know, be there and understand, oh, you know, this group, they get it they don't, you know, develop support that appropriately so I think that's one of the things we've been trying to do in DHS is try to be more designers and federal designers and put them in the mix because largely right now it's, you know, the teams are made up of mostly contract designers which is not a bad thing they're doing tremendous work but you know, having more federal designers in the mix is going to be a critical component of modernizing government. So, and I will say we're going to get to the some of the questions we have some good questions, especially a question on career path and pipeline to this work that I definitely want to hit on. Last call for if you want to say something about what design ops means to you, not getting a lot of traction on that I think, which is also maybe that just speaks for itself. I think it's complicated, but I do want to come back to. So we've hit a bunch of times on the need for community and a design community of practice. It came up to me I think Mikey just said like we're all designing forums. We always joke that we're all like everybody was on the same project. It's it's form redesign. So, but I think Mike like you're, you're having basically to like go on design safari to find the teams is is is not uncommon. So how have you worked to create design communities what's what has worked. We'll go first. Take it. Well, as I mentioned, you know, that was after going through my and finding commute folks and starting to think about like what the picture looks like. Certainly building community was like one of the first steps, because, you know, you start thinking about like how you create a program and how you start to figure out, you know, how do you start to find efficiencies across especially you start thinking about community start thinking about design systems, start thinking about all these different ways to to centralize that process. So the key building the community was the first step, because I knew there were designers out there, and they were good. There's just a matter of connecting them for on problem spaces, you know, and certainly have conversations. For me, this is natural to me because I come up through user research but finding the right people to observe research sessions and facilitate research sessions. I was part of building that community, but it was also finding who in the system was interacting directly with these underserved users. These, you know, Californians who are trying to get something done in the court system without that status of being a lawyer. And the judiciary that I have met is mission oriented and smart and amazing, but specifically the people who spend their day to day talking with people who are not professionals in the system. We're I think quickly able to grasp that design itself was a tool that could really help them do what they were trying to do. And you often saw, you know, a lot of, we've created a flyer or we've we've created an extra form that goes with this that offers some instructions and yeah maybe the iconography doesn't look sophisticated to somebody who has a design I, but there's this impulse happening there. And that's really good. And often like the things that are deeper than iconography are doing really good. And so kind of co-opting those people and having a little bit of a welcome to Hogwarts moment it's like you know what you're doing you're doing design. There's actually a ton of resources out there in this. Let me show you. There's a lot of examples I've shared a lot about building community, but one thing I wanted to also add it's just like supporting the person in the cubicle next to you kind of mentality. I'm lucky because I get to sit on a larger team, you know, at the economic opportunity. So there's a lot of design osmosis that I think goes on just from purely like being present and demonstrating ourselves like in, you know, when we're facilitating conversations, showing clean decks, sitting down with someone to like develop an agenda for, you know, a project sharing templates around, you know, project scope, scopes of work, things like that. But also, so, so you know, I feel like I've spent a lot of time just becoming really close friends with a lot of people that I work with and sharing my, my kind of practice with them, supporting other folks help get their work done just getting started, not about necessarily design just like helping each other out, you know, be more efficient. And I think a lot of that has really paid off in a lot of ways, you know, and not that I meant, I was never like, Oh, I'm going to like do just design osmosis on this person but just those other relationships and building that kind of trust with one another. Now you see folks who are who feel more empowered or more confident to do these things on their own, who are asking different questions now that aren't like basic one on one questions anymore. Like seeing people, seeing people just kind of like run, run the show is like, and just being that person that's like, you're doing an amazing job, you know, I've seen that also be part of this whole community building. So yeah, that's been a like great experience. And I remember the day where on my slack channel. It wasn't just me posting questions that folks in the community were posting questions I was like, here we go. Yeah, this is getting good here. You know, so that was a great day. Yeah. Yeah, I like just finding little opportunistic tasks to help people with. You know at some point an attorney came to me and said, Hey, we're redesigning this short form and you can't participate the committee's all lawyers and judges but you know I'm just curious you know how you like cut things up and move them around, which is something I've done to just show chunks of content. What, how would you do that with this form and I said, Oh I can provide you with some slides privately. I'm going to take a couple hours and and take an eye I look at this for you. And then, you know, you can go into that meeting and look great and interact with your peers and. Yeah. I think that's so I mean that just sounds so commonplace that like people don't fully know what design is. Why were there, what we do but they know like, Oh, you did a thing. How do I do that thing. So it is a little bit I think like the Trojan horse model right of like show you how to do this very small thing that is a very small piece of my skill set. And in the hope that you know it will incentivize people to, to dig a little deeper. We are going to turn to audience audience questions in a moment, just a reminder that power to the public is available with a discount code, which is I think believe available to people somewhere. And we also have a link to said book. So please, if, if you like this conversation. There's a lot more wisdom in both books. So, one of the first questions that I, I want to get to is, how do you do this work, how do you get to do this work. And what, you know, aside from title so the question is aside from title what qualifications are normally required for positions but like yours. I know that I think we all probably came through various weird ways, because these jobs didn't exist. So any, any advice on, you know, people who are more early career, how to do this work. I think I often say that if you have a technology skill or in this case a design skill that is strong enough for you to teach someone else, you're ready to do this work at least in that skill area. And organizations hiring for this work are usually looking for some experience in the past and some kind of commitment even if it's not professional to public sector or civic participation. So if you've been a poll worker or, you know, been on a local board or been involved in a community initiative those are all really good qualifications because in some ways community organizing is one of the models through which this happens. Oh, go ahead, Mike. Yeah, I would agree. I mean, there's, there's ways to get your feet, your foot in the door. There's also general assembly and classes like that where you can start to build skill sets, you know, and also in government, there's a new thing called digital core, which is a way for folks just out of college or, you know, folks earlier in their careers start to get footing in government, you know, start to work on, get hired and work on, you know, get really tangible product projects or projects within government space. And I think the most of us, the three of us probably have, you know, we've gone, we're on like our 12th round here of the boxing match, you know, so we're along that way to continue on there. So it takes a while, you know, to tackle the meteor kind of bigger problem spaces I guess but there's there's so much work to be done in government, all across all levels that, hey, come join us really. I encourage folks to. We need people. Oh, go ahead, Mari. No, I mean definitely plus one thing like really seeing a mission driven, you know, resume, you know, or interview, really hearing folks involved in civic participation is super key, like I look at that. I think we also really looked at, you know, our team focuses a lot on working with low income New Yorkers, who have been had pretty traumatic experiences with government and so, you know, looking at lived experience folks who have been born and raised, you know, in New York City, and interacted with government. We also really take a lot. We take that really seriously, you know, so sharing that kind of story has has been really important, you know, and I know it can be sensitive sometimes to share one story but being able to being able to feel confident that that's that's another part of you that is, you know, something of worth and value is is definitely something we look at too. So another question that this is a great question, how do you evaluate measure the progress made by your teams with regard to establishing design ops measure measures of success. Who's nailed this one. I mean, I haven't, you know, I haven't nailed it but we have a few proxies right like how many people are interested in user research. How quickly can somebody put together draft web page. You know if we've got our basic design components going and people can work on content. It's a little further forward. And, you know, ultimately, we assume that that influences the matter, the metrics for how many people say a given page or sequence was helpful. But those are kind of the big ones is is, you know, are people interested in users. Can they actually use the tools that we have or have made to build something. And how's it working for the public. The problem is huge. Are we improving services for people. I mean that's the ultimate measure right as we are we improving our services. But I think there are, I think one of the challenges internally is that we are, you know, design for design sake is not going to work right I mean just because we say it's important. So you have to tie it to key measures that are important for all the business basically and you know whether it's mission, you know, meeting mission goals or its cost savings or you know that's some of the things we were working on, you know, we've been working on within DHS is, you know, trying to make that that that tie much clearer so that folks that are internal who really pay attention to that stuff. They say okay, you can help me, you know, deliver product quicker with less support, you know, unless, and greater satisfaction among the public that are using whatever form or service or whatever it is. Yeah, I think one of the things that we need to work on this. With our office hours program for example we we track everybody that comes through so when they sign up we know their position in government you know their title, we know how often they've come in what agencies they've come from. We mark, you know, what was a specific challenge they came into came for. Did we answer the challenge. Did we provide resources what were those resources so like did you know, and like, we also look at things like. I think in our first year, more people were like what is service design, could you sit with us for an hour and like explain that. And now it's like, I need help with a journey map, you know, I need to do a service blueprint like very much more specific I need you to look at like a job description or an RFP to make sure that we are putting in the right requirements for a vendor, you know. And so hearing like that kind of question asking these days has been really great, but luckily we've been able to track a lot of the data to show at the end how many people have we engaged in and what are they doing with all that information. And that helps us at the end of the day like make arguments to like OMB around like how, how, you know, to advocate for more staff lines when they see how many people we're interacting with so that that that one's been really helpful and we need to really think about that as well. I think also within our first, our second, or actually our first year of like official launch we also hired an evaluation firm to look at this team specifically to see if we were being effective, you know, talking to different agency staff members we had interacted with to get the feels on their impressions of us and so we got a pretty good. I think a lot of people like we're really happy with their interactions with us. And so we did that in an early stage we probably need to consider maybe doing that again pretty soon. So one last question from the audience. I think we have time for so based on where your groups live within your strata of government, what structurally helped and hindered your ability to make progress towards your outcomes. So based on where you are in government pros and cons. I would like to take this one. I'll go big judicial independence. The heads of courts are judges. And they have court executive officers who help them run it but the ultimate heads are judges and judges have judicial independence which extends to administrative independence. So, basically, I mean it's a good thing for a lot of reasons but if you think you want to make a coherent design system across a court system. It is a challenge that offers you more social work to do to get more and more people on board and you really have to take the tactic of we're going to make something good enough that these people are going to be impressed by it and choose to use it there's to impose anything. That's a hindrance. I think what helps a little bit structurally in the courts was that there haven't been 10 years of civic technologists telling them what to do. And so we got to be kind of the shiny new toy like happened to civic technologists around 2009 2010. Again, in 2018. And knowing a lot more than certainly I did in 2009 2010. And I think that structurally helped us. I think the hindrance list could be long. But the couple things come to mind are the siloing of organizations, you know within government federal government especially the folks are kind of, you know, lumped into their spot in government and they're kind of hit they're stuck within that realm. And what it what it the challenge there is from a service design perspective is most government services across many boundaries, and many agencies even. And so, and so folks are tend to be focused on their specific area, but they're right now currently across government there's not a lot of There's probably some energy but there's just not a lot of roles there's not a lot of, you know, thought about how we position these design roles at a high enough level that they're able to look across more than one silo, you know, and so you get this, the services that are kind of chopped into pieces someone work well because they just have been designed piece by piece, but we're starting to look now at, you know how we can look across these services and, you know, connect from digital physical human all the way across. And so this is a piece where, you know, the siloing really kind of has an impact that's challenging on the positive side, I think, from a us to us perspective on this probably resonates with you and that is that we're not in an or chart, you know, really we're kind of independent at work chart. So we can go in and, and work at all levels, you know, meet with secretary levels, all the way down to practitioner level and so, you know, not having, you know, your place and you know, it's stuck here you know is super helpful. I think that's one of our superpowers, you know, some of us DS is being able to kind of work in that fashion, and then also getting, you know, occasionally when you need help from above you can get it, you know, kind of that leverage piece. Don't use it much you really need to know is largely, you know I think operating without it is the best way to go, not having to pull that lever. This team sits at a mayoral office or we're in a mayor's office so the pluses around that is like we're pretty high up. And so we can put, we can be involved in multiple agency projects which is really nice and that helps with the deciloing because we're looking systemically in that way. And I would say also just having that executive level leadership support from our office to just supporting service design we're at large has been like super super helpful to the point now we're like the racial justice commission for example has brought up our team as a best place and has been inspired to even think like, should we create a more centralized office of like equity and accessibility essentially and can service design team be a part of that somehow. You know so that that that conversation has been happening into this new administration which is really exciting and terrifying all at once. I think also but the, the sort of downsides in that is that we might we get mandated though so we might be on, we might be doing work and then suddenly get pulled off and say, sorry, everything, you know, put the pens down, we need you to go work on this project for the next six months. So that's very, very jarring for designers for the whole team. And so that's the part that like can be quite a struggle. But we're trying to figure out how to balance all of that out so. I have so many more questions that I want to ask but we're, we're at time so I. Marie I wrote down exciting and terrifying because I feel like that summarizes a lot of what this work is like at all level no matter what level you are in government. And I also just noted that I think Mike you had said there's a lot of social work and use the word diplomacy and how often that has come up in this conversation and how critical that is to establishing sort of whatever that may look like at your government entity. So, thank you so much to our panelists for joining us today.