 Thank you so much for taking time and joining us today for this session on creating a strong public interest technology project, applying an equity lens to pit work. There have been a number of conversations across all sectors of society about diversity, equity and inclusion, especially when it comes to tech, both the tech field and how tech is deployed. The conversations about diversity, equity and inclusion are even more important and are happening in many more spaces. So today we'll be having this conversation really examining how can we bring an equity lens to the public interest technology work that we're trying to do. How do you, as someone who is interested in the field of public interest technology, who is actively doing work in the field of public interest technology or who is putting together an application for the public interest technology university network challenge grant process. Think about applying an equity lens to this work. Unfortunately, I'm going to start off by telling you that there's no magic checklist for how to have an equity lens in your public interest technology work. This will not be a just here are 10 easy steps to being equitable in the work that you do. If it was that easy, we wouldn't need this session. Instead, we'll be spending today's session and really examining what does DEI mean. How do we actually think about the right questions to ask how to build in equitable activities and questions into the design of our projects into the deployment and maintenance of our projects as well. And so with that, let's dive in. As we go to the next slide here, I will spend some time reviewing the agenda for today. It's a fairly straightforward agenda will start off by just going over what is DEI what does diversity equity and inclusion mean, and why does DEI matter in public interest technology. And then we'll examine how do we apply an equity lens to pet. What does that mean, what are the things that we should be thinking about from there will go into how can DEI be done by design. How do you build it in to the design of the very fabric of the work that you are doing. And then finally we'll review our key concepts and we'll end with quite a bit of time probably about 20 minutes or so for question for questions. All of a sudden, if you have questions please put them into the Slido channel at any point and we will have again plenty of time to go over questions at the end. With that start with this next section of just getting started with the introduction. So first of all, as we go to the next slide. I'll start by telling you a little bit about myself. My name is Afua Bruce and I am a public interest technologist. I am a former director of the public interest technology program at New America as the director of engineering and was part of the team that co founded the PIT UN. I'm currently and have been for the past couple years at the faculty at Carnegie Mellon University, and I also do a fair amount of strategy consulting in the field of public interest technology. I've also had prior positions in government agencies and working with nonprofits all around the world on how to intentionally design and use technology to advance mission and to care for individuals. And most recently I co authored a book with Amy sample world called the tech that comes next which examines. How do we send our communities and how do we do this public interest technology work. As we move to the next slide I just want to recap for everyone here to make sure we're on the same page about what is it. What do we mean when we say public interest technology. So the definition that we use especially in the academic framework as part of the PIT UN is that public interest technology refers to the study and application of technology expertise to advance the public interest generate public benefits and promote the public good. You'll notice that this field of public interest technology this definition creates a space for both technologist and non technologists to play that interdisciplinary approach is incredibly important in order to solve the most challenging problems that we have and problems that are truly in the public interest. PIT also encompasses both tech informed policy and policy informed tech development. So looking at both how as people are designing policies as government agencies as communities as corporations even are designing policies. So people can tech thinking tech structures and the technology design and form that policy making. And similarly as people are designing different technology solutions whether it's helping to figure out how people can register to vote or how to protect one's identity and different places and protect against cybersecurity threats, what policies are informing that technology development. And also within the field of public interest technology. It examines both harm reduction and intentional tech development. So looking at how do we reduce the harms that technology can have, whether it's discrimination and technology or different emotional or sociological harms that technology can have, but tech PIT also creates space for intentionally developing technology in different ways, more inclusive ways and more empowering ways. Public interest technology this work can be done all across sectors it can be done in the government sector and nonprofit sector and the private sector, and of course in the academic sector as well. And finally an overarching thread of public interest technology work and in this public interest technology definition is that PIT work should address equity it should address challenges around justice and challenges around diversity. And as we put that together, that is the definition that we're working from today, and as part of PIT you and broadly but as part of the pit field, what we mean when we say public interest technology. As we then go to the next slide. So here's an overview here of just what it means to apply an equity lens to the pit work that we're doing. Now, over the past several years, back to when we were getting ready to launch the public interest technology university network and we can be a number of academic leaders at the time and thought leaders around the field of PIT, those initial discussions that forms the definition that I just went over and that have guided a lot of the work that's being done today under the umbrella of PIT did reference di they reference the importance of thinking about how could PIT be used to help diversify the tech field, how could PIT be used to ensure that technology works for everyone and works for a large groups of people and gets out of the hands of just a few people who understand specific nuances and specific terms and definitions within technology. So with that framework it's apparent and we'll dive into this more as we go throughout the next hour that DEI cannot be an add on it must occur from design through deployment and even into maintenance. If you are working on a public interest technology project if you designed your whole pit project, you started on the deployment. And then you're actually going to be testing the work. And then at that point you say oh well what do I do now for DEI. At that point it's too late, you've likely already found ways to make your technology, your policy inaccessible to certain groups of people or it won't work for certain people or it simply will not see or recognize people with different colors skin or different genders and things like that. And so DEI cannot be an add on it must occur from the design plays and must be intentionally managed throughout the development and deployment and maintenance of PIT work. And just to recap again here what I what I said in my opening remarks is that today's discussion really is a framework for how to think about doing this work as you do your own work to think about how you will incorporate DEI. There's not a magic checklist there are specific strategies that you should be thinking about there are specific questions that you should be addressing and answering and the design of your work. But there's not a specific just add these three steps and you magically created a diverse equity equitable and inclusive project. And so as we now go into the next section. Which is why does what does DEI mean why does DEI matter in PIT. Well, I'm going to start on this next slide actually by giving you a little bit of insight into some of the work on the next slide there. I'm going to give you some insight into what some of the work that New America has done when it comes to racial equity, specifically. And so, New America has spent has spent a lot of time over the past year and a half, really looking at what does it mean to really examine what it, what it is to do the work that New America does with a good solid racial equity lens. One of the frameworks that they're using to do that is this cultural competency framework that really gets to learning and really speaking to what is the learning that we're doing recognizing that different people are different points in the journey. And so the framework that New America has been using this cultural competency framework here starts with cultural deconstructiveness and really looking at how do you break down some of the terms that people are using some of the activities that people are using today. And then going along there to then move to cultural incapacity and then to cultural blindness acknowledging blind spots and the organization acknowledging blind spots and specific projects from there moving along this model and along this framework to cultural pre competence, and then cultural competence when people are familiar with the terms can point them out can point out the activities, and ultimately to cultural proficiency when people not just people don't just recognize the terms don't just recognize the activities, but actively do the work and actively hold others accountable to doing the work and create a culture where engaging in conversation about the work is welcome and getting in questioning about the work is welcome and that culture proficiency is then really embedded in the culture. And I think that is a strong framework in general for the work that New America has been doing but also as we think about what it means to collectively as we're looking at applying an equity lens for pit projects to make sure that you're moving yourself but also the teams of people you're moving along a learning framework and along a competency model for the work that you're doing. Now I'm going to dive in on this next slide into specific definitions of what diversity, equity and inclusion mean. You probably hear di frequently di a other terms frequently, but I want to just take a moment to just go through some solid definitions of what we mean by diversity, equity and inclusion. So you can see here, the diversity is the presence of differences that may include race, gender, religion, sexual orientation, ethnicity, nationality, socio economic status, language, disability, age, religious commitment or type of populations that have been and remain underrepresented or under overlooked among practitioners in the field and marginalized in the broader society. So here when we're saying diversity, we are looking for a diverse range of perspectives we're looking for diversity on a number of different dimensions it's not just one of these here, but really taking into account a lot of different perspectives and really the question here is actively seeking out who has been underrepresented historically or who has been overlooked historically, and making sure that that is designed for and accounted for and what you're doing. Now, equity is promoting justice and impartiality and fairness within the procedures processes and distinction, distribution by of resources by institutions or systems tackling equity issues requires an understanding of the root causes of outcome disparities within equity. And so when we look at equity, this is making things equitable, it doesn't necessarily mean making things equal in that if someone has 10 of something and two of something, it's not necessarily equal would be giving everyone the same number and additional but equity is making everyone sort of on par so giving five to the person who had 10 and 13 to the person who had to. Yes, that adds up. Thank you. And so that that is the equity that we're looking at really looking at promoting fairness and justice and impartiality into the work that we're doing. And finally, inclusion is an outcome to ensure those that are diverse actually feel and or are welcomed inclusion outcomes are met when you or your institution your project in your program are truly inviting to all to the degree to which diverse individuals are able to participate fully and the decision making processes and development opportunities within an organization or group. So here it's we are moving from acknowledging that we need to have different perspectives that those diverse perspectives, moving past, making sure that things are equitable. So everyone is sort of on an even playing field to making sure that if we've added seats to the table if we made that in table, more inclusive so to speak, that it really is inclusive which means that everyone's who's at the point where their voice is heard, their voice is respected, their voice is given the same weight, and they feel like they can use their voice and really advocate for themselves. That's the inclusion piece and so here is just the breakdown and sort of recap of what we mean by diversity, equity and inclusion. And so now as we go to the next slide just want to acknowledge here that again, the diversity equity inclusion and really also justice concepts have been embedded into discussions about public interest technology from the beginning. And I think back to that definition that we started with the phrases there about generating public benefits and promoting the public good really do tie in to these ideas of diversity, equity and inclusion. And just to recap, some of the conversations that have been said about other people who have been in the public interest technology field for quite some time. So Mark, who's a former US chief technology officer discussed how government needs to design for all and in that case, really meaning that it needs to work for everyone in the private sector and some academic sectors or even some nonprofit sectors, people can pick sort of their own target market but if you're designing a benefit system, it needs to be accessible by everyone. And so just really making sure that you're creating that so that aspect of equitable design and equity has been really important there. And you know, who is the former director of the domestic policy council under the Obama administration as a current new America senior advisor has spoken quite a bit about how, excuse me. It is really important because it increases access to services. And here again we're getting to who both the diversity and equity pieces of thinking through how pit is and what can be done through pit in order to increase access and appropriateness and usefulness of policies. In order to do that, taking into account diversity and equity, excuse me. I want to spend some time discussing some of the work of Kathy Pham, who's at Harvard Kennedy School Adjunct Lecture and has done quite a bit of leading work on product and society and really looking at public interest technology there. And Kathy Pham there discusses how equity and justice translates to code and the ability for people to be seen by technology. And then interrogating from a technology development process, what it means to actually give the right weightings to different variables in your programs to intentionally figure out what different values are between behind different fields and in different forms that you have. And what that design review process looks like what the code review process looks like, and how that ties into equity and justice and really doing intentional checks there around inclusion. Let's go to the next section now let's talk a little bit about how we actually apply these concepts we've now talked about the definition of public interest technology we've talked about the definitions of diversity equity and inclusion so let's talk about how we apply an equity lens, especially to pit. So to do that on this next slide, I want to step back actually. And when you are designing your projects, I've actually found it to be most helpful when you start by designing around values and just finding what values you have. Now there are a lot of values to choose from should be, you know, obviously around things about equity and inclusion, especially. But these six values here are actually taken from the book the tech that comes next how change makers plan purpose and technologists can build an equitable world, and these values I think really create a strong foundation and a strong framework for what can then can dictate a lot of design decisions and development and deployment decisions for pit work. And so first is just acknowledging the knowledge and wisdom of lived experience so really making sure that you value that you seek out the lived experience of the community of people in the communities who you are working with and who you're designing alongside as we were doing And secondly, I'm seeking out the participation of a diversity of people in decision making planning and building technology. And so this is taking a look at who do you have at the table is it only technologist. Is it only social impact leaders is it only policy makers really looking at who you have at that table in making those decisions. And then also another dimension of diversity as we had from a couple of slides ago, really looking for who are those traditionally underrepresented or overlooked populations that you want to make sure to have as well. And then finally on this stands as well, it's important also here to really think about different levels of the organization and making sure that you're hearing from maybe both executives and your ground level employees there. And also if you're at an organization that is serving the public to have it not just be staff but also to take members of the community there and making decisions alongside them. The accessibility here as a priority for a start is so important. And this is accessibility in the ways that you might have. Heard it before, in making sure that the website for example is accessible that it can be read by people with low vision, or things can be turned and can be read to people and be made audible. But it also means just the practicality and accessibility of doing the work itself and so if you are saying you're having a community meeting. Is it in a place and at a time where people from the community can actually make it to the meeting. This next value that we have here is valuing the multiple ways that change is made and balancing the need to address real challenges today with making systemic challenges in the future. Recognizing that there may be big change initiatives that you're pushing for, you may have big visions for your pit project, but also going to have to take some baby steps today. The last two values here are really around collectiveness and recognizing and balancing individual communities, individuals rather and individual expertise in skills within the community aspect and collectively creating a better world. Okay, excuse me. With that, we can move to the next slide and you'll notice that I didn't say in the past slide that we're just designing for all. And that's because we've used the phrase designing for all it's gotten into the vocabulary things, especially in the civic tech space. And it's really important, but it's necessary but it's insufficient as we really think about bringing a strong equity lens to pit work. So specificity matters it is difficult to design for specific communities. If you're not clear about who you're designing for. If you think of it from a technical perspective you're not going to start writing a program to. You're not going to start writing any specific tech program without an ended mind and sort of what you're going to do you're not going to figure out how to add to plus two to get to four without knowing that you want to get to four that specificity matters, and it matters in the design of your role. So as you think about bringing an equity lens into the work that you're doing for pit, make sure to be specific in the groups that you're targeting and the groups you're wanting to serve. The lack of intentional design around this inclusivity and this lack of being clear about what you're doing is really what leads to the unintended consequences that you may hear of frequently when it comes to technology. The unintended consequences in reality are often translated to brown and black people or women or different continuously underrepresented and overlooked groups are once again left out of the different technology solutions. So this is why it's important to be specific and if you are working in a community of black women to name that if you're working in a community of people who are hard of hearing to name that but to really be specific and not shy away from being specific in your design process. Now, we've talked a little bit about reducing harm and staying away from unintended consequences, but I want to emphasize that isn't just about correcting harm. It's also about proactively creating different stronger technology and policy. Fortunately, we have a lot of positive examples of how this is being done today and examples that we can really build on and take some solid lessons from. So as we go into the next slide I'll just recap briefly three different many case studies for you if you will. The first is the organization code the dream talked about code the dream in the book the tech that comes next and code the dream is a nonprofit organization that offers free intensive training and software development to people from diverse low income backgrounds in code the dream labs coders work with experienced mentors to hone those skills by building apps and technology platforms for a range of startups nonprofits in government clients. The ultimate aim of code the dream is to create a unique win-win where coders gain real experience building apps that make the world a little better and then use that experience to launch new careers with enormous opportunity from themselves, their families and their communities. And so one particular example of some of a code the dream project is an platform called so much potential so much potential is one of the many code the dream developed platforms built for and by undocumented or DACA mentored young people so much potential specifically focuses on resources for students who live in states where status presents a significant barrier to further education and so what they've done through this platform is to provide resources to provide a way to easily access those resources and then to leverage those resources resources. The key here is that they're starting by having members of this community, actually be the ones who designed this platform, and then also that helps with some of the outreach and the testing and the deployment and making sure that the technology that was designed was appropriate, and will actually be used in ways that further the aims in this case of accessing education and these educational resources to members of this community. So the first example I wanted to give is the one of the rural community assistance program. So Nathan old was the CEO of the rural community assistance partnership, or our cap, which is a national nonprofit focused on securing and maintaining water and economic development for rural communities across the country. It's the nation. It's a national network of nonprofit organizations working to provide technical assistance training resources and support to rural communities across the US tribal lands in US and although Nathan started by looking internally for how tech could have a positive drive and a positive change within the organization he began to wonder about ways to impact the farm bill actually at the time. And so Nathan and our cap partnered with New America actually and a number of other organizations to convene a day long innovation summit with a coalition of rural community advocates, nonprofit leaders and technologists to try out new methods of policy creation. So again here really setting the community and really being inclusive, really being equitable and really having diverse perspectives there at this innovation summit. So a number of technologists were identified out structure and facilitate the day and over the course of the innovation summit, this large group with representatives from multiple communities. So it came up with ideas for a number of programs, and then following this, the summit leaders of the group advanced the conversations ultimately conversing with congressional staff members about what would become the rule innovation stronger economy, arise grant program, which was signed into And as Nathan put it, taking part in the policy making process is made more effective when social impact organizations can build a coalition and advocate to policymakers with one voice. So we see here in this example with our cap again that the importance and the usefulness and real utility of having diverse perspectives and really doing that at the start in this innovation summit, then driving that through the policy making process really works well. And then finally I wanted to highlight some of the work of the New Americas new practice lab, the new practice lab works with government and non government organizations to help them understand how policies truly impact families in order to improve the delivery future design of family economic security policies. Last year, the new practice lab focused on select policies that disproportionately impact millions of families, their access to cash through instruments, such as the child tax credit access to early education and long term care, and to unemployment benefits. The NPL partnered with government agencies and support of their efforts to engage communities and solicit direct input from families and identifying meaningful changes to change program delivery from a partnership with the American Association of people with disabilities to understand and tell the stories of what the disabled community experiences at home at work and economically in the face of coven 19 to the same thing before the bipartisan select committee on the modernization of Congress on strengthening the law making process and how data can inform and improve policy to partnering with the New York department of tax and finance to better understand the challenges that present challenges to the administration of the earned income tax credit. The NPL demonstrated and shared lessons learned on how when you center and engage communities, pit in practice can really move the needle on social issues. Another thing are three solid examples of how really taking this diversity and equity inclusion into the design and building it throughout the work this pit work really paid off and really made a difference in the efficacy of these three different programs. So now let's excuse me spend some time just going through what are the steps we can take to do this work what are the steps you can take to implement this in your own work so we identified here just a simple four step process I know I said, there wasn't a checklist but there is this process here that you can go through. And so step one really defining who the community is that will use and will be affected by what you're designing. So here it's really important that you think about just who all those stakeholders are. If you are designing a type of tool for law enforcement, for example, law enforcement, certainly is who is going to use that tool, but the communities that in which law enforcement are communities that will be affected by it so as you think about defining who that community is and therefore thinking about who you'll be designing with and who you need to be whose opinions need to be incorporating into your design process, really think about the communities that will use and will be affected by what you're designing. Step two is go talk to the community. See what they want to see what they actually need. They may not speak the technical lingo or the policy lingo that you're used to, but living with the issues that they live with, they will be able to articulate their needs. And then you can work on what that translation process looks like as you move to step three and modifying any plans that you have already created based on that community feedback and that community input. Repeat steps one through three over and over until you really have a solid plan for your design, your development, your deployment and the maintenance of the work that you do. As we end on this next slide, I just want to also give you some questions that you can be asking yourself as you're putting together overall project plans for yourself and as you're putting together overall descriptions as to what the project is that you're planning if you're writing a project plan for yourself or if you're getting ready, for example, to write and submit an application to the pit you in challenge grant. So some questions that you might want to ask yourself in really then address in the design of your program are who would be harmed if the technology we build fell if the technology you build fails, who will profit and benefit the most from the technology and what technologies or services do you not currently have access to either because of availability or affordability, and what experiences or knowledge do you have that you could share for better policy making that last question. Those last two questions actually are questions that you can ask of community members and once you define that community, you're looking for stimulating that conversation. Those questions you can use for tonight took these questions also from a book that I mentioned earlier. The book actually has about 125 questions total. So if you're looking for some resources on how to have these conversations dive into the book but the these four here will will go a long way in putting together a design for a strong pit project. So let's go to our last section, which is really how can be done by design so what does it mean to really do that diversity, equity and inclusion by design. And so I'm going to give you some ideas of some specific activities that you can incorporate in order to really address the I so as you can see here on the next slide, I have a handful of specific ideas for you. So the first is listening sessions you saw in one of the steps on a previous slide that talking to the community is important so listening to the community is important, and really structuring some strong listening sessions, innovation summits, such as the, the RCAP example as well can be really useful. Second is inclusion in product design meetings, and that can be tied with open prioritizing and review meetings. One of the cases that we covered in the, in the book, the tech that comes next actually talked to the organization that was a community based organization, rescuing leftover cuisine. In the community, they started out by having people go to a restaurant, for example and pick up leftover cuisine and then take it to people who are facing food insecurity. And as they started to expand and started to use their to develop their own technology project. They put the different feature request and bug fixes on one trailer board that everyone can see. Not just staff, but also members of their volunteer community and members of the community who were donating food, for example, and then they use that and they would show their openly and transparently what was going to be worked on and wasn't going to be worked on and they were going, who were able to communicate that broadly, but also allowed for people to understand trade offs that were being made, and for the community at that point to be able to weigh into what features would actually be more useful for them or less useful for them. Also important here, as you think about specific design and development activities, testing and in the real environment so testing and test environments is great but if you've built a great solution or what you believe to be a great solution, and you test it in the comfort of your own home that may look differently than what the people who will actually be using the technology with actually what it'll look like for them on a day to day basis. So for that I think of a recent client that I worked with, and they had developed this beautiful interface style people find and access different government benefits that worked on a computer. It was designed for people in different government offices or nonprofit agencies to access on the computer so the thought was that they would go into the office beyond the computers there, and they would access it there. However, when one of the people from this organization went to the office to do a training for 20 people, they discovered that in this particular office of this partner nonprofit organization that they were working with. There were only two working computers. And so this great interface that I've been designed for the computer was completely lost or mostly lost, because instead people were trying to do what was designed for for a computer screen on different phones and phones of various capabilities instead. And so that's why it's important to really have that feedback there and do the testing in the real environment. It's also important here to incorporate some of this oversight and these guidelines and oversight boards and so as you're putting together boards of trustees or different oversight boards, making sure you have members of the community on those boards, making sure that you've got the different diversity, equity and inclusion aspects and specific people back to that specificity point represented oversight boards, and that these oversight boards you're on one or you think about who you want on your boards to engage with the work that you do, making sure that they will proactively ask these questions around diversity, equity, inclusion and really centering the work. That's really important. And then final, finally, checks for acceptance and approval. So I am a big fan of process because process instructor can actually allow people the freedom then to think about other things when they know what needs to get done. And so here it's also important to just build into your process checkpoints for acceptance and approval by community members by the oversight board, by looking at and reviewing who's been asked these questions, doing an equity check and using different equity frameworks that are out there on your pit project as well, building that in to your design process and into your deployment process is really important. And then once you've designed and developed the activities. The next step is sometimes the most fun, which is deploying and maintain and maintaining the work that you do. And so on this next line, go into just the importance about ownership and maintenance and when you've deployed the work whether again it's a policy whether it's a specific tool whether it's a curriculum, or whether it's a course depending on what you're doing, really thinking about how the community that's engaging with this is going to continue to be involved and supported, where will ownership lie, who will ask for and be responsible for maintenance or updates in the future. How will you continue to assess fit and acceptance. What does that process look like will there need to be any tweaks further down the line. What does that overall ownership and maintenance structure look like is really important. And so with that, we'll go to the next slide here in one more slide. And just recap some of the things that we've talked about today so to recap we talked about the definition of public interest technology again as the study and application of technology expertise to advance the public interest generate public benefits and promote the public good. And specifically when it comes to building a strong pit project it's incredibly important, even require that you have a strong equity lens, built into from the start into the fabric and framework for your pit project. DEI it's not just one section of the project right it's embedded throughout specificity matters, designing for DEI matters and being explicit in that and the project plans objectives and descriptions you have. And then finally also thinking about DEI through also the ownership and maintenance of projects. Thank you for that. We have one question from the attendee Debbie who's asking, can you discuss more about the interface between public interest technology and public policy. Yeah, absolutely. So public interest technology and public policy they can really fall under the same umbrella because again, public interest technology can be about specific technology development but it's also about the process in which we make policy and the process in which we roll out policy. And so as we think about public interest technology for policy making this gets into some of the examples really specifically from the RCAP example right that I went over and so that's the community assistance partnership example and how they ran a process in order to identify specific what became recommendations for specific programs in that case the rise program, the rise program for into the farm bill and so that public interest technology aspect of that was really the way in which they brought together different perspectives, the way they would in which they did some of the brainstorming and use some techniques usually used in reserve for the tech industry, in order to spur some conversations, spur some different thinking, spur some conversations and spur the development of specific ideas and then working with policymakers to then refine some of the policies and to create this program, and then handing it over to policymakers to go through that final process of actually passing the legislation that's there. The other aspect there just to recap again is some of the work that the new practice lab has been doing. The new America's new practice lab has been doing and that is excuse me. And that there is just really the importance of really centering communities really centering and the people who, again use and are affected by those policies in the work that is being done, and then tying that to and really working hand in hand with policymakers to make sure these policies are created in ways that recognize people and the policies them see people and that's how we can get to systemic change. Thank you for that we have a question. This sounds like a combination of agile user centered in contextual design principles. How much of the proposal should focus on the design process. So certainly there are aspects of agile user centered and contextual design in public interest technology so say you know public interest technology. The name itself and the field itself is fairly new but it does build on existing other fields, including the school of the discipline of STS for example, I'm in a variety of other disciplines and so I want to acknowledge that and then to this second question of how much of the proposal should focus on the design process and here I'm assuming the question is in relation to the proposal for the, for any application you may be putting together for the challenge grant process. So in the, in the, in the RFP, it does lay out different sections and lays out also I think the weights for different sections and so in the project plan conversation for example you will, or response you will want to lay out the response that you do. You will want to make sure that in the project plan for examples it is clear what activities you will be undertaking in order to engage the community for example with activities you'll be undertaking and what the process will look like in your project plan to react to some of the things that the community has said or how you will be accounting for in your project plan the activities around seeking out diverse perspectives and doing some work around in equity, building in equity and addressing inclusion into your work so we'll be looking for in that project plan description some of those activities. Jerry is asking what failures have you learned from with any. And there are a number of failures that that I've learned from, let me quickly think how to anonymize them as to protect the, the innocent or maybe the not so innocent. So I, I can talk actually a little bit about some work that that I did during some of my time at the FBI and working on on some different technology development processes and so at the FBI working on a doing a review of some different technology strategy for example, worked with a team that had been working for a long time on developing a very specific technology and putting together very detailed plans to then share it out to to the law enforcement community and to community advocates which was part of the process. The challenge there is that they had gone through nearly all of the design process and nearly all of the development process. And then we're waiting until the very end to present this to the community into the community advocates and other members of the law enforcement community. The meeting did not go well and frankly it's because all of the design and the development had been done without that community involvement upfront and so it was a fairly tense meeting, shall we say in that case, where there are some strong opinions shared by members of both sides, but ultimately it caused a much longer design and development process because that team had to go back and had to address a lot of the concerns. And, you know, as I mentioned earlier in the talk you can't just add in that at the end. And so a lot of the work had to be redone in order to accommodate and to really work with the critical feedback of the community members in that case. We have a few more so someone is asking, does a specific field of technology need to be identified in the proposal, or could the text slash policy policy to work on come from the community and change over time. I would say that the, you should provide as much specificity as possible in the application. I think that if you were going to be building in the ability for the community to, to weigh in and evolve what solution that looks like I think that's great and then articulate what this is into the proposal. If the technology is already defined and that's where you are then define and be specific about the decision that's been made and why it's been made in the proposal as well so as much specificity whether the technology decision as a question asked has already been made, or if the plan is to have a process by which the community gets involved and pixel solution and be really clear and specific about what that process looks like. I'll also just note that if you have specific questions about your specific proposal that you're putting together. We do have a handful of resources there is an email address that you can send questions to. There's also an FAQ paid, and they're also office hours that you can sign up for to get specific questions asked. I'll go ahead and chat shortly. We have another question here. This presentation focused on projects that design public interest technology as opposed to studying or fostering public interest technology indirectly. Would you discuss the role of diversity equity and inclusion for the latter. Absolutely. So, although this presentation may have seemed to have focused mostly on technology development specifically. It's certainly applicable to studying the technology or studying the field of public interest technology. And so when you think about what it means to apply this diversity equity and inclusion framework to that work as well. I'm asking a similar question so in your research design and in the research questions. I'm asking the questions and including questions about sort of who is being harmed and hurt by even the research or who would be harmed in, or helped by the findings of the research what that process looks like afterwards. Who is the community that will be receiving that research and making a decision based off of it so if it's the community, if it's you yourself. And that's going to then inform other work that your university can do in subsequent years, just being clear about that and being clear about what that process looks like. I'll be asking about the diversity of your team dynamics of who you were talking to and who is being studied or researched in that process as well. Being really intentional to ask those questions as well so that diversity equity inclusion framework does apply to some of the research design itself, as well as to who is involved and who to who is involved on many aspects of the research project that you may be putting together. I have a few more here so someone is asking. I'm also interested in the intersection between this and other community methodologies from the press or a action research and human design human centered design, sorry. Yes. So, I think, if I understand the questions. General curiosity about how the public interest technology process or sort of framework emphasizing di interplays with human human centered research and participatory research. Yes, the intersection. Absolutely. So, I think there are a lot of overlap. There's a lot of overlap here. And again, so I would think of it more as sort of public interest technology or pitch is being an overarching framework. And then as you have in pity and having an interdisciplinary nature. And as you bring in people from different disciplines, they can bring in their own tools and perspectives to solving that project to solving that program. So if you're bringing in someone who is really strong at that participatory research side, then using those tools and frameworks to really advance the pit project again the technology and non technology aspects and really seeking to generate the opportunity to provide public benefit, then that is great and you should do that as well. So, I would say their pit and the participatory research and human centered design certainly aren't in conflict and you often see them working together to execute projects. All right, and we have a request from Debbie. I want to provide some examples I think she was referring to examples of the combination of agile, the question around users centered in contextual design principles so if you have some examples there to share. So I would bring it as. So if you were doing the question, just to recap is what are some examples maybe how agile is agile methodology is used within pit. And so I would say agile is often seen as a development methodology in the tech space and for writing code delivering and more and so how it might look in public interest technology is if you are developing a specific tech product, you might choose agile as opposed to waterfall or any of the other sort of design methodologies to manage the work that you're doing. You could also they use some of the agile concepts in some of the policy design. And so, just the process of sort of iterating on different policy development I think this is part of with some of the work of at least one of the new practice lab projects I think they did some work with family and pay leave last year in the year before as well and starting with one small research sprint and going really deep in that research front and then following it on with the second research front and so using that sort of agile methodology there to really get to what are the ultimate recommendations, what are the ultimate policy recommendations and policy lovers that are to play and so I think that's just recognizing the different ways and sort of ways you can use that agile methodology to both develop technology but also to develop policy. And finally to add on to that, maybe we. Someone is asking about a artificial intelligence example. Sure. So artificial intelligence is definitely there are many, many positive uses and many aspirational uses, I might say of artificial intelligence when it comes to public interest technology but I would I would say here again, what is most some of the questions that are most important to ask as you are doing any artificial intelligence work is really being clear on what is the problem that is being solved. Is it a problem that actually needs to be solved in general, and is it a problem that needs to be solved by, by a computer as opposed to humans or, you know, used to work at data kind used to be the chief program that serves data kind which is an organization that does data science and AI and service of humanity so working with nonprofit agencies and government agencies all around the world. And, you know, some of the other questions that we would really ask at the start of the process to decide if we would even take on this AI project for those questions but again, really, who will be making decisions based on this AI is it a standalone or is it falling in to some process that a human ultimately not human ultimately manages and that's a really key distinction and making sure that it's clear there. Also really understanding who has the power to turn on system and who has the power to turn off systems is really important and so if you're doing an AI solution around. Excuse me, let's say, analyzing geospatial data to identify minds in some African countries which is a project that we worked on a data kind, and being really clear as to who has the ability to turn that on and turn it off, and are you actually going to make decisions based on what is being done, or is it just going to sort of be left to is the algorithm going to be left to its own devices and not have a lot of oversight so those are some very key questions that need to be answered, before you decide to actually engage in an artificial intelligence project especially at when it comes to an application of an AI solution. And it's time for one more last question this is a really good one related to what you just said so would you propose public interest technology projects with international slash trains national aspects to be within the scope, would you consider. So I would actually suggest you send that to to the network challenge questions email which is pit you in challenge at new venture fund.org. And maybe check the FAQ page so yes it can be used internationally there are many examples of pit being used outside of the US, I think. You can find much more advanced examples of public interest technology outside the US that inside the US. So yes, that is a definite possibility, but as far as how that will work for a specific proposal for the specific 2020 to pit you in challenge grant process. I'd say submitted questions so that we can talk a little bit more detail about your project. And everyone for joining our presentation today. If you have any more questions we will follow up with you via the resources provided in the chat today. Thank you.