 Hello and welcome to our panel on creating a responsive government. My name is Hannah shank. I am the director of strategy for public interest technology at New America. Public interest technology is poised to make big changes at all levels of government. And today we have a great panel to talk about what it looks like to create a responsive government. We'll discuss how grantees are helping government entities solve some of the most pressing human and societal problems, and how other PIC practitioners can duplicate and expand their work locally. So, on our panel today. We have Ashley, who's named my butchered, who is the executive director of the Davenport Institute for public engagement and civic leadership. She co designed and serves as an instructor for the professional certificate in advanced public engagement for local government and teaches a graduate level class on public engagement at the School of Public Policy. Dr. Little is joining us from as the director of ethics of the ethics lab at Georgetown University. Dr. Little is a senior research scholar at the Kennedy Institute of ethics and professor of philosophy at Georgetown. Her research interests include issues in reproduction clinical research ethics, data ethics, which we'll touch on today and the structure of moral theory. Dr. Sweeney is a professor of government and technology in residence joining us from Harvard. Professor Sweeney creates and uses technology to assess and solve societal, political and governance problems, and teaches others how to do the same. One area one focus area is data data privacy, and she is the director of the data privacy lab in IQ SS at Harvard. Professor Sweeney is an elected fellow of the American College of medical informatics and currently holds the privacy and security seat of the federal health information technology policy committee. And finally, Larry Susskind, the Ford professor of urban environmental planning at MIT. Professor Susskind's research interests focus on the theory and practice of negotiation and dispute resolution, the practice of public engagement in local decision making cybersecurity for critical urban infrastructure, entrepreneurial negotiation, global environmental security, and city making the resolution of science intensive policy disputes, renewable energy policy, water equity in older American cities, climate change adaptation socially responsible real estate development, and the land claims of indigenous peoples. So today, I would like to start our conversation by talking about what a response, what a responsive government would look like. In addition, a responsive government requires the ability to quickly understand the ground truth. What is the lived experience that real people are going through. What do they need and what does real help lack some other very basic inputs and most modern decision makers have at their fingertips, like easy access to real people's experiences, meaningful data, and also the ability to test solutions before first rolling them out. So asking government to completely overall the way it works sometimes seems a little bit like asking son to move four inches to the right. Where do we start and what role can pick practitioners play. I would love to start with you thinking about these questions and I will say though but please everybody I really like this to be a conversation. We have so much brainpower on this panel, and I would love to see some some interaction so please feel free to jump in but we'll start with Ashley so thinking about these questions from a pit you and perspective. As we prepare the next generation of policy leaders to think about public interest technology, many of them will be heading into organizations that don't know what pit is. So how do we help them navigate the reality that they're working against or at least parallel to established organizational culture. That's a really important thing I think for us to consider and the whole question of a responsive government I'll certainly I think there are people who are more technologists than I am on this panel. But this piece of how we connect the work of public interest technology with the culture of government, especially local governments which very widely across the country. And sometimes when we're talking about this we're also thinking of New York and LA, which in some ways have a lot more resources to bring to the table and in other ways have much more established kind of bureaucratic approaches to things. So I think that one of the ways we're approaching it at the School of Public Policy at Pepperdine University is to kind of take an inside outside lens to that and to focus both on introducing our MPP students and professors and public policy students to public interest technology, helping them to feel more comfortable working with technologists if not becoming technologists themselves there's kind of both, both people come to us. But then also giving them exposure to people who are actually practitioners in government right now and having some really honest conversations that help them set their expectations to the extent where they're looking at moving the needle and, you know, the optimism and enthusiasm of our students, but also setting them up for a place where the first six months on the job is not terribly discouraging so preparing them to navigate those very real bureaucratic structures is something that I think we forget about some of the time in in our in our work we get focused on the solutions and forget about where people will be navigating. And then we focused outward to really think about how we offer professional development professional education opportunities to current practitioners to leaders in local government so that they have a better understanding now of how, how this technology can help them, and hopefully a better understanding of how our students and others who are learning about public interest technology that like those skills are actually something that they'll be looking for and that we're beginning to make some way there so that's obviously where our professional certificate program is really focused on that mid career local government leader to try to help those who are moving up their career letter. We need to understand this so that as we bring a new generation and there's at least an understanding and appreciation of this, while also preparing the new generation. For the reality that there, that there's going to be a little bit you know it's going to look different to work with a local government than it would to work with a tech farmer, or, or even within the education field but I'd love to hear from the other panelists kind of how they're thinking about this and their own work. Well, I'll jump in. So I'm at Harvard and we've had a fantastic opportunity to really work in the space of public interest technology and sort of evolve a series of classes which students called save the world classes. And there's work that the students have done have had profound impact both on business practice, as well as on government. What are the, the one of the reasons for that is we sort of take the take the notion of government and look at it in three different ways. One is the governance of technology itself. And two is government when government designs technology and three is when government uses technology. And we have found that public interest technology has very strong voices to say in each of those areas. In the case of governance of technology itself. You know, we've had a Ron Kahana was a student who exposed vulnerabilities and Facebook and Facebook responded by changing the way it does things similarly, we've seen changes at Airbnb, and at Google, and in the government itself. So when the government designs technology, we had Max Weiss, who was able to show how our idea of using public comments, because the government had designed it, following the way they had always done it on paper. But this time just making a public available website was vulnerable to robots infiltrating and making very real using AI to make very realistic content, and thereby distorting the public, what the government thought was public opinion. Since then, government has began taking changes. Our students were among the first to point out the problems in the voter registration websites. And in 2016, and going forward. And of course, there's been a lot of work on the re-identification of data in the belief that data that's supposed to be anonymous isn't. And in terms of government uses of technology, this year we also had students who look and assess the accessibility to the American Disabilities Act of government websites and found them really quite inferior, as well as security of government websites. I use that litany because this has been very exciting. It's been exciting for the students to be able to roll up their sleeves and demonstrate a scientific fact that could have this kind of consequence and this kind of change in government. It's also been exciting to see government having to change and getting a scientific factor scientific evidence on which it could take action. This work as a body gives us a new notion of what public interest technology, one vision of what a public interest technologist can offer. That is, when they're on the inside, when these students are hired and they're on the inside, leading these kinds of policies or making these kinds of decisions, they have a much better way of looking for the unforeseen. So we realized that our view is only one view of public interest technology, it's a big umbrella, and that's part of the excitement of it and I'm sure my colleagues will give you others. Anyone else have anything else to add on that or I actually had a, so I think there's a question then or at least there's a difference between having pit people on the outside and pit people on the inside. On the outside, they're calling attention to, hey, this isn't working, hey, this is broken in some ways that's maybe an easier life. When they're on the inside, they are maybe have more power but also, or they have more direct responsibility, but they don't really have the ability to, like, just go out and say hey, these 53 things are all broken. So is the right approach, do we need people on both ends of that? Do we need people from the outside and on the inside? Is there one that feels more promising? I can maybe speak to that. So what we're doing with the Pit UN initiative at Georgetown is trying to get the people on the inside. So absolutely both are needed. But we know that leaders in government, whether it's a state and local or the federal government are being asked to make decisions, whether you get the State Department, DOD, or just working in a Congress person's office or city council, that these folks are being asked to make decisions about whether to deploy certain technologies when there is a lack of ethical norms and consensus about when it's okay to do that. And the technologies move faster than our understanding of its potential social implications. So these technologies that have enormous potential for social good also have enormous potential for peril. So we're sending people out there in the government asking them to make decisions without a lot of training in how to lead on this issue. So what we're doing is training early career folks, technologists and policymakers who in our case are headed out to work in the federal government. Ethics Lab, which is a unique team of philosophers and designers that works on ways of helping people be agile in approaching really challenging ethical issues, is working with the Center for Securities and Emerging Technologies at Georgetown to train a cohort of fellows, CSET analyst fellows, tech congress fellows and AAAS fellows, who are all about to be placed inside government early crew jobs, and doing a series of workshops with them putting them in real world scenarios where we help train them on the how to think about these issues. Not just what guardrails to understand, though that's important, but also again to really think in an agile fashion about how to spot problem areas. How to think about getting community consultation and other people's values into the mix of decision making. One of the issues we talk about in the workshops, Hannah, are these issues you mentioned about when you are in the inside and you're being asked to do something or to make a decision you're not comfortable with. How do you navigate those wars? When do you speak up? What's the difference between calling people out, calling people in, being aspirational? When do you escalate and bring somebody else in? I mean, how do you start building moral conversations around these really hard decisions? So basically our effort is to help train the pipeline. People are going to be asked to make really tough decisions that are suffused with ethical issues and give them practice before they hit the ground. If I could, Hannah, I take a slightly different cut at this. There's something about the question of cybersecurity that blurs inside and outside. We have designed a cybersecurity clinic. We prepare students to work for cities and towns so that those cities and towns can analyze their cybersecurity risks and begin to organize to take action. And the students include some who are going to likely go to work on the tech side. I'm at MIT. I've got a lot of engineering students who are interested in cybersecurity. And what they are surprised about is that the kinds of things that can help reduce the risk to cities and towns which are going to the risk to critical urban infrastructure are not technical solutions. The sources of cybersecurity risk are the people in the agency who don't listen or who don't learn or who don't know that every single person is a source of risk. If you look at how cyber attacks occur, how ransomware attacks occur in particular, it's somebody opened an email from an untrusted source. And now everything's been infected. And maybe like in Philadelphia or Baltimore or Atlanta, the whole systems are shut down so the city can't deliver services to people. And it doesn't even have a way to communicate to the people from the agency that's been attacked that the agency is not able to help them. In the last few weeks, we have stories about hospitals being attacked with ransomware and people's lives being put at stake. So I'm trying to prepare people to understand how to diagnose and manage the risks associated with cyber attack. Some of them are engineers and they need to understand that it's reverse social engineering, not encryption that's going to save that city agency. It has to teach people not to do the wrong things. It has to teach them the basics of cyber hygiene. It has to figure out what money to invest in backup systems, simple things that if you were teaching a public administration or public policy class, you'd say, oh, cybersecurity requires decisions by government decision makers, and they should invest money in it. And they think that means IT. They don't understand that it means management and administration knowledge. And everybody who's going to work in city government needs to know this, whether they're a technologist, or whether they're a policy administrator. The people who got stuck in Baltimore having to manage the attack were the assistant city council members and their staff. They were the ones who told, we're under attack, what do we do? Well, they didn't have any kind of emergency action plan. They didn't have a disaster preparedness effort. And in the middle of all this, certain neighborhoods are disproportionately hurt because in this case all the water was shut down. And water systems are gone. And they have supposed to fix it. It's not that they go to the engineering department. They said to the city council and the staff, we have to do something. People don't have water. We can't run the systems because we've lost control of the management of the system and we don't know if the water quality is okay. So we have to shut it down. Now, my point is we need to be teaching people whether they're technologists or whether they're policy oriented people about how these complex systems work and what it means when there's a technology involved. In this case, the information system being attacked and that being able to take action means engaging all the parts of the system simultaneously. The only way we teach that is with a clinic. We don't teach about it. We teach students to do it. We prepare them and we put them out in cities that want to be clients of the clinic. And the client community says help us. And we say, well, first we have to prepare students for six weeks. And whatever the background that takes six week online course intensive training and cybersecurity risk assessment. And then for the next seven weeks, they're assigned in teams of three to work for different cities and towns. And by the end of the semester, they understand why it is so hard for cities to do simple things to reduce their cybersecurity risk. They learned all the technical stuff in the first six weeks, but they also learned about working with a community, not just the technology side, not just the political side, but both. So I think university needs to be producing smart people from both who are going to work inside and outside technologists, non technologists. And the only way I know to teach them is to put them in a kind of clinical moment where they have to do what you say they're going to be prepared for, but do it while they're a student, learn by doing it, which means we need clinical educational ideas, clinical educational strategies for teaching about public interest technology. I know. That makes me wonder, and sort of broadening that out whether we're teaching about cybersecurity or we're teaching about did ethics, for example. You're, you're comparing students to exist in a world that doesn't that isn't here yet. Or maybe it is but it isn't. It's a lot of unknowns. We don't know when the next cyber tech is coming. We don't know how people are going to misuse data. We can guess, but we can only make a guess based on maybe what has already happened in the past. It's probably pretty hard to make a guess about what's going to happen in the future. So as we think about creating a more responsive government piece of that is preparing people for the world that they're going to be inhabiting when we don't really know what that looks like. And I want to focus specifically on the data ethics piece. So at the moment, because the use of data in government is pretty new and frankly in the world it's pretty new at the level that we're seeing it is it we're still wrapping our minds around what it can do. And so I recently met with a team who is doing amazing things with cell phone data. But it also gets creepy really fast. Latanya, I would love to hear your thoughts on how do we navigate evangelizing evangelizing the use of data in government while also ensuring that it's used in a thoughtful way. That doesn't move into the creepy realm. Well, I mean, first of all, the, the revolution of data data is becoming its own kind of currency. And it's clearly an incredible component in the future of decision making, because it allows us to make better policy decisions when we're on the inside, because we sort of have scientific evidence for a lot of the programs and so forth that that get pushed out right now. Right now, a lot of programs for example are pushed out. And we want to know whether or not that program was any good. We, we don't we don't we don't have really good ways of doing that so there's a need, need for government itself to want data. There's a tremendous flood of data happening. And the, the issue of individual rights and, and, and privacy versus their obligations to society, and whether or not society will protect it and to when does that turn into surveillance is has frankly been the conversation of privacy since the 1970s since the 1970s, and we haven't really come to grips with that at a point where data is becoming more and more critical. I do think it's, I do think that is a fundamental question, sort of the idea of an economy, this data economy, what will it be the, you know, clicking, sort of click through agreements and all of the things that has led us to hear clearly are not really effective as we go forward. And the design of technology and the practices that we've allowed with data have eroded many of our historical protections. So a lot of the kind of unforeseen examples that the students that our students bring forward, continue to shock us and begin to shape us into realizing we really do have to take, take ownership of this. So I'm not answering your question by giving a, an answer of this is the way forward of the solution, but I am answering your question in the sense of saying that public interest technology the development of it is critical to a few to our future. Jump in a little bit here and say that this is one reason why I think this is a really important conversation to be having in the university setting. Because this is, it's kind of the difference between talking about solutions talking about government even talking about policies and talking about governance broadly and I've been really, it's been really interesting just listening to each person, bringing the differences even to this conversation, whether it's the ethical lens and we're really having deep questions about what is ethics at an individual level at a societal level there is a philosophy conversation to be had there and it's a conversation that needs to be robust and that goes well beyond data right data does not get us. We don't use data ethically or in ethically, but data is not going to teach us ethics and so there's that conversation that needs to happen there's the cybersecurity conversation and the connecting students to practice and the, the, the sort of understanding there's a sociology piece of this that we actually behave a human behavior element of it. There's a, again the policy and governance what actually what makes for strong communities not just what makes for good solutions but what makes for strong communities. And I think all of these pieces are things that the university, having these conversations within the context of the university allows us to bring these lenses together and really important ways to where we are looking at data, but we're not just looking at and also looking at it through these different lenses and we're bringing the human elements of these conversations and unlike Margaret we're working with local governments and and a big piece of this is also just the practical conversation of what works and what actually like how do you match a tool with a solution. And we see that in the local government field all the time part of the reason we started to get involved in this is that we realize that the local leaders we were working with all of the training they were coming from vendors and the vendors are great and they have really interesting information but they're also trying to sell a product. And so having that broader conversation I think that's where the university can bring so many disciplines to bear on this conversation, so that it's not just a conversation about technology, but it's a conversation about how technology fits in with our experience of living together in community as human beings. So that's really important for us to not lose in the conversation and that everything I'm hearing from Latanya from Larry from Margaret is just really highlighting that for me right now and even your questions Hannah. So if I could add in, I love the way Ashley just put that, that the importance of the Academy, and part of the importance of the Academy is coming up with new theories new doing R&D on on technology, but part of the importance of the Academy is to work with folks outside the Academy as partners, right shoulder to shoulder. So industry government, civic society groups, and the issue around data ethics is absolutely fascinating is going to get ever more complicated questions as Latanya mentioned about privacy. So do we have the right to use or often reuse data that might never been meant for the purpose we're using it for now. Issues about even if we have the right to use the data, or the tools we're building with the data unintentionally reinforcing biases and social injustice because of how the data biases in the data themselves or in the AI or machine analytics, analytic tools for building, because we're not paying enough attention to social justice while we're designing them, but also issues around somebody mentioned the idea of the public and community. So one of the biggest issues that ethics lab works with a lot of partners trying to work on how you design data ethics solutions for privacy. So how you design data ethics solution from the ground up for social justice, and again how you design them for preserving the public trust so if we think about the context for instance in public health right very active right now with the coven 19 conversations that's all over the globe are working to try to leverage new data organic novel data sources and new analytics on data to help at drive prevention strategies how do we decide where the virus is going to be hitting. How do we a portion scarce preventive resources. Really important. We want to make sure we preserve privacy when we do that we want to make sure we don't have an intentional discrimination so that the people don't have equal access to these critical benefits. So we have to pay attention to preserving public trust in the tools that we use because sometimes some of our most powerful tools are ones that involve black box technology right well we can't explain why we have the predictive analytics we do. And if there's one lesson people have talked about in public health is that public trust is hard one and easily lost and if you don't have it you have nothing. Not to comment too much in our current state of the world but boy is that true right if we lose trust we've lost a lot. So the importance of people working on data solutions and data analytics solutions, understanding that they're ethical stewards of privacy justice and trust is really critical. I would just like, and I like to tease that and pull it in a slightly different direction. And that is, and that is the issue. So what creates a lot of the vulnerability so we, we have a whole modeling and curriculum that around how to find the unforeseen how do you figure it out. How do you put a scientific factor around it. But one of the issues that's particularly important on the inside for government, the government's not good at is teasing away when a very successful technology, a very successful use of data is inappropriate for its use in government. So this this really came up in both the conversation that Ashley and Margaret brought up because we industry is working and moving and developing great new technologies. But whether or not they're actually appropriate to be used in a particular setting is a totally different conversations ahead. I would say about a third of the vulnerabilities that our students point out each year, been in that category, where the shiny new technology is automatically deployed. But yet, it has a consequence that was not foreseen. And the consequence, because it's government is overwhelming is shocking and shocks ones conscious, and therefore becomes disruptive. I do think that part of public interest technology is close is actually filling that gap, whether you're coming to it from the kind of lens that Ashley talked about or the kind of lens that Margaret talked about, or Larry, any person engaged in these programs. We're all pushing in this kind of knowledge in different kinds of ways with different kind of pieces because it's such a broad area. At the end of the day those are the kinds of holes I just think we're all filling. I also try to build on Ashley's point about the, the role of the Academy and put a finer point on it. My sense is, when we talk about corporations we expect them to have a statement of corporate social responsibility. A few universities have a statement of their corporate so they're not for profit but or some of them, but that very few have a public statement of what their social responsibilities are. I believe they should. And within that statement, when we're talking about in public interest technology or data or data gathering or research. We teach what we call and others call participatory action research par. And you can Google it and somebody listening and study what that means. But basically it says if you're going to mess around in a place, whether it's at a skit whatever scale that the people in the place are not just partners in the sense that they're going to receive the work that you've done. They're partners in the decision about what questions to ask, how to answer them, how to interpret the first cut at the information that you've gathered what to formulate by way of prescriptions from that work. And if we don't say that it's part of the university or the Academy social responsibility to engage partners in a way that really allows them to participate and share in making decisions. Then I'm very worried that the outcomes in the products will not reverse the kind of unfairnesses and social inequities that were already in place they'll just reinforce them. And my feeling is that just because we're working with data and technology doesn't mean that the research that we do shouldn't be undertaken in a par like fashion. It's really interesting conversation and not necessarily the direction I thought it was going to, we were going to go, but I am reminded of a conversation I had, maybe a year ago with somebody who was an AI academic for a long time who is now has been in the private sector at one of the big places for maybe 1015 years. And he was making the argument that one of the big problems with something like Twitter or Facebook is that those were not incubated in the academic world first. So there wasn't a real understanding of what they could do. And that that would have come out had it had those been academic projects. I'm curious if that resonates and if there that feels like there is a pet application. Well, I'll jump in being a computer scientist. I don't think that I would push back on that perspective that the that the Academy would have led us into a different plight. It has to do with one of the questions being asked and whose obligation is it to do the asking, and what are the tools they have to enforce the answer. Each of those companies have a fiduciary responsibility to their stockholders. At the end of the day that's what business means. And those are at the end of the day. They can make claims about their social responsibility but when it clashes with their business model when things really do change. I saw that in my own work with when I did the work that sort of pioneered was the first work on the unfairness of algorithms to show that if you typed in a name given more often to a black baby than white baby in a Google search bar. You got these ads implying the person had an arrest record, even if they didn't and even if no one with that name had an arrest record. I remember notifying the company, which was one who was featuring the ads, who is a large company who you would think would be responsive not going to do any evil in the world, and we would expect them to be equally surprised. But actually action didn't happen until the news went around in the media and got to the business press, and then action happened. I remember the same thing with Facebook when Iran did the show to how Facebook Messenger leaked GPS locations, and he built a plugin that showed where you could view friends and friends of friends locations. Again, shocking the conscience. Again, we didn't see Facebook take action until until it became a part of their business issue. Now you see some examples, not because I'm claiming that the, when we do disruptive work that that's, that's, that's somehow better. It's not better what we really want, we want the companies to be responsive, we want government to be responsive, we want them to be responsive early, but to think that. But to think that somehow, if they had been a project coming out of academia, it would have been different, I don't think it's true. Because at the end of the day when they cross over to have to have that fiduciary responsibility. At the end of the day that's what's going to drive. We might push back just a little bit. I think I agree with like 90%, but I want to maybe push back on the 10%. So I also agree that it's a little overstated to say if only things were incubated in the academy all would be well. It has its own problems and poor representation and the only one slice of society that that we're all familiar with. What I think is potentially the, the, the broader insight that your friend Hannah was gesturing toward is if we incubate new ideas in a seemingly value neutral setting. I think that all we're trying to do is move fast and break things that that's the way to make innovation happen. There's a culture about what's okay that can get started that's really problematic. And then the only piece I would push back from the way that Latanya was pointing, putting it is, it's one thing to say, most many many corporations will are may not live up to their ethical obligations. That's your many people too, right. But but I wouldn't say that because they have a fiduciary responsibility to stockholders that they don't have the more responsibilities. Right, so business ethics were fiduciary responsibility or stockholders but they're limited by being a good agent and that's not merely being compliant with law because law often isn't keeping up with the potentials of the things you yourself have created. So, we want to have innovation that is developed with a socially responsible mindset that doesn't have to mean you can't make profit but it does mean you have obligations beyond merely profit maximization. And to me one of the really fascinating and important goals of the pit you in initiative is to, is to start is to try to inculcate even more the idea that innovation needs to have that kind of mindset and to start asking questions about how we develop it. Let me plus one Margaret for for pushing back a little bit on my sweeping statement and one of the best examples of what you just said you do see this change of thinking in companies Airbnb I think is a fantastic example of the exact opposite. When faced with the same kind of exposed kind of unforeseen consequence, because a lot of it's unforeseen. And so to the extent that it could be foreseen, then we should we could help them make it foreseen. But when it's unforeseen it's in the marketplace. It's kind of late and you and you end up in this disruptive cover yourself kind of reaction. The Airbnb didn't take that action they've really been amazing. They've stepped up. They've led new research within the organization itself to make sure that they fight bias on their platform, and they are to be commended so they just said as an example that it's not always just profit driven. So I can jump in just a little bit in terms of the university's responsibility. And the place to incubate things I think one of the challenges and I do think that there are opportunities to what Latanya said for more research and a more academic lens to potentially help see some of the forcing consequences right there are consequences they're often unforeseen that if we've done a bit more research we would have foreseen. I also think it's important to note that technology change it is sort of iterative as it goes and the ways that we use technology and this is a human question not necessarily a technology question. The way that we end up using them changes as we go and so I think about I actually think Facebook is a great example of this. In the Facebook generation, I had my first Facebook page when it was when you could only have it with the dot edu email address it wasn't incubated in the university as a university project but in some ways it was incubated in the university as a student project. And at the time what Facebook was designed for was to be a way to stay in touch with the people you went to school with and to build your professional network with those people so that as you moved on. You'd stay connected and it very quickly as soon as they opened it up to a more public base, it shifted dramatically in terms of how we actually used it. And in some ways, you know I still have friends from college that I do keep in touch with via Facebook and it still is the way for the thing that it was originally designed to do it still does that decently well. What it wasn't designed for, but it adapted into was this whole platform of sharing information of sharing news of sharing opinions of connecting more broadly to people we don't know all of these things. And I mean we can argue about how much people saw our foresaw but I think that the point that I want to make is that technology and the way we use technology adapts. And so I think one of the key things that we need to do and one of what it sounds like all of us are doing in the various projects is helping the people who will be using this technology whether it's in government or sort of alongside government as watchdogs. We have the right set of questions and the right set of ethical lenses, so that whatever the technology is that's released as it adapts and as it changes which it will and the way we use it well. When those truly unforeseen consequences come up that they have a framework in which to respond to that that's beyond cybersecurity. More in terms of how we use and some of the broader ethical uses of things that if people start using something in a different way we have a sense of how to respond to it. And that goes back to the whole governance question that we are, we are helping to prepare people for the difficult job of governance, not just the technical use of policy or the implementation policy or these different things. So I want to take the opportunity just to we have a little under 15 minutes last and if we would love to take questions from the audience if anyone has a question for one of our panelists just please put it into the chat box. And I want to bring it back to something that we sort of vaguely touched on but not fully. So Larry your work focuses on negotiation and consensus building and bringing that to the pit space, much of your work is out identifying relevant stakeholders for public decisions and then finding ways to help these groups work collaboratively collaboratively with government. Obviously, there is some huge overlap here in the conversation we were just having. So when we talk about shaping organizations to take a pit approach to problem solving one critical piece is being able to have a deep understanding of what the public needs. How can we build the capacity of the public to engage in collaborative decision making. Thanks, it's much more complex that I could explain or begin to answer in in two minutes but if you think about the moniker smart cities, right, we have all over the world efforts to employ new kinds of technologies information technologies and others to ensure that what's happening in the city is immediately noted or that people can put their concerns and that will allow city governments and city government agencies to be more responsive. So the presumption is that participation means information back to the system so that your needs and interest can be met. But the system that's responding is designed by policymakers and by decision makers, and unless we have a different layer a meta layer of engagement of stakeholder groups in the city let's stay at that scale to help formulate the policies and design these institutional structures, then sending messages back through smart city sensors, saying, oh, there's a hole in the road. And then that we should send the truck and fill the hole doesn't mean that the transit systems going to be designed in a way that's fairer, or more efficient, because policy decisions need to have engagement. And we can use different kinds of technologies to support the engagement of not just large numbers of people, but categories of stakeholders who can work collaboratively, not just separate lines of information flowing back some it up and tell the system what to do, but to engage in policy deliberation with stakeholders representing large segments of the community that make the systems capable or not capable, or inclined or not inclined to respond to what the technology of smart cities is sending back by way of information and my feeling is that those of us interested in public interest technology should be talking about how to use some of it to ensure that the policies in the first place are made collaboratively and more responsibly and more fairly, and it's different technologies. It's not sensors sending back information. It's online channels through which the liberation face to face deliberation can happen and it's a portion of a smarter technology for collaborative decision making, not just information gathering that makes systems of cities more efficient that I think we should be working on. So I think this is a really interesting important point that we haven't quite touched on yet, which is incorporating human voices into policy making and that there is, you know, in in technology is in the name and we talk a lot about technology but technology is very much a tool of being used for policy making, or for policy implementation, but it is not necessarily the tool. So I think that that is worth saying again and again and again. Any other thoughts? I would just jump on what you just said about a new technology can be a really great benefit for those human conversations. One example that I come back to, which is somewhat dating now, but kind of demonstrates your local background that was added in large amounts of ways. So is that we work with the city of Yau in California, right after a huge scandal, it was national news, if you haven't looked at it, I should be, it should be a movie, basically like a coffee run city. And one of the things that I really appreciate it from that is that you use, sorry for saying that my feedback is specific, let me pause. So, while we wait for Ashley to work at the audio. Technology. The unforeseen issues of technology case in point and the question is, do we produce people who are nimble enough to recover from it. And, and rooted well enough in the circumstances that that in the reaction, they'll navigate us in the right direction. Ashley's back. Not yet. Okay. And I will just put out one more call for the for questions from the audience we are closing in here to another 10 minutes. Ashley, do you think you're back or should we push push on. You're good. Right. Can you hear me. Is that sounding okay. Alright, I was just going to say, we worked with the city of Bell. After the scandal, look it up. It's fascinating. But they wanted to do public conversations. These are in person conversations, low tech conversations around some budget prioritization after essentially, there's been so so much corruption in the city. And the reform government, the new government that was elected the new government that came on the new city manager worked with a tech platform called open gov, which many local governments are familiar with now. But just a data visualization basically did budget visualization where you could look back over the years how much money had been used, how funds have been used and also do some production. And it was visuals. This is a very ethnically diverse community. Not really an ethnically diverse community is a very non white community a very, a very highly Latino community a lot of people who English was not their first language, but being able to see the visuals being able to see the graphs and charts being able to kind of have conversations in Spanish because we use dual transit we use simultaneous translation technologies, help these in person conversations to be had in a way that built trust with the community. In a way that it would have been almost impossible if we printed things out and show it to them, people would have still guess just been suspicious and being able to have this third party platform with all of the information that helped guide the conversation, along with the translation tools that helped a dual language community, have the conversations together was super powerful so I just wanted to note that that the in person, the community building piece and the technology piece, sometimes can be integrated in really interesting ways. Beyond what we think of when we're thinking of technology, kind of broadly and data and all of this stuff to make decisions. Let me just say one thing, we, it, at first glance, and actually reminds us that we're not about technology bashing. I mean, it's in fact the belief that our future and technology go hand in hand, as to how do we make sure that it leads us in the right direction and that we become the benefactors of it and not be victimized by it. A lot of our work has really pointed to democracy itself being up for grabs, as we looked at the kind of problems that we're seeing, getting bigger and bigger, in terms of our ability to execute our rights, and, and, and, and, and ensure our government. And it just seems so critically important that we pursue public interest technology right now. And I want to add to that that there was an essay recently that talked about radical responsiveness in government, which where radical responsiveness was defined as reacting government reacting to things as they are, not as we would like them to be. And public interest technology is a huge piece of that. It almost like, you know, was flashing lights to public public interest technology as I was reading it I don't know if anyone else saw that but I thought radical responsiveness, just as basically being tell the truth was a great, is a great catchphrase. I just want to be sure that when we talk about public interest technology, we don't just slide immediately to the discussions of the technology. It is not obvious what public interest means in a democratic context. And my sense is that if you bend it back the other way and said how can the technology help us understand what the public interest is in a particular context. That's a different mission. It's not, you know, and my feeling is that the essence of the public interest has got to be the product of deliberation, dialogue and deliberation, not just counting what people think they want an appointment at a time people should have a chance to can educate each other and convince each other. And so my concern is that public interest technology emphasize how what's the public interest and how can we know it better with technology, not just as a label to put on the technology so we can go about doing our technology work. Yeah, that's such an important point. In our remaining two minutes we did get a few questions and maybe we can just do a really quick speed round on the one of the questions. So, what excites you about the possibilities of technology with regard to government. Well, I'll go first I expect. Right now, we live in a new kind of technocracy, where the rules of technology determine how we live our lives. And through public interest technology that we can make sure that our democracy gets restored. Ashley. Yeah, also one of the things that I'm excited about my work is in the dialogue and deliberation field. And I think that often gets offered. I'm actually really interested in just how technology can help improve basic government processes so sort of the customer experience with government because that takes up so much of people's time and energy, and making sure that that as technology helps us do a better job with that customer experience, it can allow us to have more space and room and even emotion for the citizen and I think it's broadly not meaning not an illegal phrase but but in that place where we have ownership and we have investment and we have discussion. I think it creates more space when we're not just frustrated because how in the world could they not have told me that this was the form that I needed 18 phone calls ago, which is often I think what most people's first interaction with government still is. We emphasize that dialogue deliberation using technology is going to be key to making democracies work and not just letting them fall into his little tiny says becoming technocracies for because the technology demands it as opposed to the other way around. And I think there's a role for a facilitator of the collaborative conversation using more advanced technologies. And I think all of us in the university settings that we're in can talk about training people to be facilitators of substantive policy dialogue at every scale, using the technology to achieve the consensus or informed agreement to the extent possible in the dialogue. So not people to make the technology only, but people to facilitate the use of the technology and boy do they need to be aware of and skilled in a variety of disciplinary capabilities. And Maggie, let's, let's end on you with a. Well, I, I am extremely excited about the power of technology to advance the common good. I mean, when I think of advances. I mean, just how technology is helped us with coven, you know we wouldn't have made the incredibly rapid progress on vaccine and therapeutics without new data solutions so there's enormous potential here. And you just have to, you know, you have to balance the promise with the responsibility that comes, and I completely concur with this panel that a key part of that responsibility is actually designing it around. I don't even want to call it the end users designing it around us right and the common good. I just want to thank all of you for a really excellent discussion on a rainy Friday. Thank you so much. Thanks everyone. Thank you. Welcome. Hello everyone. This is the professional development panel where we're going to be talking to some challenge grant winners from 2019. And the title of the panel is collaborating and learning as a pit educator. This is Effie McLaughlin, I am an assistant university dean for research at the city University of New York. And my entry point into the pit you and movement conversation which I'm going to ask all of our speakers our panelists today is mostly from a professional development faculty development, sort of research development perspective I'm really interested in faculty engaging with these ideas and also developing research projects around these ideas so I hope we'll get a bit of a chance to talk more about that. And also, as David guest and started us out with yesterday on a personal note, my greatest technological challenge my grand challenge today was getting I have, I live in New York City and I have three daughters who are in the New York City public school system, who are all good professionals and all have different. They all have different online and remote learning schedules and it says my biggest challenge today was getting them out of the house so I could have under interrupted bandwidth time to conduct this panel. So, I know that some of our panelists today or some of the moderators today. You know basically read out the bios from each one of the speakers but I'm actually really much more interested the way that I would like to start out. And I'm going to introduce each one of you in turn but I'm basically just going to say your name and your affiliation and what I'd really like you to say is, you know, say whatever you think is important about your professional experience that brought you to the pit you and meeting then also because probably not a lot of the attendees today, had an opportunity to view your poster session video so if you could give really just a very brief statement about what your project was. And I'm going to try to keep this short I already had a conversation about this but we're going to try to stick to the time as much as we can so please try to keep it at two minutes and I will, if I have to break you in for time I guess I will but I so I'm going to start out and go in the order that they were on the in the program so I'm going to start out with Dr. Kenneth Fleischman, who is the associate professor in the School of Information at the University of Texas at Austin and he's also the founding chair of good systems at UT Grand Challenge as you see behind his head. And his project was the 2020 conference on undergraduate informatics education and can if you could just say, as I said a little bit about yourself, what brought you to the pit you and movement and just say a very brief overview of what your project was. Awesome. Thank you, Effie. I should mention sorry the website information was out of date when the program was made so I've been promoted since to professor. Texas at Austin sorry, but yes I'm really delighted to get to be here with such esteemed colleagues from across the UN universities. So, in the School of Information, we focus on the intersection between people information and technology but we always put people first. So the, our high school as many other high schools do builds upon and extends a tradition of service to users that comes from our roots and continued efforts in the field of librarianship. But we now expand that to a broad range of information technologies, some of which are still contained within libraries archives and museums and some of which go far beyond. So, as part of that effort in the Austin School of Information, we're launching an undergraduate major in informatics. We have six concentrations cultural heritage informatics, health informatics, human centered data science, social informatics, social justice informatics and user experience design. And so, when we learned about the exciting opportunities at pit UN and the range of different universities and colleges, all getting together around this common vision of ensuring that technology serves the public interest. It fit perfectly for our high school. It fit really well for me my background is truly interdisciplinary. So, I was an undergrad CS anthropology double major. And then I moved to the field of science and technology studies where I got my master's in PhD, and then I've been living in high schools ever since. So, we were able to organize by just barely because it was March 2 to 4. So just immediately before everything shut down informatics education 2020 in the city of Austin at the campus of the University of Texas Austin. So the collaborators on the project within the UT Austin School of Information Amelia Acker and Eric Meyer are Dean and also our pit UN representative designate and also from the University of Michigan school of information because they were two, you know, top high schools within that have that same tradition of of librarianship but taking far beyond and you know, so that included Pat Garcia, Casey Pearson, Kintaro Toyama from University of Michigan School of Information so putting together faculty members and students from 30 different universities, 10 of which are current pit UN members, 20 of which might be feature pit UN members and there were a lot of actually Eric had a lot of questions because he's also chairing the membership committee about, you know, how to be a provost or president to join pit UN so it's for a lot of interest and put you in in general, and it create a lot of information exchange, and then most the last thing I'll say about it most specifically, it led to a project that we're doing for the coming year, which is the pit UN social dust informatics faculty fellows program, which is a collaboration with Houston Tillitson University, which is the oldest university in the city of Austin it's also HBCU. And as well as with measure and capacity catalyst to social justice oriented nonprofits, and the city of Austin, as well as UT Austin School of Information, good systems UT grand challenge. And we're going to have faculty fellows from across Houston Tillitson and University of Texas at Austin, learning directly from and with our nonprofit and government partners about what it means to do public interest technology. Thanks. Great, thank you. I'm now going to move on to Dr. Mammoud Farouk, who is the associate director of the Consortium for Science Policy and Outcomes based in Washington DC, and he's also a clinical associate professor for the school for the future of innovation in society. At part of Arizona State University, and his project was public interest technology community innovation fellowship. Dr. Hello. Thank you for the introduction that's safe from from telling who I am. I, my work mainly focuses on two questions. One question is, how do we make science more useful. And the other one is how do we make science and technology more democratic so it's just, it's the democratic part. And that brings me to pit UN. And it's a pleasure to give a brief overview of the public interest technology community innovation fellowship program that we launched with the help of our grant. The fellowship program is a partnership between ASU led expert and citizen assessment of science and technology or ECAS network, and the Association for Science and Technology Centers or ASTEQS. So ECAS is a network of university science centers and nonpartisan think tanks that engages public on science and technology decision making ASTEQ is a membership based network of about 700 science and technology centers and museums. The program focused on the public part of public interest technology and on boundary organizations that work to engage them in science and technology society issues. Why, because there is a growing demand placed on these institutions to be the educators translators conveners facilitators and bridge builders between the lay public and decision makers on complex social technical issues from artificial intelligence to gene editing to climate So our the project's goal was to create a replicable scalable and competitive fellowship program where museum professionals could work collaboratively with the civic government or university partner to co create and convene a public forum on a pit issues of interest to their communities. We selected and trained 10 fellows in five communities so in Ann Arbor, Jade Marks at the Museum of Natural History work with Justin Shell of Shapiro Design to create a community forum on environmental justice in San Jose, Anya Sholes of Tech Interactive and Koring Takara work together to create a forum on Agtech, biotech and food in Worcester, Rachel Cunby of Ecotarium and Stephanie from the city of Worcester created a forum on reducing urban heat island effect and in Waco, Texas Cindy Nullard and Mayburn from the Mayburn Museum and Melissa Mullins create a forum on water challenges and climate resilience Finally, at LA, Sasha Burris of Discovery Cube and Rebecca Fredman from LA County Chiefs Sustainability Office work on sustainability education. Now because of COVID-19 restrictions, our fellows had to innovate and find online tools platform combined with creative strategies to engage stakeholders and residents in their respective communities. But they all persevered and looking at what they have built in the face of widespread uncertainties we couldn't be any more proud of them, and we are thrilled with the outcomes. Thank you. Great. Thank you Mahmood. Next up is Margaret Hagan, who is the executive director of the legal design lab and a lecturer in law at Stanford Institute of Design, and her project was pit case studies platform. Yes, thanks. So, I came to the pit UN movement as a lawyer and as a social scientist. I run a lab at the law school at Stanford where we work with legal aid groups and courts and city government to work on eviction cases or debt collection cases family law problems. I've always been thinking how to improve the citizens experience of the justice system and how to leverage technology and data and artificial intelligence to improve the services and people's outcomes when they go through court. So public interest is already a watchword inside the law school community. So it's been really great in terms of developing more interest in law schools, faculty and students about how to work more interdisciplinary across campus to leverage all of these other fields on problems and policy areas that we as lawyers and people in the justice system care about. So the project that I've been working on within the pit UN network has been a website, a case study platform in partnership with Georgetown University and to Nina Roth staying there at the Georgetown Law Center and Howard University and know how has easy who's an electric electrical engineering professor. We have student teams and all of our universities who have been scouting around within our university network and beyond to find classes that are being taught on public interest tech themes or projects. And then to student projects that have emerged out of them and we've been drafting case studies and publishing them on the platform you can go visit our prototype site pit cases.org. I'm curious to really spotlight how a public interest technology project can best be launched within a class and potentially outside of it spinning out into a new venture, and how teachers who are thinking about taking kind of a public interest technology lens or teaching a project based class can put together an effective syllabus build partnerships community organizations and teach and guide students through this process. There are several public interest technology style classes and there's a lot of ups and downs and actually setting up the partnerships in the curriculum. So our goal is that these case studies can both be used inside of classes. So students can wrap their heads around this type of project work, and in the planning stages for teachers as they're setting up a successful class. We have about six current case studies up there and we're putting more up week to week. If you have any interest in having any of your classes or projects profiled there please. There's a link on the website to let us know, but we're really hungry for more examples from teachers and from student groups. Excellent yeah I have a colleague actually that goes to CUNY law and I told her when I learned about your website I was very excited and I told her because I know she's interested in public interest. Last but not least is Susan Emberman, who is a professor of computer science at the College of Staten Island CUNY and actually formerly she was the associate dean for technology education at Central Office which is actually how I first met her. And she her project that she did with our, our library and and fiddler is entitled curricula design and public interest tech. Please Susan. Thanks Anthony. So, my project is not just done with our library and it was also done with a number of colleagues. Karen Shelby at Brook University she's an artist Deborah Sturm at College Staten Island is computer scientist and Dora Clatnick was also computer scientist at Brooklyn College. So we started off with a small village and the premise of our project is that it takes a village to create a share curriculum. Our project was to create a repository of materials that was focused on public interest tech by awarding university wide faculty mini grants to create and share pick curricular as an open educational resource. We were essentially crowdsourcing across the team university system, the resources and curriculum that we created were shared are shared through our university repository, and we have group on our comments so we created this big group and we are comments it is open to the public, you're more than welcome to link your materials to it. These repositories may the curricula available for download to anyone who wishes to download it. We've had at the current time we have about 33 pieces of curriculum up there. When I did my original video we had 18 so just to show you that you're, you know, constantly adding materials to that. And since the summer so far we've had private and seven downloads worldwide so it's been a very interesting project and very great. Thank you so much Susan. So we're doing very well so I have to say I was when I viewed all of your your videos your post your sessions that you did when you spoke about your projects. One of the things that struck me and maybe this is just my own inclination sort of in my background as a political theorist I'm really interested in the definitional aspect like we're still I think struggling a little bit with what is public interest tech and how what people, what people's understanding of public interest tech is. And interestingly, I don't know how many of you attended the Presidency Provost meeting today but I'm a lot of we're saying, you know, what's wonderful there seems idea of what come public interest tech is is gelling and there's the courses being led and we're not creating a program or a discipline or not a program but a discipline. And the thing that really struck me about these four different projects sort of how they differed was that Kenneth and Susan's projects other Kenneth you said you have a very interdisciplinary background was that in some ways they were much more sort of grounded in the traditional sense of like, as you know as mentioned one of the other panels sort of the, the digital understanding of public of technology, whereas my mood and Margaret's. It really seemed like they were had much sort of a broader understanding of technology that you bought your projects and the people that were engaging with with your projects, and particularly my moods. It's interesting with you working with the museums and the science learning setters I mean this is I guess what NSF calls sort of STEM learning and informal settings and so I guess what my question is having said all of that I guess I'm just wondering of the, the, the faculty members and the science educators and the students that you worked with. Did you get a sense of, I mean did you sort of leave the definition of what is public interest technology up to them and let them figure it out through their work, or did you have sort of a common understanding of. You know, because I mean it's again sort of my mood it seemed like there was a very broad I mean the projects that you covered dealt with, you know, smarter cities to you know what are usage to I guess I'm, I'm wondering sort of how you're defining technology. Maybe we could start with you my mood. Yeah, so our actually goal was to find the public interest and I think that's what Larry Susskind was talking about in the previous panel, which is, you know, so we actually let the communities, our partners to figure that out by asking the people. So they did an extensive the first phase was you know the topic development. So they engage with the stakeholders and experts in their communities to say if it's water okay that's what I want but what's the issue in water I want to particularly focus on. So then they developed the program a set of questions, and then they brought in the public for the dialogue so that's how it was, you know to find what's the public interest. So normally because we're coming at this from that recognizing there's a public value failure when we fail to incorporate the public. And what is that and we think that the best people to answer that questions the public themselves. So, so, instead of, you know, as academics trying to figure it out by studying the review and and doing all the things we do. This was an exploratory in that sense. Yeah, absolutely so Margaret do you have any thoughts and sort of what was the technology that you're bringing to the public interest technology. Yes, well I think because our project focus so explicitly on kind of courses and projects student projects that were already out there. We were interrogating what classes actually identify in this public interest text fear, and I would say the general pattern that we see in terms of courses are a public interest partner, meaning either a government agency or a nonprofit who is acting with a mission for the public interest, then coming to a university class partner and saying how do we fix this problem how do we kind of solve this gap, usually with the idea that technology is the solution. So as we saw more case studies come in. Yes, oftentimes the projects that come out of these public interest tech classes are data driven or artificial intelligence or new text messaging system, but not always. Sorry, I have a kindergarten or interrupting me can I take a break. Oh, of course. So can what do you. So, um, what sort of projects and discussions came out of the conference that you had in terms of sort of I guess, you know, converging on a definition for public interest tech. Thank you. So I mean very broad reaching so we largely left it up to the participants in terms of, we laid out public interest technology as we understood it from the new America website and from our involvement in the first convening. So I think that we're still building the whole concept of public interest technology and fit UN is, is, you know, in this really exciting growth phase. So there's a lot of opportunity and opening there. For us in a school of information, we're an inherently interdisciplinary unit. So we're not a discipline we're a wide range I believe we're we're rapidly expanding in the Towson School of Information with the launch of the undergraduate major so this number will be dramatically different in a few years. But last time we looked at this a few years ago we had about low 20s faculty and we had at least 10 different PhD fields across the faculty. So we find that in a CS department or an engineering department or in a core social science or humanities department. So, I mean there were, you know, I would say a third of the people who attended were at UN universities who were learning about informatics and high schools, about a third were from informatics and high schools learning about the UN, and a third were kind of wildcards. And so we had quite a mixture and when I heard the original idea of pit UN and of public interest technology it sort of struck me immediately as how much it resonated with what we're already doing in schools or information. So in some ways, I almost my reaction is part of the reason we need public interest technologies because not every university has a school of information. I mean, I think that definitely there is value in each of the different concepts but there's a lot of convergence, and it would be great to see the public interest technology university network and the iSchools coordinate in terms of thinking about how the brand of interdisciplinary and of using technology to serve people to do user centered technology, leverage data information technology to to serve the public interest and to, you know, not, you know, when you just say well here's some problem we'll add technology to solve it. In a lot of cases, it's going to exacerbate existing inequalities in society in some cases it's even going to create new inequalities in society, and our approach is quite the opposite, we want to figure out ways that we can use information technology that is people centered that social justice oriented to make sure that we have a more just and where we have we have equity and justice and liberation of society. So Susan I noticed when I was looking at your project I noticed that you had, I mean you funded the faculty projects that you funded were all the way from you know creating a module into from an existing course, all the way to creating a whole new course and also working. One of the larger grants was for faculty working across disciplines you know humanities and social science and STEM disciplines. What did, what did you find in terms of how the faculty across the different disciplines approach the idea, you know, approach the concept of public interest tech. So, basically we did not define public interest tech per se. What we did was we offered the various definitions, but from the port site and from new America and from the Heinz School also had a lot of good information exactly how you can frame it. But the bottom line was we told everybody, it's a nation field, there is no real definition, and we are the ones who are defining as we go along. And that's why we felt that in OER was a good way to go because as defining this nation field of it, we're going to take whatever curriculum and resources that we've already built and build upon them and share what we've done to, you know, extend what has been done already. And I think that's where that there the power is is that we were still in the weeds right now and still trying to figure out exactly what it is. And we did a training and I kind of welded down to two phrases, we do good, and we do no harm. Around those two ideas, we can expand upon the idea of what it is. Thank you. So I want to encourage everyone, if for all the participants in this session you'll please if you have any questions, share them in the chat you can put your questions in the chat as we go along and we hope to leave a good amount of time at the end to get to any of your questions and we have very able technical support people they're going to be making sure that we don't lose anything. So I guess one of my other questions is about. I know that many of you talked about sort of outcomes in your in your videos but I'm wondering. Actually my mood you are very sort of explicit in terms of the outcomes that you're measuring but I guess even in a more general sense. How are you, how are you measuring the outcomes how you measuring the success of these and then I guess also to the extent that if you you see, you know what's the next step where do you are you thinking about scaling up you thinking about transferring to other contexts other schools, other, you know other populations in terms of outreach outside of the Academy which many of you are already doing. What have I start with you can this time so what do you think you've got in terms of next steps for this project. Yes, thank you, so yes, and this I mean the collaboration that we have with between UT Austin school of information Houston Tiltson University measure capacity catalyst in the city of Austin emerged organically from the conference. The feedback overall was extremely positive about the conference I think a lot of people had a great experience learned a lot met a lot of great folks did some good networking saw some educational approaches that was part of in you know in launching our informatics major that will start accepting students for fall 2021. We wanted to make sure that our faculty were interacting with folks who had vast experience teaching undergraduate students specifically teaching undergraduate students in terms of how to leverage data information technology to serve the public interest which is a huge focus for the city of Austin and for the informatics major. So that that naturally led from workshops that that we had at the conference that combine different stakeholders across the city of Austin, just seem like in the good moment a really exciting opportunity just focus on how we could turn the city of Austin into a model of how multiple universities multiple nonprofits and city government could collaborate together. And academics alone can't solve all the world's problems. And I think this is similar to my moods approach that we, you know, we didn't come in saying we know everything. Definitely, there's a whole lot that academics can learn from our community partners and from government agencies that have a much richer sense of what actually can be done and needs to be done in the world. And we're much more powerful together than we would be in isolation. Great, thank you. So Margaret what about it for you in terms of just I know you want to get more cases for the website but what what do you see sort of more sort of long term would be the the beneficial outcome from your project. I think as the network as this pit you and network really solidifies and grows. Our goal is that the case studies and the resources can be integrated into it. I think. Yeah, the leaders of the network are also quite interested in having this rich set of content and resources guides examples. There as a central resource. So we're talking with the leadership about how we might fold in all of the materials that we've been assembling into their kind of central website, and also how we set up a pretty user friendly protocol to capture all of this knowledge that comes out of disparate classes, programming events, because we know it's really hard oftentimes in some universities to get a new pit oriented project off the ground or to help a teacher who's taught in a certain way to all of a sudden teach this new type of class and start community partnerships, grade and evaluate these kind of projects or even know how to support students during the class or afterwards. What I hope is that as we build more examples and more content, we can have very well structured guidance for these future teachers and future students in this space. Great. So, Susan, what do you see in terms of I know that I mean I also have. I'm a huge advocate of the OER work as well and as you know I've done a lot of work with the library in terms of building out the open educational resources initiatives at CUNY but I'm, I'm wondering sort of how do you see where do you see going forward in terms of building out your OER website I mean and I guess also maybe the, I don't know like future intersections or the con the ongoing conversation between OER and hit sort of where do you see that headed. So, well, going forward, I'm hoping to get more people to contribute to that hit website at OER Commons. Be great to link some case studies up there also. Plus one there. And, and also in the process of getting all these faculty together to do their grants. We had trainings together we actually formed a kind of community was that kind of a very holistic type of experience. So that the faculty members themselves were looking and talking to each other and seeing that there was some synergistic group, you know, relationships that they could leverage. So I think that, in addition to that faculty have said to me do you know I did this but I want to go on and extend what I've done. And, like, so there are people who created a module for a course and I wanted to revamp an entire course, or they saw that there was some interdisciplinary action that they can take within their institution so that they can have some type of maybe type of disciplinary minor created so there's a lot of avenues that we can go along in order to extend and expand. Great. Thank you. So my mood I'm wondering, I mean as I said you actually had very good outcomes data in your presentation if anyone gets a chance to watch it but I, I guess what I'm sort of curious. So I don't know if you were able to sit in on the President's and Provost panel today when Eric Schmidt and Darren Walker from the Ford Foundation and formerly from Google spoke, but they all spoke very strongly about the importance of you, you know, universities and universities as the platform for driving innovation and driving you know making real societal change. And I guess I'm wondering from your perspective with somebody that works much more closely or works much more closely with these, with the, you know, these community groups and these sort of public, you know, science literacy groups and museum groups and public science groups. I guess I'm wondering what, what do you see is like the feedback loop or the synchronicity between what universities are doing and what these these public science organizations are doing and how can they support each other. Well, you know, one of the, when we, in our, when our grant was reviews one of the things question that was asked of us is how are you going to work with the other network universities, you know, because you are going to work with the museums and the communities. And we, we did try to open it up to engage with the other universities. One of the challenges that we found was more like cultural, because this idea of, you know, directly engaging with the public. Is is is most universities or educators off, come at this from a deficit sense, like public needs to know some information and if they know that information, they're going to do what we are recommending them to do. So it, it, so there is kind of a, I feel like, although we work the part with the working with the museums and community organizations work very well. I think we need to do some kind of cultural or educational exposure about what is it to actually work with the community what is it to actually listen and not go there to say, Oh, you need to know this and I have an answer, because that this co development process you know the demand side of it is there, the supply side from the university we need to do some work on to sort of train faculty train postdocs you know in kind of how do you kind of do this bridge building engagement that would be as something that I would think would be important to do. I'm going to leave myself sorry right. Um, so I guess interesting I was at a governance subcommittee. I mean I guess some of these meetings that we are a bit in or panels and committees they sort of address the coven 19 thing that the quarantine thing head on and then other times it seems like we sort of dance around that and it's like oh look for in this virtual conference and everything is fine and he real is talking to each other just like normal. But I guess I am interested. I mean, I mean I guess in some respects I can sort of see the very direct importance and impact of say oh you are because that directly you know impacts how we teach and our pedagogy and. But I guess I would like to hear from each of you one how your project itself adapted to our greatly changed circumstances, and also again sort of the future question in terms of. I don't know responding to this and learning from it like what are you going to learn from this experience going forward in terms of how you continue to think about pit and continue to think about these projects. I'm going to start with you might Margaret because you're shaking your head. Well, I would say in the short term, you know our project is still a case studies is going full steam ahead we didn't necessarily need that much in person to get it done though everything has slowed down in terms of responsiveness and people schedules. So that's a real thing. The coven related delays, I would say though at the bigger question if our project is really motivated on how do we teach public interest technology and give students really good methodologies to do participatory design technology development ethical reviews all of these things. We don't really have that many good models for doing much of this work that depends on hands on collaborative in person trainings in a more virtual world. So we're really interested if anyone out there has models I know in my classes we've been trying out virtual design workshops virtual user interviews all of these kind of new methodologies. Before we were really reliant on in person convenings and all of this good participatory interactive kind of reckoning with technologies consequences or advantages, we're losing a lot of that in the digital sphere. So now we're really hungry to do some of the initial evaluation of how some of these coven versions of technology and project based classes are going and how we can start to define some best practices for those. Oh, if you're muted, I think. Sorry, am I, you can hear me. Okay. So Susan, I, you know that I'm very interested in an OER and how it's impacting our pedagogy and how we teach and how we, you know, approach education and learning and, and I'm wondering how you think I mean it seems like, you know, the COVID-19 crisis has brought into high relief, sort of the importance of the development of these OER materials and I just wondering how, if at all that, you know, this period of thinking about this period is impacted how you think about your work in your project. So, so I'm going to relate this to to one other project that I've done an OER and the current project so one of the projects that we've done that sort of preceded the one that I doing for Pitt, and sort of informed it was that we as part of CUNY's, one of the unique initiatives that we have with the city is that we bring industry professionals into the classroom to give a industry focused class. And part of the requirements of these industry people, these poor guys, is that we require that they share whatever materials that they use in the classroom as OER. And this way other people can see how industry is working. And according to one of my colleagues that was watching all the downloads, as soon as COVID hit, the number of downloads increased significantly. We're out there looking for materials in order to use in their online classes, especially PowerPoints and, and the homework assignments and lab assignments. These were, you know, we were, we were hungry. And to that end, because everything, you know, especially CUNY, we had it like we flipped on a dime to online learning. Many of the faculty that were working on projects for me became their workload increased exponentially. Yeah, it's hard to imagine that it does just by, you know, going online versus face to face, but when you're online, it tends the content seems to go faster and therefore you have to prepare more content. And it's definitely just a, and the way that you assign homework to new projects definitely changes. So a lot of our faculty were a bit overwhelmed. A couple of them, you know, rescinded their awards, they said they just didn't have any time I know. And then a lot of them just said, well, we just need some more time. We just want to hit the summer and just be able to focus on this without having to worry about the classes and New America was very gracious and allowed us to do an extension until December of this year. So we're still not finished. So, yeah, so it definitely affected us in a lot of ways, but it's definitely in terms of being useful for a community or an academic community at large that we are that we already have up there was certainly utilized by many, many people. That's great. Now you know I mute myself I have a tendency to make noises in time when people are talking, which is not it's not a good thing in zoom because you can't have two voices at the same time. You can so how had how did the COVID-19 crisis I know that you talked a little bit about how it felt like the, you know, the pace of everything was speeded up and you had to have you had the conference right before sort of the whole shutdown but how do you see, you know, the thinking about the COVID-19 crisis and the shutdown and all of that impacting the work that you do with your project. I mean, to be honest, I think most of it was just dumb luck from the standpoint that we had scheduled that I mean the conference center had a very limited number of dates. We were given options in January, February, March and April. The April one was right before Kai and Hawaii, which didn't happen of course it got completely canceled. And the reason why we, we instead picked the March date. We didn't have a lot of time to be honest part of our original thinking of doing it in the, you know, spring semester was avoiding the heat of, you know, Texas in the summer. You know, there became a very different reason why it wasn't possible to hold a conference in the summer. So, and we had 20 fellowship winners who flew in courtesy of the generous funding from the UN that we're able to award, which was both faculty or students or postdocs, and who were affiliated with minorities, many of whom were affiliated with minority serving institutions, or were members of groups who were underrepresented in in pit. So it's great to be able to have a broad representation of participation across the conference. So, I mean, it was actually made it more memorable for many of us because it became the last in person conference that many of us will attend for a bit, certainly have to this point and we didn't have to figure out how to do a virtual conference, which I can appreciate as a major challenge, even pulling off an in person conference with just a few months lead time was was challenging enough. Yeah, again, it made us in terms of our next steps made us think, I think the covert moment makes you think about what's most important, and what kind of collaborations are most essential in this moment. And I think that really did for us sort of lead to this idea let's build a citywide collaboration, and, you know, start something here in Austin, Texas, and then expand it out to the country in the world. Right. Well, we're, we're at the point where we're almost just take questions from the audience but my mood I do want to hear from you I'm particularly interested to I guess, working with a lot of these cultural and public institutions I mean that with your partner as tech your partner I mean it seems like, hasn't there been a huge crisis in terms of funding and I mean I'm just wondering what's how the coven crisis is impacted even. The existence of some of these institutions that are your primary partners, do you have any thoughts about that I mean just in addition to thinking about how coven is impacted your project. No that that's, that's exactly right because it wasn't existential crisis so, and our, our program was spread out the whole year, you know, so we just finished our training and now we're going to train them how to build a forum. And they were going to do them in summer and that just, you know, obviously was not possible. And, and some of, and all the museums shut down. Some of them were some of the people are furloughed, luckily, you know, one but all of our fellows made the survive the first wave some of them didn't the more recent ones. So, but what happened was that one day wanted to keep on going. They said we want to so we actually kept our training schedule the webinars. We just pushed out the actual forum convening part to the fall. And what also happened which is what I think was, you know, can talk about miracle or something like that. So what's the transformation that happened. It really became the innovators. The idea was that we were going to teach them how to do this. But when they have the challenge that they went and work with the tech tech platforms, and everybody came up with a very different unique design. And some of them I will encourage you to go and visit what they created this. This was quite remarkable using synchronous asynchronous, you know, and hybrid kind of environment. Push them to actually do something more than what we originally planned for. And I feel like we have stress tested this program. So I think, you know, if things become more normal, because our, our fallback position is face to face is very important. You know, when you sit down with a person you share a meal, you hang out together that's very different than what you can do in online. But you know, if we can go back to that and use some of the learning. We can make a more enhanced and engaging opportunity for citizens and decision makers to come together and deliver. Thank you. Yeah. So I do not say I have a question sheet and mark is there are other questions that are coming in from the audience that we can respond to because we're in the question and answer period. Oh, there's no question so far. You've answered all of their questions. Well, I mean I guess the one I mean sort of the one question that I did have for both of you I mean we were going to do we're going to do a little bit of a wrap up in a few minutes, but I guess I would be interested to find out for those of you that did more sort of like the mini grant type. Or you know conference and mini grant type things what were the criteria that you use just select your the people that participated in your project so Susan what in terms of picking the faculty and the project how did you how did you choose who was going to get the grants and do this. Well, we looked at whether the proposals connected hit with a discipline that they were actually in. And in our RFP we also requested that the faculty give us a timeline for when they're going to implement the curriculum was not just enough that they couldn't give us a curriculum as in terms of a resource, but we wanted them to implement in the classroom. And they were also required to give us a budget as to how they're going to use the funds that we gave them. And we made our decisions based on whether these three criteria made sense in terms of the proposal. We also tried to make sure that we funded proposals that covered a representative cohort with respect to full time faculty versus adjunct faculty community college and senior colleges, as well as looking for a geographic diversity within New York City. And can I know you talked about, I was a little bit so that the conference itself you said there were 100 participants from 30 colleges but then there were 10 specific awardees so who were the 10 and I know that you said that you tried to be, you know, selected diverse group but I'm, I'm curious I wasn't entirely clear on who the actual actual cohort of the 10 grant, sort of mini grant recipients were. I believe it was 20, although there may have been one or two I had to cancel due to COVID, but it was either 18 or 19 of the 20 awarded we're still able to make it despite the unusual pandemic, you know, it's just on the cusp of the pandemic circumstances. And so the three criteria that will okay so first we participants submitted abstracts for it could be a paper or a panel or a poster. And so we reviewed those first. We were looking for relevance to public interest technology. We were looking for relevance to the field of informatics broadly conceived which again at us at the ut Austin School of Information our approach to to informatics is very broad interdisciplinary so we had a lot of social scientists there we had a lot of computer scientists there as well as folks in high schools and other units of biologists and engineers and and lawyers so wide range of different folks. And certainly representation was one of the considerations so our view of excellence is that it has to include broad representation as part of that. You're never going to get the best ideas by looking at a very narrow cross section of, of participants and you want to have everyone's ideas at the table you'll have much richer ideas as a result. And then once we had done that past then we reviewed we also had the only some people applied for the travel fellowships. So many you know if someone was a full professor we had a director of a school come and so they're able to pay their own way through their, their travel funds. So we prioritize to folks who wouldn't have travel funds or wouldn't have sufficient travel funds. So the three criteria that we use we're prioritizing junior over senior. So we're focusing on students we had undergraduate masters and doctoral students present and presenting with some fabulous undergraduates from Olin College of Engineering, who presented the conference which was phenomenal. It was a conference on undergraduate education so it was great to have undergrads who are the actual consumers of the, of the product there to tell us what we could be doing better and also perhaps the year to their future educators. And we had postdocs and some junior faculty, and then we focused on minority serving institutions. So we had participants from for HSI and from to HBC use. And that was, you know, certainly funding is useful in the context that I mean whether you're talking about a minority serving institution or a community college that I mean every institution has different resources, unfortunately, so it's important that we make sure that we broad representation, even though not every institution has the same opportunities and resources that's the benefit of that that you and funding. We also want to make sure we have broad representation in terms of lived experience. So, in terms of, we asked people to describe the extent to which they felt represented in in in pit in stem in informatics and so certainly, you know, one aspect of that is like gender gender identity, sexual orientation, race or disability status, first generation college student, you know a lot of different factors veteran status that could play into that but we wanted to have broad representation across our society. And we really felt that I mean that that did help to really have a breadth of representation perspectives which led to much deeper conversations if everyone had been from the same universities and you know address the same and everything it wouldn't have been an interesting event. It was much more much richer from the different lived experiences the different disciplinary perspectives, the different institution types which really drives what undergraduate education is like at a liberal arts school compared to a polytechnic, compared to an Ivy League school so we had this broad representation. I just want to add that I attended that conference and it was great. And it was plus one for that cover it was it was and it was just before the, you know, basically everything hit the fan and like the next week we're like oh my god. And it was the last income in person conference I attended. Well, we have just about five minutes left and they had asked us to give you a little time to just wrap up say any parting comments anything you'd like to say about the conference about your project about you and about where we're going where you've been anything. Why don't we start with you Margaret. Sure. Well, I think you know I had come to this project. I thought I was very interdisciplinary before I kind of started the case study project. But really it turns out I had been in my bubble of other law school professors or lab directors and had been thinking of public interest and kind of very narrow justice or law oriented way of thinking and way of teaching my classes. I really appreciated seeing how public interest tech classes are being taught in public health schools in policy schools and all kinds of other domains and so I'd really recommend that if you are teaching the class. There's a lot to learn from other folks is syllabi and coursework how they structure their partnerships and classes. So, I'm all for more of this kind of cross sharing and figuring out, even if we are all interested in public interest tech the unique variations that that actually takes in different policy domains. So yeah I'm really interested to see how we develop more of great teaching methods and how we can share them really effectively with each other. I think it's a great place for my mood to come in here too because it's not only sort of radically interdisciplinary and getting out of your own buckle but it's also those really important connections between, you know, non governmental organizations and you know in the public and if you have any final thoughts about you'd like to share. Yeah, I think that you know what it convinced us first of all the model works you know we brought in somebody from museum world and to work with somebody from say government or nonprofit or university, and they can co create something and build that and work five out of five times, you know, so they, and, and what we also saw that there's actually a great demand for this, and we need to create the capacity and if you look at post election or in our code recovery and so forth. So we need to figure out how to empower these institutions which are kind of at the frontline. So universities, you know, to pick you and we can actually be help build those capacity and now that we have created a model, you know, all of us have different models, we can actually then, you know, replicate this and build a social capacity in different levels so that will be, you know, the long term role here. Yeah, it's exciting. Susan, what do you think. Well, I always like to quote Katie comiskey who's the grantees and is the other session. She put it quite succinctly. Oh we are is it buys very nature, it is a public interest technology. She's also said by the way if you own a cell phone you're a technologist. Well, we can argue on that one but okay. Still, I think, you know, I think it's just what it's always are defined by it is a public interest technology and it's one that we can utilize in our academic curriculum in our syllabi and share and I and I'm just really excited to see the way that people use our curriculum that we've created in addition to case studies. And how these are revised and reworked into even bigger and better educational courses and curricula going forward. Great. So you've got the last word can. So I think for us, it was really exciting. I mean, part of the strength of pit UN as we see it is the broad range of colleges and and universities and schools that represented at pit UN. And the broad representation of different institution types at most like in high schools most high schools tend to be in large public universities which are great, but aren't the whole of academia so the broad range of different, different academic institutions that was represented to so many collaborations. So we're already thinking about what UT Austin and Houston Totzen can do together to collaborate. There's an interlocal agreement between the city of Austin and UT Austin that resulted in part from the collaborations that good systems UT grand challenge and the UT Austin high school have been involved in and bringing in we're funding seven projects with the city of Austin and good systems UT grand challenge one of which won the Metro lab innovation of the month award for July 2020. So there are a lot of opportunities for doing smart city work, where it's not just about the tech it's about how the tech can serve the public interest. Thank you so much, all of you. I. So are we mark is there something that we need to do to close this out or are we good. What do we do. Okay, well thanks again so much all of you. And I think there's a few more events I hope to attend the final celebration event but I hope to see you all again soon and it's been a real pleasure hearing about your work. Thanks so much. See you soon. Thank you. Bye. Thank you.