 All right, my name is Michael Bernstein. I'm an associate professor of computer science at Stanford University, and I'm presenting on behalf of myself and my colleagues at Stanford on ethics and society review of artificial intelligence research. Now, I don't think it would surprise anyone to make the observation that AI or artificial intelligence presents societal and ethical risks. It's a horse that's already left the barn. That horse has crossed state lines, raised halfway across the continent, attracted international headlines and landed a shoe company sponsorship. AI systems have been implicated in generating in ranges of issues ranging from generating and propagating disinformation, depressing wages for workers, perpetuating systemic inequality and policing the justice system, advancing unequal health care outcomes and many others. And AI is not unique in needing to face these issues, but it is especially salient today. And we are developing but still have inadequate institutional responses to this challenge. What we need is an approach that requires that everyone pitch in, everyone participate in ethical and societal reflection early on in their research. And there's a lot of really great work happening trying to do things like opt-in approaches like office hours, which are really helpful but only impact those who self-select to participate. And other really helpful approaches such as those requiring broader impact statements in research papers are useful, but again, only written at the end of the research when it's kind of too late in many cases to change what the research is about. Now you'd think that we in the United States could use the IRB or the institutional review board to try and facilitate this kind of reflection. In the US, the common rule, the regulations that govern what IRBs are and are not allowed to do focus very narrowly on risks to human subjects. These are people directly involved in the research and specifically exclude consideration of risks to human society. If you look at the common rule, it talks about not considering these long range effects as risks that are falling within its own purview. So as a result, many IRBs consider artificial intelligence research to be not eligible for review just outside their purview. Now sometimes IRBs will take a broader lens. Microsoft research in particular has a really wonderful program that takes this wider lens, but many, many yet don't. So what we've been working on is something that we call the ESR or Ethics in Society Review, which is an institutional process in collaboration with what is known as HAI or the Stanford Institute for Human-Centered Artificial Intelligence that is trying to facilitate researchers in mitigating these negative ethical and societal aspects of artificial intelligence research. And the main idea behind it is that it becomes a gate to access funding. That is to say HAI has seed funding that they give out and they withhold the funding for the grants that they want to fund until those projects have completed the ESR process, this process that we've developed. What is this process? Well, it looks a little bit like this. Participants, researchers, PIs submit grants to HAI as they normally would. With one difference, they append a short statement to the end of their grants, describing risks that their research might give rise to as it leaves the lab or becomes policy and then articulates principles for how they might mitigate that risk in their research and show how they're planning to instantiate those principles in the research. So that gets submitted alongside the grant. The Funding Institute, in our case, HAI, does its normal merit review deciding which of these seed grants to fund. They then pass the grants that they wish to fund to the ESR faculty panel. So this faculty panel is quite interdisciplinary in its first instantiation. It included faculty from anthropology, communication, computer science, history, management science, medicine, many others. This panel reads the grants and the statements, provides feedback, suggests additional mitigations, whatever is appropriate. And eventually, as this iteration with the researchers concludes, makes its recommendation a positive recommendation to the funding program to go ahead and release the funding and allow the researchers to get access to it and begin the work. So the ESR at Stanford University, where we created it, has been going for a bit over one year. It reviewed over 40 funded research proposals in its first year. And just to give a sense of how this has been going, all of the Hoffman-E grants, these are the larger multi-year grants, and then 29% of these one-year smaller seed grants were asked to iterate with the ESR. So what happened? What's working? What's not? What's being learned? Just to give a sense of how this works, I'll focus on this case study of one project that was doing stress sensing. So for workers, being able to identify when they're getting stressed out. Now the researchers named concerns in their ESR statements saying that there might be surveillance by employers, by governments as a possible risk, but they sort of stopped there and said, well, that's all we can say. So the panel feedback asked the researchers to come back with specific research principles and designs that will mitigate those risks as the research is done and as it eventually potentially becomes a product. The ESR, a subset of ESR met with the PIs to discuss the feedback. And eventually the researchers came back with a proposed privacy preserving architecture and a commitment to use their bully pulpit as researchers to explain why that architecture was important in papers and in public talks. So we ended up doing a survey of all of the researchers who were involved in these seed grants who intersected with ESR, about two thirds of them got back to us. We did a number of interviews with researchers who responded to the survey. Again, a number of them got back to us. And then we also analyzed what was said by the ESR. So here are a few things we learned. Every single person who engaged with us, this was a surprise to us, was absolutely willing to engage with the ESR process again. No one said that they were not willing to go through this again. And in fact, about over half said that this process had influenced the design of their research. It led to changes in how they actually were going about their research. Keep in mind that these individuals may have done no more than simply writing a statement and getting written feedback from the ESR and still over half of them so that it influenced their research. The main benefits that came out through the interviews was that the ESR served as a forcing function and a commitment mechanism. It was sort of scaffolding that said, oh, you got to think about this and here are some ways to think about it. So PI here in social science said that this requirement led me to engage with my co-PI because as a psychologist, I wasn't aware of some of these ethical implications and it helped me basically have a conversation with them. Many of these interviews said that it raised new issues that they hadn't thought about. So here's a PI in engineering saying that they ended up considering flipping their entire research approach to lead with privacy, which was not their initial approach. And they acknowledged they don't have the answers but it's helped them quite a bit. Main drawback that we heard was still they wanted more scaffolding. Don't just give us an opportunity to reflect on this, tell us how to structure our thoughts. How should we be thinking about this? We kept the statement intentionally brief but they wanted more detail. So this is a representative quote where a PI in the School of Medicine said, tell us how big these issues really are. Give us a sort of Richter scale. It didn't really help us make these decisions about whether these risks were outweighing the positives or not, asking for foundational bedrock things that were in the prompt. It's worth the juices worth the squeeze but it would help scaffold the researcher rather than just having a blank page. So the common themes were that some subgroups who might be harmed in the feedback. Another theme in the feedback was are all the relevant stakeholders included? How might this be reappropriated by motivated actors and who isn't is not in the data? About 80% of proposals got new issues to think about. Generally, people felt that the ESR should be able to recommend that the project not be funded in the worst case but strongly prioritized iteration over that kind of brute force. So that's been our experience in the first year of the ESR. Thank you so much to the UN group for their support and we'd be happy to work with you if you're interested in starting in such a process at your university.