 Good evening. I'm really pleased to welcome you tonight to a conversation between two pioneering scientists about the ethics of scientific discovery. My name is Rob Riesch. I'm a faculty member in the political science department and the faculty director of the Center for Ethics and Society. I also have the privilege of serving as an associate director on the new Institute for Human Centered Artificial Intelligence. I'm really pleased to introduce someone who will come and introduce the event tonight. His name is Mark Tessier-Levin. He's a neuroscientist. He's the former president of Rockefeller University. He was the chief scientific officer at Genentech and he is currently the eleventh president of Stanford University. Please welcome Mark Tessier-Levin. Well, thank you very much, Rob, for that introduction and also thank you very much for your thoughtful leadership and for organizing this important conversation. We often look to technology to improve our lives and in these early years of the 21st century, twin revolutions in biotechnology and in computer science offer enormous promise. Within these two fields, more specifically, innovations in editing the genomes of humans and other organisms on the one hand and in creating machine intelligence on the other hand can confer enormous benefits on humanity. But they also raise urgent ethical questions and questions about their societal impact. Your presence here tonight shows your interest in these issues and also your concern and I want to thank you all for being here. Now, in the next few minutes, I'd like to briefly touch on the promise and peril that accompany innovation, the opportunities for Stanford to make a difference, and the crucial roles of collaboration and leadership play in addressing these concerns. But first, I want to thank Professors Jennifer Doudna and Fei-Fei Li for their conversation tonight. Professor Russ Altman for serving as moderator and the McCoy Family Center for Ethics in Society and Stanford HAI for bringing us all together. Now, the effects of innovation are felt all around the world, but here in Silicon Valley, we're surrounded by many of the sources and drivers of innovation. Experts here develop breakthrough ideas and build on them to create companies, products, and applications that can change lives and communities and even the planet. Stanford itself, of course, is home to many great breakthroughs developed by faculty and current and former students. The pace of that change is rapid, breathtaking, and it's only getting faster. But innovation alone isn't sufficient. Creative disruption doesn't guarantee better outcomes for society or for individuals, and disrupting just for disruption's sake is no honorable activity. As we all know, remarkable opportunities for good can also be misused, along with the good can come malicious actors, unintended consequences, and risks and real harm. Just as Stanford is known for our role in great innovations, we also have a responsibility to consider the societal and ethical dimensions of our work, and we should strive to be equally known for that as well. That's why our new long-range vision for Stanford includes a presidential initiative on ethics, society, and technology. It aims to infuse ethical and societal considerations into technological advances. Now, I understand that tickets for this event went quickly. They were all gone within about half an hour after they became available, believe it or not. That demand reflects the intense interest in how innovations like gene editing and artificial intelligence can radically change how we live our lives. Gene editing and CRISPR are already having a sweeping impact across the life sciences in biological discovery, in biotechnology, in medicine, and in agriculture. AI similarly promises an array of transformational change in the future of work, of education, of medicine, of warfare, and of entire industries and economies. These two technologies have thousands upon thousands of applications and just as many implications. I believe that this potential for broad and rapid impact is at a scale that has rarely been witnessed in human history, and the speed of the changes hastens an already urgent need for discussion and for plans. Now, our speakers tonight, Jennifer Dowden and Fei-Fei Li, are leading scientists who have taken their fields by storm. They are also both equally committed to thinking about the societal and ethical implications of their technologies. Now, I realize it's a bit ironic that the two of them are going to be on stage together this week of all weeks. It's big game week. This is a time when students on this campus are eulogizing the demise of the Cal mascot, and their counterparts in Berkeley are chopping trees. So, I recognize that it's a bit of a contrast to have a Cal professor and a Stanford professor coming together in a calm, polite setting. Friendly rivalries often produce high levels of interest and energy, and an intense focus on winners and losers. But in this fast-paced world of innovation, the definition of winners and losers are much different. When it comes to managing the impacts of these revolutionary technologies, we are all on the same team. Tonight's discussions and the questions that Fei-Fei and Jennifer are raising can help serve as traffic signals, green lights for moving forward, but also awareness when to proceed with caution, and also when to stop. For instance, Jennifer co-organized a discussion in 2015 on the ethics of making changes in the human genome, which led to a call for a moratorium pending further discussion of ethical and safety issues. Last month, in turn, Fei-Fei and her Stanford HAI co-director, John Etchemendi, argued for the need for a national vision for AI because, quote, most of the world isn't prepared to reap its benefits or to mitigate its risks. They wrote that AI is advancing rapidly, but we still have time to get it right if we act now. Now last winter, Jennifer and Fei-Fei attended a dinner that Rob hosted here at Stanford. They compared notes and realized some parallel concerns about the ethical implications of their technologies. Rob asked them to continue their conversation here tonight, and I want to thank them for doing so. I'm proud that Stanford can host discussions like these and be a part of the solutions. I believe we have the opportunity and the responsibility to help lead. Now one component of leadership is convening experts like Jennifer and Fei-Fei. Another is helping identify and developing new leaders. So in closing, I want to say that I'm optimistic about the future, both because of the leadership and commitment of established scholars like Jennifer and Fei-Fei, and also because of the emerging leaders who are building on their work. One of those emerging leaders is Margaret Guo, a current Stanford student who will introduce the members of this panel. Margaret is a Stanford MD-PhD student who bridges the two worlds of our speakers. Her work uses machine learning-based approaches to examine how changes in our DNA contribute to cancer, neuropsychiatric disorders, and other complex diseases. As an undergraduate at MIT, Margaret studied computer science and bioengineering, so you could say she is bilingual. Her achievements also extended beyond the classroom and the lab to the swimming pool. She earned all America honors as a swimmer, and she was named the 2016 NCAA Woman of the Year, an award for student athletes who have distinguished themselves in athletics, academics, leadership, and community service. Her career goal is to become a physician-scientist. So thank you for the contributions you're already making, Margaret, and thanks again to all of you for being here tonight to participate in this timely and vital conversation. Thank you very much. Thank you, President Tessie Levine, for that introduction, and I am honored to be able to introduce our three guests tonight. Professor Jennifer Doudna is a professor of chemistry and molecular and cell biology at UC Berkeley. She is a co-founder and leading figure of what has been dubbed the CRISPR revolution, and her pivotal contributions to this invention of the CRISPR-Cas9 system has not only transformed the landscape of gene editing, but also has fundamentally changed how we discover new biological phenomena. For instance, as President Tessie Levine mentioned, I studied gene regulation, how tiny changes in our DNA makeup can disrupt the delicate balance of what happens in a cell. Before CRISPR, we weren't able to study the genome at this fine level of detail, but now with a tiny pair of quote-unquote molecular scissors, we can experimentally trace the effect of a single-letter mutation and how it can propagate and lead to disease. And as the impact of CRISPR technology has grown, so has its ethical implications, particularly for human genome editing. In recent years, Professor Doudna has led many discussions regarding the ethics of CRISPR to edit human embryos and the potential to control our species' genetic future, and I think that sounds a bit like a sci-fi movie. So our next speaker is no less involved in this broader discussion about the future of sci-fi, I mean of our humanity's future, although she might be a little more worried about robotic takeovers instead. Professor Fei-Fei Li is a professor of computer science here at Stanford University and is also the co-director of the Stanford Institute for Human-Centered Artificial Intelligence, also known by the acronym HAI. During her sabbatical two years ago, she served as vice president at Google as well as chief AI scientist at Google Cloud. Professor Li has been a key contributor in the fields of machine learning, computer vision, and neuroscience. Notably, she is the inventor of ImageNet, which is a widely used benchmarking tool in deep learning and computer vision. In addition to her many technical accolades, she's also a leading voice advocating for diversity in STEM, and for shaping public policy at a national level. Last year, Professor Fei-Fei Li spoke in front of Congress regarding the growing role of AI in our society, and in that hearing, she stated, quote, we like Professor Fei-Fei Li's quotes apparently, there is nothing artificial about AI. It is inspired by people, is created by people, and most importantly, it impacts people. And that statement could well be the theme tonight, as for the first time on this stage, we're bringing together these two trailblazers, Professors Fei-Fei Li and Jennifer Doudna, as we delve into the emerging technologies in society, and what we, as the people, make of it. Tonight's conversation will be moderated by Professor Russ Altman, and Russ is my PhD advisor, so I promise that I have not received any incentive whatsoever to say nice things about him. Professor Altman is the archetype of a multidisciplinary guru. He is a professor of bioengineering, genetics, medicine, and biomedical data science at Stanford University. He is also a practicing physician in internal medicine and clinical informatics. And with this unique lens, he has become a pioneer in the field of biomedical informatics, and in particular, his research focuses on applications of computational technology to medicine and its therapeutic potential. Of note, Professor Altman hosts a podcast titled The Future of Everything, which explores how science, technology, and medicine have transformed the way we live. So tonight, we think about the future of CRISPR and AI and ponder the ethics of discovery. And before we begin, I'd like to take a step back and place these discoveries on a timescale. Not even 10 years ago, we would not have imagined a phrase CRISPRing babies for disease treatment. We would not have imagined that databases of human faces would cause so much geopolitical consternation across the globe. And we may not have imagined a future where doctors may be replaced by robots. With all these possibilities, my future occupation and many others on the line, there is a sense of urgency to ponder and to discuss the broader societal implications of these technologies. So without further ado, please join me in giving a warm welcome to Professors Fei-Fei Li, Jennifer Doudna, and Russ Altman. Well, welcome, and thank you for all the introductions. My name is Russ Altman, as you heard, and I am very excited to be here. I was lucky enough to be at the dinner that Rob hosted where it occurred to us in the middle of this dinner that this was two people in the middle of two revolutions and that the immense joy we had at the discussion 10 months ago really should be shared with the world. So it's incredibly exciting to be here. This reminds me to tell you that there will be Q&A, but it will be very high-tech. On your cards... Excuse me. There's a card that you should have gotten, and on the back of it, there's a website that you will visit with your phone if you have a phone that does this. You will go to sli.do, which is a website, and you will have a password which is CRISPR-A-I-C-R-I-S-P-R-A-I, and you will type in your question, and that will go through the miracle of technology to an iPod that I will be holding, and I will look at those questions and pick some of them. We are trying to make sure that we have a good chunk of time for Q&A because that's always the most fun and it'll be where you can fix all the mistakes I make in questions that I forget to ask. Okay. That will be later. So I'm very excited to have these two pioneers with us tonight, and thank you very much for coming. We have two revolutions that are simultaneously happening. You've heard a little bit about it from Margaret and from the President and from Rob. We don't have time to do a technical briefing on these, so I will just summarize, as you may know, that in the case of Dr. Doudna, we have a discovery that has led to an ability to edit DNA, edit genomes, pretty much for the purposes of today's discussion, at will, and also to use this technology for other very specific addressing inside the genome to do things like turn things on, turn things off, which can be applied to everything from bacteria to humans. There's your technical briefing on that. In the case of Dr. Lee, we have the beginning of a revolution in deep learning and the associated machine learning technologies that are starting to find their way into all aspects of life. They're in the background when you do Google searches, when you get on Facebook. They're in the not so much background when you do Amazon purchases and things are presented to you. If you talk to your phone or if you talk to little cylinders in your kitchen and they talk back, and image recognition applications that are kind of exploding from security to other aspects of commerce. All of this is based on initial findings that came out in the early 2010s and led in a large part by Dr. Lee. So my question, my first question to Fei-Fei, and then we're going to go to Jennifer as well. Was it obvious that the results you were reporting, back when you were reporting them, was going to be such a big deal and lead to a revolution, or was just this a paper like all your other papers, or was there an awareness? Great question. Russ, I guess you're referring to my 2009 paper, an ImageNet and the 2012 ImageNet Challenge result by a group of Canadian scientists led by Professor Jeff Hinton that won the ImageNet classification challenge where computers are able to recognize a thousand different daily objects in everyday imageries. And that was the first time that we see that kind of progress in computer science. I think as a scientist, when we led the ImageNet work and put forward this ImageNet challenge, we knew that we were driving at a very big, what we call holy grail question in computer vision and AI, which is the question of enabling computers to recognize a huge plethora of objects in everyday life. And that's an ability that humans have, but keeping my evolution took 540 million years to achieve that. And we were going after that question. So from that point of view, we knew it was a big question we're trying to answer. And when seeing the result in 2012, it was a big scientific step. And I would be lying to say if I recognized the societal implication and impact in 2012, we knew the scientific significance, but I wasn't aware of the ensuing explosive exponential increase in societies through the commercial sector and others. So Jennifer, for the CRISPR revolution, you often describe it as a curiosity-driven research. This is the immune system of some bacteria, not obviously a revolution. But as these discoveries were coming, was it obvious for you? So I think an interesting difference from what you just described, Fei-Fei, is that it sounds like in your work, you were very much, very actively going after an important goal in computer science. And I think for us, those of us that were working in the world of CRISPR, which is this bacterial immune system, very obscure, before 2012, this was a field that was populated by literally a handful of labs around the world that were studying a very sort of esoteric area of biology. And so in my own work, we were investigating how this worked. I was fascinated with it because of the evolution of a system in bacteria that I thought paralleled something that happens in human and plant cells, and I was trying to understand those distinctions. But for us, when we did this curiosity-driven project that led to an understanding of how this system could be harnessed to trigger changes in DNA, and we realized that it could work in any type of DNA, including in genomes, that for us was kind of the light bulb going off and saying, oh, this project is clearly going in a very different direction than where it started. And for me, did I realize initially that it would be a big deal? I would say that it was very clear from the work going on in the genome engineering field in general, which was in existence before the CRISPR technology, that this was an important thing to be able to do to manipulate genomes. It was just that it was very difficult to do it up until that point. And I think for us, the really kind of exciting connection was realizing that this really esoteric thing we were studying was merging with a very important area of biotechnology in a way that would be broadly enabling. And that's, you know, so that was true. Now, could I have predicted CRISPR babies and everything at that point? No, for sure not. But I think it was clear to all of us that we're working on this, that this was really an exciting breakthrough in biotechnology. So I just love this. But one of the things that you both have said is you knew it was big, but the degree to which big is big and then there's really big, and the degree to which it would take off might not have been clear. One aspect of how it took off is the fact that we're hosted today by a Center for Ethics. Is the ethical implications of the technology. So maybe starting with you, Jennifer, did you have a moment where you said, holy cow, this is now getting into things that I wasn't expecting in terms of ethics? You were a bench biochemist and now there's these amazing ethical questions. How did that come to your attention and did you hesitate to engage? Okay, so first of all, you're absolutely right that, you know, I'm a biochemist and somebody who's always done very fundamental research and for me, I think it was really, what happened in the field of CRISPR and genome editing was that, you know, we published a paper in the middle of 2012 describing how the system could be used for genome editing and before the end of that year there were already several academic papers that were submitted to journals that were describing how it could be actually used in practice to engineer genomes and then throughout 2013, you know, it was clear that this was just taking off incredibly quickly and by early 2014, there were already scientists who were using CRISPR to engineer the genomes of animals including monkeys in the germline, so making heritable changes in monkeys and even though before that I had sort of occasionally thought to myself, gee, I wonder if this could be used in humans and in human embryos, that I think it was the monkey work when that was published in early 2014 when I thought, holy smokes, you know, there's just no reason to think that people maybe even right now are already starting to do experiments of that type and I think that was really for me the moment when I thought I need to get involved even though and I was quite reluctant, I have to say. I felt like this is an area I don't know anything about but I felt a real responsibility to engage in that discussion and start trying to put the word out that this is happening, the technology is moving very, very quickly. Regulatory agencies, governments are not even aware of this yet but in the science world this is a big deal. So I want to hear your story about this Fei-Fei but let me mention you've already heard something very interesting that I'd just like to highlight which is Fei-Fei mentioned the paper in 2012 and Jennifer just mentioned the paper in 2012 and I think when we went back, these two papers occurred within a couple of months of one another in totally different literatures, in totally different journals but this 2012 I think was fall or late summer, that's a big moment in this whole story and it's an amazing coincidence that we're within weeks of these papers being published. So Fei-Fei, was there a moment or is it less momentous when you said, uh-oh, this is ethical and there's ethical issues and do I want to get involved? Yeah, so ten months ago when Jennifer and I had this private dinner conversation at Stanford, I heard a summary of what you just said and the thing that struck me the most is the striking parallelism in timing that the moment 2012 started the first paper in Deep Learning Revolution. I still remember that fall I flew to Florence, Italy to announce the ImageNet Challenge result I think it's in September and the paper came out in December and starting that point it was a parallel story by 2013 the Deep Learning Technology is taking the academic world of computer science and AI by storm you're getting just many paper submissions and publications in computer vision in natural language processing, speech recognition and also possibly a small probably not so small difference between biology and AI is by 2013 the commercial world, the tech companies are moving fast into Deep Learning we're starting to hear Deep Learning startups already getting acquired by Big Tech and by 2014 which early 2014 the moment that kind of hit me by surprise is suddenly in the public media we're hearing people talking about AI as summoning the demon and I remember Bill Gates Elon Musk, Steve Hawkins all went into public media and start to talk about the amazing concern about AI and the evidence was this new Revolution of Deep Learning which is really incredible to me because probably less than 12 months ago I felt my whole career was in a field that nobody cared about and just within a year suddenly our field is the demon that has been summoned and I started really thinking deeply about the connection between humanity's future and the field that I went in as a scientific curiosity and 2014 was a defining year personally for me to get into this just like Jennifer said it was reluctant from the personality point of view I wasn't looking for that angle but it's really that sense of responsibility that I was part of this generation part of this group of scientists that brought this technology to this point and now it has a serious impact on this future what should I do and it was that kind of realization similar to yours that brought me into this So these two revolutions I don't think I'm putting too fine a point on it to say they both really put the nature of humanity and fundamental questions about humanity in the focus in the case of CRISPR we now have the ability to modify ourselves I'm not saying we should or will but this is a capability to re-engineer to reprogram the human genome In the case of AI we have an ability to have machines that perform as well or better than us at tasks that were traditionally considered uniquely human tasks of recognizing faces and many others So these really go to the question of the future of humanity and choices that we need to make So you both said at some time in 2014 you both had this very parallel realization that this was more than about technology and you became in fact in some ways public intellectuals it's an old fashioned term but this is what you are now and my question is how did you prepare or what prepared you to engage in a set of conversations that were entirely outside of your formal training I presume were mostly outside of your formal training and how did you even think about engaging in these new audiences and new conversations Jennifer There's an easy question Well in my case I had a lot of hesitation I have to be frank Like you Fei-Fei I felt a personal sense of responsibility on the one hand on the other hand I thought there are people who do this professionally and maybe we should leave it to them to think about this and I really felt deeply that the scientists who are involved in fundamental research of the type that we had done with CRISPR needed to engage because in the end who else really deeply understands the science So I actually the first step that I took was to convene a meeting in the Napa Valley with scientists who had been involved in our field a very important meeting that was held back in the mid 1970s called the Asilomar meeting which was at Asilomar and was held to discuss something called molecular cloning which was back at what many people considered to be kind of the birth of modern biotechnology was when people had figured out in the 1970s and a lot of the fundamental work was actually done here at Stanford with Cohen and Boyer and others that you could also made some contributions Thank you for pointing that out I appreciate that I'm wearing Stanford colors tonight but you know it was in the 70s when this happened it was an exciting time when people had figured out how to cut and paste pieces of DNA and that triggered a big revolution in biotechnology a lot of early companies in this field like Genentech and others that got started around that time but people at that time also recognized that there were some real or at least perceived risks to the technology and that Asilomar meeting addressed those risks and it was kind of the idea of scientists coming together and trying to ask can we police ourselves can we control our own technology that we're developing so in that same kind of spirit we organized a meeting in early 2015 with a couple of the scientists that had been at the Asilomar conference Paul Berg from Stanford and David Baltimore from Caltech and then a number of other folks some of them are here tonight and it was that for me was the beginning of my own education really about how you even approach a complicated question about technology where it's going how do we educate people how do we educate regulatory agencies how do we think about it ourselves about where we would like to see this going what role should scientists be playing in all of this so that was hugely helpful and frankly it's I've been on a trajectory ever since I feel like a student again in many ways of kind of learning how to think about this and how to approach it So this is a constant part of your life now This is a part and you consider it part of your job So as you realize these challenges how did you manage it Feifei So first of all I totally agree with Jennifer it's a constant part of my learning I still feel like a student in the topic of ethics and you know machine learning fairness and all this In 2014 we just talked about this personal realization that that I feel that sense of responsibility What is slightly different from the world of biology is that CS is a much younger discipline that does not have an ethics sub area at all and I didn't know who to talk to there was it was probably the lack of my own knowledge but I don't know who was doing CS ethics or AI ethics just nobody talks about this So what I kind of gravitated towards was a very important topic both personal to me but also connected me to the humanity issues of AI was diversity inclusion because when I hear these people on radios or podcasts or TV talking about summoning the demon and terminators coming next door I was thinking I was really asking myself what is the fundamental issue here is they're worried about the drivers of this technology going against humanity and creating that kind of adversarial creature or piece of technology that would be harmful for humanity or society and as soon as I landed on the question who are the drivers of AI I realized the real problem in 2014 is we have a serious crisis in the representation, human representation of this technology and that's the lack of women lack of underrepresented minority and 2014 around summertime a PhD student at that time in my lab her name is Olga Rusokovsky it was her last year of PhD and she came to me and said she was bothered by this problem of lack of human representation and she wanted to do something together so we decided that we are going to pilot a summer program at Stanford AI lab to invite high school students from underrepresented backgrounds mostly women to come and study AI for a few weeks and invite them into this technology through a human centered perspective and encourage these underrepresented students to become tomorrow's leaders fast forward 2015 similar to your NAPA meeting was the first summer camp at Stanford for what now we call AI for all program and that was the first class and then we did it again in 2016 and the reception of that program was just so so positive and we realized we have to scale this nationally so 2017 we established a national nonprofit called AI for all and start to putting out these summer programs for to invite underrepresented students underserved community to take part in AI study and research and by 2019 this summer we have 11 campuses or 11 programs in North America we've graduated more than 500 alums of young women racial minority low income class students rural students and we're still scaling that journey to increase the human representation diversity inclusion of AI was the starting point of my own personal journey in advocating for ethics ethical guideline and treatment of AI it's a fascinating and really almost brilliant idea that the way to start this is to create a bigger tent and then to let the other issues play out with a group of people who are more representative of the society at large and it doesn't seem like that's happening are the voices being heard are those young people are they considering this part of their mission going forward so Russ the truth is I can sit here for hours to talk about the amazing students and alumni of AI for all we've been doing this for four years so the first class coming out of Stanford are now freshmen in college and we have some incredible young people now advocating for AI ethics machine learning machine learning fairness AI policy so the mission of AI for all was really about making sure that tomorrow's technologist and AI thought leaders are much more diverse and thinking through the human lens but I think what also happened along the way and it's not just my own effort is that the world of AI especially academic AI is starting to wake up to this to this important technology and its ethical implications so now the talk about an efforts in AI ethics goes beyond diversity and inclusion it includes many different angles now so I want to move to a topic that I think is actually quite related but it will sound different which is the industry take up of your technologies I think both of you have seen a very rapid take up and I'd like you to just characterize your impression of how that's going does it raise concern or is it rewarding and reassuring what's the status of CRISPR in industry and are the roles of academics and industry clear are they competitive how are things playing out and are you happy with what you're seeing right so in the world of CRISPR so in 2013 there were already a number of companies that were getting going and investors that were starting to sniff around and sense that there was something big happening and so there were several companies that got launched that year I think by around the end of 2013 and today there are three of those companies have gone public so they are now publicly traded and one of them actually made a big announcement today about having actually used CRISPR Cas9 in two patients that have blood disorders, sickle cell disease and thalassemia and reporting some exciting results granted it's two patients but you know I think that sort of is a testament to the very interesting very rapid advances that are being made with this technology already in the clinics are already seeing benefits to patients this is not in the germline this is treating individuals but I think it at least you know looks so far looks like it bodes well just for those of you who are not biologists germline modification would be one that would then be passed on to your children and their children so it would be fixed or at least in the human population in your offsprings it's also possible to make changes to the DNA that's local to that one person but will not be passed on to their progeny and so that's a big difference in both ethical and technical circles yes yeah absolutely and I should be clear that the vast majority of companies that I'm aware of right now that are using CRISPR are using it for you know making changes in individuals but not in the germline not for heritable changes and then a couple other comments on what's happening commercially so as you alluded to in the beginning your technical intro CRISPR is a technology that is broadly impacting all of the biological sciences really and so it's not just in biomedicine but it's also impacting agriculture it's impacting synthetic biology and companies that make reagents and all that sort of thing so I would I think that if you were to look at many established companies that work in the fields of agriculture and these other areas that affect biology a lot of them are using CRISPR technology now as a tool as something they use in the course of their product development so that's going on and then in the area of advertising there's very frankly commercial interest in this I think that companies that are involved in egg freezing in reproductive technologies in vitro fertilization are certainly paying attention to this that's my perception based on the kinds of phone calls and emails that I get from entrepreneurs who are thinking about that space and I think that will remain a commercial interest going forward it's very hard for me to know right now how quickly that area will develop and certainly I maybe will discuss it here in this conversation but you know it's an area that is there's a lot of of issues technical but of course also ethical when we start talking about making heritable changes and especially selling that to people making claims about it and charging money for it you know these are changes that affect you know could affect people for you know their whole sort of family structure in generations is the academic is there a good equilibrium between what's going on in industry and what's going on in academics I'm kind of anticipating some comments that they may have is there any problem with for example brain drain from academia do we still have a good ecosystem or has there been a tilt that's worrisome I think there's still a very good ecosystem between what happens academically and research that goes on there and what happens in companies however I've noticed just in you know this is very statistically small numbers but you know I've been running an academic research lab for 25 years the vast majority of students that I've trained in my lab in that period of time have gone in the past at least have gone the academic route largely a few of them have gone in the industry in the past but a lot of them stayed in academic science in different ways and just over the last two or three years I've seen a very interesting change where my very best students and actually last year I had five students graduate with their PhDs from my lab in the end all five went to startup companies and all five are doing CRISPR related work but in startup companies and I think that says something it's small numbers but it does say that some of the very most talented people that are coming out right now that see the potential of this technology feel like the best way that they can do creative work in this space is to do it in a startup company that's got a lot of capital coming in a lot of smart people that are you know focused on solving a problem and they've got a technology to do it and they just want to get in there and get it done and in some ways that they feel that biotech is a better place to do that than in an academic setting yeah so Fei-Fei I imagine you might have things to contribute on this issue what is your sense of the equilibrium between academics and industry what kind of industry where are we I suppose industry has heard of the word AI by now so I feel like the world before 2012 and after 2012 especially after 2016 after AlphaGo is just turned upside down for academia in many ways because I've also been an academia professor for almost 20 years actually no 15 years and plus PhD 20 years and most of my professional life and scientific career this is just about scientific curiosity companies are doing similar things but by and large we're doing basic science research I feel like some of the problems that my lab was working on before 2012 industry was not even interested in granted image that paper in 2009 was a poster a small poster in a conference that most people didn't pay attention to so and then deep learning revolution came and within month the onslaught of industry attention just took the whole world AI our world by surprise especially academic world now no matter what metric you look at whether it's the amount of investment by big companies in machine learning, deep learning AI or the number of startups peer reviewed academic papers published by industry labs or the number of students going to startup an industry career versus faculty career all this metric is showing the intense explosive interest investment of industry in the area of AI and added on top of that this technology deeply depend on two important things one is compute which means the chips, the GPUs the CPUs, the machines and the other one is data and both elements of AI are now more or less by large in the hands of companies that have the money to buy the compute or create the compute and have the data so the world of AI research and R&D is definitely now heavily tilted towards industry but having said that I mean we're in Stanford and Russ we work together in HAI and just Stanford AI labs there is amazing amount of work happening also in academia and frankly I think many interesting work and forward looking work that is not necessarily of immediate commercial value are happening in academia that I feel very excited about but I think the reality is that the commercial world has just really swept this whole change the landscape of AI. So often when people talk about academic research and industrial research in the past the sense was industrial research is a shorter timeline with deliverables because they're on a quarterly reporting schedule whereas academic research would be a longer horizon is that still a thing in each of your fields or are those boundaries being blurred? Yes I think in AI the boundary is being blurred because some of the companies especially tech industry recognize the importance of this technology so the leadership is willing to invest in longer term research within industry and willing to invest both in terms of timeline as well as freedom and that enables researchers in industry to work on more longer horizon problems that used to be traditionally academia and that really blurs the line of course I still say that those of us working in academia do not have product pressure we do not think so much about commercial endpoints so we still have that freedom but I think by in large the line has blurred a lot in AI So Jennifer in the crisper industry world is the great activity and I mean great in magnitude activity that's happening is that leading to any issues to bottling up of IP and potential long term problems with academics and new discoveries being in some way frozen or more difficult because of intellectual property landscape Well in the crisper world of course there's a very public very famous you know patent fight that's ongoing that's kind of always going on in the background but the interesting thing is that it really hasn't stymied you know I would say scientific advances companies have gotten started I hear about new companies virtually every day in the space and you know lots of billions of dollars in capital have been invested in crisper commercially so that's going on of course academic labs are carrying on and they're not getting seasoned assist orders from companies saying you're in in general you're violating our patents you're violating our patents because that would be a chilling I would guess that would be very chilling correct and I think that could happen in the future you could imagine if there are exciting products that come out of crisper as people anticipate there will be that will be the kind of thing that could happen in the future but currently it's at the level of research so I think for the most part you know whether research is going on in companies or academically and I think in our field what really happened and I'm sure this is true it sounds very parallel to what you're describing that things started off very quiet but they took off kind of exponentially and that's absolutely what happened with crisper and so what was a very sleepy field with just a handful of people working a few years ago is now hotly competitive lots and lots of people working you can't open up a journal these days about reading a crisper paper and you know and I think that there's a whether you're working in an academic lab or whether you're working in a company there's a feeling of I gotta get moving because other people are out there working and if I've had an idea probably somebody else has had the same idea so that very much is sort of agnostic to whether it's academic or commercial but I do think that and I'd be curious how you think about IP in the AI space but certainly in the world of crisper there's this sort of fight over the foundational IP but then beyond that so many companies and so many academic labs I mean you can't imagine it's probably thousands and thousands of patents maybe it's tens of thousands now that have been filed on ideas that have come out of crisper and things that people are doing whether it's new technologies or whether it's applications of crisper and so that's just driven this huge kind of you know layer of intellectual property that's been developed and what will be interesting to see and we're just kind of living through it right now is how that will play out in the future as all of that leads to actual products that come to market and have real value and then you know how those patents will sort out it's not clear yet how that'll happen. So it sounds like a good story of everybody's kind of publishing and there's right now a sense of not too many barriers to discovery and innovation how is it in AI is the great company presence in AI is that are they publishing are they making their algorithms open or is there a sense that there is a certain bottling up of things and lack of transparency. Great question Ross and Jennifer you mentioned IP so the world of computer science is a little different at least just from my perspective that IP has not been at the forefront of the world I live in because first of all we open source a lot of our codes our data our papers as soon as it's written before peer review these days we uploaded our archive so and even the company research groups tend to do that of course so from that point of view unless I'm naive I don't think we're going to find some company calling us and say you infringed on our IP that kind of bottleneck we're not facing in terms of so again like I said they do have in industry researching AI the resource the compute resource and the data resource and the engineering resource are significantly more than academia so that differentiates it differentiates the kind of research the differentiates the kind of projects for example AlphaGo everybody knows about it is an engineering feat that it's much harder to imagine that academia would take on the amount of just engineering of that system that it's much more likely and indeed it happened in industry so that's a little different but what they do want to say is that one thing I find very hopeful and participating and also observing with a lot of curiosity is that more and more of my colleagues and including myself because of this intense industry pressure of their research we're differentiating in a different way academic AI research right now is moving into a much more cross-disciplinary fashion that would like my personal research I work with the doctors in Stanford hospital ethicists and law scholars or behavioral psychologists in a way that is harder to be done in industry and it takes full advantage of the academic exactly and I'm starting to see that at least at Stanford as well as at Cal we see that happening and I'm actually I'm actually very very excited by that movement in academia so we're getting very close to the time when I want to get to your question so maybe I'll end with this kind of general question which is revolutions such as the ones that you've helped lead will happen in the future again from your experience over the last I guess seven years how has society set up to respond to them have we learned things or do you believe that there are things we should be doing to prepare for the next kind of inevitable wave of innovation since you've been in the tempest I'm wondering what advice if any you would give policy makers or governmental officials ways to manage these explosive innovative technologies it's a big question I recognize well I guess two aspects first I think we haven't seen the ending of the AI story at all it's just the beginning and I think we need to recognize that as we push forward this technology we've got to work just as hard or harder on putting the boundaries the ethical guidelines into this the development and deployment of this technology if there is one piece of not advice but wish for thinking I can give to policy makers or just leaders of our civil society at least from technology perspective I would still come back to invest in the people investing the diverse inclusive group of people who should be together at the driver's seat of any technology and discovery because by investing in that most diverse group of people we ensure that human representation is maximally there and whatever we make together collectively is in the interest of the maximum quality I would just add to that I would like to see more scientists in congress in government I mean it's very interesting thank you yes I was really struck when I went to the first time that I went to Capitol Hill to talk about CRISPR technology that I met with Bill Foster who pointed out that he was he pointed out the same thing to me when I went to God he's very proud of it you know but this is a problem so I think we really need to advocate for more diversity in our representatives including some scientists I can't help but think that you probably live in different districts and I'm thinking well okay so very good so I want to remind you SLI.DO with a event code C-R-I-S-P-R CRISPR-A-I all one word is where the questions can come in I'm now going to go check to see if there are any questions there are whoa what's the biggest criticism facing CRISPR today and how do you answer it I guess it's the one that you hear the most well let's see the biggest criticism of CRISPR I guess the biggest criticism CRISPR babies have gotten a lot of attention in the media and I guess the thing that I hear about the most is I don't know if it's a criticism it's really a question about that but the flip side of it is people who write to me I virtually every day I get at least one email a day from somebody who has a genetic disease themselves or genetic disease in their family and a lot of these are I'm really honored that people share their personal stories with me it's really touching and it's not a criticism but it's just a desperation really that's being expressed it's how do we accelerate the promise of this technology so that it can actually help my child at the time that they need it or help my husband or help my mother and so and I feel this very personally myself because I feel this in addition to the as academics we always feel there's competition in the field and I've got my papers published but I feel it very differently now I feel like patients are waiting patients are waiting and they can't wait very long so we really need to get a move on and get this technology to a point where it can really really help people so I feel that urgency all the time this one is I think for Fei-Fei although we can all chime in there's concern about the future of deep fake technology so these are the very realistic but fraudulent is one word simulations of people speaking or their faces what problems do you anticipate and how can they be mitigated yeah so deep fake is like Russ said is okay let me take a step back creating imageries and videos and documents that are not real has always been there right we can even Photoshop fake pictures but the recent AI advancement especially a particular family of algorithm has really accelerated and lower the bar of creating fake imageries and speeches and videos and already there's a lot of concern about this information and how that not only participating manipulating public minds and manipulating people as well as influencing the process of democracy and even more severe the severe consequences especially if this technology is being used and exploited by both nation states but non-nation states actors so mitigating you know I come to law we just like any technology there should be laws to regulate to govern the malpractices of any piece of technology or tools and to incentivize good practices so already there is a lot of conversations and work starting to be done at the intersection between AI and regulatory policies and laws and I think we need to you know speaking of urgency we need to accelerate that and make sure that's happening and I know from your work in HAI that you've had boot camps for congressional staffers to try to increase the chances that they write draft laws that are helpful and not harmful to everybody. Yeah we have boot camps for staffers we also have now at Stanford cross-listed classes between law school and CS and multiple ones looking at this in different angles. The next question is a little bit of a palette cleanser it says does the fact that Dr. Doudna's name can be read as do you DNA suggests does that suggest that we are living in an AI simulation where the AI coders have a very good sense of humor are there any comments? If I had a nickel for every time someone's pointed out that about my name I'd be a returner there's a question I was going to say from anonymous but these are all anonymous but it kind of goes to a question that a comment that he made how do we encourage students and faculty to undertake the kind of interdisciplinary work that will be needed to kind of address the ethical and societal challenges of both these technologies and encouraging them to avoid being stuck in an academic silo? Yeah well I could take a stab at that so I was actually fascinated with what you just said about how your own work has evolved it sounded like from being focused more narrowly initially to now being broad enough that you're spanning many of the schools represented disciplines here at Stanford that go well beyond science I think that's extremely interesting my own work I would say hasn't quite to that extent sort of been that cross-cutting but I think it could be in the future and it probably should be and so what I did a few years ago started the Innovative Genomics Institute in the Bay Area which is right now a UC partnership between Berkeley and UCSF and that institute in addition to fostering applications of genome editing is really focused around the future of the technology very broadly speaking so thinking about the legal aspects of it the ethical aspects of it the educational opportunities and challenges of genome editing so increasingly we use that institute as a way to do this to try to encourage people to come in we have a lecture series that isn't like a typical science department's lecture series where we have lawyers speaking we have people in sociology that are speaking we have people business people it is pretty broad so I think that's what I'm trying to do to spur that kind of interaction yeah so I guess I can speak both personal and also about the work at HAI so personally I just always find interdisciplinary work most fun even in my earlier days as a scientist crossing neuroscience and machine learning computer vision is where the frontier of discovery is and nowadays I find that if we want to really work on questions that matter you know either to people's lives or to the implication of the technology it naturally lends itself to the interdisciplinary boundaries and also just like you Jennifer I think at Stanford we sense so much appetite and demand by our faculty and students to be integrating in different ways that were traditionally not happening so this is why at HAI we're crossing ethics history, law design thinking, medicine education, sustainability and just in less than half a year of our launch we're seeing incredible research projects and courses and affinity groups and dialogues happening seems like this is this is what the moment calls for now I just want to thank you all I have like nine screens of questions and I also want to apologize that we will not get all of them but I will try here's one in a pluralistic world and I think a lot of people worry about this in a pluralistic world that is rapidly changing and highly decentralized why should we expect any alignment on ethics around the use of CRISPR or AI and I think people are thinking about local, regional international differences in how people think about priorities and so is how do we think about this pluralistic world and the chances of having a or should we think about a uniform ethical approach to these technologies no well we talked about this a little bit dinner before we came in here yeah it's a really great question I actually think about this a lot I think that and actually I was talking about it with President Tessie Levine the same issue because we were talking about the fact that at least in our field in biology as I alluded to earlier in the 1970s there was a meeting in California the Asilomar meeting that convened scientists to think about how to control the science of molecular cloning that really did set an example of how a scientific community could come together to try to police itself with the use of the technology and it wasn't perfect but it was an interesting example of how that could happen and I think it set a really good precedent for the things that happened later in the biological sciences including CRISPR however now so that was in 1975 now in the mid-2000s the world of science has really changed science is much more global than it was at that time there are many more scientists than there were at that time technology like CRISPR can move around the globe incredibly quickly for a variety of reasons it's really been enabling in a way that made the technology advance very very rapidly with all the different labs that got involved with it quickly but that also means that I think it is really challenging to think how you could build the same kind of community-wide consensus around how it should be utilized and I think I'm not sure how to do it I guess my hope is there will be enough work internationally by highly respected scientists that we can at least have a, if not a global consensus, at least a global framework that sets a set of principles that will become the basis for regulatory agencies globally to act but whether that will really happen remains to be seen I agree, I think pluralism doesn't mean we shouldn't cooperate, doesn't mean we shouldn't listen in respect to the differences either it's between individuals or groups or countries I think when Jennifer and I had a conversation a few days ago we actually talked about what would be our magic one to wish in the world of CRISPR and the AI and both of us pretty much said exactly the same thing if there were a magic one then there would be an international framework where different groups can come together to agree on certain important things of principles of CRISPR and AI but also agree to cooperate in different ways that matters for their own different groups so kind of related to that there are a ton of questions and that's why I'm going to be asking it because the idea of every technology has a dual use that is to say the beneficial use that many times the inventors think of and then the nefarious uses that other people think of or maybe the inventor as well and so I'm getting a lot of buzz here about warfare scenarios dual use for both CRISPR, bioterrorism as well as AI in the sense of weapons and automatic weapons it's almost unfair but how do you get yourself around and do you have any thoughts about this dual use issue and has it impacted you in significant ways and do you worry about it so my undergraduate degree is physics and inevitably I think about the nuclear physics and Manhattan projects and all that so you're right that every technology is a dual use is potentially can be dual use I think this is why we need to talk about it this is why we need to have ethical principles this is why we need to have laws international framework so it's even more important that given how powerful these technologies are that we anticipate these potential usages and dangers and try to get ahead of ourselves as a society as a country as a species and it speaks directly to both of your previous comments about an international framework would be very helpful in that setting I would guess yeah absolutely there's I don't know if you wanted to add to that go ahead so there's also a lot of questions about do our young scientists and young engineers do they get the training they need in ethics for the future and if so good and if not what needs to happen they're just a young scientist a young engineer they're trying to acquire technical skills they're trying to become virtuosos at their field they're working very hard it is not easy to get them to put their head up and say I would like you to think about the ethical implications they say yeah yeah yeah I'll do that at the end of later when I have a big discovery like these folks then I'll start thinking about it is there a move towards changing the way we teach ethics in professional education so I want to share with you one moment and then answer the question so I took a sabbatical at Google in 2017 and then went on to the larger half of 2018 2018 was exactly the year many people called the tech lash happened and I remember in the room with a group of very young Google engineers as some of them and when we were talking about the adversarial impact of technology and AI and I literally the most memorable moment was two young engineers one woman was looking at me because I was the leader in that group almost in tears and said to me Feifei we have great we are PhDs from top universities we have great education in computer science I know machine learning inside out I can code TensorFlow but when my friends and my family talk about these issues you know where there is fairness or weaponization or privacy and I see the new stories of the technology that we could potentially create can harm people I don't even know where to begin to think about this I don't even have a language in my head that I can just even speak of it I had zero classes or zero training in this yet now I'm a privileged engineer software engineer machine learning engineer machine learning scientist in tech industry creating this and I really felt that moment struck me so vividly and I recognize that I was towards the end of my sabbatical so I was returning to Stanford I really speak to that sense of mission I had returning to academia as an educator that we absolutely need to prepare the current and the next generation so that no one in five ten years or even two years would be in the same situation and look at me in tears and not knowing what they can do about it because they had zero preparation and here I right after I returned to Stanford I really want to say that I just watch in admiration and give a lot of credits to my colleagues like Rob Rich Miran Sahami Jeremy Weinstein and Tino Quayer many of my colleagues across Stanford campus from ethics from law from computer science are coming together and start to teach classes like this Rob your class had 300 students sign up last time it's offered and so the short answer is it's not enough but it's happening and we need this to be happening in every single university and college campuses and possibly even earlier part of our K-12 education I would say in my field in biology we're probably not as far along oddly enough as maybe you are in terms of education about ethical impacts of technology this is just my personal perception of it because what I see at Berkeley where I teach a very large undergraduate class in molecular biology for sophomore level undergraduate students and we don't get into ethics really at all we focus on we talk about CRISPR but we talk about it from the science perspective of it and all the other developments that have happened in biology we're talking about the science and trying to educate students about that but not about the broader impacts of the science and how we think about it and partly it's just because we have a packed curriculum there's a ton of stuff to teach but I think it also is just a cultural thing in our field that we're still very much in the vein of creating scholars who deeply know their subject the science and subject to find pretty narrowly I would say rather than saying we need to create people who educate people who are going to be knowledgeable broadly not only about the details of the science but also the impact of the science and I'm not quite sure how to do that and also you kind of alluded to this but what I see also is that a lot of the students that come into our field you know kind of don't necessarily feel drawn to the bioethical questions and they feel like yeah I'll worry about that later if I do something that has ethical implications I'll worry about it then but right now you know I want to get my PhD published my next paper or something and I don't want to take time away from that to think about these these bigger things so there are a lot of questions on a lot of topics but I think as we end in the last four minutes there's a lot of questions about equality and justice they're not all positive but I want to ask you tell me the dream scenario for how each of your technologies will lead to improved justice and improved equality and fairness in human endeavors because I would prefer to end that way there's a lot of more negative implication in some of the questions so in AI my dream scenario right so on equity and justice this is a technology that can to start with can call out justice it has enough powerful algorithm to detect the pattern of bias pattern of injustice and pattern of unfair whatever treatment of people or situations so that's one positive usage this is also a technology that can lower the barrier of entry and lower the barrier or cost it can make medicine more accessible it can improve access of education medicine, transportation potentially financial resources this is a technology that can improve productivity by augmenting people not replacing people and my dream industry for AI to help is government you're scared so but it also is a technology that can really help us humans to tackle global issues like climate change environment like medicine education so these are the I can go on and on but these are dream scenarios but they're not necessarily just daydreams we can work towards that and I know many of my colleagues are working towards that at dinner we had a great discussion about publicly available data from the government that's not particularly sensitive traffic you pointed out this is where the academics and other public interest groups can do work on this data and it's freely available and it won't be bottled up by companies necessarily and so it represents truly a great opportunity so dream scenarios for CRISPR so CRISPR I think there's two sides of the CRISPR coin in a way so on the one hand CRISPR is a truly democratizing technology that's one of the reasons why it has taken off as fast as it has it puts a powerful tool in the hands of anyone who has a little bit of knowledge of molecular biology for better or worse but it's meant that this is one of the reasons it took off the way it did and why we're on this trajectory is that labs around the world have been able to get a hold of it and use it to do research on essentially any biological system and get information and start sort of driving forward their research programs at a pace that wasn't possible previously so that I think has really been a very positive thing for the most part the flip side is that when you start to think about the applications of CRISPR and let's just think about biomedicine with the announcement today for example about being able to use it to imprinciple cure sickle cell disease I mean this is phenomenal but it's not going to do a lot of people a lot of good if that technology if that cure costs two million dollars a patient so what I am very... I mean it's crazy and so we're really and this is one thing that I think academics can really contribute to this is to think hard about how are we going to take this very exciting development and the way that it could be deployed biomedically but do it in a sustainable affordable way I think it will require more technology development but maybe things that companies won't want to invest in because they will take a little bit longer to do it so I think that's one thing that has to be addressed and then we didn't talk about it tonight but I'll just throw out there that I think one of the biggest impacts maybe the biggest ultimately impact of CRISPR will be in agriculture it will be in the food that we're eating and I think that again has the potential to be widely democratizing if handled appropriately and that will happen both in academic labs but also in companies over time well I think we're at time I want to thank you both for both your scientific contributions your contributions to the ethical and societal discussions of these technologies your time today I want to thank our hosts our host organizations and I want to thank all of you and your 90 questions which didn't go answered I thank you to all