 Okay, well, welcome to our consortium series on Ethics in Research and Biotechnology. I'm your host, Nsu Hyeon. I am the director of research ethics and a faculty member at Harvard Medical Schools Center for Bioethics. And I'm the director of the Center for Life Sciences and Public Learning at the Museum of Science in Boston. As many of you know, who've been following this series, we try to bring together both the science and ethics of emerging biotechnologies and emerging issues in research ethics in general. And many of you already know that the topics we've covered, such as human genome editing, stem cell-based embryo modeling, even our last session on solar geoengineering, all scream for the need for public deliberation, public participation, public engagement. How best to do that? Well, I decided it would be a good idea to include into the series this topic of deliberative polling and bioethical controversies. So let me just go over some of the ground rules before I introduce our guest. As many of you already know, this is the interactive session. So we're going to have a Q&A portion at the end of Dr. Fischer's presentation. During the presentation, if questions come up, please use the Q&A box at the bottom of your Zoom screen, enter your question. Only use the chat function if there's a technical issue you need to get in touch with an administrator. Okay, so the question is going to the Q&A box. After the presentation, I will then pose questions to our speaker and we can then have a back and forth with the audience. So let me introduce our speaker for today. Dr. James Fishkin is joining us from Stanford University where he's a professor of communication, professor of political science by courtesy and the director of the Center for Deliberative Democracy. He's a fellow of the American Academy of Arts and Sciences, a Guggenheim fellow, a fellow of the Center for Advanced Study in the Behavioral Sciences at Stanford and a visiting fellow commoner at Trinity College, Cambridge. James received his PhD in political science from Yale and as well as the second PhD in philosophy from Cambridge University. Ever the underachiever, he's only the author of four books, five books. His books are Democracy When People Are Thinking, When the People Speak, Deliberation Day with Bruce Ackerman and Democracy and Deliberation. He's best known for developing the concept and the practice of deliberative polling, which we'll discuss with us now. So please welcome James Fishkin. James, it's great to have you. The floor is yours. Well, fantastic. Let me pause for a second while I share my screen. And, where is it? There it is. And, oh no, I'm beginning. Sorry about that. I wanna make sure that I have the sound. So I think you wanna end your show and then click on your first slide and start from there. Yes, you're right. Let me do that. I should say that those are my books about deliberation. I had, the first half of my career was about ethical and political theory on other issues. So, but once I thought of this deliberative polling, I could think of nothing else. And so, oh no, that's the end. Okay, that's not gonna be good. Sorry, I'm missing something. Well, now you've got a quick tour. That's the, we do Shakespeare comedy version of the presentation. Okay. So, oh, but I haven't shared my screen, have I? No. I really messed this up. All right, let me just get back to the Zoom. I will, share my screen. And start the show from there. Yes, okay. There you go. I will set. And I noticed that 15 people have joined during the time that I messed up the technology. I'm from Stanford. I'm supposed to be master of this technology. But- People are giving obsessive deliberative democracy, so that's- Yes. So thank you for the kind introduction. I'm delighted to be here. I am going to talk about methods of public consultation, as Insou said. And the only slight embarrassment, which is in full disclosure in the advertising piece, is that we've been thinking about a project precisely in this area for years, and have developed proposals for various collaborators for it. But this is one of the few things, few projects that I've developed that has never happened. So Insou asked me to do this. So I'm going to talk about the method and why it would be appropriate. And we can take it from there. So I'm going to draw on many projects in other fields that we have conducted. So, the the idea of using public deliberation on bioethical controversies was in fact endorsed by the presidential commission that was Damien Gutman's commission in 2019, where I was privileged to give testimony, but I was far from the only one who advocated this. And the commission said that the process of I'm sorry, but my headphones aren't working. Is my headphone working? Yes, you're all set. Sorry about that, continue. Oh, okay. The process of democratic deliberation is especially useful for the types of ethical questions we face in bioethics in which solutions have complex empirical as well as moral basis about which people can disagree. So I'm going to talk about other issues we've done which fit those criteria. And just recently I published in this Hastings Center special report on gene editing in the wild that also made the case for broad public deliberation on ethical questions. And indeed it was not just me as one of the contributors, but the Hastings Center overall that made this case. So if there's broad public deliberation, it could take many forms. What is the form that it should take and what is the ultimate goal of such public deliberation? And in my approach, the design is dictated by the goal of finding out what the people would think under quote, good conditions on quote, we're thinking about the issue. And I want to talk about why that would be needed, how to select the people, how they deliberate and how to make this practical. So there are lots of public consultations and I want to distinguish the method of selection on the top row from the kind of opinion that you may get out of it or by what public opinion, I just need the opinions we find in ordinary life where people have not gone through some process to think about it in depth. And refined is a word sort of a quaint word that I take it from actually from Madison who talked about for representatives, the discussions in the Senate and the Constitutional Convention and other areas refining and enlarging the public views by passing them through the medium of the chosen body of citizens. It's the refinement process that is equivalent to deliberation, but we'll talk about what we mean by deliberation in a moment. My colleague and collaborator, Norman Bradburn from University of Chicago, coined the term for self-selected well-public opinion as a slops back in the days when radios would tabulate call ins. So we called it self-selected listener opinion polls and the term has now become a widely accepted term. And I'm gonna say something in a minute as to why we don't wanna consult the public through slops. You could have a self-selected discussion groups and indeed the Kettering Foundation National Issues forums and other groups foster public discussion with self-selected groups. And that's great in terms of the impact on the citizens but it doesn't represent the public. And then there are non-random polls that there's a, there are broadcasts on TV like Spokane should do listener polls all the time but they're self-selected and they don't really mean anything from a scientific standpoint. And then there are small self-selected groups, citizen juries and other groups, consensus conferences where the Danes would advertise in the newspapers to get people to volunteer. Now you might say why not just do conventional public opinion polls? And it's interesting that Gallup when he launched the public opinion poll on a national basis at least in political matters where the 1936 general election gave a speech afterwards that the public opinion poll would bring the democracy of the New England town meeting to the large-scale nation state. And his aspiration was that it would bring representative and informed opinion, which is my aspiration too. But so if you have a scientific sample you clearly get over some of the problems of self-selection although of course there's a degree of self-selection in any sample conducted any place but say a prison. But how aware and how much are the participants and how thoughtful are they? So Gallup's idea was the public opinion poll would bring the democracy of the New England town meeting to the large-scale nation state. And there you are in New England but my colleague both the arena wrote about how the town meeting has been destroyed in New England in most cases by the growth in population and the size of the towns. My friend Jenny Mansbridge did a study of a Vermont town which had 200 and some participants which she gave a pseudonym to that most towns are much larger. In any case, his Gallup's idea was that radio and newspapers there wasn't television in those days would send out competing points of view and the people would think it over and their views would come back in the form of public opinion poll results. And it would be as if the whole country was in one great room he said. And there's a sense in which he was correct but the whole country in one great room it was so great, it was so big that nobody was really necessarily paying much attention. And audience democracy and an audience we've all gone to talks perhaps even this one where you fell asleep in the middle of the talk. So they pay much attention to the details. So an audience we need deliberation as an active process not just a passive process of exposure. So what I call deliberative polling puts the whole country in one room but it's a room of a human scale by having the scientific sample deliberate and we improve it slightly but in the standard way now of having stratified random samples rather than the quota samples that Gallup use that we have the sample deliberating. Now of course in a sense you could have everybody or you can consult everybody as in a referendum that's with that or Bruce Ackerman my former Yale colleague and friend and I wrote a book called Deliberation Day about the idea of getting everybody to deliberate which now we are beginning to inch towards with technology and I'll tell you about that at the end but I'm gonna focus on deliberate polling. James could you maybe move a little bit away from your microphone because your sound is a little bit harsh and distorted? Is there a way to make the sound quality a little bit clearer? Oh well I don't know I'm using a headphone. Okay. I'll do my best I would have to exit I think to. Yeah it's just a little bit distorted when you're speaking because it sounds like you're it's a little too close to your mouth wherever the microphone is. Oh well hangs down from my ear I could I could take off the headphone and just speak. No I think that's fine just you make it continue. I'll try not to speak so methodically. Yeah just to say something about the problem of self-selection isn't that a beautiful bridge? That picture? That's the bridge that connects Buda and Pest in Hungary and the country of Hungary decided to have famously decided to have a contest for naming it and they asked for votes on the internet and they were very surprised when an American comedian named Stephen Colbert decided, saw that picture and said that's a beautiful bridge I want it named after me and he got 10 million or so self-selected votes and so he won the he provisionally won the voting. I think when I last looked there were only something like seven million people in Hungary so obviously there are a lot of people who may have voted twice or people who certainly a lot of Americans who voted and so the Hungary decided that they the government decided that they would change the rules and say only if someone only if someone spoke Hungarian could the bridge be named after them and so the he then went on TV and showed off his new Hungarian lessons and spoke Hungarian fluently enough to qualify so then they changed the rules again and decided it would only they could only name the bridge after someone who was dead so he withdrew but it shows that self-selected participation with mobilization and all kinds of interests involved is not it's very hard for that to be representative of the population in question represents the people who are motivated to turn out and you might say that's part of democracy in some way but it's not part of our assessing what the answering my question which is what the people would think so there are basically three problems with public opinion as we find it. The first is made is rational ignorance as Anthony Downs coined the term in 1957 if I have one vote in million or one opinion in millions why should I go to the trouble to think about complex public policy issues by individual voice my vote my opinion will not make any difference and I've got other things I have to do I have to do my job I have to provide for my family whatever it is we all have plenty of things where we can our time can be made more useful and on general public issues and that's unfortunate because we would like citizens to be well informed but that's the condition of the citizen in large scale mass society. The second is that many of the opinions that we solicit in polls there's some controversy about how many but on many complex topics they are what the great public opinion researcher Bill Converse called non attitudes or I call them phantom opinions. The late George Bishop asked the American public what they thought of the Public Affairs Act of 1975 and in polls some of them supported it and some of them opposed it but it was fictional there was no public affairs act in 1975 so the Washington Post famously decided to ask the public celebrate the 20th non anniversary of the non existent public affairs act of 1975 by asking people what they thought of the repeal of the public affairs act of 1975 and they asked, they spoke a sample telling some people that the Republicans wanted to repeal it and some that the Democrats wanted to repeal it and they got quite different results but of course it didn't exist in the first place so it couldn't be repealed. Now since that time a further problem has developed or has been accentuated but it was always there and that is when we do tend to talk to other people about public issues we tend to talk to people like ourselves from similar social locations and often with similar points of view and the news now that we have social media and we communicate with our news feeds we are often in our filter bubbles where the algorithms will feed us views that we agree with and we can often avoid those that we disagree with and we can choose cable news channels that we agree with and avoid those that we disagree with and so in our filter bubbles we're fostered our communication with the like-minded so the whole presupposition of a liberal democratic society that falsehood would be self-correcting as John Stuart Milne explained in his four cases for why you should tolerate disagreement and falsehood will tend to break down if people never hear the correction they never hear, they never engage with an evidence-based substantive discussion with each other and so we tend to persist in we're becoming more and more polarized and some of this polarization is a long partisan lines. Now the method that I employ and that we have employed quite widely now involves speaks to these issues. It involves balanced information. We have a, on the issue that's going to be discussed we have a briefing document and then also a video version of the briefing document but the briefing document lays out some proposals for action and the pros and cons of those proposals that have been vetted by an advisory group that represents the different points of view on the issue and that actually is a good part of the work in doing any deliberate appellate is preparing the materials and vetting them and that involves a lot of rewriting until everybody is satisfied on the advisory group. So we have the balanced information we have small group deliberation moderated small group deliberation but I'm gonna say more about that in a moment the small groups then engage in extensive discussion of the proposals and the pros and cons and any other pros and cons that they think of and then they formulate key questions that they think might need answering and those go to panels in plenary sessions of competing experts who represent different points of view or competing policy makers usually both who may have views about the issues and then the process is repeated going through the agenda and this usually takes place over a weekend and then at the end people take the same extensive questionnaire that they took at the very beginning. I should have mentioned on first recruitment we have an extensive questionnaire about the policy options and about the pros and cons that we can identify beforehand. And we also will tend with permission of the participants to tape all the small group discussions and in fact with new technology we have the transcriptions instantly and we can do automated text analysis of the arguments because we wanna see what people conclude and why after they've really thought about it. So and where possible we have control groups who did not deliberate pre post control groups. So these are actually controlled experiments and we're interested in deliberative polling is different from many of the other methods of public consultation because we're concerned to have a sample that is large enough that we can analyze the demographic and attitudinal representativeness of the sample in a statistically meaningful way. So many of these so-called citizens assemblies do not produce their results or citizens juries do not produce their conclusions with individual confidential questionnaires and their sample sizes range from 25 or so for the citizen juries from 50 to 150 for the citizens assemblies so-called which have been conducted in Europe and in Canada in various places including the US and then also but our projects go up to a thousand persons or 500, 600, 700 and very quickly we get and that's overall and then in each small group of 10 or 15 people can really discuss but the aggregate results are statistically meaningful we can analyze the change. So we have the representativeness at the start and why do you want it to be representative of the start in attitudes as well as demographics? Well, if people come in already convinced then the results may not reflect what the public would think because the overall public is very likely not to have firm views on most issues, particularly relatively complex issues and so you wanna see where they would go and people with different demographic groups may have different interests in the issues and we wanna be sure we covered that. So then we engage them in the opportunity to consider and really discuss the issues in an evidence-based way. I really think that it's the discussion and we've had some controlled experiments where we showed that it's the discussion not the exposure to information merely but it's the discussion. If you expose people to information that they disagree with at the start you may just trigger a negative response but if you engage them in a discussion with diverse others you often get very surprising changes and in any case we also measure knowledge to see if the people have become more informed and there are certain distortions that we're very interested in studying and have not found in the deliberative poll and we now have a paper that's been accepted by the British Journal of Political Science which looks at a large number of deliberative polls and an even larger number of small groups because each deliberative poll may have 20, 30, 40, 50 or as in the climate change project I'm going to discuss, 100 small groups and we look at the weather, the movements are in the direction supported by the most advantaged, the most educated, the men rather than the women, the rich rather than the poor and nothing like that is the case. There's no such pattern of movement but the more advantaged not imposing their views on the less advantaged, everybody is participating equally. Anyway, let me see, oh, here we see, where this lists 110 projects in 34 countries, actually this is out of date, it's now more like 115 projects and I think the number of countries is even larger on every inhabited continent. Even we've done this all over Africa, Ghana, Uganda, Senegal, Malawi, Tanzania, we've done it in Asia, European wide, I'll show you. So I'll show you a few such cases but we've done this all over the world in developed countries, developing countries and we've done it, oh, this was the, I'm not gonna play the video for this but this was the very first project and it was on, it was in the United Kingdom, it was in Britain on crime, we didn't have Northern Ireland so I won't say it's the whole United Kingdom but it was about, and I remembered very clearly goes all the way back to 1994, soon after I had the idea of consulting the public in this way and a woman came up to me who was a spouse, a company or husband, they allowed the spouses to come as observers but you had to be selected to be in the sample so the observers couldn't participate but she said she wanted to thank me because in 30 years of marriage her husband had never read a newspaper and from the moment that he was selected to come to this, she said, he began reading every newspaper every day and he was gonna be much more interesting to live with in retirement and to me that encapsulated that we were providing a response to the problem of rational ignorance, that is he was effectively motivated to prepare and to think about public issues in preparation because instead of one opinion in millions he had one opinion, he was gonna have one opinion in a discussion group of 10 or 15 and in a sample of three or 400 and he was, and we went back to these people a year later and they were still even more informed than they were at the end of the weekend and by the way they did change their views on many contested issues, we published that also in the British Journal of Political Science and there's a video on our website because Channel 4 made a two hour program about it but that was the launch of this whole initiative and it led to all kinds of things. That's the European Union Parliament building but those people are the sample of the first of two European wide deliberative polls conducted in 22 languages you can see the headphones of the people and that's the plenary session with the questions to competing experts, the small groups where 10 or 15 people also with headphones and it was very complex to work out the pivot languages for the interpreters in order to cover all the languages. Now, this was a series of projects on the provision of electricity in Texas, we did eight such projects and this is a case where there are, it's a combination of empirical questions and values and where the deliberative poll had a direct impact on policy because at the time that we started this, Texas was last among the 50 states in renewable energy and by the time we ended, Texas shot from number 50 to number one and what really caught the attention of the Public Utility Commission who cooperated, who helped organize the projects along with the eight then regulated electric utilities was the percentage of the public willing to pay more on their bill in order to support wind power or wind or solar power but it was basically wind power at that point and now, of course, the cost of wind power has gone down to the point where you don't have to subsidize it but at that point, it was just people were willing to pay $1 more, $2 more, $5 more, something more on their bill and so the Public Utility Commission assessed everybody 25 cents and you might say what is that but when you have millions of quarters that the 25 cents really sparked the development of wind power in Texas and it just has kept going so by 2007, Texas surpassed California as the leading state in wind power and it's only increased its lead among the states wind power all over the Davis Mountains and other places in Texas has Texas is the leader in the US and one of the world leaders and it's because of the public brought into this on reflection when before many people had not even thought of wind power in fact, the initial questions in Phil Converse's study where he proposed the non attitudes came from questions about public utility issues where the public couldn't even remember in a panel study from 1956 to 1960 what they had said the previous year and so Converse concluded that people were answering questions but there was no actual opinion there just buried randomly from year to year over the four years so I took particular satisfaction since Converse was one of the people who helped us bring the deliberative poll to the United States on a national basis that we were able in that policy domain of what to do with electric utilities we were able to show that people instead of non attitudes could formulate thoughtful opinions that were consistent with their other opinions as they changed their views before and after deliberation sometimes we had very controversial issues with ethnic differences in we did a national project in Bulgaria about the condition of the Roma and this was I mentioned this because it was consequential for policy that is the Roma children went to school that were segregated and there's a lot of prejudice towards the Roma in both areas there is throughout Eastern Europe and in this project, the percentage of the people to everyone's surprise who were willing to close the Roma only schools and bust the children to schools with everybody else went from 42 to 66% and the percentage who wanted to preserve the current Roma schools which were terrible schools on all the evidence went down from 46 to 24% well, our Bulgarian partners tell us that this national project which the prime minister attended and was interviewed in the New York Times about was the catalyst in the long struggle to get the Roma only schools desegregated which they are now desegregated and now here's an example of another issue in Japan which was also policy consequential where the Japanese were about to privatize privatize their pension system and you might say pensions are quite different from the areas that this group is ultimately interested in bioethics, nevertheless, it's very consequential and very complicated but the ordinary polls were, the Japanese called the private account system that was being proposed, which is like the Chilean system where the funded system, I thought misleading me because everybody would have their own fund and support in public opinion polls for that was it's about 70% in many public opinion polls about 70% in many polls at the time the government was about to act and in our poll at the beginning it was 69% and when people deliberated, it fell precipitously to 35% and because the public in the deliberations when they realized that they would be responsible for investing their pensions in the stock market they didn't like the insecurity they didn't like the risk of having their retirement vary with the, they wanted something secure so rather they supported increasing the consumption tax which was very low, it's so like a national sales tax if provided only that the money would go into the pension system and you know how hard it is to get people to support and increase in taxes in any case, this project had a lot of BDS support Asahishin Boom, which is sort of the New York times of Japan featured it a lot and NHK was a partner that a public broadcaster and the government at the time dropped the proposal for the privatization and the next government that came in continued that and so this had a clear effect on the policy making. Japan also did after the Fukushima disaster commissioned officially the other project with our partners at K.I. University was funded by the equivalent of the National Science Foundation there but this project at the National Deliberative Poll on energy after the Fukushima disaster was explicitly sponsored by the government and interestingly the government did everything else before they did the Deliberative Poll they did ordinary polls but they couldn't tell how much the public really understood the issues from conventional polling and the issues were very complicated and then they had open town meetings so-called around the country and we're very surprised that the anti-nuclear activists showed out in force at the same time that the electricity companies showed out in force and they yelled at each other and so the town meetings all around the country were a kind of circus that was covered in the press around the country but it didn't enlighten very much so then they did with a scientific sample they did a Deliberative Poll with our collaboration with our colleagues at K.I. University in Tokyo and we got results that they found useful. By the way, nuclear energy is another thing that's continued in South Korea we were very surprised that President Moon decided that since he was stuck with two nuclear reactors when he came into office that were partially constructed he announced that there would be a Deliberative Poll and that would have the final say on whether the reactors were finished or not. His party position was basically anti-nuclear but there were consequences for climate change and for the cost of electricity if the nuclear reactors did not come on board they had to import fossil fuels and so it was a difficult choice and the root of deliberation is really weighing the competing arguments and in this one everyone was surprised or a lot of people were surprised that the people in the sample moved to support the completion of the two nuclear reactors and then they've gone on to do some others as well since. Another unusual example which again is a little far field from bioethics but I think it's very consequential. We've done some projects in Mongolia and Mongolia has changed the law on constitutional amendments so that before they are permitted to do a constitutional amendment they have to have a national sample with a Deliberative Poll and here's the Deliberative Poll of about 700 citizens selected meticulously by the National Statistical Office the same office that does the census and with an extraordinarily high response rate people were brought from all over the country to the capital Ulaanbaatar and to the government palace which you see there and the sample is gathered for picture under the statue of Genghis Khan which dominates not only the parliament but a lot of the political culture in at least is a revered figure in Mongolia and they did actually make some consequential decisions as recommendations to the parliament and the parliament took them on board and according to the speaker of the parliament the decisions of the Deliberative Poll were absolutely crucial in their fashioning and amendment which then actually passed the parliament by two thirds so we successfully helped them to change the constitution. Now, how much time do I have left? I may have talked too long. I'd like to get to the Q&A in about 15 minutes. Okay, good. Okay, so I will zip through some things. Here's what I'd like to do. So this is a project called American One Room which we did on the eve of the primary in the eve of the primaries in 2019 and we had NORC at the University of Chicago provided a sample of more than 500 registered voters. We discussed in depth for a weekend these five issues and by the way, the New York Times did us the favor of putting the photos of all the participants in the newspaper with permission of course saying this is what America looks like and look how they, after a weekend of deliberation how they changed on some of the most controversial issues. For example, look at immigration here. This was published in the Times. Support for forcing undocumented immigrants to return to their home countries before applying to live and work in the US. The Republicans before deliberation supported that 79% after deliberation, only 40%. That's a 39 point drop. That is an earthquake in public opinion. Support for DACA went way up. Support for reducing the number of refugees allowed to resettle in the US dropped from 66 to 34% et cetera. We got a lot of very large changes in the opinion and the Democrats also changed in a very large way. Remember, this is before COVID. So support for opposition to universal basic income went up dramatically. Support for a government funded baby bond for use in education or other purposes which they thought was a kind of giveaway went also down 40 points. Support for Medicare for all went down 14 points from 70% to 56%. I think as people thought about the problems they didn't wanna give up their private health insurance. So the Democrats changed dramatically. The Republicans changed dramatically. I don't necessarily agree or disagree with any of these results personally but they had coherent reasons which and we found that overall the deliberation sharply depolarized extreme partisan polarization as people came to a view on the substance of the issues. Now we did a similar project on climate and energy and we use technology that I'm gonna describe to you. We had almost a thousand deliberators and a control group of 670 and they did it online and we got very dramatic, we had a great sample that NRC at the University of Chicago got us and on 70, there were 72 substantive questions and 68 of them changed significantly and 66 out of the 68 changed significantly in the direction of doing more to combat climate change and once they got over the threshold question of whether climate change was real and whether humans had a role in producing it, again here the action, a lot of the action was with the Republicans who on all of these questions about 35% would thought it was real or would support doing something on getting rid of coal or supporting innovations for electric cars and all of these, as I said, we had 68 with 72 proposals so about 35% before and about 55% afterwards it was very consistent and so the Democrats tended to go, started high and went even higher, the independence went higher and the Republicans went higher from minority to majority as they thought through the complexities and many of these carbon capture and sequestration, biofuels, we went through all of the and strategies for reaching net zero, it was very complicated, I'm not gonna go through these slides since I've already talked too much except investments in solar energy, nuclear was surprisingly supported by almost everybody or it was not a partisan issue and what I wanna mention is that we use this platform collaboration with my colleague from Management Science and Engineering, Ashish Gowell, who has a center called the Proud Source Democracy Team and then my center for Deliberative Democracy, we've developed an automated platform and what this does is it allows a group to moderate itself, you can see it in the bottom screen, whoever's speaking has the picture enlarged and it randomly assigns the participants to a small groups of about 10 but it does a number of things, it has a speaking queue, people who wanna speak volunteer for the queue, each speech is limited to 45 seconds although it can be briefly interrupted if somebody has to make a point directly. It shows the pros and cons for that proposal on one side of the screen and it asks the participants when they think they have covered the agenda for that proposal and would they like to move on or would they like to discuss it further and it moves through, moves the discussion very efficiently through all the proposals. It nudges people who have not spoken, invites them to speak, it intervenes if people are being uncivil to each other and it orchestrates their decision about the most important questions that they wanna ask in the plenary sessions as they invite people to enter a question then others to edit the proposal and then they vote with ranked choice voting among the different proposals until they get it down to two questions that are suggested, which are passed on to the moderator or the human moderator of the plenary session. And we use this voices of the future is a national sample that we did of young people on public policy on political reform issues. We used it nationally in Canada. We used it nationally in Chile on pensions and healthcare reform, the project commissioned by the Chilean parliament and heavily covered by CNN I should add. We've used it in Hong Kong and in Tokyo and in the climate change project we used it. And because we have the transcripts it is really facilitated are getting the qualitative as well as qualitative material about the project here. Let me just show you this two minute video about the, this is a two minute video that shows you just a little bit about the platform that I wanna say the implication of the platform is that when we find that people deliberate together particularly on issues they disagree about as well as issues they haven't thought much about we help create more informed citizens who have a greater sense of efficacy who are much more likely to vote who are much more likely to think about the issues when they vote. We have data on that that I'll come to if you ask me about it. So we'd like to spread this. So we're now going to open up use of the platform to civic groups and we've got an undertaking with universities and colleges around the country to spread deliberation. And we think eventually we wanna build up to and we've just gotten some raise some money to experiment with scaling with the mass public. We think we could have thousands, tens of thousands. There's no limit to the number who could deliberate on the platform. So we'd like to actually change public opinion in whatever way the thoughtful and informed participants arrive at through the platform. So that's our scaling activity and we're gonna start experimenting with that on the issue of climate change but we're also gonna use it for other issues. So I'm gonna just show you this short video which gives you a picture of the platform and it draws, it excerpts particular people who were talking on the platform but it's about climate change. But climate change is a contested issue. It's, we turned out to be a surprisingly partisan issue we found 33 of our 68 proposals exhibited extreme partisan polarization in terms of the distribution of opinion on one side or the other. And so we think it's worth scaling. So let's see if this video works here. Good morning, I'm Eric. Hi, I'm Ray Anne, I'm from Tennessee. Hi, my name is Gann. Hi, I'm Gary. I'm Barbara. Hi, I'm Lauren. I live in Southern California. I have Sean, I work in aerospace in Washington. Good morning, I'm Watson Harvey. Thank you all for joining today. This discussion is an opportunity for everyone to learn. I want to encourage everyone to speak freely. Please respect and listen to each other's opinions and try not to interrupt others. I do think we need to reduce greenhouse emissions but I'm not sure how practical it is to totally eliminate a lot of these things. You guys can hear me right? Okay, good. The hollow promise of net zero carbon pledges and the key message here is unfortunately there are no standard guidelines for achieving net zero carbon neutral emissions. And it feels kind of disingenuous for us to try and regulate other countries when it's still a matter of debate here. Try to convince me. Maybe I'm a minority of one on this panel but there really is nothing that human beings can do to tilt climate in one way or the other. The tax breaks for Tesla owners and those types of policies have come into places because we don't have a lot of low-income people making our laws and policies. As a libertarian, I hate to agree with you. Coal is going very naturally on its own. It's so slow. There's other things, gasoline car cars. So really the issue is not whether it will happen but how can we speed it up? Most people are saying that a tax will be missing. It's unnecessary. I don't have that answer. I'm just throwing that out there. Most people most of the time are not motivated to become informed about complex issues. Furthermore, if they do talk about these issues or consult the media, they consult congenial sources like-minded people. So they never hear the other side. They thought, they deliberated, they weighed with one another and they changed on a great many of these items not only in statistically significant ways but in substantively, often very large ways. So you see these amazing individual examples and anecdotes of people changing their minds not because a politician is scolding them to do so but actually the opposite that they're being empowered to learn about this information for themselves and come up with an emergent opinion that may or may not differ from the party line or dogma. I joined the project hopefully to learn and to maybe contribute some things and I did learn. I found out that there were a lot of people who had my concerns. When we first started, we felt like, oh boy, this is not gonna work because everybody brings their ideas to the table but we did come to a conclusion that yes, this will work if we work it together. Okay, so I'm available for questions, Insou. Fantastic, thank you so much. And that was such a breathtaking array of examples of deliberative polling in the past. I'm gonna just start things off with a concept you brought up at the very beginning that was intriguing, right? It's the concept of rational ignorance. Look, I'm busy. I have my own opinion. I only have one vote. Why would I spend all this time hearing these other opinions and hearing about these other factoids when I'm busy? So how do you get over that tendency that people may have even in the deliberative polling approach? I mean, it looks like your example's involved like a day or two of commitment. How do you get over the rational ignorance inertia? Well, you have to convince people that their voice will matter in this in a way that it would not otherwise and that it will be very enjoyable. They will get to meet people very different from themselves which normally people never do. So we normally pay a modest honorarium in the face-to-face projects you saw. We were also paying for all their expenses and so that was the free trip and the rest of it. But in the online projects, we pay a modest fee but it's just to help facilitate the participation of the economically less well off. And so it's certainly not the reason that people participate. We convince them that the issue is important and that their voice matters and we usually try to find a context where something will be done with the results and also that the media will cover it. And we have a good track record in getting a lot of media. And so people participate for a variety of reasons as long as you make it easy for them. And I think there's a lot of people who are tired of being manipulated and getting the people trying to sell them something or convince them to buy something. And if you appeal to them, well, can you help us in thinking through these public problems and thinking through how the public, which ones the public will support and which ones they don't and your voice will matter to this project. Your voice is really important because you've been selected and if you don't participate, it will be less representative. And we are persistent enough. We often can get them to start and once they start, they always finish. Almost nobody drops out. They find it very interesting and intriguing. And that applies to the face-to-face and to the online and the Chilean 400 project which was done with this platform because it happened during COVID. It actually ended up being the Chilean 500 but they picked the name 400 against my advice but they had 500 participants. The people were crying at the end that they had to stop even though they'd spent a whole weekend because they bonded in their groups and they became fast friends and they exchanged their contacts and they wanted to have reunions and et cetera, et cetera. So we find that we can get people to participate or at least try it and if they do, they like it. So questions are coming in. I'm gonna first start with questions that kind of again have to do a little bit more with deliberative polling in general and then get to questions that relate more to bioethics. So Judy Friedson asks, what are the critical components of conducting a deliberative poll and what is your definition of a successful outcome? Well, I had a slide about that but let me just say that the first thing I ask if an organization contacts us and wants to conduct one of these is a clear statement of the issues, the policy proposals that might be considered and a committee that can vet briefing materials that are balanced and evidence-based about the pros and cons of each of those proposals. And once we have that, it's very easy to construct a questionnaire about those same proposals and from the pros and cons, the possible explanatory variables that might explain the reasons why somebody might support or oppose those proposals. And then we're up to the races. Then we just have to get away of getting the sample and a stratified random sample of the population in question. It could be a city, a state, a country or as I pointed out, the entire European Union but then you need a method for recruiting the sample and getting them to participate and collecting the data before and after. Now the criteria for success involve the representativeness of the sample and the, I won't say specifically that the opinion changes are a criterion for success because there may be a case where people have an impression that they will lean one way on an issue but it's an uninformed impression and then after the process, they're well informed about the competing arguments and they might or might not come out the same way but if they come out the same way, it's based on reasons at the end. So reason-based considered judgments but also the easiest way to communicate this is when we have substantial opinion changes, substantial and statistically significant opinion changes so that we can say it's not just some arbitrary change but you saw the magnitude of the changes that I saw and many of them were quite astonishing and so the opinion changes and then we have questions about knowledge and we almost always get significant increases in knowledge. We also get significant increases in mutual respect and in trust questions and we lower affective polarization. The extreme dislike between Republicans and Democrats usually diminishes significantly in these processes and then so these are all criteria for whether it's a successful deliberation. James, do you measure how long this change persists in people? So we have big shifts. Do you see whether that is kind of like the warm, fuzzy glove I've just gone through this process, I bonded with my deliberators does that last, does the change in view last? Well, no, and yes. In that American One Room project, we often don't have the money to go back but in that American One Room project that you saw, the face-to-face one, we went back at the time of the election in collaboration with the New York Times because they wanted to know how the people voted and we also had a survey around the time of the party conventions. So we went nine months and a year later and many of the policy attitudes went, reverted back a good deal, not completely and among the most, those who took the most extreme positions, there was still significant persistence of the changes because those who took the most extreme positions moves significantly closer together but something else very dramatic happened which we've now presented at the American Political Science Association meetings and now submitted it for publication because, and the New York Times published it so I can talk about it and it was very surprising, took them a while to publish it because they checked it out because it was so very surprising. The voting intention, well, let me put it this way. The people who deliberated compared to the control group and we got very good because NORC was doing it, they got very good completions from both the participants and the control group and this is now posted on our website as the American One Room election follow-up. So the participants were much more aware of the campaign as it unfolded. They continued to become increasingly knowledgeable. Those who changed their views on the weekend, they also had a much greater sense of efficacy that they thought their voices mattered and those who changed on the weekend moved dramatically to support Biden rather than Trump regardless and so the control group got the election in a national results almost perfectly. I think I can't remember three and a half percent, four percent for Biden over Trump. But the treatment group, the participants were, I think almost 29 points, it's on there. I mean, a tremendous gap in terms of support for Biden instead of Trump was almost, it was so big that people wouldn't believe it. Now, and we wondered, it was just COVID but if you look at COVID, COVID was a factor for everybody. COVID was everywhere. COVID didn't distinguish the participants from the control group. So anyway, there was a big gap and the participants went dramatically for Biden rather than Trump and the actual details are there on our website. I'm now forgetting and I don't want to misquote it but it was gigantic. We then checked in balance tables, there's the ratio of the participant's sample, though it's not attrition that explains the gap and we sorted that out in the paper pretty clearly. It was the effect of the deliberation and we think we have a causal mediation analysis where in our view, deliberating in the deliberative poll created a kind of civic engagement, mediating variable which the people just became more informed, more aware of the campaign, had a greater sense of efficacy and took account of their policy positions more and those are the people who voted for Biden rather than Trump, particularly in the broad middle range where in the control group, many of them didn't vote at all but in the treatment group, they were motivated to vote. Having participated in this made them much more likely to vote and it was those people in the broad middle who often don't pay attention and it was somewhat more women than men but it was everybody really. So we had a tremendous lasting effect. In fact, so tremendous that, but the Times checked it out, the upshot team checked it out and then they published it and they published the pictures of the people again in the paper, so the people who participated got the pictures in the New York Times twice. So yes, this can have a big lasting effect and that was part of our case for wanting to scale the deliberative process. Yeah, I want to move into some bio-exfoliated questions but a couple have come in that I think follow up very nicely what you just said. So Alan Regenberg, hey Alan, he wants to know, the participants who are involved in this, is there like a process where they might, you might be planting seeds for broader discussions, maybe with them bringing home these deliberations to their friends and their family. Have you looked at the sort of impact of your projects? You know, sort of the sort of like, you know, their participants bringing the conversations home and continuing it with their friends and family. No, but we like to do that and indeed that's part of the plan. We've just gotten a lot of money, a lot of money but a significant amount of money to experiment with and prototype, scaling the deliberations to much larger numbers and with those resources our hunch is that since people are always saying they love the process and can they bring their friends and family, can they recommend it to others, we're going to test that to see if they would recommend it to others and see if we can scale it to much larger numbers of others. This is not something we would ever do for a deliberative poll because for a deliberative poll, we want to be very careful who's in the sample. But for scaling deliberation to the broader public, it's essential for spreading this virally. So your question foreshadows the research that we want to start doing now and we're going to be doing controlled experiments where that's part of the aim. We're going to see if the people are willing to recommend and follow through and if we can then scale it. So that's our next frontier. Okay, one last general question before the bioethics questions and if you can answer this briefly. Anna Lewis Hiana, Anna, she asks, about how much is the cost to do a deliberative poll and what are the main components that drive that cost? Well, first you have to distinguish the country. I mean, the projects we did in Uganda and Ghana and places like that, the costs were quite minimal by US standards. Those were face-to-face projects. The cost of a national US project, most of the cost is taken up with airplane travel and hotels and food. So if you do it online, you eliminate all of that. So then it's a question of the... So with the use of the platform, it's actually quite cheap, except depends how much it costs to get the advisory group going through a lot of meetings so that they approve the materials and the rest and then the cost of basically the survey firm recruiting the sample, whatever incentives we paid to the people. And so we were able to do a much... And then of course, this will also vary. Well, in any case, there's clearly a few hundred thousand dollars for a national sample to actually deliberate and a control group to do a... I mean, the climate change one, we had a thousand people instead of the 500 people and we had a control group and the rest and we had incentives that we decided to pay them. And we paid $300 a person for the weekend. But we could... There are some places where we could pay less. I think in Canada, they paid significantly less and it was very effective. And so the costs have varied. That was also down on the platform. The costs have varied, but it's clearly gonna be a few hundred thousand dollars. But you have to tailor it to the situation, the country and the cost of recruiting now to get a good sample. Okay, so I wanna move to some of the ethics and bioethics questions that have come in. There are a lot of them. So if you can provide brief answers, if possible, we'll try to get through as many of these questions as we can. Takunda Matosi asks, and this is I think a good question for some of the original ethical uses because obviously you've been dealing with public policy questions, political issues. But Takunda asks, instead of, I'm paraphrasing, instead of maybe looking at a consensus building and decision-making, do you think some of these deliberative polls can actually get that quote on quote something like the right answer ethically? I think he means ethically, correct answer. Is there a role for using deliberative polling to arrive at correct answers to ethical controversies? Well, I'm not an epistemic to liberationists. There are some who are, but I'm certainly not a relativist either. What I am is I'm facilitating the public will. And some people will interpret that as getting the right answer. But it's the right answer for a population after they really thought about the competing arguments. So there are people who will bring arguments and as they weigh the arguments and revise their own views. So it's public opinion and public will formation. So it's very Habermasian in spirit, but I don't want to take the step of saying that it's the right answer, but it's the answer that really has purchase on this population that they are willing to buy into as what they would approve. So I spent a lot of years writing, doing ethics and political theory. So I, but I'm not gonna take that last step. I'm gonna say this is the right answer for that population that they're willing to endorse. And here are the reasons that they think have weight. And that should be good enough for policy making. Okay, great. So Barbara Canig, hi Barbara. She has two questions. I'm gonna start with her second question first. Do you think bioethics questions present any particular challenges as opposed to other topics such as climate change or environmental policies? I know you haven't gone into the bioethical issues. Yeah, but do you think bioethical questions pose particular challenges for this? Well, yeah, they do pose challenges, maybe risk challenges, but we deal with risk all the time in other policy areas. I think that I think there are, sometimes we do projects that seem to turn on what we call empirical premises if you do this, that will happen. If you do that, this other thing will happen. Well, depending upon the risks and benefits of those causal claims and whether how transparent they can be or how much it's a step into the unknown, that can complicate the analysis. But apart from that, I think that there are, most of our deliberations have to do with competing value-laden goals. And I emphasize value-laden goals, not just the values in general, but the goals of policies that have and the competing values that attach to them. And I think the bioethics questions fit into that quite well and that was what the Presidential Commission concluded, which I take as a, which I've used in trying to raise funds for this kind of project, because I think this is a really important area. Yeah, Barbara just added an important point. She says, identifying persistent disagreements may be another important outcome. But let me move to our other question. Jason, can you come up with any examples that people have used the deliberative, not your approach, but a deliberate approach to a bioethical issue that you think is a good case study? Well, no, because most of the other processes don't collect the opinions in confidential questionnaires. They have a group spend a lot more time than a weekend, sometimes up to a year writing a report that is a shared consensus document. And so that makes it impossible to study the opinion changes. And I think it exposes the people to social pressures to go along with the group as in getting a verdict in a jury. Now, I actually written about the jury and it's an important institution in our civic life. But I think for this purpose, you really wanna protect the autonomy of the individual opinions. So I don't actually, this is one of the reasons that I support this particular model rather than the other models. So with our model, we have only a weekend that we have the confidential questionnaires and we have the opinion changes. And we do a lot to try to protect the opinions from the social pressure that you get in small groups. And that's why the small group analyses have turned out so well. Yeah, some of the thoughts that were early work in deliberate democracy, I'm thinking here about Amy Gutman and Dennis Thompson. They have a stricture that only people can bring their publicly accessible reasons to the deliberation table. So you can't appeal to faith claims about God, things that cannot be scrutinized from other participants in a publicly accessible way that don't share those same faith commitments. How do you deal in deliberate of polling with core religious differences, belief differences that are not the kind of thing one can easily put there out on the table for discussion for others to access who don't share these fundamental core faith traditions, right? So I'm asking like, what about the role of religion? So I'm please sort of affirm. People bring their religious convictions all the time into the discussions. But they do so knowing that they're in a diverse group. So they have to sort of retranslate them so others can understand. And usually there are ethical claims that are mutually intelligible. I don't believe that Amy and Dennis have ever actually done any experiments in deliberation. They've just written about other projects. So I wouldn't know how to police the, you can't bring your religion into the discussion. I wouldn't know how to police that. People say all kinds of things. They tell stories about their experience or what they learned in church or that's, but they have to, because they're in a diverse group, they have to make it intelligible to people who don't go to their church or don't, and they do. They manage to open up and try to communicate sincerely to others and it works quite well. So I haven't found any difficulty there. But let me point to an area where I think there may be a little bit of a unique challenge in the bioethics context and just get your thoughts on this, okay? So you've done all your work, I believe in political and public policy questions like energy and climate change that pretty much affect most of us, right? So that seems to fit very nicely in a democracy theoretical framework where those who deliberate have the moral legitimacy of what they believe or what is the output of that deliberation process because they're going to have to live with the decision somehow. Like it's kind of a self-governance model. But in bioethics and many of the context that like the need for public participation comes up will be on, for example, very specific scientific issues like bench techniques that, should we put human cells into an animal embryo? Okay, well, I can't think of the average American will be affected at all by the outcome of that decision. But it's more like asking the public, what do you think about the morality of doing something? It's not really the same thing as asking, what do you think about a policy that could affect you as an American, right? It's not that kind of issue. It's more of like making a deliberative judgment about whether or not it's right or wrong to do something at the bench for which you may never ever intersect with personally in any way. Do you think that's a problem for translating or bringing this approach to a bioethics context? Well, I would select bioethics topics that have some public policy implication. You see where there are potential benefits and potential harms and where some of the benefits, you know, my philosophy PhD supervisor at Cambridge many, many years ago, Bernard Williams read an essay on the idea of equality where he talked about, oh yeah, he was a marvelous, marvelous teacher, where he talked about if people could select the genes of their, if there was a method where the rich could select the genes of their children, how that would change the, it was just a sentence in there, but how that would change our ideas about equality and should that be permitted, that got me thinking way back then about that issue. But there's societal implications to the kind of society we have if the rich can purchase smart genes for their kids in the poor can't. So it may not be a direct harm or benefit, but it may be an implication about the whole, the kind of society we're gonna end up living in. So I would certainly begin the initiative in bioethics where we can identify some public policy implications with the pros and cons or the broader social implications of the kind of society we want. I think that's the perfect note to end our session and we are out of time. I've really looked forward to seeing what work you do in the area of bioethics. I'm really looking forward to that. So I would like at this point to thank our audience members for joining us in this session. Our next session next month will be on microfluidic modeling of the human female reproductive system. So get ready for that one. I'd like to thank Ashley Trautman and Helen Sethanitis for their help in organizing this session. I'd also like to thank the Center for Bioethics at Harvard Medical School for sponsoring this event. And again, thank you for joining us. Have a great weekend. See you next month. It's a great pleasure.