 Well, thank you to everybody for joining. My name is Caitlin Penya. I am the Director of Operations and Programs with the Center for Election Science. We have an exciting event for you all today. First of all, we'll be joined by Aaron Hamlin. He is our Executive Director for the Center for Election Science. And then we also have online with us today Jean-Francois Lathlier. He's a researcher at the French National Center of Scientific Research, and he's a professor at the Paris School of Economics. His research is focused on democracy, particularly on voting rules and voting behavior. And then we've also got Herod Egershine. She is also a researcher at the French National Center of Scientific Research, and she's a professor at the University of Strasburg. Her research domains are history of economic thought, welfare economics, and experimental economics applied to voting. So these two worked with Aaron on a nationwide polling study in 2016 around the presidential election, and that's what they'll all be discussing today. So I will go ahead and hand it off to Aaron. We're very excited to have Dr. Egershine with us as advisors. They're wonderful and it was amazing to be able to work with them on the 2016 project. So maybe just to kind of start us off leading in that direction. Herod, would you like to maybe summarize and talk about what we were doing with the project, the kind of methodology that we did in terms of the questions that we were asking about different voting methods, what we were looking for, that kind of thing, just to kind of get us started off? Yeah, so that's a very vast issue to start with, but I will try to to cope with it. So yes, first of all, thank you for having organized this event. So yeah, as you just said, I'm a researcher at the French CNRS, the Center for National Research, and since 2017, I'm beginning to work with Jean-François in particular, Jean-François Lalier on voting methods and with an experimental perspective. And so a few years later, Aaron asked us and me in particular to join the Center. After an invitation by the World Forum for Democracy, where I have talked about our results in France because we have conducted many experiments in France since 13 years now and a little bit more for Jean-François, actually. And so difference with what we'll talk about today regarding the 2016 election in the US, because we have studied data but focused on our survey. And what we did in France is a little bit different actually because we are studying data collected from field experiments. So experiments conducted during actually the near the official station the very day of the French presidential election. So that's a little bit different. Of course, regarding the analysis of the data and stuff like that, we can have the same kind of reasoning and that conclusion because it's essential premature to speak about the conclusion of our results now. But the methodology is really different when we cope, of course, with survey than with data collected from the field. And one of the things that you had mentioned that I think it's important to point out, both you and Jean-François have an immense amount of experience doing voting methods research. So when you go on Google Scholar and you're trying to look at comparisons between voting methods using polling or exit polling, there's not a whole lot out there in terms of like direct comparisons of different voting methods. But when both your name and Jean-François's name come up quite frequently, particularly with the French studies that you referenced. So maybe you can go into the type of maybe Jean-François, if you want to go into the type of voting methods that we looked at during the 2016 election. During this poll, the main objective was to study approval voting because it was conducted by the center. So it is really about approval voting. And because it is in the U.S., the idea was, of course, to construct contrasts the outcome of approval voting with first as a post method. Still, the survey was a little more sophisticated than just comparing simple plurality with approval voting because it also involved other questions asked to the respondents. Namely, how would you vote on what you call the, I think the rank choice model, rank choice voting model? How would you kind of sincerely evaluate the candidates? So that was these four methods, in a sense, were studied. And on top of that, there was an experimental design with this survey, which was to have some participants voting on one set of candidates, a small set of candidates, and the remaining voting on a larger set of candidates. This idea was, came out from theoretical research on voting methods, research that points that the one key feature of the voting, different voting methods, is how do the results vary when you add or remove candidates who might, in any case, be losing. But the fact of adding candidates that are losing may change who is a candidate who is winning. So these kind of issues have some importance in theory. And so this survey tried to tackle also that issue. So, Harad, with the issue of adding and taking away candidates, perhaps you'd like to talk about the long and short list of candidates and how that played in with what we saw. Yeah, so I don't know if it's possible to share something right now. Yeah, I would be proud of you. So that's here, but that's not the right slide. Regarding this issue, the right slide is here. Sorry. I think that's the one. Yeah. You've come quite prepared with slides, Harad. Exactly. The slide, not the slide, but we have tried to understand why the importance of changing the set. So as Jean-François just said, the long set with nine candidates and the short set with four candidates and to understand, in fact, why we obtain different kind of results regarding the sets we are dealing with. So maybe I should come back first on the results now or I don't know if everyone has the results we obtain in mind or... We'll leave it up to you. I think you should record briefly the main results. Yeah, yeah. So that's another slide right here. Yeah, so that's the results we obtained with three of the voting methods Jean-François just mentioned. So plurality, approval voting and range voting. And so here you have the candidates, the nine candidates. So in the short sets you have here four candidates, Clinton, Trump, Johnson, Stein. And in the long sets, you have all the nine candidates included. It's the same for approval and the same for range voting. So here you have the main results regarding plurality with the short sets, plurality with the long sets. So the person and it showed the percentage of voters which vote for one or the other candidate, the same for the long sets. For approval, it's a little bit different. In fact, it's a percentage of voters who approve of one or the other candidate. That's why the total can be up to more than 100%. And for range voting, in fact, it's an average grade each voter, each candidate. So here you can see that an important result of this study is that as you can see under plurality with the short set and with the long set actually, Clinton is the winner of this election. So first change regarding the official and national results, of course. Regarding approval, it's very interesting as well because you can see Clinton is the winner with the short set. But there is a tie with the long set between Clinton and Sanders. The results are very, very close. So there is a tie between these two candidates with the long set. And for the range voting, the results are even more interesting for the long sets because you can see that there is no tie anymore between Sanders and Clinton. Sanders is the winner of this voting method. So the range voting method, Clinton, Sanders, sorry, is the winner followed by Clinton, followed by Trump. So just to stress, of course, how to put into light these results is very interesting per se because you can see very clearly that if you change, in fact, the voting rule, you obtain a different winner. And that's a very important first point. And now shall I continue? And you, with rank choice voting, which isn't on this particular screen, also chose Clinton for both the short and long set. Is that correct? Sorry, I didn't fully understand. So rank choice voting isn't on this particular one, on this particular slide. But that also chose Clinton as well, correct, for both the short and the long set. I don't know if you have rank choice voting on there as well, or if there's another slide. Yeah, sorry, sorry. Yeah, it's another slide. Sorry. Yeah. So that's the second slide with the results, sorry. So that's the instant results you mean, the instant results. So here you have Clinton, yes, at the winner of this election. So here you have Clinton at the winner of the instant runoff followed by Trump, Sanders, and then that's the result for the long set. And so we have here our main results in this very simple table. So here you can, I mean, visually see the difference of winner regarding the short set versus the long set, and of course, regarding the different voting rules. So like going back, when we're looking at, say, like, either the grading voting method, range voting, or the other cardinal method, approval voting, one of the criticisms within those is that either selecting only one candidate or in this in the instance of range voting only using the extreme scores. I had seen on one of the slides that you had was a frequency distribution, looking at approvals per ballot. And you may also have one on range voting. So that may be interesting to look at in terms of being able to see how respondents within the study used the grades and decided how to allocate their approvals within their ballot. And that's a question from Keith. So thanks, Keith, for your question. So maybe I don't know if you would like to. Yeah, you mean the, yeah, so that's we have, I have two slides here regarding the way, in fact, people grade or the way people choose to approve of one or the other or not to approve of one or the other candidate. And there is an interesting connection, by the way, with the study you made regarding the the primary you sent us yesterday. So here you have the the distribution of candidates approved per ballot. So that's for approval voting, of course. So you can see here and we have a long this kind of long discussion regarding this issue. You have here the so this distribution and so the average number per ballot for AV. So approval voting is 1. 1.40 24 sorry for the short set and 1.73 for the for the long set. So it's not a lot. In fact, it's not very important, meaning that more than three out of four candidates voters choose to approve of just one candidate and two out of four choose to approve of only two two out of four, sorry, choose to approve two candidates, one candidates for the long set. So we had discussion regarding this very specific point, because yeah, we thought that it's not very important, at least regarding the French experiments, which turn around 2.5 approvals per ballot, which is a little bit more. And we had discussion regarding this point. I don't know if Jean-François wants to add something maybe on this issue. Well, yes, maybe I can answer on the issue of the maximal grade used by voters on the range voting. So the question is that how many people in a range voting system are going to use the extreme grades? So first, there are two extremes, the maximal grade and the minimal grade, the minimal grade when the system starts at zero, which is often considered the case under consideration. Then especially when there are many candidates, these zero also is a grade chosen by voters who just do not know the candidates or who know that they are very minor candidates who for some reasons are not interested in the race. So there are always a lot of people who choose the zero grade. Now more interesting is how many people use the maximal grade. So the theory goes that in fact all people should choose once at least once the maximal grade and that no one should ever use any intermediate grade. And that is clearly something that is not observed in the data. All the data we ever gathered on various selections always show a smooth distribution of grades so that it is not true that people use only maximal grades. This is what we observe in these experiments. Now of course you can try to save the theory by saying of course but these are only fake elections and in practice people will not do that. Of course that's a possibility but at least for the time being we have no indication that that might be true. I would like to ask that to add a point and important point with respect to range voting is that the issue is quite different if there are negative grades. We did try to test these kind of things and not alone but the presence of negative grace completely changes the picture. If we want to talk about that we could but it takes us maybe a little further away. Well do you see how do you see using like say negative grades as being material versus just using a simple scale that starts from zero to some positive end. But again we observe that. So the question was raised because there are these theoretical answers. The theoretical answers are priori answers that tells you well if you use for instance the tags zero or one like under approval voting all the tags minus one plus one it should not matter okay because why after all why why because there is a mathematical possibility that if everyone who votes zero under zero one turns to one under one minus one plus one then it makes the difference but the problem is that people do not do that. So with respect to in the last years we studied a lot of this by trying to have during the same election people using different scales okay. Our plan because in all these theory approval voting is kind of central so what we did is that we try to contrast each time approval voting with one or two other methods. Okay it's a benchmark case but as a matter of fact we do observe that negative grades have an impact as they are they carry some symbols that are different from the zero grade and that's it. So the discussion should we use negative grades should we phrase the presentation of approval voting which is binary choice as saying that if you don't vote you vote against this is something that is not well settled okay it is not clear I can give you my feeling but there are arguments for and against for instance reasonable people tell me that they believe the the mere possibility of saying that you are against okay as some democratic value and people do not think about that but once they they are given this possibility they realize that it's very important and so that's an argument in favor of using negative grades. There are arguments against that and I think an important argument is that in these systems you must be prepared to use these systems with many candidates including many candidates that are not known by the voters and that are not known for good reasons because they are odd candidates that and so many people will not express anything about these candidates okay and naturally they will use a zero if there is a zero for for these candidates okay so it makes the comparison with candidates that are known that receive negative grades a little bit strange because you compare people who say nothing with people who say something so it's a bit odd if you want my feeling about that after many open questions I thought that voting is so simple that it solves this kind of dilemma you know Harad when you showed the earlier slide and the frequency distribution for the number of approvals per ballot even for the long set like you said it was 1.7 approvals per ballot so given that like even with as few approval voting as as few approval approvals per ballot as we saw what kind of impact did just did that did that have compared to just being allowed to choose one so did being able to choose that extra on average 0.7 per ballot for the long set or 0.3 per ballot for the short set what kind of effect did that have yeah and yeah it does have an impact because as we just speak about about the results and the comparison of the results between plurality and approval so so you can see that in spite of the fact that 1.7 one man say yeah but it's not a lot and it doesn't make much difference but actually when you look at the results it's clear that it makes a very important difference as you can see here with this slice again this slice with the result which is important of course you can see that it makes all the difference between the plurality and and approval because the winner is different so the and we obtained this kind of results as well in France in our experiment in 2007 we got a different winner in a way between the the true run system which is this the official voting system in France and the approval voting and we obtained as well this kind of results in 2017 with not directly with approval voting but with different kind of range voting so even if people change only a bit their behavior it shows that it can change a lot the results and and so in terms of changing the results you you referred to an earlier friend study I assume one of those is referring to instead of Sarkozy winning Bahru winning instead is that one of the ones that you're referring to yeah exactly yeah in 2007 we found a different a very important difference yeah regarding the who could have been the the president of the french president and it was Bahru that is a centrist candidate who could have won the election on the approval voting yes instead of Nicolas Sarkozy um and uh John François perhaps you can answer this one so one one finding that seems to come up uh a lot so like saw it in uh one of the german studies and also seeing it repeatedly in the studies that you conduct that you as well as Farad have conducted with the french studies and that is the reflection of support that candidates get who don't win uh perhaps you can talk about that yes so uh obviously the the the main criticism of plurality voting or single name balloting is that people have to not lose their their their vote and therefore refrain from for candidates do not stand a do not have a chance which has an as an obvious consequence that these candidates do not get their share of votes in essence okay so uh this is um I guess uh an important point uh in for you in the US uh with uh plurality voting uh this is uh also true I think it's important to to to realize that it is also true in the in the runoff system despite the fact that because um the the presence of the runoff you can if you wish a vote at the first round without strategic consideration and not uh having any fear of wasting wasting your your vote and say okay I do uh I vote with my heart for during the first round and in the second round where the choice will be really serious then I will vote uh in a purely strategic manner and uh this is something that is often heard this is something that is at work in how people think and practice their their voting in this system but still it is not the case for everyone and it is just uh wrong to believe that uh people at the first round of a runoff vote sincerely and and therefore that the candidates have their fair share of votes so I could elaborate on that but but I mean I mean this is something that is very well known in in countries that practice uh runoff system this is something that is discussed uh uh every time in the in the families and in the bars before the elections so so these issues of strategic strategic voting at the first round of a runoff is something that that is a political reality I mean and I think this is something that you might have in mind in in the US because I know that you you are also discussing the the idea of having instant runoff okay and the argument goes that with instant runoff you can vote for the candidate you you you like because if he's not uh if your vote is not counted for that guy it will be counted next well that's not going far enough because in countries that do use this kind of system people do have to to to to to vote strategically even uh only at the at the first round okay compared to that the uh systems where you really ask to to to give your opinion to each candidate independently from the other candidates then you leave open the the possibility that uh each voter uh just say what he thinks about each candidate or say whether he approve or not each candidate uh whether or not this candidate stands a chance to win or not okay so it's not an absolute truth but uh this mechanism is at work in the mind of many of voters so that at the end it seems it looks like the picture that is given by this method is to all the candidates including the ones that stands no chance of winning is more true to to to what people think and uh Harad since you've you've you've kind of so prepared with all these uh slides um perhaps you can so um one argument of different voting methods is like this idea that uh third party and independent candidates candidates that aren't quite as mainstream can get that reflection support um how do you see this uh playing out with the data that we saw with uh ranked choice voting approval voting and range voting uh for uh independent and third party candidates relative to how they were reflected under um the plurality voting method oh and you're you're muted at the moment so yes you you mean uh with this um with this study um not sure exactly how to uh uh so so I can different kind of candidates we are dealing with with what and how my uh I mean uh for for example um and the 2016 election some of the third party candidates were uh folks like uh particularly johnson and stein were the main third party candidates in the 2016 election uh so how did voters ability to use something like approval voting or range voting or even ranked choice voting how did that affect uh the reflection support that these candidates got relative to um the choose one voting method as well as relative to an honest assessment yeah it seems to me that uh regarding this little candidate right uh it doesn't change um much uh yeah okay yeah so it depends in fact of the the kind of candidates uh we are dealing with uh regarding the the this study specific study uh I accept of course the change a very important change regarding uh bernie sanders uh it doesn't seem to me uh except you if you have different opinion please cut me but uh it doesn't seem to me that's the the role of this lead very little or very minor candidates uh is very different uh regarding this different voting rule probability approval or range voting or even instant runoff voting it doesn't seem to me that we have put some light on these issues uh I have here maybe just this slide but I think it's too it's not very appropriate to show you this um it in the difference uh regarding France actually is that some uh little very little candidates uh have much more lights uh with approval or with range voting uh that's the case for little candidates such as under probability or under the torrent system such as uh Jean-Luc Mélenchon for instance or not Jean-Luc Mélenchon but uh uh or if I should say sorry or Olivier Besançon in some elections in some previous elections so they got much more uh approvals they got they got much more support and it shows that they have really a role to play in in the political landscape and in the in the in the country or in the people are voting and so on and so forth but here for the U.S. presidential election I'm not really sure that we have we could show with the data we got that this little candidate played a real role except except Sanders except Cruz but uh yeah for example like uh with under say plurality voting within the short-sighted candidates when they're only Clinton, Trump, Johnson, and Stein uh it looks like uh Johnson uh gets 8.25 percent under under the poll uh whereas uh for under plurality voting whereas under approval voting it looks more than than double so so it seems like uh perhaps here uh they're they're still they still don't appear to be contenders so it looks like they were never really in the in the race as a as a competitive sense but thinking more about the reflection the support that they got so like Johnson going from uh 8.25 to over uh 20 percent uh and Stein going from three and a half to uh just over 11 percent and thinking about that like so here we're seeing that relative to priority voting and approval voting also thinking um it's the metric here for for ranges we're seeing here might be a little bit hard to compare uh but we may also be able to look at that with relative to say uh ranked choice voting say looking at how well a candidate did uh before they were eliminated with particularly looking at Stein and Johnson for instance yeah yeah yeah that's true but um um I think that the the main point of this result is not really here but yeah yeah you you are right of course but I have a I have a figure somewhere with this gap of scores between approval and plurality actually but I'm not sure I have it here but uh we see that uh even of course there is a kind of mechanic uh increased uh for this uh candidates it remains uh it remains kind of low uh as comparing uh with the the change of behavior with the with Sanders for instance that's it's a very important difference and for this other candidates it remains uh much more much less impressive I would say but I have a figure here but I'm not I'm not sure I will be able to to find it right now um and that's and that's how in fact we can uh define the different kind of candidates actually that's typically with the we can say the gap of scores uh that's the kind of study we have we have conducted in this paper we can define the kind of candidate and uh regarding how uh uh the how important is this gap between approval and probability arrange voting priority and so on and so forth and that's how we can define the kind of candidate we are dealing with and here uh John's with this definition in mind uh or with this idea in mind uh it seems that uh even if there is a some difference uh between Johnson between approval and probability and so on and so forth Johnson still remains uh a little or candidate according to to this uh definition but I don't know if we can continue or pursue on maybe I can say a word on this sure on on on this issue so with many uh candidates of course the it opens the the possibility of more renewal of uh political ideas and uh the appearance of new uh parties and things like that and and it has been a major political at large event in the recent past the emergence of green parties in in Europe uh in in the US of course it's uh impossible to to to see that because you have really two party system for the moment that you are more or less blocked with that and in uh in France it's a little bit the same as in the US we we we have these these rules whereas most of other european countries have system which make it easier for new parties to to to uh to appear and this is what happened in uh for instance in germany and in several other countries and the political landscape in these countries and therefore the the the landscape of ideas has changed now in France we we we observe that when we use uh approval voting the green candidates uh really happen to the same importance as they have in in uh in other european countries but uh they are bad they they cannot make it uh in in politics due to uh uh you single name balloting okay so i think we we saw uh in uh in France we we we use um the comparison between uh approval voting and single name balloting what has been predicted by by by the theory we see the advantage the statistical advantage of these kind of rules that can uh widen the political spectrum and this is i think the the the real particular importance of these feature of approval voting to to give a fairer chair more fairer chair to to to so-called small candidates so i think we can we can approve but we can really observe already that in in this kind of data and uh haraad with uh with this data that uh that we're seeing were there any uh was there anything um that you found like surprising relative to other data or uh going future into other studies are there types of questions that you would like to see uh asked and you and you mute it at the moment haraad um yes i i just say that i mean one of the surprise it seems to me uh learning this data and that's why the the results of the the survey you conducted with the democratic uh party is interesting uh at least in my opinion uh is um sorry i come back on this on the number of candidates approved by ballot as i just i was saying that we were a little bit surprised by the fact that uh the number of uh average um candidate approved by ballot remain a little bit low even if it changed the the results but still and um that's one of my that was that was my one of my interrogation regarding uh these these results as compared to to what we've obtained uh with the the french experiments and uh we wonder if it was a a kind of you specific us behavior uh i mean us voters would be less able to vote for more than one candidate or one or two but not more and so on and so forth and with our results you just uh posted regarding the uh the democratic primary we obtained kind of very very different um results uh i mean i mean global statistics with the the average number per ballot under approval voting very important number actually i just noted one of your results it was um four point nine candidates uh approved on average per ballot so that's uh very important of course and uh so that's one of the of the thing we can i guess uh put into light uh as a comparison between this study and what you just obtained that's what what one of my point of interrogation before going into other folks questions kind of a follow-up uh to that and perhaps uh Jean-François you can answer this one uh in terms of that uh discrepancy that Harad highlighted with the type the number of approvals per ballot that you see with some of the french studies uh that that were done versus what we're seeing here um in terms of thinking about explanations uh for instance like leading up to the election in 2016 there was a Gallup poll where uh response were asked whether they even knew who stein and johnson were at two-thirds people said that they didn't know so so in terms of the media attention uh that third parties and independents are are are able to get in the us relative to france and i'm i'm less aware in terms of the uh media attention that uh third parties and independents get in france so perhaps you can um highlight that as as a particular explanation yes yes absolutely so so so uh the situation is completely uh different uh in uh in france and in the us with respect to that so in france too we have often uh we have usually more than 10 candidates something like 12 12 candidates say uh for for this most more important elections among those there are uh candidates that are not known or that are not serious there are um maybe each time one or two and all the other candidates are very well known including those who have very low scores for instance uh we have always uh extreme left uh candidates trust guys candidates these guys are make uh they have all almost no votes very few very few people vote for them but see they are perfectly known and uh their image their ideas are perfectly known so uh this is completely completely different with the the polls that we we have in the in the us where we face this this forum um i think that it should be important to to to have um experience experiments uh done in the us maybe in the local elections with with uh more than and uh with more candidates uh known by the can by the voters if this exists i don't know actually you you have a problem with the uh implementation and research on that well voting uh in in the us which is that uh these uh rules shape the the the party structures okay and the party structure in the us is really uh defined by the current voting rule okay this is a basic uh take away of political science uh and if you want to change the voting order that you don't know maybe yeah and uh kelly maybe can highlight some of the other questions that we have from some of our viewers so i think we were able to cover some of the questions that we're already in here but if anybody does have questions feel free to go ahead and um put them in the chat uh there was a question from michael weinbaum he was asking about why the percentage that clinton and trump got in our poll under plurality voting why would that differ from the actual results of the election so i think it it it's not so different it's not so bad because this was a poll uh made uh before the election something like two weeks before and so the the week leading into the election yeah one or two weeks so so things change even in the last days of the elections so uh yes uh we don't have exactly uh we could have if we if we would have rescaled the the the the ballots observed in order to to match the exposed what we know exposed but of course we didn't do that so uh the result is different it's quite close uh the fact that it's quite close proves that that the sample was good the sample was sort of not that and people uh answer these questions the way we have them to answer please tell us what you would do in the election if it was great thank you um tommy was also asking he says sometimes score voting allows leaving no rating for a candidate right that way their vote doesn't contribute to the candidates average score so no rating versus zero rating are two different ballots yeah yeah so we can answer that that because uh um uh sometimes score voting allows uh what what that means it allows so the the definition yes so so so the issue is how uh are you going to to what are you going to say to the voters uh what is will be the form of the ballot okay what do the uh voters will understand of what you tell them how they would uh infer what they can do from the form of the ballot okay and how you count the the the the votes the ballots after that okay so it's it's a it's a more complicated issue that that than it looks this kind of this kind of question so uh would uh is no rating different from zero zero rating okay uh you cannot decide on that okay you can take a you can make a decision but then you have to and say well well i i for instance you can write in the in the legal definition of the rule that uh it is the same thing or that it is not the same thing but then you have to to check whether or not people understood that and whether or not they can have accepted that okay and we we looked at that quite in uh quite in details and uh try to to frame the ballots in different ways in different places where we we did the experiment and things like that and the answer is that there are no good solutions to that there are no good solutions for instance it's not true that if you just write down well uh ballots uh will be uh if you if you write nothing it will be counted as the smallest uh value or it will be counted as zero it's you you cannot uh be sure and in fact it's not true that people immediately understand and behave that way that's not it doesn't work so so you have to cope with these uh issues of uh understanding and uh i think this is a problem for uh a negative voting it is a problem for negative voting as a matter of fact you have always uh uh an important part of the voters that uh use zero as what the grades they give uh if they don't know as they don't want to grade okay and they uh are confident that the zero will be no point okay so um the strengths i think the strengths of uh approval voting and range voting is that it can be understood by the voters as a quantitative thing we add the points okay even if you don't say what you are going how are you are going to count ballots that looks like point voting uh people will infer true or not that that's what you are going to do okay and that is true also for that the simplicity of approval voting you just count you add the one the ones okay uh and uh in this framework uh zero is just zero okay and negative is something else you cannot say that negative is taking things away it's uh different yeah it's interesting how much psychology goes into voting systems right yes you you must face the fact that psychology pops in yeah absolutely um we do have another question from Colin he asks so how does this research inform 2020 and beyond what research is taking place this year or in this us election cycle um so maybe Erin you want to um note what we're doing and then um Rod and Jean Francois if you if you all have thoughts as well uh sure i can uh highlight real quick and what we're looking at uh for the uh 2020 election as well as what we've done in the primaries and uh of course uh want to hear more about uh uh the uh the work that uh Leslie and Dr. Egersheimer are doing for uh for France as well as uh how we'll be looking at the 2020 election right right now for the so we've already done a lot of the uh democratic primary uh election using polling using a lot of the same methodology that we've done in this 2016 election which is comparing uh four different voting methods having a control measure the control measure we did for the primaries was both this um honest assessment scale as well as an honest ranking um to be able to look at kind of an honest uh be all winner or honest condorsay winner as well as an honest kind of high utility winner so we we've already done that within the uh democratic primary uh looking forward uh um working with uh uh um Dr. Francois and Rod again in the 2020 election um uh doing really a similar type of setup and like it's still kind of early so it's it's hard to know uh all the all the details in terms of uh what will be going on what what I I kind of hope uh and I'm sure that uh Jean-François and and Harad are also hoping it's for some kind of interesting election so right right now in the U.S. uh for the general election it doesn't look particularly interesting um so uh only recently has uh the uh a third party candidate uh with some kind of name notoriety uh jumped forward um but if there are only two candidates it's not a very interesting election and even if there's a third candidate or more they need to have some kind of traction to to make the election interesting so for for research purposes kind of hoping for an interesting election to be able to uh to work with um but uh uh Jean-François and and Harad uh what uh what kind of things are are you looking at and are you thinking about with both uh uh U.S. elections as well as uh French elections or elsewhere uh maybe Harad you can you can lead us off uh yeah yeah yeah so uh you're right uh Aaron regarding yeah the 2020 election in the U.S. we'll see uh for the moment in France we still we are we say very much focused on the on the big uh elections in France or the the big rendezvous that is the the French presidential election so we still have a little bit time to think about it here uh two years um probably and um but yeah for for now uh i'm continuing to we are continuing actually uh to work on the on the data collected in in the in the previous election in in France and so we are continuing as well to to work and I hope to to be able to continue to work with uh with our room as well and and but we still have work to do on the on the data collected in 2017 actually in France so that's uh and in parallel uh i'm conducting as well some experiments in um uh in the lab and so not in the field but in the lab and uh in the lab on voting uh different voting rules and it's as an actual but uh it's uh very um and lighting as well because um uh it shows uh much more precisely uh this psychological trait uh in fact of the of the voters and uh they behave very differently uh in the lab as compared uh comparing with the behavior in the field and that is especially true uh regarding uh strategic voting uh and regarding uh what Jean-François was talking about that is the the the the scales with negative grades and that's very very um very interesting to compare in fact this uh two kinds of uh of behavior lab and field uh focused on on voting issues or voting methods yes so um research also is uh is going on in the important field of uh proportional representation and and voting rules uh under proportional representation which is a very vast and complex uh uh matter uh i i believe that uh we we've done a lot we with the uh main elections uh so the presidential elections uh in presidential system countries like the US or France of course are are the most most attractive ones they are the the most important but i i guess that if there are things to to to change the uh voting systems uh it will also come uh from uh other voting rules being used repeatedly by many people in in various circumstances so uh studying uh voting uh in less important settings but still important ones uh be political elections or uh elections and decisions uh in different settings is uh also valuable both for research and and for uh popular science i mean and for for culture in in general thank you um all right we've got two more questions and then we'll go ahead and wrap up let everybody get back to their day or their evening um so Keith earlier asked a question that he thinks maybe needs a little bit of clarification so he says his question is if a large fraction of voters score any candidate at the maximum not if every candidate is scored at maximum or minimum um oh wait keep can you clarify that because um here i'll unmute you that way you can ask go ahead so um there's this kind of two issues one is that everybody votes max and min and that's kind of viewed as a bit of a flaw the other thing is if no if a lot of people don't use the full range if they don't use the maximum value at all those are kind of and you'd want that you'd want people to kind of normalize relative to the candidate so you'd want them to have somebody at the minimum and somebody at the maximum so it's it they're kind of inverse problems really does that make sense yeah so so of course it's it's different depending on on the scale so we we use a small or large scales of of range voting so um with large scales and we used two large scales one is 0 20 that may may be strange to to you but that is very natural in France because it's the way kids are graded in schools okay so 0 20 is very natural things to do in France and 10 is a pass normally 10 is pass fail is at 10 okay and we also used a continuous scale which in fact is is possible when people vote online because you can click on on a line and it's also possible when people use paper and pencil because what you ask them is is to to mark a point on the on the line and then you can scan these and automatically get so with these large scales okay you really can ask the questions how many people will go to the maximum we many and there are people who go to the maximum for some candidates okay but the global picture is that they they tend to use a lot intermediate grades okay even if with these very large large scales and of course with small scales as for instance 0 1 2 the question is a bit different okay so the question is a bit different then you can ask the questions will people use the grade one in the grading system 0 1 2 and the answer is yes many people do for many candidates I don't know if you just answer the question maybe I just can add something regarding how people use intermediate grades and I was just talking about it in fact two minutes ago when I sorry I put I was talking about the difference between lab and field and what is interesting precisely and strategy and the issue of strategic voting and what is very interesting in fact when one compare this to a kind of of studies is when can see that people actually use intermediate grades much more much more in the in the field than in the lab in the in the lab even if you propose a 0 20 scales for instance as I was saying about that people will focus on 0 and 20 they won't use intermediate grades they they understand very quickly in fact how to vote strategically or to use this this grade this scale but with with the real election or with experimental election at least they are using intermediate grades so the so we can confirm through lab experiment that people know how to use maximal grades they know how to use it and they know how to vote rationally or strategically in a way but they prefer to vote through intermediate grades so that means that they they want to to say what they really think and not to just to vote strategically rationally if I may all this is pretty much consistent with what is known in in political science that most people vote sincerely that strategic voting exists a non-negligible fraction of of people do so okay but they they they they are not the majority and that's that is what what we see also in these experiments there is nothing much new I think here so the point is rather that voting strategically or sincerely doesn't mean the same thing on the different rules that's an issue okay thank you so much for that explanation all right we've got one final question from Colin he asks what an approval voting potentially enable a third party who isn't running to run and receive substantial support so if no substantial third candidate runs in 2020 plurality general that's not evidence that no third party candidate could have run in a 2020 approval general so essentially he's asking doesn't doesn't approval voting give give third parties a better chance at running so yes I think that that's all research on voting points to to to that I don't know if there exists proof in this matter but I mean it's I'm confident it is true all right well thank you all so much for joining us Sean Francois and Herat I know that you both are all the way over in France right now it's probably around your dinner time we're getting close we really appreciated you taking the time to chat with our supporters and explain the results of the election and I know that we're all looking forward to seeing what we what we do here in 2020 with the general election and and the studies that we're planning to conduct okay so thanks for for the questions I see that people some people have been asking whether the data was available actually there are plenty of data available on this on these things the data on fresh elections we are plenty of data that can be studied it's not immediate to do so because the what we did we always as Herade said work not with representative samples okay so it's difficult to to to study cities and the advantage of what has been done at the center with this pool is that it is done with representatives representative samples so it's much easier to use okay but still this data exists and I guess as soon as the the the paper we we we we have in writing on on on the last presidential election in the US is published the data will be available I'm in favor of releasing data immediately everywhere so don't hesitate to contact us if if someone is interested