 So I want to welcome all of you, thank you for coming. There are a lot of you in attendance and for good reason. We have a lot of exciting stuff today. And we have a wonderful guest, Kapi Nyi. And for those of you who aren't familiar with our work at the Center for Action Science, we empower voters through voting methods that strengthen democracy. We look at all kinds of voting methods and we have a number of events talking about these different types of voting methods and how to look at them. And one of the interesting parts here is that sometimes voting methods can be a bit complicated. So we're lucky and fortunate enough here to have Kapi Nyi with us. He was going to talk about some of his work about how that's been able to clarify the way that we look at voting methods. And it's a pleasure as executive director for the Center for Action Science to be able to introduce Kapi Nyi. So I'll give his introduction. So Kapi Nyi is a software engineer with a background in election systems from his graduate studies in computer security at UC Berkeley. While there, Ping happened to be in the right place at the right time to participate in a voting machine security review commissioned by the Secretary of State in California. The results of which led to the de-certification of all voting machines without paper trails in California in 2007. Since then, he's worked in humanitarian relief with Google and Doctors Without Borders and more recently in progressive politics at Tech for Campaigns. So it is my pleasure to welcome Kapi Nyi. So thank you for joining us. And I, before the call about how much of a personal fan I am as well of your work, I know getting into this space and as something more and more about voting methods, sometimes it can be kind of complicated and being able to have folks like you out there to be able to be able to take some of these sometimes complicated ideas and be able to create visualizations with them to be able to kind of give our working memory a little bit of a break in some cases to be able to think of the kind of higher order of working with it and sometimes just to be able to understand what's going on to begin with. So maybe to start, we can talk a bit about your history working with election security. So we could talk about that and then go into some of the diagrams that you've developed. So how did you get into looking at election security and some of these voting machine issues? Yeah, thanks, Erin. Thanks for that really glowing introduction. It's a real pleasure to be here and to see all your faces. Some of the names in this room actually I recognize and am excited to actually meet. Not going to meet you in person. So it's a real pleasure to be here. So I did my graduate school at UC Berkeley in computer science, came out here from Canada and initially was interested in system security and human-computer interaction. So sort of the intersection of usability of computer systems and security, which tends to be a problem area for... It's a very common problem area. Things that are highly secure are often hard to use and people often sacrifice security in the hope that it will make things easier to use. And so there's a real question about how to make things that are both usable and secure. And as it turned out, at the time I was in graduate school, there was a lot of controversy over the use of electronic voting machines here in the United States, particularly in California. And a number of academics, computers, scientists and others had some pretty strong positions and really strong concerns about the reliability and potential vulnerability of voting machines. And that's how I got into thinking about voting machines as a sort of offshoot of my security work. And at the same time, because I was really interested in usability and information architecture, I also had a side interest in information digitalization. And so that led to some of my other work, trying to make things more visible and understandable. I always thought that there's a big gap between the information, the data we have and how people are able to understand what that means. I saw with some of the work you mentioned focusing on making sure that voting machines had a paper trail. Like a lot of times when we think about election security, some people look at ideas of like internet voting or online voting, but then you have ideas like man in the middle attacks and then other types of vulnerabilities. What were some of the issues that you were concerned with when you were say like looking at some of the voting machines in California in terms of issues that you wanted to make sure that weren't present in actual elections? So in that specific case, so there is, I should maybe just give a little context. So the research that I was doing was about sort of end-to-end security that is like, when you run a whole election, how do you actually know that the votes you've counted and the results you have from counting the votes are actually the votes that people cast. And that was a little bit broader and ended up sending me in a certain direction. And it just so happened that during that time when I was doing that work, this political controversy was also happening. And so the Secretary of State of California at the time, Deborah Bowen, decided to, I think it was pretty unusual. I've never actually heard of something like this happening before. She actually ordered the voting machine companies to hand over their source code. They're all proprietary systems and they all still are. That's one of the criticisms of them is that our voting systems run on software code that nobody can see or audit. And so she actually ordered them to hand it over to the Secretary of State's office. And she commissioned several computer science professors to investigate that source code. And it so happened that one of the professors she tapped was my advisor, David Wagner. And so he looped in me and some other graduate students to sit in a windowless room for a few weeks on the Berkeley campus with this super secret source code that we couldn't take out of the room. We just had to go in there and like page through it to look for bugs and security vulnerabilities and write a report about it. And that was actually really exciting. It was pretty cool to get to look at source code that at least in theory, no one outside of those companies had seen before. And it was both shocking and entertaining to discover the volume of bugs and issues that we found. When you say shocking and entertaining, you mentioned some of the bugs, but was there anything in particular that really came to mind in terms of like that you just really didn't expect or cut you off guard? Just, I mean, there were, there are pretty pervasive problems to write. So the, you know, the system we looked at, we looked at source code, I'm just reading the questions in that chat here. So we looked at source code on computer screens, but you know, we had to write up a report on what we thought might, you know, might go wrong when these things were actually deployed. And there are multiple systems involved, right? There's the direct recording electronic machines where you punch the buttons on the touchscreen. There are machines for scanning paper ballots. There are machines for taking all of that data and aggregating it. So in this case, they had a cartridge, a memory cartridge that you had in the machine. It would record the votes when you use the touchscreen. And then they had to like put those things together and sum them up on a PC. And then they had PC software for the election administrators to manage all of this data and tally it up. And every single component of the system had pretty glaring security flaws. For example, one that I just remember off the top of my head was that the tallies that were added up on the PCs, right? The program that was for your desktop computer to add up the tallies was on a Windows local network and everything was transmitted in the clear over the network, including the password. So if you're on the same network, you would just become the administrator and change all the election totals. So, you know, very, very basic issues. And then we also found some sort of more interesting issues that maybe were a little more far-fetched that were sort of interesting and that they were possible and that it turned out components that were supposed to be inert basically, you know, just contain totals for reasons we don't really understand. They decided to invent their own little interpretive language which could run code on those memory cartridges which made it possible to infect the memory cartridges with the virus. So in theory, technically it would have been possible to introduce a memory cartridge with a virus, you know, put it in a machine, have it infect that machine and have it infect the next memory cartridge put in the machine and have it infect the other machines and change the voting results. So, you know, there's a fun report. You can read about this if you like in which we go through a lot of these issues. Now, I think one question I think a lot of folks and people's minds is, of course, all of these issues were fixed, right? Is that the answer that you have for us? Sorry, I was just replying to post a URL in the chat because people are asking where to find the report. So the URL I just posted is the URL of my page that has a bunch of different voting information on it and the security review is one of the items in that page. I'm sorry, Aaron, could you repeat that question? So you mentioned being able to identify a number of security flaws within the voting machine software. So I think a lot of the, what's on people's minds at the moment, given that statement, it's like, of course, all these issues were fixed. Is that correct? Not at all. I mean, we, of course, strongly recommended that they be fixed and we actually made a bunch of recommendations in our report. The public version of the report, which you can see at the URL I posted, is only part of the report. So we wrote it in two sections where we sort of documented as many of the issues as we could in a public fashion and then took all of the really sensitive information, like the actual passwords we found or the actual vulnerabilities and how you might exploit them and put all that in a private appendix, which you did not see, including some recommendations for how to fix these things. And we have no way of knowing that those recommendations were followed. We did not get to review the machines again. The conclusion the Secretary of State reached was simply that it was not a good idea to rely on the correctness of the software. And so that, I guess I just wanna highlight like one of the principles that most of the computer security experts in the field of voting will refer to, I think it's the most important principle with respect to voting, is the principle of software independence, that any system you design that's gonna be processing votes with software has to have some mechanism for recovering from errors in that software, such that you can show that the result is independent of the software failing. And so the Secretary of State concluded that the only way to really be sure was to record everything on paper. And so now in California, we record all of our, I mean, we still count them with machines, but we record the votes on paper. And looking at this, have you learned anything else about other states in terms of like how they do things? Well, I guess like within voting machines, there are only a few companies that have, that they use voting machines at the US. So like, do you suspect like that these issues are more problematic value that, so do you think like California is an outlier here in terms of these issues with what you've seen here or given how voting machines are utilized through these handful of companies at the US, like do you see this as perhaps a more pervasive issue? So I think the issue of flaws and potential vulnerabilities is pervasive. There really aren't, there are no real significant incentives for the companies who make these machines to really harden them or even audit them for security. They're private companies, they just sell these machines to election officials and make claims. And election officials usually, they're stuck with the same machine that they used for several years because they have a lot of processes developed around it. They don't wanna retrain people, getting a new system would be a lot of work. And so often they use machines that are really old, often they don't even have funding to buy a new machine. So the incentives are really, really messed up. So California, I think at least in 2007 was ahead of the game in the sense that Secretary of State was actually like fairly technically aware, which, as you know, it's not something you can say of all politicians. So we were really lucky with Deborah Bowen and she understood what source code was and what the importance of it was, why it was significant to have it audited. That's already like a few steps ahead of the game, I think. So in many other states, there are still, it's still totally normal practice to record votes with no paper trail. So like there are other organizations out there like Verified Booting and other academics like Matt Blaise who look at these types of election security issues. Given your experience with California, if you could just say like a particular policy that would be nice to be implemented, that would really address a lot of these problems. Like what kind of policy, like if you could just kind of wave a wand and then there's the policy here, what kind of policy? There's, I mean, fortunately in a way or, I guess the good news is that there is a pretty simple, like a pretty straightforward recommendation to address these problems that almost, pretty much everybody agrees on. It's not like there's a lot of controversy in terms of like in the academic professional space. Matt Blaise is actually, I'm a big fan of his too. You know, I worked together and he was one of the investigators on the source code review in California. But here's the thing to remember if you're lobbying for a policy, for example, or talking to an election official, there's two things that you need. The first is you have to have a durable paper record of the votes, whether that's the actual ballot that was marked by the voter or a printout that was verified. You know, the voter actually looked over the printout and confirmed those are their correct votes. There has to be some kind of physical record of the vote that is managed with a physical chain of security from where it is created to where it is stored. And the second thing is that you need to establish software independence by auditing the results of the election according to that physical record. And there's a paper on this, came out, I think it was like maybe, oh age or so about how to do what's called a risk limiting audit. So you have this question, right? So you have, you know, 100,000 ballots or something like that. Okay, now how do you decide how you're gonna audit? Are you gonna recount them all by hand? I mean, it's really expensive. So what you do is you establish a maximum probability of error that you're willing to accept. Let's say, you know, I wanna be 99% certain that the outcome of the election that I got when I counted the votes was the correct outcome. And so you can work backwards from that using that probability of error, that limit and your knowledge of what the margin of victory was in the election to determine how many ballots you need to sample and audit. So if the margin of victory is very large, right? Then you only need to sample a few ballots to confirm that like, okay, the person you selected as the winner is probably the winner because there would have to be a lot of errors in order to reverse that victory. On the other hand, if the margin of victory is really small, it's close, then you need to sample more ballots. And so you can calculate how many ballots you need to sample and then randomly sample them, you know, audit them against the actual counter results. And that gives you some, you know, actually like quantifiable assurance that you got the correct result in the election. And we're actually seeing that come into play in some places like Colorado, for example, I think they were one of the pioneers there. Yes, yes, that's right, yeah. So the most important and successful things, policy changes, I would say, in some are paper trails and risk limiting audits. Before we go and transition into the diagrams, is there anything else that you want to highlight with some of the election security issues and also related a little bit to our audience? Are there elements of that that you see being a bit different when you're talking about different types of voting methods or different types of data, whether it be or no data or cardinal data and how those things change? Yeah, yeah, I was noticing in the chat, Rob Lanfeer, if I've pronounced your name correctly. Hi, yeah, really glad you brought up that point about some ability. So, exactly, if you can sum, if your election method is summable, then you can actually do smaller audits that give you useful results because you can sum the results into little precincts. If you're Lanfeer, okay, great. If your voting method is not summable, then you can't determine the outcome of the election without collecting all of the ballots in the entire election. And so that makes it much harder for you to do a useful audit. And in terms of precincts and mobility, I think the voting method that comes to everyone's mind in terms of not meeting that particular criterion is instrument off voting or a rank choice voting. Yeah, I'm of the opinion for a variety of reasons including the diagrams that we're about to talk about, but also other reasons. I feel quite alarmed and concerned about the popularity of instant runoff or as they call it rank choice voting that seems to be taking hold in a lot of places in the United States. And yeah, it's concerning for many reasons including the lack of some ability. And I feel you're paying too with the naming the idea that they would take an entire class of voting methods. But we're all with it. So transitioning into the diagrams that you created. So you've got, from your website, there are really two that stand out here. One, which I think a lot of people think of with the E diagrams are two dimensional. But there's another one that looks at political ideology and frequency of votes where you can move the distribution. I think that one is maybe a little bit simpler where you look at ideology on like a one dimensional plane versus a two dimensional plane. And so perhaps we could talk about the one with fewer dimensions first and then we can go up to two dimensions. So you have the vote simulation tool, perhaps we could talk about that first. And I'll give you a moment to share your screen if you like. Yeah, let me share this window and see if that is visible to everybody. I'll try and just arrange that so that you can, so it scales well for the screen aspect ratio. So what do we look at here? Okay, so first of all, I just wanna say this page has existed for a while. And if any of you have gone to it and found it to be broken, I apologize. This was actually written way back in the dark ages when people wrote things in flash. So this is a flash animation. And I only just discovered this morning that there are the New World Web Assembly, people have written a flash simulator that you can use to make flash animations work again after Adobe deprecated it and turned it off last year. So this is back to me working again. And what we're looking at is an extremely simplified idealized political map in which everybody's opinion is expressed as a point on a one-dimensional spectrum, just a left to right, all the way left or all the way right or somewhere in the middle. And we know that doesn't actually fully represent the way people actually vote, but it's a nice visualization to at least help people understand how we're doing these simulations. And then we make an assumption about how people are distributed and imagine that they're distributed along a bell curve on that spectrum. And then we imagine we have some number of candidates whose positions are also indicated, described using a single dimension. So this represents the red candidate and this is the red candidate's position. This is the green candidate and the green candidate's position. And the shading you see up here is imagining what would happen if each voter, voted for the candidate that was closest to their position. So everybody on this side, all these voters vote for the red candidate and all these voters vote for the green candidate. And this dividing point is exactly halfway between them. So what's going on here is given this distribution, the way it looks right now, you can see that the green candidate would win because roughly three quarters of the voters would vote for him. So that's this part, the green shaded part of this bell curve as compared to the red shaded part. And you can also see that correct by clicking on the particular voting method. That's right. Yeah. So this is a description of how people will vote and then underneath it, each of these bars shows a particular voting method and then what the outcome of the election would be where the position along the bar is the center of the distribution. So the fact that this is green over here means that in a plural election, if this center of the distribution is over here in the green region, then the green candidate will win. And if it's over here, the red candidate will win. So this is pretty intuitive for, I mean for just a two candidate election, all the methods behave the same, just depending upon whether the center of gravity of the electorate is closer to the red candidate or the green candidate. But once you introduce more candidates, then it gets more interesting. So I'm gonna add another candidate over here. These are our favorite elections. Elections with two candidates are so boring, we don't wanna see those. Yeah, they're also really problematic to have only two parties, but that's yeah, that's another serious problem our system has. Okay, so I'm gonna add this yellow candidate and turn them on. And so now you see that the behavior is a little bit different depending on the system. So again, each voter is voting for the candidate that's closest to them. So now there's a segment of voters over here that votes for the yellow candidate. And as you're probably familiar, if the candidates on the sides can steal away enough of the distribution, they squeeze out the candidate in the middle, even though maybe the candidate in the middle might be acceptable to more people. And you can see that this problem is worst in plurality elections. You can see here that the width of the green bar is narrower or wider depending on the particular voting method that's being used, even though they're all seeing the same election. That's right, that's exactly right, yeah. So in this particular example, I've just set up here, right? Where the center of gravity is located right about here, where the circle is. The red candidate is winning, even though the green candidate actually is closer to the center of opinion. And that's because the green and yellow candidates have split the vote on their side. When I... This is interesting too, because I think like a lot of times when people think about vote splitting, they think like, oh, like there's that fringe candidate coming out of nowhere, coming from the extreme left or the extreme right. But here, like, you're not seeing that. Like here, you're seeing a squeeze from the middle. Yes, yeah, it can happen in either situation. It's, you know, plurality is particularly sensitive to it. And so we run into this problem a lot in our current system. And it can even be the case that you can set up a distribution where I just squeeze this in here. The green candidate actually has no chance of winning it whatsoever, even though they're in the middle. Approval and condor say voting have the same behavior, which is that the... You actually get the behavior you might expect, which is that the candidate who's closest to center of opinion wins. The board account, which maybe most of you are familiar with, I'm guessing, where candidates get points based on how they're ranked. Do I need to explain this, Aaron? Do you think folks know? It's helpful to maybe give a quick one. It's quicker to explain than an instrument of voting, at least. Yeah, yeah, that's true. Most things are. So the board account is a method of counting votes in which each voter ranks the candidates. Let's say I rank them, you know, one, two, three. And the candidate with the lowest average ranking or lowest total ranking, depending on what you require people to fill in their ballots, wins. And that method actually favors the centrist. So it actually gives the centrist even more of an advantage, the opposite of what happens in plurality. And now someone else is pointing out a funny thing that seems to be occurring here with instrument of voting. That is, as you move closer to a candidate, it seems like it's hurting that candidate. And when you move further away from a candidate, it seems to be helping that candidate in some instances. So you see this kind of like going back and forth in terms of who wins within the bar on instrument of voting. That looks kind of funny there. Yeah, so this is really strange. I mean, this, I mean, I would call this like. Pretty messed up. In that. You know, if you're, let's say here, for example, let's say this represents the public opinion, you know, two weeks before the election. And you support the green candidate. So you campaign for the green candidate and shift public opinion toward the green candidate. That can actually cause them to lose. And that's the non monotonic behavior of instant runoff voting. That's, that's often criticized that people can actually hurt their favorite candidate by ranking them higher. So one of the other comments I've seen and recall that your simulation tool out for it, which is, so when, when I've looked at studies that have asked the question like, okay, well, what does the distribution of voters look like in terms of their political ideology? Even when you add other dimensions, it follows a normal type of distribution. But for a sake of argument, what happens when we change the distribution from normal distribution to say like a uniform distribution or a bimodal distribution? Oh yeah, a bunch of, a bunch of things happen. So I just added a fourth candidate here just to make this, just to show how messy this is, but let's, the colors are just for a second. So it doesn't get too confusing. But yeah, I can change the distribution. This is a bimodal distribution. What this looks like with two humps. And you can also ask for a uniform distribution. And in this uniform distribution model, we still get this non-monotonic behavior in instant runoff. It's actually even a little bit worse. You know, the green candidate has very little chance of winning, despite being, you know, probably the best candidate to, to satisfy the needs of most of the voters. And has these tiny little regions that are way off on either side. And one of the questions that we're seeing from, from Joel is asking about the different voting methods in terms of which ones can exhibit this non-monotonic behavior versus others. And as we can, I mean, see right here, the only one where we see this kind of back and forth pattern is showing up with the, with instant runoff voting. So, yeah. So you can, is the only voting system that anyone ever really talks about that has this property. It's kind of amazing that it, it qualifies. Interestingly, although this isn't on there, a traditional runoff would also have this property as well as being non-monotonic in certain situations. Oh, I guess that's true. Yeah, that's true. And that is a system that people use. Any other requests in terms of fun pictures that we can look at, like, back and forth the color scheme that we can see under instant runoff voting from this one? Just that, as you introduce more candidates, it just gets more complicated and the candidates in the middle generally lose out. So here, you know, the yellow candidate has no chance of winning at all. And you can even create situations where, here we go, where, you know, the, the outcome of the election just kind of changes all over the place and it's not even really sensible. This, you know, I consider this disqualifying and that like it doesn't, it just doesn't really make sense to have an election system that reflects public opinion in this way. There's no intuitive reason why, why this should happen. Maybe to give IRB a little bit of credit because like, we do criticize it quite a little, quite a lot in, in our work as well. Would it be possible to see maybe for simplicity, just three candidates and look at a, maybe a typical vote splitting scenario that we might see where there's like a, someone like close to the person on the left or on the right. Oh yeah, right. Okay. So, right. So in a situation like this, for example, red and green are splitting the vote on the left. And that gives an advantage to yellow on the right. Yeah. So, so here, like, I think this is like, normally when I think about where we can kind of throw instrument off voting a bone tends to be in this type of scenario where we see a third party or independent who really doesn't have very much support and you're throwing it about only with regard to winner selection, not actually how it captures information for the candidates themselves, but just in terms of choosing the winner. It seems like when a candidate doesn't have very much support, it seems to address this address that kind of problem really well. But as we saw a moment ago, whenever that candidate becomes more competitive with the other two candidates, you can just kind of throw all that out the window and you start to see these other types of oddities such as that center squeeze effect where you can cover the distribution right over that candidate in the middle and they sort of won't win or some non-monitonously where you can actually have a candidate to get more votes and wind up losing. Yeah. I guess I just want to call out a couple of things about the simulation here. So one of the idealistic assumptions is that the voters have complete information about the candidates and they're just voting for the best one as if, you know, every voter was perfectly rational. And it doesn't, there's no way to express in the simulation that some candidates might just be better at campaigning than other candidates or just might be more, you know, have more name recognition, for example. And so the idea of major and minor candidates doesn't really show up in the simulation as well as it could. Although this is probably about as close to an illustration of the effect that Aaron is talking about that you could have. The thing I guess I want to point out is that, you know, again, approval and condor say split the difference exactly in the middle as they, as one imagines that you would or you want them to. And in the plurality first pass the vote, first pass the post system, the yellow candidate has the biggest advantage. So IRV reduces their unfair advantage but it doesn't eliminate it. Is there anything that you'd like to point out here or any questions about this particular simulation tool before we add another dimension to the mix? I don't have anything else I think, although I see a number of comments going by. Is there? Yeah, I guess Aaron I'll leave it to you as moderator and person who is keeping track of the time. Sure. If you see anything that. So there's one question about the types of inputs within the model. So what you can see here on the top left hand of the screen is the type of ordinal inputs or grades from. That is assumed given the distribution layout of the of the voter distribution that you can kind of manipulate back and forth there. So it's using that information to infer all the ordinal rankings. That's right. Yeah, so it's making it's making again the assumption that voters will rank the candidates in order of closest to their opinion to the furthest away. Well, maybe we can start to look more in a second dimension in terms of political ideology and see what this looks like in two dimensions. Yeah. So. So in these visualizations we're doing exactly the same kind of simulation. We're even assuming a similar distribution. It's a normal distribution of L curve like this. But there are two degrees of freedom on which voters and candidates can differ. And so there's a lot of text here just describing how these were run. So opinion is represented by a point on a plane. And then the way that the simulations are run, we just scatter hundreds of thousands of voters at random according to that bell curve distribution actually, you know, count the election and see what happens. And then we color the point at that spot. Just like along this bar, we would color the point along the bar. So now we can color in the entire plane. And so here's a simple example where you've got three candidates. And the candidates are arranged at the corners of an equilateral triangle. They're equidistant in the plane. And you color all the points in the plane. Then maybe. And I realize this is perhaps a bit late, but perhaps like we can quickly go over each of the five methods that are being described here. I'm assuming it's better late than never. Yeah, yeah. Yeah, I'm assuming a certain. A lot of people in this, in this audience are going to know, but just, we get some newcomers to who want to be fair to them. Great. Yeah. So, okay. So we're comparing five different ways of determining the single winner of an election. And in this particular setup with the candidates in this position, they all look the same, but. So the methods are a plurality election in which everybody gets to vote for just one candidate. And then the winner is the candidate who gets the most votes approval in which everybody can approve or disapprove of all the candidates. So you can think of that as voting for any as many as you want. Or you can also think of that as giving a thumbs up or thumbs down to every candidate. And the way that's run in the simulation here is that there's a number of candidates within which a candidate is considered acceptable and the voter votes for the candidates that they, they find acceptable. The board account I described earlier. You rank the candidates. In this case, you rank them based on how far away they are from the voter. And they get more points. If they are a higher in the ranking. They get, for example, they get. If they are a higher in the ranking. You could, you could call it like they get one point for being ranked third, two points for being ranked second, three points for being ranked first. And then the candidate with the most points wins. Condorcet is actually a family of voting methods. It's not a, it's not a single method, but it's, it's a property that a certain class of voting methods has. And that property is that if every voter ranks the candidates in the order of their preference. And the winner of the Condorcet election is also the winner of every head-to-head match. If you were to compare, you know, every possible pairing of two candidates, A versus B, B versus C, C versus A. And you compared who ranked A high, higher than B versus who ranked B higher than A. And that determines who wins that matchup. You do the same thing with every possible pair of matchups. The Condorcet winner, you know, for example, if it's a. The winner would be A if A beats B in that one-on-one matchup and also A beats C in the one-on-one matchup. So it's absolutely clear that A has to be the winner because, you know, if, in comparison to any of the other candidates, they're considered superior by the voters. And then finally the, the fifth diagram here. I've called this the hair method. It goes by several different names for the United States. It goes by instant runoff voting or currently marketed as ranked voting in which each voter ranks the candidates. Again, we're assuming the same input. So they're ranking the candidates based on the, how far away they are. So the candidates that are closest to the voters are ranked first, furthest away are ranked last. And then the way you combine all these rankings is you do this complicated procedure where you look at all of the ballots and you put them in piles based. First of all, you just ignore everything except the top rank and each ballot. So who's ranked first? So you put the ballots in piles based on who's ranked first. You count them up. And then if any pile has more than half the ballots in it, that's the winner. And if not, you take the smallest pile and you go through all the ballots in that pile and you cross off the top ranked candidate. And then you consider the next ranked candidate on that ballot and move it to that pile. Then you repeat the procedure. And once you have, if any pile is now, more than half the ballots, that's the winner. Otherwise you find the smallest pile and then eliminate the top visible rank, not yet crossed off rank on those ballots by crossing it off and then looking at the next one and moving that ballot to that, that pile. So there's sort of gradual consolidation of the ballots until there's a pile that constitutes a majority. I do have a question about Condor say here. So when we look at ideology or issues or degrees of freedom in terms of just one, you can't have, in that case, there's always a condor say winner. But when you have more than two issues or dimensions, now you can have your condor say paradox where there isn't necessarily someone who can beat everyone head to head. So here, so in the first one, we can expect that there's always going to be a condor say winner when there's only one dimension of removing the distribution back and forth. But here we have two dimensions, an x axis and a y axis here. So here, how do you, what tiebreaker or you mentioned that the condor say method is a family of systems. Is there a particular tiebreaker they use when there wasn't a condor say winner here. So, as it turns out, okay, so if you allow voters to rank the candidates arbitrarily, then you can end up in a situation where there is no condor say winner. But if you in this particular limited case where you're on a two dimensional plane, and you're using Euclidean distance. You always have a condor say winner because that there's no that there's always a possible ordering, just because of the flatness of the space if possible to mathematically prove this but it may be a little bit a little bit much to go into here. It's just because of this the limited the the simplifications involved in doing the simulation that make the possibility of non of having no condor say winner go away. And you've been in this scenario you're saying. Yeah, actually, and David has made a good point here. That's true just because I'm using a Gaussian distribution and dependent, you know, if I use the different distribution then a cycle might actually appear. Interesting. Yeah, I think you're just kind of exemplifying like another rationale for like when we look at things different, look at these voting methods under different lenses, it gives us a sense of what their behavior is. Yeah, and I guess I should, I want to step back and just say for a moment like how it came to me that, you know, this would be a good thing to try it's that, you know, I'd seen all of these debates on the internet about all the different voting systems and people, you know, arguing in favor of instant runoff or arguing in favor of approval. And, you know, every, every team so to speak, you know, team condor say would have like their examples of like particular elections with specific voting counts, you know, specific inputs that would produce interesting or better or worse results. And team approval would have, you know, their examples and team instant runoff would have their examples. And so, you know, there'd be a lot of debate about like, okay, well, you know, sure, you contrived that example in which this weird behavior happened. But surely that never happens in practice, you know, that like almost never happens or that always happens. And so then you sort of get into a debate about who's examples are more realistic. And so, you know, that made it kind of hard for me to grasp what actually makes a method better, or not only better in a particular situation, but like, how's it better behaved under change, right, if opinion shifts in a certain direction, you move a little bit, you perturb the vote, or you campaign for somebody, you know, what effect does that have, and it's hard to imagine what that is when you just look at a set of numbers. And so I really wanted to get a more holistic view of like, okay, what's the sort of behavior on a larger scale of the system. And that's what led me to make these diagrams. So is that part of identifying like, well, when you're trying to analyze something, it's very challenging to do so unless you have a common kind of metric or playing field where they all abide by the same rules and are all in the same scenario to compare them. Is that kind of what you're getting at? That's part of what I'm getting at. I think it is a valid criticism of these simulations that the space of possible elections that we're visualizing here, you know, is a limited set. It makes a bunch of assumptions that don't necessarily hold in reality. And so I see this as helping you get an intuitive feel for how these systems behave. And it's much easier to get that feeling by looking at a visualization of data that's shown altogether and its relationships shown than by looking at individual point examples. Sorry, the difference between, you know, looking at a bar graph and looking at a table of numbers, right? So you look at the table of numbers, but that you have to sort of do a lot of mental work to see that there might be a trend, for example. But when you plot it on a bar graph, the trend is much more visible. And so this is a way of trying to collect a lot of examples to give you a feel for the system as a whole. Maybe you can show off some of these examples. Yeah, right. Okay, so things get a lot more interesting when we don't have the candidates arranged in this very nice balanced fashion. So here I try to set up where the blue and green candidates are really close together. And they split the vote as we were similar to the example we were looking at in the one-dimensional case. And as you might expect, in plurality voting, the middle candidate is squeezed out and has, there's no actual way for them to win in the situation. And the same thing actually happens in instant runoff. The other systems provide a blue candidate with a chance to win. So the two-dimensional analog of a fair boundary exactly halfway between two candidates, which we saw in the one-dimensional case, is the perpendicular bisector between those two points. And you can see actually, it's actually possible to mathematically prove that this boundary here between the red and blue candidates actually is exactly the perpendicular bisector between these two dots. And the same is actually true of Condorcet. But these lines are actually in the same place. And the fuzziness that you see here is just an artifact of the fact that I did these simulations by randomizing the locations of the voters. The same fact we talked about with Borda is also visible here. So the center candidate blue has a larger win region. You can actually see this is actually biased quite a bit. It gives a lot more of this region to blue than to red. So the difference between the two is in keeping with the way Borda gives it advantage to centrist candidates. Yeah, I think with the previous tool, we saw that same kind of effect where the opportunity or the width of the middle candidate was wider compared to the other voting methods. That's right. Yeah. So then I just tried a bunch of different examples here. And then you got some interesting and strange shapes. So in a two-dimensional scenario, what non-monotonicity looks like is non-convexity in this case. So the win region for the green candidate is non-convex, which means it has these gaps in it so that certain paths traveling across the two-dimensional plane will cause you to enter and then exit and then enter and then exit the region in which the green candidate wins. And this is the... I was just going to say that the boundary between blue and red, actually this is sort of the correct boundary and that it's halfway between blue and red, but then the green candidate gets involved here and it just sort of messes things up. And here this is the votes putting scenario where you have a candidate that's kind of on the edge that is touching near one of the other candidates. Is that correct? So, I mean, that's also... The vote is getting split by these two candidates, although the previous diagram was the one that was intended to illustrate vote splitting in which the blue candidate has no chance. And then this one was intended to illustrate the non-monotonic behavior of instant runoff. This convex region. I guess like with that as you're moving towards a candidate so like you can imagine being on like the bottom right or even like the, I think maybe the bottom left of the candidate there and moving towards the candidate and seeing like the voting center get closer to that candidate and yet it's really killing that candidate's odds of winning even though more voters are actually voting for that candidate. That's right. That's exactly right. So, I mean, you could imagine in a fictional universe in which people actually were running an election like this and you were a green supporter, it would actually be strategically the correct thing for you to do to try and push public opinion into this weird corner over here and not actually campaign for a candidate, which seems ridiculous. And then there's some other examples here of different things I tried to use more extreme example of a non-monotonic situation. And when you add more candidates, things just kind of get more complicated. So when you have four candidates, even when the four candidates are uniformly arranged, you already have some really weird behavior. It looks like a good one. Yeah. It just becomes highly unpredictable what you're going to get. It's a very little relationship to what you would want. I think if you wanted to actually represent the will of the voters. And, you know, more ridiculous things happen if you play with it. So something I've wanted to do for a long time, but I've never actually gotten around to, was to make an interactive version of this where you can move the dots around just like you can move around the candidates in the one-dimensional version. Back in the day, computers weren't fast enough to run the simulation in real time while you're moving dots around. But I bet you could do it today and I bet they would be fast enough. Yeah. So there's just lots of weird things that can happen. And so this really kind of cemented it for me that, you know, instant runoff as a method is it's not just sort of imperfect in the sense that it's like almost good enough. It just has some fundamental flaws in its behavior that don't make sense and, you know, can't be easily corrected. And you mentioned computers nowadays, likely being able to perform some of these simulations in real time. So it's probably worth a hat tip to Nicky Case, who has developed a lot of these types of sandbox tools through. By the way, as Nicky points out through inspiration derived from you. So you were wondering if Nicky's inspires. I can show that off. Nicky Case does these amazing interactive visualizations. That explain concepts in a lovely way. And done a little bit more. Did one of them. Here we go. Yep. On voting systems. And if you scroll all the way down to the bottom, you can see a shout out to copying you as well. Right. Thanks. Yes. Yeah, I just wanted to show that here we go. So this is, this is the interactive visualization that Nicky made, where you can actually just move around the candidates. Oh, they don't turn into smiley faces when they win. That seems very unlike Nicky. Yes. So I think that's what I have to say about these visualizations. I have learned in the intervening years that people have talked about these visualizations. I have learned in the intervening years and used them and even named them after me, which. Feels like. Kind of surprising and flattering. But I'm very happy that. It's of interest folks. And to the extent that it's helping people understand how these things work. I think that's good. Excellent. For those in the. I can go ahead and look out for those. So. One person asks about other different types of. Voting methods. So. For instance, there are a number of. Cardinal methods that are often. Being invented. How do you see, like, how would you imagine some of these? Right now. So I guess, like, as an example. Would you see like other cardinal methods such as like score voting or. For voting or three to one. Would you see those like. Looking a lot like. How. Approval voting and board account and condors say look on here. Or like, would you imagine them looking a little bit differently. I think so. Those are really interesting systems that. I wasn't familiar with at the time that I. Made the simulation. And so it would be really interesting to see how they fare. One of the things that makes that. A more interesting problem. To simulate is that some of the systems like score voting. Give you more degrees of freedom. Than. Are really accounted for in the simulation. Right. So already. You know, For example, the assumption that I made about ranking the candidates in the order of how far away they are from the voter. That. Eliminates the possibility of. It eliminates a degree of freedom that would have made it possible to rank them in a cycle. For there to be non. For there to be no condor say winner. And. In score voting, you can assign every candidate to score. And so you would see different results depending upon which algorithm you chose. I expect though, under most. I mean, this is just a guess. But I think that if the. Distributions were normal. And the scores that voters gave. Were. Linear and including distance. And so you would see different results depending upon which algorithm you chose. I expect though under most. And so you would see different results depending upon which algorithm you chose. And including distance. Or perhaps even. Yeah, perhaps it will probably, probably even happen for like. You know, distance squared or something like that, just a, if it were a monotonic curve decreasing. You'd probably see the same results as approval. In the sense that you, you know, the dividing line between two candidates would probably be half the same. And so. One. Chris Hubbard asks about some of the. Peculiar boundaries. That. You see an instant runoff voting. Some that aren't perpendicular bisectors. You have an explanation for like why. Some, like you're seeing like some of these behaviors that are even perpendicular bisectors that are being. I don't really, I think there's, yeah, there's some interesting math that could be done there. To figure out, you know, analytically what those boundaries are. I have not done that math. Often, I mean, a lot of the segments of those boundaries are perpendicular bisectors. You know, some of them are, but they might be perpendicular bisectors between different pairs of candidates. So for example, this, this line here. You know, bisects green and red. I'm not sure about this line here. Yeah, I don't have really an intuitive guess for, for where that came from. So. I see. My little Lonnie has a question about. Whether this would work with cumulative voting and as like a brief recap. These simulation tools are being used for single-winner methods of voting is a multi-winner method where you can stack multiple votes on a particular candidate, but kind of stretching that question a little bit. Can you imagine using other types of visualization tools to be able to look at multi-winner methods, whether they be block type systems or systems that have better geared more torture outcomes? Because that's, I think you got an open space there. Yeah, that would be super cool. That sounds like a really like tricky and challenging and really interesting design problem. Both in terms of how to design the simulation so that it is realistic enough to be useful, that it, you know, actually represents something that will, will give you useful information. And then also how to design the visualization and the user experience so that you understand what it means, what the outcome means. Oh, I just saw a comment from Keith Edmond saying he made a similar multi-winner system. Is this a visualization that Keith is talking about? If there is, I'd love to see it if you're really cool. I'll try and take a couple of these. So just, just quickly. So David asked, do I have a rationale for the log normal threshold that simulated voters use, that distance threshold? That was basically arbitrary. There isn't an objective reason for that. Okay. Okay. So I'm going to take a couple of comments. I'm going to take a couple. Keith has sent a link to his code, which I'll be excited to check out. I noticed a couple of comments about arrows theorem and the conclusion he reached that it was not possible to come up with a voting system that satisfied all five of his conditions. I want to just like get on a soapbox for just 30 seconds here and say, I don't have a voting system. I don't have a voting system. I don't have a voting system. So. I interpret errors results. Differently from the way that a lot of people seem to quote it. They sort of quote it as an impossibility theorem and sort of summarize it to say, Oh, well, there's no perfect voting system. Or, you know, every system has a flaw or a paradox of some kind. And I really don't see it that way because. I haven't been able to come to make that proof about. So. The content of the proof, like the result of the proof really says something about the criteria that he chose. It doesn't say something about all voting systems. Right. It says this set of criteria, the five criteria or six criteria, depending on how you count. Are not mutually consistent. And that's really all that it says, you know, and then you're, you know, the interesting design question is what criteria should matter to when you're designing a voting system. I mean, any engineering discipline, you've got a bunch of different criteria, a bunch of different desiderata that you have, and you've got to trade off how important those are, what value they have to you. And, you know, Kenneth Errow found one case, one particular set of criteria, where it was not possible to achieve in the absolute a system that met every criterion. And so that simply says, okay, well, you need to make some trade-offs. You need to relax one of those criteria. Maybe some of those criteria are not as important as others. And of course, they're entire classes of voting systems that Aero's theorem doesn't apply to. So that's how I like to frame it. It's really a theorem about the criteria that he shows. Yeah, I think that's a good way to describe it. And I'll also include a link to an article on our site where we talk a bit more about Kenneth Errow's theorem, as well as to an interview that I was fortunate enough to have with Kenneth Errow before he passed. So, but I think you're speaking well to the idea of a kind of classical mindset that we've, that a lot of academics have taken towards voting methods, which is a criterion-based approach, which is we say some set of criteria are important. And if a voting method fails it, then it's no good. And if it passes it, it's great. But to pass the voting method criterion, it has to do it every single time. There can't be any failures. And so it's not saying anything really to the extent of how badly a particular criterion is being failed. So if it messes up, like how far off does it get? And also saying nothing about the prevalence or the the likelihood of a particular criterion being failed as well. Yeah, that's exactly right. I'll say that. So I actually encountered this, I was doing this work and thinking about voting systems, voting methods like this, before I started looking more deeply into election security. And that really broadened my perspective on elections from being in these theoretical systems where you have these ballots and you're just trying to get a result to these messy real-world systems where voters have to get to the polls. They need time off work to vote. They may or may not understand the ballot or the balloting system or the accounting system equally well. There are concerns like some ability that are entirely practical concerns as to how to run an election. And those things influence things like how vulnerable is the election to different kinds of attack. What kinds of incentives do politicians have to make regulations around voting? And so as you step back, you sort of zoom back, there's just a lot more concerns that you need to take into account that are really significant and worth trading off against. One that I find particularly concerning, for instance, interesting about instant runoff is that it seems to significantly increase the number of spoiled ballots because it's more complicated to mark an instant runoff ballot. And it seems that not only are there many more spoiled ballots, but the spoiled ballots are predominantly in regions of voters of color or voters in lower socioeconomic classes. That's a huge effect. I mean, if you suppress the vote by 1% or 2% consistently for people of color, that could easily swamp many other effects that you might be concerned about. I'll go ahead and put another article in the looking at the kind of fundamental question of what makes a voting method good? What are the particular types of qualities of a voting method that are important? And as you mentioned, like there are a number of them, highlight some of them as well. Maybe go into some last minute questions before we wrap up here. I was asked for a citation for the claim about spoiled ballots. Is there one linked on the page you just referenced or? No, the article that I referenced just looks at the fundamental question of what makes a voting method good. I don't believe it goes into that particular issue. I can dig around. I have come across that before and I can probably find one for it. Is there anything that you'd like to share with us or plug as we're wrapping up here? It's been, yeah, I don't have any specific things to say beyond what I've conveyed. I'm again really happy to be here and it's an honor to get to speak to all of you. And I'm really, really strongly supportive of the work that CES is doing on approval voting. I hope that it catches on. It's really, yeah. I wish people could see all the advantages of it and how its simplicity and how cheap it is to run, how simple it is to migrate to an approval system, how much easier it is to do that than an instant run off system. It would be really great if we're more popular. And so I'm glad that we're talking about this and learning about it together. Well, like I mentioned before, you've got a lot of admirers among us. Myself included, very appreciative of all the work that you've done. And I see one person asking about the discord as well. So as a reminder, we do have a discord that folks are free to join on. I'll also share cupping news website. We can find lots more information on voting and other projects that he's doing. You can find that in the chat. This is also a link to our discord, which I think you're welcome to join us there where we talk about all the campaigns that we're doing, including looking at places around the Bay Area where we are, as well as some of the the wonkier stuff as well. So kind of wrapping up here, I want to thank Coppigny again for his time and being able to share this awesome work that has really brought a lot of eyes and attention to voting methods and being able to illustrate ideas that are really kind of complicated and being able to make it so that they're easier for us to understand and look at really in creative ways and being able to really see different aspects of what makes a voting method good or bad. And your work has definitely helped there quite a lot for a number of us. And there's a reason why these were named after you because folks appreciate it and they feel valuable. And I appreciate it too. And also plugging in the organization, the serve production science, if you've liked this event and you would like to see more events like this, then you should make sure that you're subscribed to us on social media on our newsletter. And also, we are a 501c3 and we would love to help you make your tax deductible contribution or if you can't deduct, we would still love your contribution so that we can be able to put these events on in the future as well as to make sure that we are able to empower voters in cities and states throughout the country so that they can use a voting method that is able to encapsulate their values and make sure that they have a voice. So again, I want to thank everyone and especially Kapingyi for joining us today. Thank you so much. Yeah. Thanks everyone. Actually, I'm just curious, Aaron, do you want to say anything about CES's like are the current initiatives you're doing in particular jurisdictions to promote approval loading? Anything that's like close to passing or? Sure. So our biggest wins have been in Fargo, North Dakota, which we are particularly proud of because that happened immediately within a year of our initial funding. We hired staff and got approval voting passed in North Dakota all within a year. So we were particularly proud of that speed run that we had done for democracy. And then right after that, after we got initial funding, we saw approval voting get passed in St. Louis. And to kind of toot our own horn a bit more with that initial win in Fargo, North Dakota, we went from not having staff to getting a method implemented that had not been implemented anywhere in the country ever to its first. So we were just another way of just highlighting how how precious that that win was for us. And then as a result, being able to see these elections carry out, being able to see the folks in Fargo, being able to win without like a sliver of support from the vote splitting that they were used to, being able to see the voters in St. Louis, whereas previously you had a number of candidates suffer from vote splitting within the black community, being able to see in the progressive community and being able to see those votes really being able to be more accurate and being able to see candidate support in a much clearer light without having to go through all that vote splitting that St. Louis was used to. And also being able to see like their previous mayor, who won as a result of vote splitting, the moment that mayor found out that approval voting had passed, she's like, I'm not going to run anymore. Seeing the writing on the wall that vote splitting wasn't going to allow her to come through with the victory again. And we now have campaigns throughout the country. We have over 50 to 80 individual cities where folks have shown interest in building chapters. We have a chapter system for cities and states across the country, in particular, like looking at places in Colorado, looking at Seattle, looking throughout Texas and Utah, really all over the place. So we are really, really excited to again, thanks everyone for helping to bring us where we are. And we are happy to celebrate with you as we move along and are also excited to highlight the other players in this space who have helped us to get to where we are and be able to see really how voting methods play out and for us to have a better understanding of it. So and I just use that again as an opportunity to thank Kapi as one of those individuals. Awesome. Well, we'll go ahead and sign off everyone. And again, thank you so much. Thanks for joining.