 Hello and welcome back to the FEM channel. Our next speaker will be Rajiv Gauray. Until last year he was a professor at the Australian National University. And his talk will be about how the counting of elections with complex voting schemes, if that happens electronically... ...and verifiably. After the talk there will be a Q&A session, a live Q&A session here, to ask questions... ...and post it here in the hashtag FEM and in the hashtag AC3FEM... ...in the hashtag ROCKETCHAT and the hashtag FEM and in the FEDDIVERS and on Twitter... ...is the hashtag AC3FEM... ...on the native of translated... ...and then you can have the simultaneous session. And now enjoy the talk. Thank you very much. Hello everybody. My name is Rajiv Gauray. I used to be a full professor at the Australian National University. My name is Rajiv Gauray. I was a professor at the National University of Australia. It is a joint work with Minhad Kahl, who was a doctor at us. who was a doctor at us, just before I go ahead and all of the stuff that I'm talking about that I'm doing here in the proofs are available here. Everything I'm going to be talking about, the code, the verification, the proofs are available here, if you want, okay. But where am I going to be talking about? I'm going to be talking about complex voting as a way for a transferable voting scheme that can show the results of, for example, the transferable voting options with the computer so that the results, both formally and publicly, are verified by all. Okay, so that's the overview. In the beginning I'm going to talk a little bit about what the end-to-end verification is. In an electronic voting system it means then I'm going to explain what the transferable individual settings are. Then I'm going to talk about how it is done in Australia with the computer. I need to know the flavor of everything rather than the depth of the data. I hope you'll bear that. I'm more than happy to have a short chat session after this. I'm in electronic voting teams. The goals that are noted as in the viability system and the way that they achieve it is by three states which will feed into each other. So the first one is called cast as intended and the idea is that a voter should be able to verify in some way that the electronic ballot that he or she is casting is actually what they intended. So the idea here is to limit the aspect of ballot because when you put a paper ballot in the box you know exactly what you're putting in the box. When you do electronic voting via some sort of computer terminal you don't necessarily know what the computer is actually casting on your behalf. So cast as intended is to make sure that your electronic ballot really is what you made it to be. And there are cryptographic ways to record it as cast says that you must be able to provide public evidence that the ballot is not tampered with in transit. So it's all great. The electronic ballot is what you intended. But if there's a man in the middle attack then the man in the middle can change the ballot in transit. And finally there's a tally in the report. So the idea is that a voter can verify in the middle attack is tally. In other words tally didn't just throw the ballot away. Right now these three things naturally form a cascade so that if you have cast as intended and then recorded as cast, tally didn't just throw it away. You have end to end very well. And there are politically available voting systems that try to guarantee these things. Now most of these electronic voting systems are cryptographic or some sort of hashing or some sort of Now tally is a calling is all about a sort of hashing so that you know that your vote is tallying. But what if the vote counting program is tallying? So what you really want to try is that it is tallying correctly. And that's what we're trying to address in this word. And the idea is called the voting counting program has to produce a tallying script. Right? Some sort of evidence of what it was doing. And then what we do is we try and guarantee that if the tallying script is correct with some notion of correctness then the result is correct with the appropriate version of it. So the voting counting program has to produce a tallying script. Right? Some sort of evidence of what it was doing. With the appropriate version of it. Okay, so there are two notions of correctness. One is the correctness of the tallying script. The other is the correctness of the count. And what we do is we formally try them to get up with this implication in line in tallying too. Right? So that's formal implication which is a formal implication. And then that claim is that it's trivial to write up the tallying script rather than having to demonstrate that. Okay. So the idea is, what we verify is that this particular run of the voting counting program is correct. And the one that counted your ballot or the one that counted the tallying script is the one that counted your ballot. So this idea of this running script will come up through my slides, through my slides, through my slides, through my slides, through my slides, through my slides, to see how the current running script is correct, through my slides, so it's a new intake on notification. All right. So what we do in the voting, the exact level of transferable voting... So it's basically a method for setting out a filling in and counting things. So, you have course to make it electronic. You still have to have an electronic... method of the scope of what we've done. So we have a paper ballot. And for example, like this one here, it says that it's a paper, but Charlie one. This can look like this on paper. Here, there are four candidates. And I have chosen a position of Charlie. And Dave of position two. And Alice on position three. And Bob is not interested in me. And depending on how that works, it could be a guilty vote or not. For example, all of them have to be filled out. In some cases, it has to be done. And then it would be unfair, because you have to put four in and in the other cases, it would be correct. So in Canberra, where I am, we just have to have a Robson rotation, too. And that means if you just go to all the same candidates, then there are the so-called donkey voters, the donkey voters, who would just write one, two, three, four from top to bottom. And all of these votes would go to Alice. But we have quite a lot of donkey here. And then there are quite a lot of these donkey votes. And that's why there is a new sorted version of this vote in all possible permutations. And then the donkeys will be distributed everywhere, the main thing is that the donkey is distributed everywhere. And then the most important thing we have to do here when we're counting is we have to find out if a certain quote is reached. If the quote was not reached, then we have to find the candidates with the least votes and the strokes. And what do we do if it's not decided? And how do we transfer a vote? And with cyber security, artist security has nothing to do with me, but there is a formal verification of the algorithm. And I'm going to take a look at this election process again in the individual. Here are all the technical terms here. And I'm going to let you read them. So I'm going to go back to the value of a vote. You can do less than or even one. Okay. So here is the transfer value of a vote that can be less than one. We tell it all the time. So what does that mean? That means that we create a pile of votes to Charlie's pile. And we spell it to Charlie's pile. And that's a table for Charlie's pile. And put it into wherever the votes are the highest number, the lowest number, first. And then for which candidates we always sort out the votes. And in this case, we have four tables for all of Charlie's and Dave's. And in each of these tables, each vote on Charlie's pile has a one and each vote on Dave's pile has a one for Dave's. And all of them in Alice's pile have a one for Alice's pile. That means we first sorted out the first preference. And what if nobody selects and then we have this quota that we've determined, then it's a vote. But if no one's voted, then we have to find the weakest candidate and eliminate a candidate. And we have to transfer the votes that we first counted on each of the votes. So what we're going to do now is we're going to say, well, Charlie is going to vote. And what we're going to do is we're going to vote on Dave. But normally, he's all hope. He's not going to vote on Dave's pile because he has full vote. So he's giving them more power. So he's giving them more power. So he's giving them more power. So he's giving them more power. So he's giving them more power. So the idea is that we continue to give them a full vote. Dave will vote. And then he'll get more power for this vote. And we have to eliminate this value. And then this value is a little lower than one. And I'm going to do that again. Let's say if the number of the remaining candidates is less than the number of seats we want to give, then we can just select everybody. And that's then also done with this process. I'm going to explain a little bit about the details. Here's, for example, a example. We have four candidates, A, B, C and D. And we have five ballots and we have to select two seats. And there are these five votes that have been eliminated. One has A, one has B, two has D, three has A, one has B, two has A, one has B, two has A, one has D and two has C, two has C and one has one C and two has D. And first of all, there are no share transfers. And we also don't make this automatic speech but we all vote if there are less candidates than seats. And what we are doing here is we have a Droop Quota by the number of seats plus the total number of the number of seats plus one, that's here three. And so what does that mean? And that's the five-divider, one-point, the whole number, the bigger it is than that. And then we have five by two plus one. And it's one plus two. And as soon as I'm going to do that, I'm going to count all four first and then I'm going to go there and count all four seats. And as soon as you check that you can count all four A and all four D and all four A, one for D, one for C. And that's what I just wrote here, so one for A, one for D, one for C. And what we're going to do now is we're going to need to We have to see if someone has filled the quota. Yes, A has filled the quota. A has three, and the quota is two. And A has more than even the quota. So A is valid. So what we have to do now, we have to decide what we're going to do with the additional ones. What are the additional ones? The additional ones are one. Because A only needs two of these voices to decide how the quota is going to come. A has three, so it's one additional. And now we have to decide what we're going to transfer these to. And there are different possibilities. And I'll tell you later again what I'm going to do. But right now, I just want to keep it simple. That means all these voices are exactly the same, all these voice numbers. So what's going to happen? So we're going to have two of them because they're all the same and we're going to transfer them to the third. So the third voice number now goes from the A-staple to the B-staple. That means B is going to get one more. So what's going to happen now? Now we have three that have the same number of voices. No one has voted. No one has filled the quota. That means now we're going to have to distribute the weakest candidate. So we find the weakest. And there are different possibilities and that makes a difference. So what I'm going to do now is I'm just going to make this voice because it was transferred is less valuable than the non-transfered voices. I'm going to do something but there's still a lot of better ways. But what I'm going to do now is I eliminate the voice for the ballot. So I eliminate the voice for the ballot. So I eliminate the voice for the ballot. So I eliminate the voice for the ballot. So I eliminate the voice for the ballot. And now D has two voices. So we had B first. D is counted. And then we have B-staple. And then the voice for the ballot. That means D has two now. And then we have the quota. And then D is also voted. And that's the end of the vote because we're going to have two votes. And we have two votes. We have one-staple. And C is just still hanging around. And that's neither voted nor counted. But the winners are A and D. So what I would like to do now is I would like to tell you a little bit about the sad situation of electronic voice counting in Australia. We don't have an official electronic vote. We have the Australian election commission and I would like to talk about the number of voices. On the highest level, we have the Australian election committee. The code is needed. And D is responsible for the two national elections. And it has a proprietary code. It can't be publicly reviewed. And there is a freedom of information request to publish the source code. And that was rejected by the Electoral Commission because of the security and because of the arbitration commission. So we have the default security. And it was coded in the house. And security was no longer valid as a defense argument because it only works in a closed system. But the arbitration trust was still an argument. And that's why the freedom of information was still rejected. And now you ask yourself why the highest election authority in Australia has a proprietary and confidentiality reason. And it's because the $19 million is earned in the way that elections are held. And this business model doesn't want to lose it. That means they refuse the public the right to see the code from the side with which the official representatives are elected because they want to make a profit on the side. The Victorian election commission is the state where Melbourne is and they also have a proprietary code. And as far as I know, you can see it. But as far as I know, no one has really examined this code. So the best trust is in Australian Capital Territory. The code was developed by an e-vax with C++. It has been used since 2001 and four times. But in the last year, the sale code was available on the website. But in the last election, the 2019 election, it was a secret. And the entire code was available when you sign a disclosure agreement. Where it says, you shouldn't publish it. But as an academic, we can't work like that. And then there's the election commission in New South Wales. There's Sydney in the province. And they have at least a few years published detailed functional requirements to say there was a letter from a law expert at the Queen's University of Technology. And she said that the functional requirements of the legal language of the code were certified by an Indian company. And the very first line says we give, if you look at this certificate, then it says in the first line we don't guarantee that the code really does what it's supposed to do. We just have this and this and this and this and that. And as far as we can see, it's all okay. It's a proprietary code. And we asked if we could see the code and we rejected it and it's not available for examination. So, how can we do that? The idea is we look at three aspects, namely the artifacts that exist and the examination that we can do. And if there's no examination, where is the trust that we have to set blind in the entity? And there's the law text that the camera and where it says in this election process, the vote must be counted. Then there's the capital territory, the election commission and software improvements. The company made these functional specifications. They're not published. But maybe if I really would sign a non-structure agreement, I could probably see them. That means we have to trust software improvements and the election commission. And software improvements made the code in C++ and they don't want to publicize it and not the cell code in any case no more. That means we have to trust this audit of the company called BMM and the company in Canberra that certified this code which is called BMM. And why should you trust this company? What have they done? BMM is a company that audits game machines. And I think in Germany they also have these one-arming bandits, these money-game devices and then you throw money in and then it works and then you have the jackpot. In Australia, the law says they have to repeat a certain percentage of games, for example 85% has to be played again and this company audits these game machines, game machines and if they really repeat 80% of the money and if we audits game machines why not even choose software? And so they did that and the first certificate what they did, it says we don't guarantee that it really works as it should. So here we have the law. It has a functional requirement for 47 pages and it's published and somebody made this computer code we don't know who, the code is proprietary, you can't see them but now we have to trust this company Bearlasoft and on the first page it says we don't guarantee that the code really works as it should. So what does the situation look like? In the last 20 years, we have found a lot of mistakes in the ACT system and someone else has also found mistakes in the New South Wales system. And the first bug that we found was three bugs in the, that was a bounce error in a foreshadowing and for four, one, up to and so on. For the candidates, this and that. And either they had a minus one or a plus one, an off by one error. And one of our doctoral students worked at Google Zurich and found out that the code was normally really good. If it was a straight number of candidates, but it went completely wrong, if there was a straight number of candidates. And I don't know what exactly happened, but we found this bug three days before the people in 2001 would want to tell the live-in-vice. We like this nice letter back from the election commission saying thank you. We gave them this error, they fixed it, but we got a nice letter from the election commission, like I said. So we are, thank you for telling us about this simple error, it was okay. And they asked now, if this simple error was there, do you know that there is another one? And they said, no, no, no, we are not sure, but it's okay. Of course, a little later, my colleague found another bug. That's this one here. There was an initialized Boolean and various compilers were able to give different results. Another colleague of mine, Jeremy Dawson, has the implementation new written or has his own implementation written. And when we checked the results of the 2010 election, then we noticed that all three papers in a slightly different kind of world and what we noticed was there was a vote. And none of the three had the same votes. So there were three candidates who all had the same votes and none of them had the same votes. And one of them had to be elected as the worst candidate. And actually the previous round, I think all these situations, that you should go back to the last round, what does that mean? So long until all candidates have the same number of votes. And Jeremy's approach was to say, and what does that English technically mean? It's that all candidates have the same number, then it means that they all have a couple of different vote numbers. And what that actually means is that all of these have to be different in some ways. And we have to go back until this is not the same and this is not the same and this is not the same to the other one. But the ACT quota didn't do that exactly. And he wrote it so quickly. Thank you very much. And we went back to the election commission. And they said, it's fine. We have, I think, found a problem. And they wrote us a letter back. And thank you. But we just defined that the meaning of this text, that's exactly what our quota is currently generating. And we can't do that. That's the election commission. And somehow, how bad are these errors? And for each of these bugs, we were able to generate a possible choice where this bug had changed the result recently. But the commission said that it had full trust of a candidate winning. And the NEMAP group has, last but not least, also found a bug in the News of Males code that reduced the chances of a candidate from 90% to 10% here. And the first question we should ask here is, why are we even talking about probabilities? And when it comes to choice, that it deserves more attention. And we go back to the slide here, where when I had to transfer one of these election papers, what we did was, we just talked about them, because there was an overshoot of NEMAP. And we had to transfer it to a NEMAP. And what we did was, we just transferred one of them, because they were the same. And what the News of Males does is that it's all fine if you just select one of them and transfer it to this one. And that's all nice in this poll, because they're all the same. But if they're all different here, then more important is that we transfer it to a NEMAP to see who's going to be elected. And what's even more important is what it means. You can't make any new statements about the election, because the additional generator, of course, could deliver other results at the same place. So what did my colleagues do? And they just made this election a hundred times. But in the real election, they noticed that one of the candidates had a 90% chance of winning. But in the real election, there was a bug. And he had the chance to win this person at 10%. So, you know, this poor person couldn't make the election, because the three-month period was already over. So this person potentially lost the election, just because of a bug. He couldn't make the result anymore. I would try to do it better, but you probably need something called formal verification. I don't want to talk a little bit about formal verification and what that means. What formal verification does means. So I'm going to talk a little bit about that. So formal verification started in the 1960s during the AI dreams. So the researchers in artificial intelligence have dreamt to automatically perform math on a computer. There's a very nice paper by one of the authors called John Harrison. And he currently works for formal verification of their code. So what they found is that, you know, automating mathematics is the ultimate problem. And so mathematics is the ultimate undecidable problem. And so humanities is the ultimate undecidable problem. So you need a kind of human behavior. And they tried it 20 years and noticed that it had to be interactive. It's not automatically automated. The first example was the reason why you want to automate mathematics is that you take any math that was proven by Appel Haken from the four-color theory. He says you can take your favorite map and you can color it with at least four colors. So Appel and Haken came up with two different limits that have the same color. So Appel and Haken found a proof in 1976. And that was about 4,000 times in code. And in 1994 it was the best example that I know is the pension bug from 1994 that cost half a billion dollars. And John Harrison used to work for Intel for a long time. And the next example is called the Kepler Conjecture. This was a proof that John Harrison also worked before he went to Amazon. So this is the Kepler Conjecture from Thomas Hayes. He says the best way to stack oranges is in a sexy pyramid. The proof was made in a market. And there was no proof that this is actually the most efficient way. And he found a proof but it had 200 pages of written notes and an example. And the experts said we are not sure about that but it seems correct. And the idea of the formal proof is that we use computer to test this proof based on logic. So we give the proof to the computer with a special language and the computer checks if the proof is correct. So the problem is that the computer checks are correct. So there are many proofs that are currently developed by the human brain. And I'm going to do that using an example. So let's take an example of odd and even numbers. So what's the definition of odd and even numbers? What is the definition of odd and even numbers? The definition of an odd number is a number that can be divided by two. That means a number is straight when it is written that it is two times a different number. So for example 0 is 2 times 0 2 is 2 times 1 2 is 2 times 1 and so on. That's an odd number and an odd number are sitting here in between. Now I want to prove a lemma. If the natural number n is even then the natural number n plus 1 is even. So how can I do that? Well, if the natural number n is straight then the natural number n plus 1 is even. We assume that n is even and we take some k and we take what n is straight. We assume that n is 2 times k so we know that this is true. What we are going to do now is we just add on both sides 1 and then we get that n plus 1 has the shape 2 times k plus 1. So what do we know about the numbers that have this shape? Well, it's a definition of odd. So n plus 1 has to be odd. What I want to show you is an example in a computer script where a computer program called COC is an automated proof assistant that proofs it so it looks like and of course I can't go into the details but the idea is that we import some sort of arithmetic which is already defined for the natural number n which is just if a k exists so that n is 2 times k and here is if a k exists so that n is 2 times k plus 1 then n plus 1 is odd and this is the coding of the lemmas. Again, I can't go into the details because it's a command and I'm not going to go into the details but here is a command and the idea is to write the comments what happens in this step and the idea is in the end I just write qid and the computer program checks all these steps and it says yes, that's correct or no, this step is wrong do it right and the idea is that to do it interactively and I can't go into the details next question is why should we trust this machine-checked evidence what is it now it's based on a research institute in Ria and France developed a code it's about 50,000 OCaml code but it's been checked by many people and that's all public and yes, we have to trust some of these things but we can actually check them and the idea is to use this framework to do formal verification here's a map of what we did we took the law text that's the 40-page English text of the law manually extracted the rules and modulated a full computer and we tried to keep it as simple as possible we don't try to make it as simple as possible but a kind of abstract machine in logic then we have this law text in text of some notion of correctness this thing produces a correct count with the appropriate notion of correct count if we know that this is correct a correct count means I can't tell you the details but I'll come back to that later but the COCT proof system is so good that we can generate these certificates and if you write a proof in COCT and COCT accepts it then we know that it's true and why should we trust it? it's all public you can ask a choice expert and you can ask a mathematician to check the mathematics and the only thing you have to trust is the certificate that we publish and you can check that what we've done so far we have also used the Schulz method which is used by Linux for example if they decide which direction you should fork the code and we have exact breaking calculations and if you can count up to 10 million in 20 minutes for 40 candidates and 20 places and that's where we go and in a third year you can just write a program that shows that the certificate is correct and in the ideal world you don't have to trust the software or the hardware because the proof proves that if the certificate is correct then the count is correct and if we have a wrong count then the certificate is wrong and I'm going to talk about the minimum and what we do here is we have a minimal abstract state of state that has three types of states initial states and then there's an end state where we know the winner and then there's several between states and I can't explain the details but these between states have a lot of different data that's a two-play list and then from one list to the other depending on the defined states and the states corresponding to the numbers strokes transfer, transfer waves and winners and then it goes from one state to another state and that's described in a logic and the idea is we prove the minimal conditions that it corresponds to the rules and I can't show it in detail but I want to give you a simple example we're going back to the idea of natural numbers and what I want to do I'll show you a code that forms the natural numbers in COC and that's COC here and 0 are my natural numbers and the big O are my natural numbers and a big S is a function that takes a natural number and gives a natural number and what does that mean? for example 0 is in big O and 1 is S from O and 2 is S from S from O and every natural number is a string and it has a certain number S and this complicated thing is implemented here and there's a starting list of the voting and there's a final list of the candidates that are elected and there's a complicated structure in between that I can't explain in detail and the minimal instance is given by the definition of these rules and these rules that fill the conditions and the conditions are divided when do the rules have to be applied and in COC we now show three rules namely every every transition reduces the complexity and we reduce the complexity of what is left over that means in the end we reduced the complexity to 0 and there's the life and these are the conditions that you can't give that blocks there must always be a transition when you always have a lot of voting numbers then this process always terminates and what we did is we have this code extraction from COC and COC extracts the code I can't explain it in detail but what we did is we created a program that creates a certificate and what is a certificate? a certificate is one run of the automatic system and it's easy to write a program that checks this certificate and I'll show you an example again to the natural numbers and I claim that this certificate is this certificate of a correct addition and what is that? it's just a stack a stack and I add something to something to something and in the same way this is a certificate that was given by our voting program and this is a voting certificate A is better than C is better than B and it moves somehow through this list until you have a winner and this is essentially what's happening and the claim is that to check this certificate you only need simple verification and these are the rules that are applied to go from the next stack and there are rules here ADR0 and ADRS so what we're going to do is we're going to define an example and I'm going to give you a simple example we define the natural numbers as I said and the simple ADR0 has to be bigger and why is that? if I add 0 to X then it's X and if I add 0 to X then it's X and the middle is always 0 and when I say X if I add Y to X then I give it Z and if I add X to S from Y then I have S from Z and if I add Y S, Y and S, Z then it's correct and what's the duration of that and the sentence claims that a correct duration is a correct result and I claim this is a correct duration so is this correct yes, that's right because it's something and the same thing and 0 plus 1 is 1 and if that's a correct duration then we only have an S0 and an S and that's the step the duration step and that's also a correct duration and 1 plus 1 is 2 so what's the end result 1 plus 3 is 4 and what we have to check is that what's the end result that I defined up here and what is the progress for elections that means we have these logical guidelines from the election process and we publish a mathematical proof that this was used correctly and this software is probably wrong but it has to produce a correct certificate and that's what we mean if we say a correct certificate and the software may not do that but the government has to publish all the correct books and then we have available opportunity to check everything they just look at all the correct certificates and they can write a program to check that they just have to do the rules what a correct certificate is and the guidelines to implement this pattern matching so what we have is a lot of certificate reviews and it's very unlikely that all of them keep the same bug so what do we do now we make the choice all check it and somebody says the public certificate is incorrect and what we can do now is okay the public certificate is incorrect but at what point is the mistake we said well we can check so there is a wrong number of that kind but someone else said public certificate is wrong so I will say public certificate is wrong tell us where it breaks and someone else said okay tell us where the rules are get the academic certificate someone else might say let's go to the academy or somewhere and they ask to check someone else could say the correctness criteria or the encoding of the math is wrong and here you can ask give us an example where is the law and what is the correctness and in the end what is the correctness if the correctness is buggy the result implies incorrect and we have got all these thousands of people who are checking the certificate so what it says is that the bug only interested in the correct result and also interested in the program and this one of the program the result can still be checked or the vendor has to be in the program but the result is not going to be tested so this one is correct so it could have been a bug in the program or it could have been hacked but in this result it was not relevant and this result is still correct okay, what have we done now we have the transfer of individual mood that makes the correctness and you don't need to trust the hardware or software because there are all these certificate checkers written by in the context document and you don't need to trust the hardware or the software because there are all these certificate checkers because what is not is that if you publish about the only disadvantage is that we have to open all the selection and that is a problem because it is known if you publish the selection then the selection is called a certificate checker which is firmly correct with respect to this amazing thing using a different thing in the certificate checker which is checked in C with a different interactive provider so can we do this to balance because in Australia so the message that I want to and the question is now can we still do this that would come around this attack and now the conclusion that I would like to have in Germany is that it is possible to build such a system thanks for the talk no one that before you heard about individual mood do you think that the German was a specific time as a reminder you can ask welcome to the Q&A session now hello and welcome I am positively surprised about that we actually have some questions about money public code in Australia or if there are if there are any public money public code in Australia or how do you do this international cooperation between these movements so the first one is about cooperation the first one is about international cooperation and as far as I know there is not the only cooperation between academics and me there are groups in Karlsruhe and in France in India and there are some other groups in the world who want to push and move to accept this technology and the biggest problem is that the elections are not acquired on sophisticated software right so they are not like you know the space commission they go out and create that so they don't say well okay they go out and create that so they don't say well okay yeah and what was the second question what was the second question so I think it was more about the public how does the attention to close to this code so I think it was about how does it get the time code in public what happens here is that it is a strange a strange conflict of interest is that the commission has built a lot of trust in the last 100 years and the last thing they want is that their trust is undermined so what they don't want is they don't want the academics to jump into the air and say there is a mistake in your code and there is another mistake in your code but what they prefer is the manufacturer guarantees that everything is in order trust us that someone will only change if there is really a big accident so that a candidate loses his choice and goes to court and then the choice has to be repeated it costs millions of dollars and it is very embarrassing and it will only change if a catastrophe happens and what we found all over the world is that there is a lot of academics who give me a square jump and say we need verification and that is for the reasons that I just explained absolutely absolutely it sounds like security security it sounds like security it sounds like security it sounds like security it sounds like security in the CEL module there were some requirements in the code many to avoid for example the state solution I don't know everything you told us is that you tested it and all tests were done and you have to see where the choice comes from they used to have the right number and the camera has 250,000 and they had to quickly count to give the result and in 1996 there was a verification and there were three votes and candidate A was the winner and then they had to count everything again and then it was different and then candidate B counted only five and then candidate A counted again and I don't know how it ended but it took three or four weeks so at that time the end result was not counted and that is why the election commission said it has to give us a better chance to do it with computers but what I didn't notice is if you do it with computers then you have all your own single-core and you have to trust that this individual error point you have to trust and the next is you have the so this is the classic thing of form of error and the same as forbidding of it then we can go with something wrong then you can get everything out there is no evidence that it is a bad logic error in the form but in the end we have a proof that a correct certificate means a correct number and that means if there should be a problem in the form of error we can say that this error in the form of error we can check the proof and that is the scientific peer review yes string theory is maybe wrong but all tests in the moment that are currently being tested they exist and all experiments that are being tested yes I think the next question is not really meant well frankly I don't know I wouldn't trust it but in Germany I think I learn the right things and before diploma is not trivial I trust computer computer students in 3rd year from Germany ok I think a lot of people would have thought about this ok next is to which grade do voters trust in these election machines can you answer that yes in Canberra we don't have an electronic vote with a tablet we have you go to a election and a computer with a screen and then you press the button I think it's a touch screen and you touch the screen to create the correct number and then I say and then you get another friendly vote and say thank you your vote is given trust us and what people should ask is where do you know that the vote on the screen is actually the one that was given where do you know that the machine doesn't make a mistake with the vote and only one candidate unfortunately the simple voter doesn't know that he should ask this question and what they are asking I can't check a paper vote that's why I trust the election commission and I don't know enough about computers to know that what is on the screen but I trust the election commission another one wants to know whether and how you distinguish if you vote right so if you remember right at the beginning I can't even verify vote but there was this test game which was cast which was cast which was cast which was cast which is the vote and the vote which was left and cast which is registered and in the electronic Are you able to give the vote to the re получ into the end result it was the engine which was cast then some have And the tablet sends the voice over the internet to the election commission. And the election commission has to prove at some point that the voter can check that the voice has not been changed. And the election commission has to check the results. And that's the traditional way of doing electronic voice tasks. There are systems that can do that. But in Canberra, we have a voice cabin. That means you give the voice in the cabin, in private, on a computer, by saying that it really works. And the voice is then somehow transferred to the election commission. And they say it's all right and everything is safe. And then you can say that with your program, that they don't show us, and they assure us that everything is all right. But in Canberra, we don't have any end-to-end verification, because we have to trust the election commission and the manufacturer. And in all cases, we have to trust the manufacturer. We have to trust the manufacturer. We have to trust the manufacturer. We have to trust the election commission and the manufacturer in all steps on this path. Is that an answer? I think so. So, in the end, you treat the problem and let the others do it. OK. OK. So, another one. And you have to have a complicated cryptographic process to secure the first steps. Another question was, in Germany, we have bizarre methods. Any ideas if it's a commodity system or a solution for a situation? Yeah. So, is that also a solution for all situations? And the easiest answer is I don't know. I haven't tried it yet. But my colleagues in Karlsruhe, for example, there are theoretical informatics and a formal method group in Karlsruhe, who are the right people who could understand the German system. I'm sorry, I haven't looked at that yet. OK. And I have a counter-question. You said that the German system is a bit bizarre, but this system, which I showed you, is also quite complicated. Is it even more complicated than this contractable individual voice message? That's a good question. I haven't looked into the German system, but it could be in the local ones. There are a lot more complicated systems. But that's just my unqualified answer. So, there are probably a lot more qualified people. Yeah. So, we know that this process is one of the most complicated. And there's the social choice theory, and where mathematicians have compared various choice processes. And people have thought out many other things alternatively to do it fairer or better. And there are at least 10 different processes that we have looked at. And this is one of the more complicated ones. And everything depends on, for example, how you solve equality, how you decide which one is the worst candidate, and such things. For example, in the ACT system in the camera, it means that if everything is the same until the beginning, then the choice-maker can throw a coin. Okay, so someone noted that... Oh, that's a random answer. That's also pretty bizarre. Okay, someone said that I'm going to get these very special bizarre systems, especially in Bavarian countries, in the South and South. Yeah, I've heard that Bavarian is a bit unusual. Okay, then the last official question. How is the certificate generated? And the French has already missed it during the talk. And the certificate is just a naive expression of the process of the state automation. And I mean, we could do it strongly, and we could also... we could also shuffle it, and we could take the right number and randomly create a certificate and claim that the certificate would be correct. Maybe we don't even use a program for that. And then we just have a thousand aphids that create random certificates and after an hour or so we say, so this is the certificate. And yeah, I won. But it doesn't matter, because the evidence says that if you can verify the certificate, that it's correct, then the number described in it is correct. That doesn't matter if we use a random generator. We don't do that, but we could do it. Okay, thank you. One last question we had in the studio. Could you explain a little bit more what the problem with the public choice was? Yeah, that goes back to the mafia in Sicily. And that's why it's called the Sicilian attack. And what you want to make sure that Raj gets elected because Raj has a choice, for example, for the mayor. And you want to know that Raj is elected, because Raj is a criminal and you can deprive him and do what you want. And what you do is you tell the voter, I want you to vote on this very complicated, certain kind of vote. For example, Frederick on three and Frieda on one. And Raj on 17. And what do you do? You're on five or something like that. And then what you do is if you publish the ballot, and then you look at exactly this weird ballot, it's the right number. Frieda 17 and Raj 17. No, it's the right number. As you know, if this is not the right number, then you know that the voter didn't do what you wanted. And then you can fire a house or shoot their children, or whatever you want to do. And in Australia, in the election campaign, it's very plausible. Because, for example, we have elections where there are 43 candidates and you can and so in so many possible combinations and once the ballot is public, and we ask you to vote so and if these voting votes are made public, then we look at this one very strange perverse voting and then we are sure that we distribute the candidates very conveniently, and we want them to win. Imagine that he wins. Does that help? Yes, I think that makes it a little more understandable. It's through the complex basic assumption that every election is actually secret. Yes, on the bottom of the complexity of the system you can be clearly identifiable. Yes. Then that's the last question. Thank you Raj for your talk and also for the very detailed answers. And thank you very much. Okay. And now on the 5th channel the last one today is the Herald News Show and tomorrow it will go on or we will continue at 5 pm with a lecture in latex. Thank you. Yes, thank you very much for your attention and for the lecture about formal verification of electronic vote and was translated by Pink Trispetscher and I don't know how to pronounce the name. Thank you. For feedback please use the hashtag c3lingo. Thank you very much. Thank you.