 I'd like to go to the next presentation, and it is Machine Voting the Bulgarian Experience. So Alex Stanov is the CTO of Information Services for JSC, and been a software developer starting in the late 90s for a wide variety of projects from specialized hardware drivers to large scale information citizens for private and public sectors, including e-government services, management, and smart cities. And so since 2003, Alex has been a leading computer processing of all election results and referendum projects in Bulgaria. So we're going to learn what goes on in other parts of the world as well. And as a consultant for the Central Election Commission of Bulgaria, Alex is the primary author of technical and security requirements for election machines used in Bulgaria. So welcome Alex to the stage. Thank you. I'm happy to see so many interesting faces for the processes in Bulgaria. I hope we can share some ideas, problems, and solutions. So we already know that my background is IT technical background. I've been for many years a software developer, then project manager. Then I have some IT security and penetration testing background until the end. I had to lead the technical side of computer processing of the results. So this put me in some unique situation where I know IT and I know the election processes in Bulgaria. So this was very distinct. Of course I'm doing all our stuff and as you can see here, my github. You can see the other things because this talk is a bit different from my other talks here in that corner of my head, which are more technical ones. So first let's start a bit about Bulgaria. It's a country here in Eastern Europe. It's part of European Union. We have a population of 7 million people. And it's the size about the number of Nevala. So what you're seeing here can be safely transferred like just one state here in the US. So we have 265 municipalities where they do different kind of elections. And we have around 12,000 polling stations. Naturally we have elections for president, for our national parliament, EU parliament since 2007. We are a member of EU. And we have local elections. This year we already had other European countries elections for European parliament. And in just a couple of months we'll have local elections, which are way more difficult than others. So let's first start how we vote now and what are the processes here, how we put there the machine voting. So generally speaking, you see two ballots. The first one is just majority candidates. This is ballot for president, I think. And you have here the party, you have the person, the candidate. And you just have to scratch with X or V the chosen candidate. On the other side, you have the preferential ballot where you first choose the party. And after that you give a preference to choosing from your candidate via these circles here. And if the candidate collects enough votes, he can push and be elected. If this party gets enough results here and puts, this can go here. And this generally is a good thing because it was for another majority. But it's highly sophisticated and makes non-trivial the processing of the results. So when people vote in the e-day, in the end, the polling station members can count in the ballots, the paper ballots, and they fill polling station protocol with the results. This is how it looks like just the first pages. This protocol looks pretty clean, but in reality we receive this. You see a lot of scratches, a lot of corrections here. And in the end you don't know what exactly happened in that section of course. If we are not so sure what happens, they open the backs with the ballots, recount, and et cetera. So putting machines in this equation. All voting machines are offline. We don't have online machines, as in most states as I understood. When the voter goes in the polling station, he can decide if we will vote by paper ballot or in the machine. And after that they fill the protocol with and the results are written from the machine on the USB drive. And this is where it gets really crazy. So the polling station members have to soon by hand the results from the machine with the results on the paper voting and they have to put it on these protocols. After that all this is transferred from polling station to IT centers where the data from these paper protocols are entered by operators. In fact we have two stage of data entry. On the left you see where the polling station members go and go with USB drives from the machines and with the protocols and give them to operators. They do first entry. After that in separate IT center deployed in different place it's re-entered again. So you can catch the eventual discrepancies between those both databases. This is a lot of hard work which sometimes happens for about 72 hours. In the e-voting in Bulgaria we in Bulgaria very much like and love to do experiments with different things. So back in 2009 we have these machines. You can see here the magnetic stripe card reader, the touchscreen and I don't know why but they put two UPS devices because half power is really a paramount. So the good thing was that this happened only in nine polling stations from you remember 12,000. And we didn't have two big problems with this. After that we continued with the experiments and back in 2014 we did it with 100 polling stations in machines, then with 300. And if you can see there we fall to 50 because from this they started to use the results not just as experiment but we use the results for real. This was the machine back then that we're using and maybe the important thing here is that Bulgarian government decided not to buy these machines but every time we rented or landed because they're not so sure who can keep them. Will they be needed if there is a legislation change because all these were experiments and et cetera, et cetera. So from the experiment what we know with these kind of machines the first version were installed with all dated Windows XP. When somebody screamed about that they put it Windows 7. The way that machine is activated to allow somebody to vote was with you see this cable here. On the other end you have just a button which is unlocked which is pressed by the polling station officer and the machine unlocks. So no other way. The problem here, yes the machine prints paper slips and you pass these slips here so you have paper audited trail. Of course we have the slips but we didn't have the legislation that requires those slips to be counted so of course we have a count but yeah we have the trail. There were some issues with the machine missing flash drives and of course very low machine voting usage because generally the campaign had to be in a very narrow time and nobody understood that if we can use the machines at all. And of course there were some polling stations where the machines were not powered on. So in the beginning of 2019 the Central Election Commission tasked me to draft new requirements for the machines because most of the people understood that this is a problem the way we're doing it and we have to have something more current against current threats. So in the requirements we put very strict hardware and crypto algorithm requirements. We put that the machine can't have any external communication interfaces so the air gap principle we were kept. We required to have a secure boot so we know that nobody changed the machine in transit. We required a full disk encryption of course against this tampering again. We put very hard requirements about what should be in the OS image, what drivers, what components should be stripped. So we have... I'll get to that. Because we're still on the stage of requirements. After that the guys who lent the machines have to do a certificate authority and the only installation data about the candidates about the parties must be signed and the machine have to decline to what... to refuse to what assigned data. And in the end we have activation with smart cards to control and three voters are seen here. So when the voter decides to vote he gets one voter card. It can unlock the machine and vote with it. The vote must be signed the electronic ballot with this smart card. After that he returns this card and the next voter can vote only with one of first two cards. So they rotate and we avoid this interesting button that you saw on the previous machine. Of course the usual requirements for VGPAT slips counting, very detailed voting process descriptions that have to be implemented are also in place. We wanted to see the code and we had requirements about unit tests that have to be covered, the coverage of testing etc. All the things that we are doing when we are developing, trying to develop high quality and secure information systems were in place. So what happened? Because those machines were rented again they came with Wuntu Linux. Of course they were smart card activated with external batteries. Here is the smart card reader and here is the printout for the results. As you see it has places for physical clips where you can clip it with tamper evidence. This is how the machine works. The problem that we had here was that nobody wanted from the government institutions to audit if these machines really adhere to the requirements. So the vendor says yes, they are fully compliant by the requirements but we didn't have somebody to check that. But all it was we were obligated to do and we took the team from Bulgarian Academy of Science and Sofia University and under the supervision of the Central Election Commission they did some audit which is classified. We haven't seen that and they said we are good. So what happened? Just several months ago back in May we landed the machines again. This time 3000 machines. No audit. The time frame for the project because of those high requirements was very narrow and it was practically impossible to do the audit in proper way. A lot of polling station workers missed the training. I didn't know how to work with this machine and we went into coordination hell basically between different institutions that have to deal with these machines and their physical and logical protection. Some statistics from 23 machines from 3000 we missed the results data. Just 26% of the voters used the machines despite the previous several experiments. 26% of those who voted. I have to check but it's not really important how many 100 or 1000. It's just a percentage. Generally in Bulgaria we have high percentage of voters activity. Of course it's a war on EU elections but yeah it was nothing different from other ways. So 5% of the machines weren't used at all and in 10% of the machines less than 10 people voted on the machines because remember you can choose to vote on the machine or to vote with paper. So we detected a lot of human errors because we take the polling station protocol and we take the USB data from the machines and in about 2.6% of the polling stations you had more votes received from the machine than summing in cup from those on the machine and the paper. I'm not sure if I said it correctly but this is really a big problem. From the preferences that you see we detected errors in about 40% which is really high. What that happens you see this paper slip it comes around here so the polling station officers have to read this and to submit by hand with the paper ones and to put it in polling station protocol and this is very very error prone to as we saw it. Just for information the project cost was about 5.1 million US dollars which was the landing of the machines and services that were received. The aftermath from this was a very wide political and media debate about if we should continue with the machine voting. There were a lot of concerns about the security, the result's accuracy and of course the price of this machine voting. In the meantime of this debate on 15th of July it just amounted from today back. We have the largest personal data leak in Bulgaria. Somebody hacked the machine in our national revenue agency and put a lot of personal data of Bulgarian citizens. Of course on this debate it gets a bit negative about machine voting because you can't protect the national revenue agency how you will protect the machine voting etc etc and on 3rd of July just half a month back we had the legislation changed. We will not have machine voting on upcoming WACO elections which are to be held in October this year but for our other elections after this we will use only machines in polling stations with our 200 voters. So go figure I really for myself I don't know how this was decided in the end. But yeah and that's all. Was that because of double voting or double counting? Total votes for what? I had the problem that total number of votes was more than the balance plus the machine thing. So that obviously a problem but was it because something was counting twice or was it because some people were able to vote twice? No, it's not concerning double voting there were no double voting it was just a human error when you sum the result from this long paper slip with the results from the ballot box and they have to write it on this protocol and those are about 3200 numbers that you have to sum and of course humans made mistakes. Yes please those? Well in those were missing in 23 machines which means 23 polling stations. In our legislation we can have a polling station with more than 1000 voters this is tops and when I say missing results we have a paper protocol from this polling station which says that there was a voting there but for some reason which can be they couldn't find the USB flash drive the USB flash drive can be read after that or some other issues we just don't have the data from the machines. But I don't think that for this 23 polling stations a lot of people use the machines anyway. Some of those were part of those that were not used at all. Yes please? No there is no EU regulation on using voting machine. There are some just very broad requirements about voting process but every country in the EU decides by itself how it will vote what processes and mechanisms will have. We have a bit more complex requirements concerning the EU elections which are held in Europe for one week but generally this is up to every country to decide. Well we're not sure because there was not any extensive research on that part and the politics like to dribble with that if the machine voting is secure or not of course half of the parties want machine voting, other half say this is fault and we don't have to use it. So it's different, it's really different. We have some parts of the country where we have very high usage of the machine voting so it's not evenly distributed everywhere. But in some other places we have really small awareness of that. Yes please? Well the requirements says that we have to have a paper audit trail and all the data that comes on the USB drive from the machines have to be signed with the smart cards with of course the certificate and the smart cards have to be transported by different means not with the machines in complex but from different people yes from different teams. All this is in the requirements but we really don't know or at least I don't know because I haven't been everywhere and of course I haven't did the audit and I don't know the audit results and that is always okay but I believe these contours were in place. I hope these contours were in place. Yeah always connected yeah. Okay people can't vote where they decide, they have to vote in the polling station that they can vote and where the machines is decided by the Central Election Commission so you can choose and in most cases you know you can use the machine when you go to vote not before that. This must be done by the polling station members and maybe not yes there was a campaign about that. So okay sorry guys if there are more questions I'll be on the back. Thank you very much.