 Welcome to the only public third party assessment of voting equipment in the world. Okay, third party testing of election equipment in the world. What do you want to call it, Matt? Okay, third party opportunity to examine voting equipment in the world. Leave it to the academics to be more specific. So I had some great conversations yesterday with some amazing Americans who are deeply concerned about protecting our democracy. Noah Pretz who's here, Amber McReynolds, Neil Kelly, Secretary Padilla in the state of California. And there's dozens of other election officials who are coming to DEF CON this year to try and figure out how they can better secure their elections bar by room to who's in the back from Michigan. So first of all, I want to give a round of applause to them for taking this so seriously. Then after I had those conversations, we got these two fucking letters from the Secretary of State's Association and the largest vendor of voting equipment in the United States, basically saying that this is bullshit. So I'm not hiding behind the corporate veil. I'm not hiding behind the DEF CON Twitter account. I'm not speaking for hire, Matt. This is me, Jake Braun, standing here, just one measly human out of seven billion standing here in a jeans and a t-shirt. And I'm standing here to say, fuck you, you fucking Luddites. I'm a deeply patriotic American, as is everybody else who's doing this. We are deeply concerned about securing our elections. And as I've said thousands of times, nothing that we say is going to make the public have more faith in our elections. Only something we do is going to make the public have faith in our elections. And what we're doing here today is identifying things that local election officials can do to better secure their elections. So one thing I do want to address, before I turn this over to Hari and Matt, the actual technologists who know what's going on here, is this claim, these polls that are cited, that what we're doing is going to undermine people's faith in the voting process. And they're not going to vote. Right, thank you. So a couple things. First of all, I've worked in politics for 20 years working on campaigns and elections. I'm actually not a technologist like these guys are working in politics. And I know how to read polls. And what I know is over half of the people in that poll are talking about undocumented people voting in our elections. And that has anything to do with what we're doing here, right? The other, the remaining people who are actually concerned about voting integrity and the technology involved and so on, they're concerned for a good reason. And it's because of Putin, not us, that they're concerned because Putin has actually attacked our democracy. He's the one who hacked into our voter registration databases. He's the one who hacked into our election systems and could possibly hack into our voting machines. So what we're trying to do here is actually give election officials and the public an opportunity to understand that people are taking this seriously and we're actually trying to secure their democracy. Because again, as I said a minute ago, election officials and others can say all they want that everything's secure. But the public isn't going to believe it. They're only going to believe it when we do something to make it more secure. And that's what we're doing here today. So with that, I want to turn it over to the true heroes here, Hari and Matt, who actually are going to be helping people identify things to fix. So thank you all very much for coming here. Thank you for coming. Matt and I, we were running last year, the voting village, and I'm giving a little bit of 10-minute reflections before turning over to Matt. First of all, as Matt said, we are not doing a systematic research. We are not doing evaluation. We don't do any of that. What we do is we allow people to explore, discover and freely exchange the facts they have found. It's all about being educated, be informed and find out the truth yourself. Last year, probably the one thing would really make me happy. A lot of things, but one particular thing was the local election officials who came in and hacked the machine they use in their own jurisdiction. Because that was their first opportunity to take a look into the machine themselves. The thing which makes this all possible is DMCA Accent. Right now, we have a favorable law which allows Bonavida real security research done, exploratory events like this possible when we can take a look into the systems and reverse engineer them and look how they build, what they do, without problem with the DMCA as long as we do it in a responsible way. And we honor the safe harbors needed. So hence, we do put the limitations what people can do, but everything is according to the DMCA Accent. I welcome you all to come to the village. The village is not us. It's not Matt and I or we who help people to do discovery. The village is you guys, everybody who come in who participates. You are the people who do the discoveries. You are the people who learn the facts. And Matt has been saying very accurately that for 15 years, a very small group of people like us had been allowed to take a look in the voting message. And we have been sometimes under NDA, sometimes without NDA, sometimes the reports are deeply redacted and what we can say. But it's still the underlying issues trust us. That's wrong. Nobody should be trusting us. Who knows, are we honest? That's why it's important to everybody to discover themselves, learn themselves, do their homework and that's what we are for. We are for education and discovery and free exchange of data. Please come to the village, have fun. I turned over to Matt. Professor Place. Thanks, Harry. So I just want to say a few words reflecting not so much on what we're doing, but on what we're not doing. And first of all, thank you so much for crowding uncomfortably into this much too small room. Next year, they have to give us the giant room for this, obviously. And your commitment to democracy shows by the fact that you care about this. I've been working in voting technology as a computer scientist for, you know, 15 odd years now, maybe more, depends on how you count. And for really, for most of that time, I've been spending it being increasingly horrified by the disconnect between what computer scientists and experts in the field understand is necessary to do to build secure and trustworthy elections, worthy of a world-class democracy, and what we actually do, the vulnerabilities in place in these systems. And in particular, being horrified by the level of kind of denialism on the part of people with a vested interest in the existing systems. Really for the last two or three years, I've been optimistic for the first time in the time that I've been working on this, that we're actually making progress, that we're actually winning on voting machine technology. In particular, DREs, a lot of the machines that we're seeing, are not obsolete yet, but they're well on their way to being obsolete. The touch screen voting machines that only create a record of the vote inside the machine. States are finally moving away from those machines. We have formal understanding of two incredibly important concepts. One was invented by Ron Rivest, the R in the RSA public key algorithm, called software independence. The idea that a voting system shouldn't be dependent for its integrity on the integrity of any piece of software. And that that's a design criteria that should be incorporated into systems. And the other is a technique called risk limiting audits, which is a way of statistically sampling when you do manual recounts on systems. States are moving in the direction of building software independent systems that use technology like risk limiting audits. An example would be optical scan paper ballots that are physically captured and leave an artifact of the voter's vote that has nothing to do with software that can be recounted by hand if necessary. And we're actually winning in moving in that direction. We're not there yet. And one of the things that we can accomplish and hope to accomplish with the voting village is to broaden the community of experts from the privileged few like Harry and I who've been allowed to look at these machines to the broader community of hackers and technology experts who can understand exactly how vulnerable these kinds of software dependent systems are. So we're doing an important and I think critical service to democracy. I remember 10 years ago when we did the assessment of the ES and S machine when we were invited by the state of Ohio and the state of California to do assessments. I remember thinking this may be the most important work that my students and I ever do. And I think that actually was wrong. This may be the most important work that any of us in this room ever do. We're actually on the cusp of moving toward more secure voting technology. Now I'd like to say one other thing about what we're not doing, which is that our focus here is not exclusively, but it's probably disproportionately on voting machines. And one reason for that is that there are only a relative handful of voting machine vendors. There are four major vendors in the United States of voting machines. There are a handful of different models of voting machines from each vendor. And these are the things that we interact with. These are easy to understand where the risks are of tampering with an election. And it's an area where there is real progress needed to making these systems more secure toward things like optical scan paper ballots. But that's not the whole story. There's another big piece of this and that's the back end systems used by counties to manage things like voter registration databases and ballot creation. And the counting and tallying of ballots and the reporting of results and the management of other election operations. There are about 3,000 counties in the United States that are responsible for managing the elections in those counties. And that means that there are about 3,000 different systems administered by different sets of people with different amounts of competence and capability to defend them against attack that are as important to the operations of elections as the voting machines that we're seeing. So one of the things that I want to caution us to do is to be very enthusiastic and zealous about understanding how these machines downstairs work. But let's not declare victory when we solve the problems that are there. They're also very important but probably less sexy security vulnerabilities in these back end systems. And in fact, Russia has gotten a lot of attention as an example of a new threat, the state actor who wants to disrupt confidence in an election. And in fact, if we look at what happened in 2016, it looks like even though we have demonstrably insecure voting machines, a lot of the focus of the Russian interference on the election was on back end systems and registration databases and so on. So one of the lessons that I hope people will take away from this to bring back to their local counties who are buying the voting machines and managing these back end systems is that we need better election technology. We need better voting machines. We need better voting technology. And we also need to get resources to pay attention to securing these back end systems. And that's a less interesting but equally important problem. The other thing is that I want to just give three words of advice. The three words of advice and I'm like a broken record on this. People ask me, what can I do from here to get involved in securing elections and voting technology and making contact with my local officials? And I have three simple words of advice. Become a poll worker. Maybe that's four words. I'm not sure. Every local jurisdiction that conducts in-person elections needs people on election day to help manage and run the polling places. And they're called poll workers, election judges, various names. But everywhere where there are in-person polling places in elections, those people, overwhelmingly senior citizens, are drawn from the community. You basically get paid a token stipend for the day's work in most places, enough to buy you lunch and maybe parking. And a huge advantage of this is you actually see how an election runs. You learn the little details of all the procedural things that happen when people vote. You get to help voters. And you get to meet the people in your county who are managing all of this, making the purchasing decisions about what equipment to use and protecting the back ends and the voting equipment. And you get to learn what their problems and what their concerns are. And that is just an invaluable first step. So there's almost certainly wherever you live, still time to do this ahead of the November elections. Find out what it takes in your local community to become a poll worker. You will learn at least as much as you learn in the voting village here over the course of this weekend. So Harry, why don't you come back up? We have a little bit of time. We're hoping to just kind of open it up to questions generally about what to expect this weekend and so on. Let me just make a quick overview of what is around us and what's going on. First of all, I thank you, Matt. One thing I would a little bit modify would you say that you need better. It's like America deserves better voting system. It's not anything less than that. And one thing is, again, I want to underline. If somebody would try to explain me everything I have seen, I wouldn't believe it. That's why you have to see it in real life. Now downstairs where we have we have e-poll books, we have voting machines, and we have a cyber range. Cyber range is a virtualized environment of exactly what Matt said, a election environment. There is a voter registration database downloaded from Ohio. It's protected by similar systems. It was built by a help of a very great election supervisor. The environment is similar to what it real is. The question is, can you defend that database? Can you attack the database? That is a training environment where you can go and train how to defend and how to attack that database system. And there's a lot of other things going on at that. So that is in a corner, the huge screens. That's a virtual election office, which you can play around. It's not real, but it's very similar and real. And you can see how to defend the vote. Then there's the voting machines, e-poll books, and then we have the roots. A cell where teenagers, I think it's from 8 to 16, are doing election related challenges, which are mock-up voter election reporting databases. In the United States, it's still recommended in certain places to email a ballot image, not very good idea. So there's a challenge which for us adults would be stupid, but for kids, fun. So, opening the floor for questions. So I don't want to call, since I don't have the definite list in my head, I don't want to call anyone out by mistake. But there are about 15 states that still permit DRE machines, these touch screen voting machines, without any voter verified paper audit trail. And the problem with those machines, the problem with that scheme, is that if the software has been tampered with prior to, during, or after the election, there's no way to know what the true votes were. There's no way to conduct a meaningful recount. Fortunately, the number of those states is decreasing. So we're making progress. It's not enough progress, but we are making some progress. We also are going to be running in afternoon. We already have set up a few machines, but we are going to run a mock election in the touch screen machines used in the United States, and about 20 states in a lesser extent, 15 states. And I'm absolutely certain the outcome will be honest. Yes, so I think we're conducting the election between Benedict Arnold and George Washington. And we'll come up with some other candidates later in the weekend, but we can see if George has a chance here or not. Any other questions about what to expect this weekend? Yeah. So it depends, again, on which state and which county you're in. They also need election judges to help count there, in most cases. I mean, this is a huge temporary workforce, whether it's mail-in ballots or in-person. When it's in-person ballots, it's a little more prominent. There are more people involved, but they often need help on the back end as well. So contact your county or state, depending on which state you're in, voting officials and find out how you can get involved in that. There's an excellent chance that you can. Okay. Yeah. Right. So I was asked to repeat the question, I think, yeah. So the question was, are there trade-offs here? Are there benefits to touch screen voting systems? And this is one of the difficult conundrums here. Nothing is, you know, this is not an epic battle between good and evil, where everybody who has a different opinion from you about this is malicious. There are genuine benefits to using computer technology for casting ballots. In particular, people who need accessibility help benefit greatly from touch screen interfaces that might have audio capability, the mobility impaired, have things like SIP and PUFF interfaces, so they can vote without having to trust somebody to mark their ballot for them. Fortunately, those benefits don't necessarily require DRE voting machines. There are technologies called ballot marking technologies that use a touch screen interface to help mark a ballot, and that ballot then gets put into the optical scan system, so you still have the paper artifact coming out of the machine. So there are ballot marking technologies that yield essentially the same benefits for accessibility as DRE machines without the dependence on software that DRE machines give us. So, you know, there's room for progress there. Yeah, hi. Yeah, so the question was involving ballot marking devices that produce a barcode rather than a marked ballot of the traditional kind. So, yeah, it's absolutely possible to do ballot marking in a terrible way. And unfortunately, there is at least one example of a ballot marking device that produces instead of a marked ballot that a human being can look at and understand, a barcoded ballot, which a human being, you know, unless most of the attendees here probably can, but most of us can't look at a barcode and just decode it. So, you know, it is possible to do this badly. Fundamentally, that design is sort of incompatible with the concept of software independence because you need software in order to interpret the barcode. And so those wouldn't meet the requirements for building a software independent election system. And you can't do a risk-limiting audit based on a human being looking at the ballot and determining what's been marked on it. Yeah, the question really here is auditability and understandability. So the problem here is that they are fundamentally, even if you have a VVPAT, which is the small printer aside printing your choices, we have a number of studies that that doesn't work. People are A, not carefully enough to check that paper, but also there's a protocol of cancelling that ballot, which is on a printer after you have left the station. So that really doesn't work. So it's all about how you audit the election. And it's not important to audit the machines, it's important to audit the results because the results are the ones which matter. We don't have one of those ballot marking devices here. Maybe next year. So the question was ESS, which is probably the largest vendor of voting machines in the country was allowing remote access to their back-end systems with PC anywhere. And not everybody knew that. So it was a particularly egregious example of exposing critical election systems to basically anybody on the Internet. So that was pretty bad. The question was, do I know of other anecdotes where that's happened? So probably the best example of that was last year at the voting village here when somebody immediately turned on the wireless Internet interface to one of the voting machines. Fortunately, a voting machine that's now been decertified everywhere. And it turned out that there was a Wi-Fi interface in the machine. It could be turned on. And very quickly, the machine was recrolling the voters. So remote access has the advantage that is very convenient for administrators. And so I would not be surprised if we don't see more examples of this sort of thing. The ESNS, which you said is not alone, they use PC anywhere. Other vendors use the real VNC that this practice of having remote access has been explored and found out in the wild. So they're not alone. It's also the case that in a lot of cases counties rely on election services vendors to help with things like ballot definitions. And the connections between the counties and those vendors, there are going to be the 3,000 counties and there are going to be 3,000 different configurations. So the question was, across the spectrum of the attack surface for a voting system, you have the voting system itself, the casting of ballots, you have the ballot definition, you have the counting, you have the back ends, you have the registration databases. What's the highest exposure? And I think it really depends on the threat. Historically in the United States, there has been a history of fraud in elections and that history is overwhelmingly not national elections but local elections. People trying to get elected to mayor or dog catcher or what have you who want to throw the election a certain way. And that's the traditional threat that these systems have been designed to the extent they've been designed to resist the threat. That's the threat they've been designed to resist. Over the last three years, we've seen evidence that that threat model is not rich enough to express the level of threat from things like state actors. And if you look at, you know, a state actor, they may not be interested in selecting who wins the election for dog catcher. Their interest may be simply in disrupting confidence in the outcome. They may not care who wins. What they care about is that people don't agree on who won. And that is much closer to the problem of denial of service. And that's not an attack that these systems have even been designed to withstand. Things like the voter registration database. If people show up at the polling place in the morning and their name isn't on the list of registered voters and they get turned away or have to get a provisional ballot to vote, that's a significant disruption. And if that happens at a large enough scale, people may question the legitimacy of the outcome. Now, if you're trying to steal an election, you don't want that. You want to get legitimately, you want it to look like you've been legitimately elected. But if you're a foreign power that's seeking to weaken a democracy, that's a really great way to do it. The problem is, you know, those kinds of attacks can be performed on back end systems that are often directly or indirectly connected to the internet and depend on really fundamentally insecure platforms. So, you know, I think it really depends on what threat you're looking at. I'm going to dodge your question slightly to say, worry about everything. Yeah. Yeah. Oh, you break my heart. So, the question was if you find a device, if you find a flaw in, say, a medical device, there's an easy path, right? You can report it to the FDA. The FDA will use its authority to put pressure on the vendor to fix the flaw. And, you know, there's a pathway to get problems fixed. What are you, is there a similar pathway in the case of voting systems? And the short answer is not really, except public pressure, right? Disclosure, sunlight is kind of the best disinfectant here. And it's made particularly difficult by the fact that, you know, these systems, the update cycle for these systems is very slow. They have to be certified. Patching something is not a matter of just rolling out a patch. And so, there's resistance to even admitting that a problem is a sufficiently bad problem to justify fixing it. So, it's a real toxic environment here. Back when Matt and I and various of other people, when we started 2005, 2006, 2007, and we reported to secretaries of state everything we found, Everest report is 318 pages, is the redacted version. We thought this will get fixed. It's now out in the wild. A secretary of state has published these vulnerabilities. These must be fixed. The truth is that the same software versions are run today and the same software version, in certain cases, are still sold today. Exactly the same versions which we reported 2007. That's a fact and it's very sad state of affairs. What you'll find this year is someone going to an actual polling place and exploiting that device as an insider threat, probably, unless the nation's state after sending 1,000 operatives into the state of the impact of election, which I think is a high land level, unlike, because if you explain the insider threat for the, like, the dockhead. So the question was, was the insider threats explained to secretaries of state? First of all, 2005 when we started, a common practice was called sleepover. Sleepover meant that the voting machines were sent to homes of poll workers up to two weeks before the election day. And the poll workers were bringing them, not two men rule, bringing them to the polling place. That was one insider threat, I think. Today, it's very interesting when we go and see sometimes in a real election place, we see the password literally on a post-it note on top of the server. Again, the commentaries, the Russians are not going to come here to see that password on a post-it note. But the threat is their insider. It's the Joe or Mary, whose risk-crandled employee and wants to disrupt the thing. So we have been in a number of ways, tried to explain all the different various versions of insider threats. And for example, the sleepover practice, thank God, is starting to be dimmish. People are starting to understand, at least put the temporary evidence seals everywhere and make sure that they are not in somebody's garage unattended for two weeks. But there's still a lot more to do. And the insider threat is always a big part of this problem. Even more with what Matt mentioned is the local service companies. In a number of states, up to over 95% of the jurisdictions are actually programmed by uncertified third-party companies, which are locally, literally sometimes a strip mall, 12 dogs, two little guys and a dog in a strip mall, with a website telling exactly who they are, what are their homes. So basically companies which don't have any security consciousness. That is a, from the point of view of jurisdiction, they might qualify that as external threat, but as a system when this is an embedded to the very fabric, to the very DNA how the elections are run, I consider that as a new insider threat. And in my opinion, probably this watches domestic threat towards the integrity of elections in all the aspects. Yeah, I should also point out that one of the criticisms of efforts like this to look at the vulnerabilities in these systems is that, well, you have to be this electronics expert. And you know, people take pictures of the voting village and they show people with voting machines open and soldering irons and oscilloscopes and so on. And you know, that's what you need to do in order to discover the flaw. But you know, you do that offline. You don't do that on election day. You do that before election day if you're interested in attacking one of these systems. On election day, you're going to, you know, weaponize an attack in a way that you can do it very quickly and very efficiently without having to do, you know, to open up the machine and do whatever to it. So I think one of the things that we have to realize is that a lot of these things that look like they require extensive access to voting machines can actually be done with the access that a voter gets when they're privately in the voting booth. You know, for example, the ESNS iVotronic machine has an infrared interface and a magnet. And if you have a palm pilot device and a magnet and you know, the harder one to get will be you'll have to go and get this obsolete palm pilot device. You can, and magnet who knows how they work, but you know, you'll be able to do with the interface of a voter and the privacy of the voting booth, you can essentially reprogram the machine, you know, restart the election and the election, do any of the administrative tasks. Now figuring out how to do that is not something you're going to be able to do while you're in the voting booth, but if you know it in advance you can quickly, you know, you can bring enough equipment in with you to do that very easily. So a lot of these things that look like they're, they require insider access are really attacks that voters themselves can perform. So we need to wrap up soon. I just want to say a few things. First of all, this year every machine we have in a village is in use and it will be used in 2018 somewhere in the US. The other thing is that when we have been doing the demonstrations, both Carsten who is going to show here the win vote, which was remotely hacked on myself, we have been using on purpose win vote because it's decertified. So nobody can make the claim that we are helping somebody to learn how to attack. So that has been a choice. After criticism we didn't bring win vote anymore this year, so now everything is in use. And Carsten is going to show how the win vote works. But that is a, that was a choice for us. And one thing which is important, exactly what Matt said, we have been and the security community of wonderful researchers, for example Alex Haldeman have found out how you can do a wholesale attacks. In a way which wasn't really thought as, which was dismissed as a science fiction sort of a while ago. So again, we are as a people who discovered these vulnerabilities, we know always that we are not the first ones. Everything we find, we solely believe that somebody else found before us and just was quiet about it. Yeah I think one of the things that I remember when we did the Everest study, the budget for hiring everybody and shipping the machines and everything like that for each vendor was about $300,000. And we spent about a month, we worked cheap because it was a professor and grad students. But still for about $300,000 and most of that was stolen by the university as overhead. We were able to discover a 300 page report listing exploitable vulnerabilities that could be used to sway an election. Think about the amount of money that gets spent on elections in the United States, even forget about presidential elections, even state and local races. That amount of money is a tiny drop in the budget. The bucket compared to the overall budget. We spend way more on advertising and campaigns than we do conducting the actual elections. So to a large extent the defenders here are overwhelmingly outgunned in terms of the resources that they have available. Which means that we need robust technology that can withstand real attacks. And it also means we really need to think as a society about giving election officials significantly more resources as technical threats have become more and more prominent. 2016 is not going to be the last time that we see state actors attacking election systems. And do we really expect random county in the middle of the United States to have the resources to defend against the Russian military intelligence apparatus? So on that cheerful note, please spend some time in the village, learn how these machines work. We have a great lineup of talks today in this room. The next will be at noon, a lunch keynote with state and local election officials. So please stick around for that. And we'll have a whole bunch of interesting talks in here and wander down to the village and thank you all so much for coming and for being interested.