 As you can imagine, Accurate spends a lot of time working with DRE machines, trying to figure out ways to make DRE better in theory, tries to find ways to limit the damage of DRE in practice. And before I go on too much further, I'd like to point out that auditing might not mean the same as what you think it means. When programmers and geeks talk about audits, we talk about code audits. We talk about looking at each line of code and making sure that it's correct. That's not what we talk about when we talk about auditing an election so much. You might audit the source code for DRE like that, but when it comes to auditing an election, what you really want to do is to verify each step that was taken and verify that the outcome of that step is correct. I clearly need to thank the National Science Foundation first for funding this research. I need to thank all of you American taxpayers for funding the department, which funds me. Also a couple of the code monkeys in UI Accurate, Patrick Holly and Tristan Tita. Now, first off, before I begin, who here thinks that DRE is in principle a really good idea and we really, really need it? No hands go up. Okay, one hand. How many people think that it is a priori a bad idea and we should stick with paper as long as possible? A lot more hands are going up. The good news is both sides are right. The bad news is both of you are going to have to change your minds. So in order to help facilitate the changing of minds, I need to talk for a little bit about the history of voting. Most people think of the secret ballot as something that just, you know, sprang into existence at the time of the Big Bang. That's actually not the case at all. In the earliest days of this country, all ballots were public records. You went down to your local courthouse. You talked to your local clerk of court. You put your hand on a Bible and you swore your vote. It was entered into a public record. You were associated with it. When paper ballots began to become popular around 1796 in Tennessee was the first real American experience with paper ballots. Newspapers would have ads taken out by political parties in which they would print out all of their own names on the ticket. And they'd say, hi, tear this out, take it to the clerk of court and just tell him that's your vote. This is not something that we recognize as voting today. It took technology to change things. And just so that you guys don't think I'm pulling one over on you, this is a painting from George Caleb Bingham. It is a painting of an election in Salinas County, Missouri in 1846. I'd like to point out a couple of things to you if I can. Can anybody guess what they're talking about? Fixing the election. Fixing the election is an old American pastime. It is honored in its own infamous way. These guys were local party bosses. They were standing by there to make sure that everybody in their company voted the way the boss wanted them to. So by modern standards we had considered this election to be incredibly, riotously unfair. And in fact it was. And things did not change for a very, very long time. It wasn't just party bosses who could intimidate you. It was your alcoholic domineering father. It was your best friends. It was your circle of friends. It was your church. It was everything else. This wasn't voting. This was a high school popularity contest. This is no way to run an election. Despite that, we didn't think that anything needed to be changed up until about 1890. In 1858, Australia invented the secret ballot. Thank you, Australia. Do we have any Australians here? No? Well, damn, because you guys today are doing something else right too. The secret ballot may seem really simple, but it's not. It required a lot of changes to the way we did voting. It was very, very expensive. So before, all you needed was a clerk of court who had to be paid a certain salary and he had to record votes down. No problem. Now you have the expense of voting booths. You have the expense of printing ballots. You have to pay people to count the votes. You have to pay security to guard the votes. Everything else. This resulted in a huge increase in the complexity of the system, but it was on the whole good. Now that said, it also killed off third parties in America for once and for all. When a party is pre-printed on a ballot, there is a huge social impetus. It's more convenient to fill in a checkbox than it is to remember, so how do I spell McCain anyway? I'm sorry. Oh yeah, Schwarzenegger. That's a great one. Keep in mind that if you misspell the name, your ballot could be thrown out. Anyway, despite being invented in 1858, adoption was very slow. It wasn't adopted in the United States until 1888, 30 years later. Shortly thereafter, we had more explosions in voting technology. Lever action voting machines and punch card ballots became common in 1892. We started using hollereth machines, hollereth cards almost immediately. People know what hollereth cards are, right? Yeah, we call them punch cards, the same thing IBM 360s used. Mark's sense ballots date at least to 1916. We can trace it back further, probably, if we want to. There have been no major changes in voting technology ever since 1916. This means that for 90 years, it has stagnated. Our voting technology is based around ideas that were old when man walked on the moon. They were old when the first computers came out. They were old when World War II was fought. Now that said, we shouldn't necessarily throw it out. Just because something is old, that doesn't mean that it's bad, right? If it was, we would be throwing out all of our cobalt legacy applications and all of our banks would be handling our credit cards using Java written in the last 18 months. Who thinks that's a good idea? I'm glad to see there are so many wise people here. So do we need to change? I think the next slide will speak for itself. Yep, yep, it keeps on getting worse and worse and worse. We are beginning to see the kinds of failings that we would see from any other system which is being stretched past its limits. What happens when you have a light duty web server that suddenly takes a slash dotting? It begins to fail. What happens when you have any process that is suddenly ramped up far beyond the volume or the workload that it was designed for? It begins to fail. And that is exactly what we saw in the perfect storm of Florida 2000. Everything that could go wrong went wrong. We're talking butterfly ballots. We're talking predominantly black men being disenfranchised from the vote. We're talking 1500 Orthodox Jews in East Florida suddenly deciding to get up and vote for Pat Buchanan. What's up with that? Okay. So who here thinks that Gore won the election? The vote in Florida, the popular vote in Florida. I see a lot of hands up. It's hard to ask who thinks Bush won. We would see very few hands go up. Yep. However, they didn't exactly talk to very many voting analysts because we're going to see later on the signal level in that election was a factor of 300 less than the noise level in that election. So given all of this, given the incredible failing, failure, total disaster that was Florida 2000, can we say that it's electoral malfeasance or can we say that it's a flaw inherent in the system? Exactly. There is no incompetency which cannot be turned into an attack and vice versa. Now, first let's talk about the major problem, the biggest problem that we faced in Florida. It wasn't in overmarked ballots, undermarked ballots. It wasn't in the disenfranchisement of people. It wasn't in the Palm Beach recounts. It was time. Elections operate under a very, very tight time frame. At a certain date, by law, those election results must be certified. If they are not, then you have major problems, possibly a constitutional crisis. It took Florida two weeks to count the ballots. That means that they got the ballots counted around mid-November. That was when the first certification came out. That meant that if there were any challenges to take place, they had to be completed, finished, done within the next six weeks, or else. Because in six weeks, the next president was going to be sworn in. So even though it's reasonable to say that, eh, two weeks, nine million ballots, this is okay, it's not okay because that's two weeks that Gore didn't have to present challenges. Speedy counting is essential to the fairness of an election. Paper ballots simply do not scale well to a nation of 300 million people. The internet disagrees. The internet is not run on paper. Canada disagrees. Canada disagrees. Canada does not have 300 million people. Distributed counting is one of the things which does occur. However, for various reasons, the United States is a patchwork of many different legislatures, which some of whom have much better electoral laws than others. So while it is certainly possible to have some legislative effect on this, having a single legislative standard across the entire United States saying you will use distributed counting is impractical at this point in time. I'm sorry, it would solve confusion? Yes. Yes, basically. This is a state issue and the states have broad latitude in how to handle it. So, now let's talk about errors in the counting process itself. 39,000 votes for Gore, presumable votes for Gore, were spoiled by somebody voting twice for president. If somebody has voted a straight Democratic ticket, has voted for Gore, but has also voted for Pat Buchanan, is it more reasonable to think that this person voted for Gore or that this person voted for Buchanan? We can't say. Actually, I think it's a state one. These are only overvotes. I'm not talking about votes tossed out for no reason. Only overvotes. Because this goes to a flaw in paper ballots themselves, a flaw in our current application of paper ballots. Likewise, 15,000 votes for Buch were spoiled in the same way. If we were writing a web page in which we said only choose one item, it would be very, very easy for us to throw logic into that web page that would enforce this only one item. It's the entire idea of a radio button, right? There is no radio button on paper. And the risk of human error in this step is absolutely huge. At least 54,000 people in Florida had their votes discarded, thrown out as being unreliable, as there being no clear intent because of human error. And in my book, that makes it a fairly massive problem. Now, who here has been to a Damien Conway speech? Okay, in that case, I'll skip my Damien Conway impersonation. He usually says, but wait, it gets better. It actually gets much, much worse. Old-fashioned voting technology, even when voters vote perfectly, has an error rate. Studies in Arizona, Virginia, and elsewhere show upwards of a 1% error rate in the counting process. In Arizona, it was shown to be 4%. In Arlington, it was shown to be around that same thing. Some counties in Florida were reporting upwards of an 8% error rate for certain kinds of errors. This is just simply insane. A 537-vote margin of victory versus 90,000 votes, which were simply counted erroneously. We know they were counted erroneously, because that's the error rate of the system. It gets worse. Printing ballots in lots of languages is expensive. It is wasteful. For that reason, it limits how many languages the ballot can be printed in. And thus, people who don't speak the language are implicitly disenfranchised. Yeah, I know a few cabbies who would possibly be counter-examples to that. No, you don't. There is a big difference between the standard of literacy that is required for a citizenship test and functional literacy in English. The literacy tests for citizenship are at a very, very low standard. The literacy test for voting means you need to be able to open up some kind of voter's guide. You have to be able to read technical directions and understand these. These are two completely different thresholds. And for that reason, large districts need to have localized ballots for their minority ethnic communities, which are not fully acclimatized to English. I think that when I was living in San Jose, we had ballots in nine separate languages. As you can imagine, that's an incredible amount of overhead. We need to do something to limit that and also to increase our ability to present the ballot in new languages. I'm sorry, what? Think about Japanese. Japanese uses hiragana and katakana. So that's an example of a language that does not romanize very well. Exactly. I'm sorry, what? Yes, exactly. Exactly. Exactly. We're beginning to talk about political questions, not technical ones, so let's keep going. Accessibility is the next major problem with paper. Blind people have a hard time using mark-sense ballots. Wow, imagine that. You can't see to bubble in the particular spot. Quadriplegics can't use punch card ballots. Maybe if they hold the stylus in their mouth, but that's kind of difficult, one would imagine. And saying that, well, you should be able to ask an election worker for help. Great. So we're now going to ask election workers who are political appointees. We are going to invite them into the ballot, into the booth with someone. This is not necessarily a good idea. It may be better than the alternative of having them completely disenfranchised, but we still should not settle for this. Paper ballots are a technology. They're a technology just like any other technology. Technologies will always, always get supplanted over a long enough time frame. Paper has a lot of drawbacks which we simply cannot tolerate anymore. We can't tolerate the slowness of the count, modulo what someone in the back said about distributed counting systems. We can't tolerate its error rate. We can't tolerate its inaccessibility. We can't tolerate the expense of localizing ballots. That said, we have a lot of experience with paper ballots, so let's not throw the baby out of the bathwater just yet. Paper is non-volatile. It's tamper-resistant, tamper-evident, long-term storage. We have ballots going back to 139 BC. We can still recreate Roman Senate elections from 139 BC because there are museums that still have the ballots. Now, we think that's kind of funny, but think about it from a historian's perspective and think about it from the perspective of historians in the year 4006. What are they going to think about DRE records? They're going to see a CD-ROM and they're going to say, where can I find a machine to read this? We need to preserve as many of these good features of paper as possible. And in many cases, the only way to do this is to go back to paper. This is what I meant when I said earlier that both sides in the debate were wrong. It wasn't that DRE was bad and paper was good. It was that paper was intolerable, but a pure electronic solution is just as intolerable. So let's start talking about the aftermath of Florida 2000. The Help America Vote Act is the most comprehensive overhaul of the voting system in history. In one single swoop, Congress outlawed lever-action machines, most forms of the punch card ballot. It outlawed a lot of stuff that we have relied on for a very, very long time. It also threw money at the states, huge amounts of money to election commissioners who were used to shoestring budgets and said, hi, one of the rules for you taking this money is that you spend some of it on DRE machines. Now election commissioners are not paranoid DEF CON attendees. We are all paranoid here, right? Exactly. I was debating wearing a shirt that said, you can trust me, I don't work for the government, but then I realized I work for the NSF. Yes, I do. So election commissioners are not technically oriented people. So when D-BOLD, ESNS, Sequoia, these other vendors came around and said, we here, you have money, we have solutions, let's talk. Basically a lot of inferior solutions got adopted. These are currently kind of entrenched. We need to figure out some way to minimize the damage. So the fight against DRE, I am very sad to say, is over. It's here. It's not going away. It's been legislatively mandated. And quite frankly, we as geeks are far too small of a constituency for Congress to listen to us all that seriously. Also, nobody likes a griper. On the other hand, if we can constructively engage the process, if we can say, we can make this better and we can do it for free, that people might take us a little bit more seriously. Bad hardware can be surmounted as long as we can detect errors and tampering. This sounds like a really simple statement, but it's as the reporter from WIRED pointed out to me, hand waving. It is hand waving because how do we detect errors and tampering? On some level, we are stuck relying on the DRE vendor to provide an accurate log of events. And that's true. It is absolutely true. On the other hand, we have to do the best we can with what we have. And that means, when possible, subjecting the audit logs to withering scrutiny, extensive statistical analysis, looking for patterns that are out of the normal in order to try to trap incompetency and program. We are unlikely to trap malfeasance this way. Then again, how many of us have seen genuinely malicious code and have been affected by it? And compare that to how many people have had windows crash-offs. Incompetency is the real threat here. Incompetency is the biggest problem with DRE. So, a lot of us think that open source is inherently auditable. It may not be. And because that's such a controversial thing to say, I have to draw a clear delineation between auditability and transparency. Transparency just means, can everybody see what's happening? Auditability means, can everybody prove what's happening? Let's say that you have a complicated program which is completely open sourced. You feed in inputs, you get outputs. The core of that program is so hideously complicated that you would need a Ph.D. in mathematics just to be able to read it. The source code is transparent. You can see what the code is doing. On the other hand, you can't really prove what's doing because you don't know what's happening. Don't believe the hype. Open source will not magically make things better. Open source is a way to get what we want. It is not the end and end of itself. A lot of voting researchers nowadays are suspicious of the open source mantra. A lot of voting researchers think that disclosed source is more important than open source. Do we really need the legal right to modify and redistribute our changes to DRE? Well, that presents huge problems with the certification process because you can't just walk up to an election official and say, hi, here's the latest ISO for the DRE machine, please install it. This software has to go through an extensive process of inspection, certification, all kinds of paperwork in order to make sure that when something goes wrong there's a single throat to strangle. So for that reason, our mantra of modifiable redistributable source is really not that important here. Disclosed source is unarguably necessary. Moreover, the set of qualified election programmers is really, really small. Writing code to handle elections is incredibly difficult. It is intolerant of even the slightest failing. I've been doing this for about two and a half years now at the graduate level, and if I were to ever tell you, hi, here's my new DRE system, I want you all to go running, screaming for the exits, especially if it's written in C. So given this, we should not believe that an open source software is going to produce good election software out of absolutely nothing. That's living in fantasy. Australia already has GPL's DRE software. How many people have ever looked at the source to that? How many people here think they could understand the nuances of Australian election law? Clearly, just saying open source it and let the community develop it is not a sufficient answer. The central goal of auditing is always to ensure the system works correctly. This is true whether you're talking about a code audit or whether you're talking an architecture audit or a tax audit. Auditing means make sure it works right. The most important thing that I am concerned with right now is error detection and recovery. Right now, we are entirely dependent upon DRE vendors to, number one, create an audit log and, number two, inspect that audit log and tell us if there's anything wrong. We can't do anything about number one. What can we do about number two? Now let's talk briefly about open source and why we are not going to see open source solutions any time in the immediately near future. How many people here have heard of open voting? It's a good website, openvoting.org, I think. They have some very good ideas. Unfortunately, they're also very limited in some ways. This right here is the basic architecture. I'm sorry, what? Where is the mic here? Ah, right. As you will notice, I hope you will notice something. I have a master's degree in computer science. This means that I am exceptionally dumb and should not be allowed outside without supervision. Okay. So, the basic architecture here looks fairly nice. It's clean, it's understandable. The voter signs into a terminal and makes selections. Once the voter makes those selections, an electronic version of those selections are sent to a central processing center. A paper ballot is printed off. The voter verifies the ballot in some way and they have already included plans for accessibility here. The voter casts the ballot in the ballot box. The paper ballots are tallied and reconciled with the electronic audit trail. So far, so good. Except, well, we'll get to that in a moment. It is very conceptually simple. It relies on commercial off-the-shelf parts. Clearly, it's an improvement over complex, Byzantine and under-documented DRE systems. The system sanity checks the errors the human is likely to make. If you try to vote twice for president, it'll tell you, dude, you're screwing up. Please fix this before you go on. And the human verifies the correctness of the computer's output. This is good stuff. That said, there are problems. First off, this is not really an end-to-end voter verification scheme. Because once the voter places their ballot in the ballot box, how does the voter know that their vote has been tallied correctly? Now, the defense that a lot of open voting supporters say is that, well, in our current scheme, once you put the ballot in the box, you don't have any further ability to verify it anyway. So where's the problem? But I don't think we should settle for something that isn't an improvement over the existing way of doing things. We have ways to do end-to-end voter verification. I think we should probably adopt them. And secondly, when it talks about reconciling the ballots with the audit trail, now, I may be misunderstanding open voting here. I have to grant them that. On the other hand, this is what David Mertz of Open Voting said at OSCON a couple of weeks ago. That electronic audit trail, or trial, as I think it said in that slide, is just another copy of the exact same information that's on the ballot. So this isn't an audit trail at all. It's just a redundant data path. It is not actually telling you what went on inside that machine. It's not actually telling you what decisions were made and how the computer acted on each decision in the process. So given that those electronic records are redundant, what happens if there's a conflict between the two? What happens if somebody loses a box of paper ballots and thus the count between the two gets out of sync? Well, the answer is you go back to the audit process and you use the audit process to verify the outcome of the election. This is fairly standard. In Florida 2000, whenever you saw a picture of somebody studying a ballot very closely with a magnifying glass, they were auditing it. The audit process is, in most schemes, the recount mechanism. But, wait a minute, so our recount mechanism, which is supposed to be more trusted, is entirely electronic. And our initial vote count is going to be the slow paper route. Something about this doesn't make sense to me. Since the data recorded is identical in format and content and everything else, the only difference is the media to which it's recorded on. Which of the two data paths is canonical? What happens if there's a discrepancy? How can we use the audit trail to establish a definitively correct record of votes? Clearly, there are open questions with open voting. And in defense of the open voting guys, they freely admit that our scheme has problems. They freely admit this. They believe, again going from David Merz's talk at OSCON, that they can overcome these problems and have systems ready to go in 18 months. I think that's probably optimistic. I think it's optimistic because bureaucracy, paperwork, certification, it's slow. So, do not expect open voting to be ready any time in the next couple of years. This only underscores the importance of auditing proprietary DRE systems, of coming up with tools to do that. So, if we're going to talk about audit logs, auditing these to get some kind of results out of it, well, wouldn't it be nice if there was some kind of standard for it? A lot of people think that Oasis Election Markup Language, EML, is the way to go for this. But this is a false promise at best because EML is meant for election reporting. It's not meant for election analysis. The difference is night and day. If you take a look at the XML on the left side there, that is a fairly minimalist EML document. Incidentally, I apologize to anybody who lives in Santa Clara County. I am not implying that all of you are communists. That's just the way the slide came out. In other words, not my fault. So, XML is inherently too verbose for what we want. An election log needs to be small and terse so that a human being can go through it line by line if they need to. If you don't trust the election reporting tools, if you don't trust the audit tools, if you don't trust anything and you want to go back to the actual original records, it has to be in a format people can read. Most interested parties cannot read XML. I can't read XML without a parser. Also, EML only tells us results. It doesn't tell us the decisions that go on at every step. It doesn't tell us if a voter canceled their ballot. It doesn't tell us if the machine ran out of paper. It doesn't tell us any of these things. These are important things for auditing and EML doesn't provide it. So, now that we've established we have to have audit logs, now that we've established we have no good open tools for audit logs, that means we're stuck. We are stuck needing to help de-bold. Trust me, that doesn't stick in anybody's craw bigger than mine. We need to help de-bold because they sure won't help themselves. We need to help de-bold because democracy demands it. We need to help de-bold ESNS Sequoia. I'm not picking on any one vendor here. We need to provide open source tools, transparent and auditable tools to facilitate the auditing of elections. Now, please don't think that I'm dismissing the importance of end-to-end voter verification, voter verified paper audit trails, anything else. These are great ideas. I wish more people would adopt them. Instead of getting them adopted in the next two years is pretty much nil. The likelihood of being able to come up with open source tools to help audit elections in the next two years is pretty good. And in fact, we're already started on that. The National Election Data Archive at USCountVote.org prides itself on being a public repository for election data, including from public DRE machines. That's kind of nice for what we want to do. Props to NIDA. And in fact, the existence of NIDA makes it possible for the University of Iowa to work on NIVA, the Non-Proprietary Electronic Voting Analysis Toolkit. When it is released, it will be released under the terms of the BSD license. It will provide simple human-readable outputs. It'll be written in Java. Incidentally, the reason why we're picking Java is because the Java virtual machine is certified by many governments as being suitable for high reliability operations. I'd much prefer to do it in Python. And ultimately, it'll be turned into a web application so that you don't have to do anything yourself, except get the electronic format of the audit logs, upload them to a NIVA web server, and you will get back a nicely formatted document outlining what happened in that election, what things were statistically improbable, what parts of election protocol were not followed, what precincts need additional training, things like that. That said, thank you, this is not going to be easy. There are massive, massive legal challenges to be overcome. Some states under heavy influence preventers are outlying just about any independent analysis of DRE. Is anyone here from Florida? No? Oh, great. Talk to your legislators. I'm sorry, what? Oh, the irony. The irony. I am going to be telling people about this story for years. So what we have are some legislators like Florida are attempting to ban independent analysis of electronic voting machines. I'm sorry, what? That is most likely true. I have not personally experienced that, but that is most likely true. Florida is trying to get a vendor created, quote, independent, certifying authority group and only allow electronic voting experts who are members of that group to inspect machines. Makes a lot of sense to me. Lawsuits are also a great way to stifle development. The good news is that as an academic organization, we have a lot of First Amendment protections, and we are also, strangely, get protection from the Digital Millennium Copyright Act, which, depending on how you read it, has a fairly large safe harbor provision for, quote, bona fide security researchers. You don't get much more bona fide than NSF funding. So the technical woes are much easier to overcome. The formats are proprietary. They are under-documented, under-specified, under everything else, but they're usually plain text, and that's very good news for us. That means that we can ultimately attack this with nothing more complicated than a context-free grammar. The biggest problem is that no two companies use identical formats or track the same data, and that means that we are, at this point, stuck writing custom applications for each particular voting machine we want to work for. We are going to be trying to create a more general framework later on, but at this point, we don't know how to do it. The best stuff that we have is a parse of ESS, iVotronic proprietary logs. We have been allowed to use results from Webb County, Texas, primary elections for purposes of our research. And using only the audit data, we are able to completely recreate the entire voting history, right down to vote counts, right down to tallies. It serves as a very useful second source for validating the ESS-supplied closed-source auditing software. And we also picked up several flaws in the election protocol. Let's take a look here in just a moment. In Pre-Sync 4, which is what we're looking at right now, you can see a tally at the top on down to individual DRE machines, how many ballots they were issued, how many ballots they actually cast, additional hardware that was used for them. And most interestingly, let's take a look here at Machine 5145873. It was opened at 7.20 in the morning, which is a little bit unusual, because the Webb County polling places were supposed to open, I think, at 8. Notable, not unusual. It got zeroed, all the votes were wiped from it, shortly before, shortly after it opened. Then it was zeroed again, and again, and again, and again, on throughout the day. Who wants to bet that at some point along the line, well, I shouldn't say that, I'm supposed to be a non-partisan, non-political election analyst? I'll just say that, you know, zeroing a machine and wiping out the votes on it just after noon is probably not something that's in the election protocol, right? The people who gave us the Webb County primary results are people who are currently engaged in litigation. They came to us because the audits, I'm sorry, they came to us because whatever software they were using to recreate the audit logs could not give them the same vote count twice. Yeah! So they came to us and said, could you please help us? And we said, yes, sure, if you'll let us use the data for research. And they said yes, and we found this stuff. Now, you might wonder why we are doing it in such a simple HTML format. And the answer is simple, because we can print it out. When historians from the year 4,006 opened up a time capsule and see a big sheaf of acid-free paper, then they are going to know right off the bat, hi, this is old and we can read it. Thank you. So, in 200 years, nobody's going to have a copy of PiGTK or Java 1.5 or anything else. We have to support a paper audit log. We have to optimize our tools for paper output. Now, that said, it's HTML. More accurately, it's XML, the style sheet added. So while it is possible for people to have any kind of data backend that we want on this, our focus is still going to be paper outputs. If you want to help with NEVA, talk to your local commissioner of elections or whatever the name is of the person in your jurisdiction who is responsible for these things. See if you can get audit logs. In some places, like California, it's really, really hard. In other places, like Florida, it's really, really hard. In other states, like Iowa, they're public record. So, email the logs along with your location and the summary system used to my email address right there. Also, please report them to NEVA because NEVA would certainly love to have them. Right now, NEVA is barely in a 0.1 state. It only works well for ESS iVotronic systems. And for that reason, we haven't prepared a tarball. It is not ready for prime time yet. That said, we are committed to keeping the source open. So if you want to inspect it, look at the subversion repository right there. That brings us to the end of the talk and I have a couple of minutes, not much more for questions. Right, right. This is something that is hotly contested on the legal front right now. Depending on how you read open records law, audit logs should be open. Depending on how you read trade secret law, they shouldn't be open. So this is something that's getting hammered out in the courts right now. California is debating this. So was recently debating this, et cetera, et cetera. So I'm sorry I can't answer your question better, but that's fundamentally a legal question. And since we have someone from the EFF in the audience, I will just politely request that people not throw any more legal questions at me. I am not a lawyer. I do not even play one on television. If you take legal advice from me, you're on crack. We have a lawyer. Ask her. Well, it's either that or else it was the fact that my hotel room was directly underneath the party last night. Either way, I'm out of my gourd today. Any other questions before they kick me off? Hi. It seems to me that we're focusing on the wrong issue in the sense that what really needs to be secured is the local counts. And I would propose a system involving index cards and fourth graders. And then everybody would know exactly what happened. You'd have your paper record. You'd have your local total. Those totals could be published. Then anybody anywhere could use their own complicated software, complicated systems to compute the results of the total election. I haven't seen anybody talking about, you know, why we're spending billions of dollars on a technical solution that could be solved with index cards and fourth graders. The answer to that one is it's in the audit logs. The data that you want is in the audit logs. If the audit logs were to be made public and accessible, then exactly what you're talking about could happen. But why? Why do you spend all that money, all that time on... You're asking a political question, and I'm supposed to be non-political. Sorry, I can't answer that question. But ultimately, you need a system that any person can go and look at it and say, I understand how this works. Absolutely. You'll never be able to do that with a computer solution, ever. It is unlikely. That's correct. Anyone else before they kick me off? Okay. They're kicking me off. Thank you very much, guys.