 Hi, my name is John Odom and I'm currently the city clerk in Montpelier, Vermont in that capacity I'm also the election administrator and Why is this then a year for that? We've got our our own statewide primary coming up in about a week and a half So even taking a little time to record this is kind of challenging I'm also Certified ethical hacker. I've You know been in charge of a couple networks and been a database administrator for various nonprofits so I tend to have a little more hands-on knowledge than most of my colleagues in the industry, but Not as much as you might think because my life's a little busy and I don't get to get to practice this stuff very much Anyways, I want to bring something to Folks who participate in the voting village's attention and it's not something that has been ignored But I think it's something that deserves a little more attention To make the point about how concerned I am I'm going to tell you a little story And this is not exactly on topic, but I want to use it to make a point Several years back in Memphis and some of you may have heard this story There was a local election mayor was being elected, you know various local officials officials and one clever person decided to compare the Turnout from the tape and most tabulators generate Some sort of tape some sort of physical record of the time listing the votes compare that number With what was reported through the GEM system and I know also a lot of folks who've been participating participating in voting village share some concerns about that system Well, the numbers did not add up according to the tape 546 people in this particular precinct had voted and Oh, I'm according to the tape and the system only showed 330 Okay, you know they looked at this they looked at other precincts. They found the same problem especially in In districts with heavy minority populations Big problem, it's the problem we talk about at Defconn, but I want to present another scenario to you Consider this happening on a statewide level Not district by district not precinct by precinct not through any one individual network of voting machines, but an entire state There are statewide systems that manage databases. This is mandated by the Help America vote act Which came in the the wake of the 2000 presidential election voting debacle, you know the hanging chads the Numbers that didn't add up and it all went to the courts and it went to the Supreme Court After that there was a bipartisan Group that got together Federally mandated and came up with several recommendations. Now generally, they were very good recommendations I don't want to disrespect Hava at all one of the recommendations understandably was that states should Be working on their voter rolls through one centralized system One centralized statewide database that would contain all the voter registration information But obviously you can see where I'm going with this. There are concerns about or we have to be concerned About the security of these systems. So these statewide systems Don't all necessarily just hold the voter registration information A lot of them including in my home state of Vermont are actually election management systems Election administrators will report their information for for election might reporting Sometimes we will work directly into these systems to manage it to create our reports to create our elections and Local elections and manage them directly out of that. So there's a lot going on with these systems They are very important and local administrators have come to really depend on them Well, how are they doing? Could be better. I want to talk about the the famous Smoller report Now tucked away in that is this little gem that I'm going to read here part of it It says in addition to targeting individuals involved in the Clinton campaign Mueller's operation also targeted individuals and entities involved in the administration of the elections Victims included us state and local entities such as state boards of elections Secretaries of state and county governments as well as individuals that work for the entities They also targeted private Technology firms responsible for manufacturing and administering election related software and hardware such as voter registration software and electronic polling stations Now here's the scary part the report says that they targeted that foreign actors Targeted state and local databases of registered voters using a technique known as sequel injection By which malicious code was sent to the state or local website in order to run commands in one instance in Approximately June of 2016 The working group was able to compromise the computer network of the Illinois State Board of Elections by exploiting a vulnerability Presumably a sequel injection related vulnerability this gave them access to a database containing information on millions of registered Illinois voters They the group extracted data related to thousands of U.S. voters before the malicious activity was identified all right So beyond just the obvious scary here You know, okay a lot of you folks will know that code injection is a very big deal code injection is You know, it's where most of the hacks come from these days, but sequel Injection is Something we have been aware of for many years. We know how to harden against it so my question is Why wasn't it already hardened against it? Sequel injection is very easy one line fed through to a database from a simple login screen can get you in We know input validation is the solution. So where the hell was the input validation? What that says to me is and you Again to refer to voting village in the last few years What's gotten a lot of attention was the simulation of the statewide voter databases got national coverage that kids could sit down at this Simulation and they could hack right into our dummy statewide voter registration system well, of course the pushback and all the Yelling and hollering from the secretaries of state where that this was a phony simulation That their systems are actually far more secure somehow. This was set up to be hacked to make their point well If sequel injection is a way to get into this stuff I would argue that those systems those those dummy systems we made up maybe were not Loosen accessible enough So All right, so let's talk about Habba to help America vote act And I'm gonna read a little bit from it Part of the mandate is each state acting through the chief state election official shall implement in a uniform and nondescriptory manner a single uniform official centralized interactive computerized Statewide voter registration list defined maintained and administered at the state level So states have no choice But to do this and again the population is what they are It's completely understandable So what do states do? Well, several do in-house development In Vermont we used to our first database our first election management system out of the box out of Hava was an internet-facing Fox pro application It was precrued But these days and and other states currently are still doing in-house solutions states like Colorado Illinois Kentucky here you could put together something pretty nice if you knew what you were doing Obviously then the networks they sit on the municipal networks are potentially vulnerable too, but if you can get in by sequel injection why bother Okay, but more and more often you see these states using vendors and these are vendors, you know, it's a like any niche Application any niche market you're gonna get niche vendors who pop up specifically to serve that market So Before this we had somebody call around. I wanted to know what states used what How many of them used one vendor versus another how many of them were designing in-house? I don't know if I can really say this but word got out that we were calling around and there were high people who were less than thrilled and Defensiveness in public systems Should really make us all uncomfortable now I want to look at one particular vendor right now because I have a little more first-hand familiarity with it Called PCC Now here's a list from their website of their Current clients Okay, now factoring out for the consulting only options here That's 15 states doing database management and in election application hosting for voter registration election night reporting So who is PCC? Well, this is part of the problem. Who knows Go to PCC's website and no staff is listed Only board officers not even the whole board Now I didn't do research on people who did show up. I probably should have I could have but I didn't I don't want to knock them without any basis or unnecessarily then and You know, they seem to be perfectly reputable people. I didn't see any the one I did look at I didn't see any obvious big political connections CEO Tom Amber he is a big shit was a big shot at Central Square technology For example, and they were recently Had a major hack a major card attack Now, you know, I don't want to beat them up too much for a mage card attack. They're good but mage card attacks generally use JavaScript injections, so it's something that could be hardened for and With election systems, there's just no margin for error. You've got to be ahead of these games So I wouldn't say that was a big concern of mine, but it does raise my eyebrow a little bit But let's look at some of these I proposals that PCC organization organizations like PCC have put forward to the states to try to get their business I want to show you the one from Delaware and Just pieces of it. It's of course very very long, but I'm gonna show you here a typical page from the publicly posted proposal From PCC to run their election systems Okay, when you look at this, this is what you'll see Not much. I You know, I didn't do the measurement and everything but from my glance I would say at the most about 20% of this entire proposal is visible. So that's me being generous Now of course I understand companies have proprietary information. They have proprietary stuff that's standard for any RFP You expect that But come on Why even posted at all? I mean, this is something made available to the public You know public records, but it's not I mean, it's it's it's almost a joke if I didn't know better I'd say it was almost passive aggressive But what bothers me the most in terms of the redactions is all the staff is redacted and this is typical So just like the website, we don't know who's working on this and that bothers me a lot Because people have their own interests. They come from backgrounds partisan backgrounds Non-profit backgrounds. I I think it's reasonable and I don't think it necessarily, you know Reveals any particular corporate secrets that we could have some idea who the people are doing this stuff either in an Individual state or even at the company proper now Financials are also redacted. I know that's a thing. That's very standard, but I would argue that it shouldn't be I think our right to know Trump's any embarrassment or discomfort of big companies and publicly we might not want one that's on the edge of bankruptcy and We might want to see that so so here is the problem. There are very few companies doing this and They are opaque. We don't know who they are I mean, that's basically it. We don't know who they are and that's scary. That's very very scary at least to me and it does matter in Georgia they had a a Recent problem at debacle involving their voter roles involving the voter registration Now this wasn't exactly what I'm talking about but it makes the point about how Badly you can screw up an election simply by screwing up voter roles You can disenfranchise people in a big election crush They're just not going to get to vote or they're going to have to fill out an enormous amount of provisional ballots, which Honestly might not necessarily get counted the way they should be Now during a court case involving this whole debacle A lot of Security is a lot of vulnerabilities in the PCC system Was was brought to the attention of the court After that George has decided to pull back their contract ran out and they decided to pull it back in house So that's a pretty unusual step to take and it it shows you just how concerning those vulnerabilities were Now You know, we've got our public officials doing this Can we count on our public officials to be straight with us about this stuff if there's a problem? well Obviously not and again, I don't mean to knock secretaries of states, but they have their own interest They have an interest in getting reelected and that means they have interests in looking competent Now some of them like to talk about internal security a lot You know, we've upgraded this to make it better We've got better voting machines to make this better. We're we're doing our due diligence within the sphere that we control In order to, you know, do a better job and get reelected They don't like to talk about the potential for things outside Right that and that means they don't necessarily like to talk about the walls that they've built that they're responsible for that are You know the dividing the the firewall pardon the expression between the voters and the outside world where you could have malicious actors And again with the Mueller report, we're talking about advanced persistent threats. We're talking about state actors, but not necessarily So secretaries of state have a vested interest in saying everything's rosy and everything's wonderful. So that that is a problem So let's look at some of the other systems. I've been picking on PCC, but there are other systems out there And folks might recognize voting village might recognize one of the other major probably the other major election management system that's out there It's from ESNS ESNS or our old friends for years voting village has been hacking their machines and they more than any of their companies have been the most belligerent Last year, I believe it was they actually had folks sort of roaming about trying to make people uncomfortable about hacking suggesting they shouldn't These were the last folks to come around and say the voting village had a point and it was only after so much coverage again I think it was last year. These folks are not good partners. They are not reliable partners in our experience I don't want to get sued for slander here, but in my opinion based on what we've seen these are not good partners And we will remember how quickly the National Association of Secretaries of State last year was right there to defend them on their own terms On ESNS's terms and in some language that looked a little bit much like their own words sometimes So scary scary stuff and are these things we can test in voting village the way we take apart the voting machines? Of course not. Of course not. We can make our own dummy systems like we have and we should and there's a lot to be made there But we can't go and test these systems. It's hacking. We can't hack, right? So I've thrown a bunch of terrible stuff at you here The question is what do we need? Do I have any solutions here? Well, first of all transparency, transparency, transparency, transparency. We should know who these people are. We should know who runs them, what their interests are We should know what their background are. We should know, we should need to know whether these folks even have the competence to do what they claim to do And I'm going to talk a little bit about the Iowa debacle where during the caucuses for the primary caucus this year the Democratic Party was using these little specially designed custom designed apps, mobile apps to report the results of the caucuses to a central place You all probably heard about this. They were a disaster. They were a disaster. It took a very long time to sort of rebuild the mess that they created and actually generate a final voting tally It was a big embarrassment to the Democratic Party. They were going to be using the same systems in Nevada and then they pulled that out Now what I would say the biggest problem conceptually with that application was that it was made by, and you see this a lot, made by folks in the industry who made a lot of personal connections You know these were folks who had worked for the Democratic Party and IT stuff and they decided to go out on their own and they made a crappy product But the crappy product was bought up because oh we know these people. These people are in our industry. They're in our world. We know them. We trust them So I don't know if the same thing isn't going on with some of these voting applications. PCC, who knows who ESNS hired. That is a real real problem in this industry So we need to know who's doing it because we need to know if they're competent, if they're reaching out for the best people or if they're just reaching out for people who are connected with the company Which gets us back to another point. This needs to be opened up. If we've got one or two or three companies doing this, that gives those one or two or three companies a lot of power We need to have more genuine RFPs get this out there and I'm not saying there are angelic companies out there but mixing it up a little would help Now second, we've got to test this stuff. I'm sure the secretaries of state do pen tests on their systems but it sure didn't help for 2016. I don't know how a pen test misses a sequel injection attack There are pen tests and there are pen tests and again the secretaries of state have a vested interest in not finding vulnerabilities I've seen some of this. The pen tests tend to be minimal. So we talk about standards for voting machines. We should have standards for these systems and reports to be publicly available. We need standards for penetration testing And that includes testing for social engineering all the way down to the user level, all the way down to the level of the election administrator who has their own account and talks to this database system A thorough pen test is going to include tests for social engineering and you don't generally see that. Especially when you consider any kind of code injection notwithstanding the biggest problem will always be malware You can't really audit these systems. So we need those standards and again transparency Yeah, you have to redact most of the pen tests. Sure, and that just makes sense. But we could at least be able to see the executive summary of these pen tests as a public document So I would argue standards. I would argue again transparency. Those are the big two words. Standards, transparency. That gets us a long way Now I cannot stress how much of a problem this is. If we're talking about voting machines and systems, which we love to talk about and we need to talk about, a presidential election on those terms would be hard to tank just by going after voting machines But not these systems. There may be thousands. I think 7000 is what I read. Localized voting machine tabulation systems You know, that's a lot to get into and hack, although not for a local election and local elections are every bit as important as national elections. It's all democracy Although again, not impossible. But when you're talking about these online databases and these internet facing user election management systems, we go from 7000 targets down to 50 50 targets. That's a lot more appealing. It's a lot more dangerous. And these systems interact. Okay. First of all, most states you're going to see some kind of connection or some kind of ties to the DMVs to the Department of Motor Vehicles, you know, to, you know, we always have to check off for do I want to be registered to vote to There are there are states like Vermont. I'm proud to say that have automatic voter registration. So those systems have to talk to each other some ways. There are safe ways to do it. There are unsafe ways to do it. We don't know, again, how they're doing it. Tax departments. Some states even connected their tax departments. So obviously any big extensive network to network that they're going to be as strong as their weakest links. And those weakest links could be in the statewide networks talking to each other. The weakest links more often than not are the users. You know, in New England, you can have election administrators wanting running jurisdictions of as few as 70 people. They don't have a lot of good equipment and they don't necessarily have a lot of sophistication in how to, you know, do proper hygiene guard against against social engineering against spearfishing. I mean, if I were somebody I'd take aim at one of those folks and I'd go right after them. There's also Eric, which is, which is something I don't know about Eric is a statewide system, which now covers about half of the states. I think it's roughly 25 with more looking at it, whereby states interact their databases so they can track cross state registrations. As it is, you know, it's it's been a challenge for those states to track to take someone off a roll in one state because they registered in another state. That's a challenge. It's a weakness that a lot of folks have made a lot of a lot of fuss about and honestly, they probably should that stuff has been, you know, a matter of sending a piece of paper or an email from one secretary of state's office to the other. So obviously, you're going to have systems like Eric popping up. Now, my understanding, which is limited is that in Eric, you're not having a situation where the statewide databases are talking directly to each other, which is great. But obviously, again, there is the malware issue. Malware can ride along with all kinds of things. And at any given time, a third or more of the malware out there in the wild could be zero day. So, you know, it's the same problem we have with antivirus systems. They can only be so up to date. They can only be so current. So anyway, I'm not going to say I hope I didn't scare you. I hope I did. That's why I'm here. I think that's why a lot of us are here. Our two to scare people into action and to scare people into making things better. Again, I'm not trying to knock anybody down. I'm just trying to draw a lot of attention to this problem. And this is a problem we could go into a lot more technical detail on and, you know, I could talk for, you know, an hour or two if I wanted to. But I want to keep this accessible. I know for the election administrators who were watching this and also I only have about 20 or 30 minutes to do it. But it's a conversation I think we need to have a lot more of. Thanks. Thanks very much to listening. Thanks for voting Village for having me. And hopefully I'll see you next year, maybe even in person.