 All right, well, thank you for everyone for joining us. In this particular bit, I'm going to be introducing myself very quickly and then our panelists. So I probably should have done this at the beginning of the day. But I'm Maggie McAlpine. I've been working in election security now for about nine years. As an auditing specialist, I've worked on a few things like the Estonia eVoting report with the University of Michigan. I worked as an advisor for the Secretary of State of California. There was a limiting audit program in 2011. And these days, I work with advising states on their cybersecurity initiatives and generally just helping them out. So this panel is how journalists can, I should know this because it's my panel, but how journalists basically can help with cybersecurity. And we are joined by three amazing panelists. And I'm going to introduce them quickly. And then I think we're just going to jump into discussion rather than doing like an opening remarks thing. So first in the middle, we have Kevin Collier, reporter with CNN. He covers the intersection of cybersecurity and national security, including efforts to safeguard election integrity. He has previously worked with BuzzFeed News, Vocadov, and The Daily Dot. On the end, we have Kim Zeta. We're very honored to have her. She's been just institutional knowledge, you wouldn't believe. She's a longtime cybersecurity and national security reporter for various publications, including Wired Politico and The New York Times Magazine, and is the author of the book Countdown to Zero Day, Stuxnet, and the launch of the world's first digital weapon. She has broken numerous national stories over the years about NSA surveillance, digital warfare, WikiLeaks, and the hacker underground, and has been one of the nation's leading journalists covering voting machine and election security since 2003. And closest to me is Eric Geller, the cybersecurity reporter for Politico. Eric Geller is a journalist on Politico's cybersecurity team. His primary beats consist of cyber policymaking at the White House, the Justice Department, the State Department, the Commerce Department, but also he regularly covers election security, data breaches, malware outbreaks, and other cyber issues affecting the government, private sector, and society at large. And wanted me to mention, Politico now has an election security tracker that is just implemented. So just to dive right in, I think the first question we were gonna ask, well actually, yeah, let me pass this over so it's just, wait, okay, no, I'm not gonna do that. Yeah, I gotta ask the question first. So I'm used to having more microphones when I do panels, but so one of the first questions I thought, just to kind of jump right in, and I thought might be relevant to this crowd, is what should hackers do if they discover vulnerability in the election world? How can they reach out to journalists and what can they expect in the process? I'll start. Well, I mean, so a lot of work has been done on voting machines already, and Maggie participated in some of the most famous reports back in 2007. And you should look at those online and see what already has been discovered. So if you're doing work, you don't wanna repeat some of the work that's already been done, but there's still a lot of work to do and there are new voting machines coming out. There are a lot of election security experts who've been doing this for a decade, including Maggie and her husband. Sorry, is he husband? Sorry, sorry. Harry Hurstie. The people running the conference map blaze, I would suggest if you find something, reach out to those people, get context for what you found or what you think you found, they can help you understand it. And the benefit of that, the bonus of that is these people also have expertise in how elections are run, the process. And so if you find something, they can tell you, well, there are mitigations for this or they'll tell you the process that is used in elections to actually check this. And that helps you when and if you decide to go to a voting machine company and disclose this, there's, you know, they have just recently launched bug bounty programs, that's just gonna be interesting to watch to see how these actually work, if they work the way we expect them to work in the rest of the industry. And you're gonna wanna be armed with information because if they try and tell you that something is not a bug, you're actually going to wanna know, if they tell you, well, election process will catch this. It would be good for you to have that knowledge. If nothing more than have Matt Blaze or someone else give you that knowledge, so you know what it is that you have and what can be mitigated and what can't be mitigated. And then come to a journalist or even come to the journalist first and I can help broker that with you, with Matt or someone else. But I'm always interested in hearing about election security issues, vulnerabilities, anything like that. But it does help if you, and save time for everyone, if you have a good idea already, if something has already been covered or something's already been found. There's not much I can add to that, except I would strongly recommend going to a journalist who you do trust, who any of us works or someone who covers this rather than an institution itself. And then also, as a source, you do have a bit of power. You can say, look, I wanna bring this to you, but can we talk off the record? Don't quote me right now. Let me talk through this idea. Let's have a conversation that I'm not, I don't have to fear that I'm gonna be taken out of context or something. You're allowed to set those terms with a reporter, and it's probably smart to do so. We just also add that most of us are not technical people. So understand that you're gonna have to do a lot more explaining than you're gonna have to do if you're talking to Matt Blaise, if you're talking to one of your fellow researchers. We like to think that we osmotically pick up some of this technical knowledge along the way, but for the most part, we need things explained to us. And that also helps us explain it to our readers who, as you can imagine, 99% of them are gonna need everything explained to them. Even, you know, I write mostly for sort of a government, you know, Capitol Hill industry audience, but even those people are very typically high and mid-level, not the people at the ground level working on the technical details. They're the decision makers, and they need things explained to them too, just like the general public. So talk to us as if we are, you know, what we are, which is a conduit to the people who need to see your work, so. Well, I gotta thank you for that, because you lead me perfectly into my next question, which was, how to convey technical topics to non-technical audiences and do them justice? I mean, I think you have to start at, you know, you have to abstract out from the vulnerability that is presented to you that the researcher comes to you and says, I found a way to do this, and it's not supposed to be done. It's not supposed to work this way. As a journalist, you have to go to them and then say, why does that matter? What's the end result of this? What does this look like in this case for the voter in the polling booth or for the election workers sitting in front of the EMS? What does this look like for the end user of this product? Because ultimately, people who are reading my stories about vulnerabilities don't really care that, you know, this is a problem with the system and everybody said it was secure, but it's not secure, and you're not supposed to be able to do this, but you can. What they care about is, what does that mean in the real world? People in this room are probably very interested in the vulnerability for the vulnerability's sake, in addition to the outcome, my readers only care about the outcome, and so it's helpful for me to be able to say, you know, so-and-so found a thing that could let people do X, Y, and Z. That's typically how we describe it, is what is the outcome of this vulnerability? And, you know, as somebody who knows more than most of my readers and constantly has to scale back what I'm presenting so that they're not overwhelmed, I would imagine that for researchers out there, you also have to scale back what you're telling the journalists because we probably don't need to know 50% of the sort of things that you find interesting about this bug or about this problem. What we need to know is the things that we can explain to people so they get the implications, and so, you know, do a little bit of that scaling back, make sure that you're explaining it to us in a way where even a really stupid person, and none of us here are stupid people, but it needs to be something that a stupid person could understand, or someone, let's say, with average intelligence about cybersecurity, right? Those are the people ultimately making decisions that your research can affect, so make sure they can understand it. So the difference is context, right? So a security researcher, oh, sorry, I forgot that you're not mic'd. So the gentleman asked, how is it different from the researcher writing the article instead of the journalist? Which is a fair question. And the difference is that the journalist brings in the context of how does this vulnerability fit into the political or administrative ecosystem that we're talking about? So the journalist is out there talking to the election officials, whether about this specific issue or about other issues. The journalist is talking to people at DHS and the FBI to understand how this fits into law enforcement and defensive activities, critical infrastructure protection. The reader does not just need to know, you know, here is the thing that might appear on, you know, the CVE page in the system that tracks vulnerabilities. They can go there if they're smart enough to wanna see all those details. Most readers wanna understand why this matters to them and how this fits into the things that they're doing on a daily basis. And journalists are sort of the translator of that for a real world audience, for people who are not, frankly, in the DEF CON bubble. If I can add, if you have a vulnerability, you have an issue you wanna bring up, sorry. I wanna add that if you have a vulnerability, you have an issue like this that you do bring up to a reporter and you wanna say, write your own technical paper aside that, I mean, you absolutely should. I mean, I think, sorry. It's how do we as reporters convey concerns better than someone who, for example, has a law degree and is an ethical hacker. I think, I don't know, I kinda wanna echo Eric here. It's context, it doesn't mean that we do have better perspective, necessarily. It's that we have a, you know, we work in an institution that has a mass audience that we work as translators, basically, from you guys to the public. That doesn't mean you're, you know, I don't think that has to stand in opposition to your role, but yeah. I mean, it sounds like a question just in partisanship and how to mitigate that. No, all right. How do we, can we table that? Can you think of a way to shorten that one and we'll table it for the actual Q and A part and we'll do that in about 10 minutes, sorry. So let's just go on to the next question here. Thank you. So actually, but I can maybe use it a little bit as a jumping off. So elections are an interesting and somewhat unique problem in that even just sort of, and this was the argument for a long time, even discussing the possibility that an election could be hacked was not discussed for a while or at least was criticized when it was because simply casting the doubt on an election is a form of attack in and of itself. So like, how do you balance reporting on vulnerabilities, not blowing them out of proportion, keeping thing, but also that level of responsibility around alarmism too? So I write for both mainstream media and I write for tech publications like Motherboard. And so the, I mean, when it merits it, I will take the story to Motherboard for a lot of technical details. And I know who my audience is, right? I know the audience wants those details and they understand them and it's not gonna get taken out of proportion. But then it gets picked up by other media and they will take the top level of it and kind of run with it. So there's only so much that you can control, you try and have as balanced of a story as you can to get that first story out. And so that's established. And then if it gets picked up and sensationalized or errors get put in or whatever, you still have that initial factual story there and people can come back to it. People complain a lot about headlines and the story can be very balanced and not sensational and then a sensational headline gets put on it. And reporters don't like that because it undermines all the hard work that you've done and all people do is focus on the headline. So I'm just putting out that as sort of context for you that even if a story seems sensational from the headline take some time to actually dig into it because it may not actually be that sensational. And we do try and push back on headlines that we don't think properly represent the either the content or the story or the level of what that vulnerability represents but we're not always successful because publications don't wanna put time into a story if you're not gonna read it. And sometimes you have to create a headline that is actually going to make people read it while still being a truthful headline and not sensational. So my anecdote about this is when I was here last year I was writing for Buzzfeed, angle on it and I said, I wanna do how people like Matt and Hari are who've worked on this issue for 15 years or God knows how long and now all of a sudden they have a microphone in a way they had not in previous years and how does that change how you raise the alarm about issues in election systems? And so I would like to think I mean you can Google it and find the story I would like to think I presented a fairly nuanced view about that but it was a longish story and so my editor said I'd like it if we have some sort of anecdotal lead and I mentioned that well we could throw a line in there about the kids that were playing on a version of a Secretary of State website and so the headline became a 10 year old just hacked the Secretary of State's website discuss and it was by far the most widely shared story of my career like millions of people and that was great and how many just shared that headline and took away the wrong idea how many read the story and took away what I would like to think is the right idea I don't know but that is I think it's how we share information and God I know it's not perfect but I don't know of a better one. So jumping now on to maybe and by the way I love anecdotes if you have any stories you also want to do you've also done some great journalism reporting lately so if that finds a way into the discussion I certainly won't mind. So one of the questions we had discussed was how do you, there's sort of a delicate way and an indelicate way of saying it so the delicate way is how do you juggle competing claims by various aspects of this election officials or security researchers but also there is a less politic one what do you do if somebody like just is completely random example I promise a vendor were to simply lie to you about what's the truth versus what they say in there you know PR is happening. So I'm sure Kim has stories but I just very very briefly wanted to say I have a great example of this which is as mentioned we did just launch a tracker that looks at every county in the country that uses paperless voting machines and we tell you where they are in the process of replacing them and we also looked at the states that are doing this at the state level almost all those states are trying to get to some form of paper I know that ballot marking devices are controversial so we'll just leave that aside but the one state that said it was not replacing the paperless voting machines that it does have is Oklahoma which has gosh I don't remember it's the eScan AT there for voters with disabilities most people do vote on paper but I contacted Oklahoma and I said we're gonna put this on the page we're mentioning that you use these paperless machines and the spokeswoman for the election board said they're not paperless they do have paper I went to gentlemen who used to work at heart who actually see the audience here and I went to someone at the great group verified voting and I said is this true? They said no it's not true these devices do not produce individual paper vote records went back to the secretaries the election board spokeswoman I said here's the information from the experts and she never responded to this day they have not addressed this point about the eScan AT but what she did say before was at the end of the voting session each machine prints out a piece of paper that shows all the votes cast on that machine and that was her view of a paper vote record to her that meant that this machine wasn't paperless and so you do have this problem and I saw this at counties too where there just isn't this basic cyber literacy and I sort of see it as analogous to climate reporting in the sense that there are people who have spent their careers doing peer reviewed research have produced falsifiable claims that could be falsified if they were not true and nobody came out and demonstrated that they were false and so that is sort of the emerging consensus in the expert community of what's true and what's false and when you have somebody else who doesn't spend their life doing this who comes in and says the experts are wrong it's not my place to say that person is wrong or is lying but what I do do is I make sure that I emphasize here are the people with the expertise here's the consensus we probably use the word consensus more than almost any other word as a euphemism for the people who typically know what they're talking about and that's because for various reasons it's not our place to say liar, idiot I'm not appointing an idiot here but that's not our job but we do have to just sort of convey the people who are typically the experts the people who are typically not the experts and that's my contribution to that I let my sources say that the vendors are lying so I have been lied to multiple times the first time that I was lied to was all the way back in 2004 covering elections in California the voting this was the when we were still paperless machines and trying to get paper California was the first and there were some disability groups lobbying for the vendors and I asked one of the disability group guys have any of the vendors giving you any money and he says no absolutely not and then six months later the New York Times writes that they got money from one of the voting machine vendors so I did write that in my story that they told me this and then six months later the New York Times write that that's what I do election systems and software wrote a article for the New York Times about modems and remote access there were some people in Pennsylvania who found who were asked to look at the machines by a Pennsylvania Board of Elections and they discovered that the machine had PC anywhere on it this is the backend election management system that programs the votes and ballots before elections tabulates the votes afterward and that PC anywhere was being used by well they saw a log that showed that it was being used for several hours the night before the election and that was the voting machine vendor or whoever a contractor doing troubleshooting so I went back to election systems and software and I asked them and I started looking through contracts and the contracts actually say particularly one in Michigan actually talks about installing PC anywhere they say we might sometimes need to do technical support and we can't send someone to the field and so we'll want to dial in so we want you to install this PC anywhere okay it's written in the contract and I call up ESNS and I asked them have you ever installed remote access software on your voting machines and they said no never we've never sold or installed on your voting machines that story came out in February 2018 Senator Wyden sees the story and sends a letter to try and get them on the record about this I mean it's on the record in the story but to try and see what they'll say to a lawmaker the answer then as they come back and they say well we installed remote access software on a few systems between the years 2000 and 2006 and that came out and I wrote a story on that I think in March, April so we're now two months later and then NPR speaks with ESNS in September and they asked the same question have you ever installed remote access software on your systems and they tell them they installed it in 300 districts, jurisdictions so we've gone from no absolutely never what are you talking about how can you even suggest that to a few systems but it didn't really matter it was long time ago in 2000, 2006 to 300 jurisdictions okay so that's what we're dealing with and multiply that by the information that they give to vendors if they're lying to journalists they're lying to vendors and that's the subject of a story that I published this week where vendors have been telling the public for years they've been telling their election customers that these voting machines and those back end election management systems are not never have been never will be connected to the internet in fact they are and in fact it is in the request for proposal documents that they give states the problem is that states look at it and they don't know how to interpret it they don't see there's a firewall there there's even actually in a document that ESNS gave to Rhode Island it actually shows the modem transmitting votes from the voting machine on election night these are optical scan machines shows wireless modem nice little cloud and then it says internet and then a firewall over here so I don't understand the disconnects how that can be in a document and yet they tell the public and they tell election officials they're not connected to the internet and then the story that we found today was not only are they connected to the internet because they'll say they're only connected for a few minutes they're only connected over cellular modems only cellular network the story that I wrote last year was actually cellular networks are internet but they'll say it's only a couple of minutes so it doesn't matter the story this week was no they've been connected for years for months and years they've been on so it's a problem and sometimes the ESNS after I did the ribbon access story they put out a press release with facts and they said again and it's very carefully parsed so what do you do in a situation like that you don't wanna say right out you're lying and you've lied to me and you're lying to readers and election officials so I took a screenshot of it I put in red my responses I annotated it where they were saying this I said this is not actually true this is what the real situation is they're saying this here but they're carefully parsing their words this is what it really means and that's how I tried to combat it so that it stops it at the beginning because once they get that out and everyone just keeps repeating it in media interview after media interview because that's what will happen the election districts take the lead of the vendors and take their talking points and then it just goes on so that's how I try to combat it so I think now I would like to open up to audience questions but I will do so with a quick caveat it needs to be a question not a statement and it needs to be under 10 seconds to deliver if possible so I will cut you off it's not because I don't like you I just need to keep this moving so would anybody please raise your hand and I'm gonna ask the people who answer to repeat your question otherwise I think I can hear people Ion yes my question is when I'm reading the reports the last piece of information I never find in that is what is the manufacturer and what type of product is that's not in any of the journalist's stories it is my pet peeve so should I repeat is that funny so the question was basically I'm annoyed that journalists never say the make and model of the voting machines that are purchased when they're talking about a county buying equipment so I will say for myself as someone who's also annoyed by that I always try to mention it you see a lot of local news articles that's like the local reporter is at the county in a lot of states call it the commissioners court that's basically like the local decision making body and they approve the purchase of paper based voting machines and it never says what exactly they bought and it is extremely annoying and in a lot of those cases I think it's that the local reporter maybe doesn't have the sort of the dexterity the cyber security knowledge to understand that for you folks out there reading that story it actually matters what that model is so you can determine is this really a paper based voting machine is this a voting machine that I just tested last week and then I know it has some problem yeah that's a huge problem and I think it just comes down to you know we put that in because we understand why that matters but a lot of people writing about these decisions are writing about them as part of normal everyday county business not necessarily expecting that people like you and the audience are gonna be reading these stories for sort of very technical and really national security related concerns I mean there's a wonderful source for this for even you if you don't see it in a story and you know this, Ayan verified voting has a verifier tool online it can be a little outdated he's got this great tool that they put together published on Monday about what states are moving towards what systems but the verified voting historically it's great you can go back to I think 2004 2002 if you ever want to see what voting machine was used in a jurisdiction during the 2004 elections or something like this and it drills down it's you can drill down by state and then all the way down to county and they will list the voting machine that's used in the polling place the voting machine that's used for absentee ballots that are mail-in ballots they'll use often time a different system for disabled voters and they'll list all of that there and then they also have links to the voting machines themselves a page describing what the capabilities of those machines are I can't recommend verified voting enough it should be if you're interested in voting machines it should be a primary source for you they have everything there just briefly add if you do want to see our page it's politico.com slash election security you can see where your state or county is in the process of buying a new machine we are going to eventually add what actual machine it is we had to pick specific fields to show at the beginning we do have that data about what they're buying so politico.com slash election security all right next question so the question was how does one reconcile the threats that are out there that everybody understands to be out there with the reticence on the part of senate majority leader Mitch McConnell to allow election security legislation on the senate floor there's no good answer I have to be very careful about how I describe this but the majority leader and a lot of particular republicans believe that these are decisions that should be made by the states and counties and I'm not saying this because I agree or disagree I'm just telling you this is the reasoning it is, as journalists I think we pretty much all find it jarring to hear on the one hand leader McConnell and others say we should listen to the senate intelligence committee's conclusions which were released a couple weeks ago one of which is paper ballots are the most resistant to cyber attack to reconcile that with leader McConnell then saying we are not going to put on the floor the safe act or other bills like it I will say just as kind of an armchair commentator one thing I found curious is that the democrats in the house the first thing they tried was HR1 which if you don't know was this big grab bag bill that had election security but it also had same day voter registration and elimination of things like voter ID laws and the result of that whatever you think about that bill the result is that it is very easy for the republican leadership in the senate to say election security is part of a democratic agenda to change the laws that protect our country from fraudulent voting and again I'm not endorsing that statement I'm saying it allows that statement to be made because the first attempt at this in the new congress was part of something bigger and they quickly figured out that this was not maybe the smartest idea and they went to HR2722 which is just election security but now McConnell can say this is part of something else and that is part of the reason that those bills have not yet made it to the senate floor I believe As reporters we have to if we're reporting on a conflict we have to say this is how one side says to them and we have to let the other side respond we owe them that but at the same time we also have an obligation to readers to call out bullshit and I don't know of anyone who doesn't say in stories that relate to it now McConnell has blocked this and experts resound there's a consensus experts resoundingly agree that we need paper ballots I will say I personally deeply regret not writing a story earlier that election security had become such a partisan issue in a way that defies logic that is a regret I have Not too clearly I mean because working at CNN I'm thinking that you have probably a lot of competing interests when you are doing stories how much influence do you have to do that so no one is paying attention let's say at CNN about this politicization and all that how successful, how effective can you be in steering I mean the thing I just said about you include a line about McConnell is blocking all legislation I've included it in I think several stories and it's honestly I honestly don't know I mean I think a lot of people no I don't watch TV this is off the record right we're not I honestly don't know how why I think it's kind of widely accepted literally every story I've ever had it in it's no editor has mentioned it it's just yes of course that's a thing that's part of the story and it goes through the process the editing process I'm intentional about what you were for and I can give you a little context so I recently took a lot of the country who were there citing sensational media I think you guys did really great work but we need to do more research so I'm curious like do you have a conscious intention with what you cover or how do you decide the angles that you take of what you do? I don't know if there's a good answer and I am positive I've missed things I get so many more like insane people tips than regular people tips and things get lost and I know that's a fact and if you have something you know is good keep at it or try somebody else you know don't give up one thing that I find kind of difficult in this process it's just perpetual difficulty is as reporters you have to tell stories you know you have to there has to be some element of a thing is happening and we don't always have a full picture it's messy sorry I don't have a good a genuinely good answer here somebody else so I I'm always overworked I mean there's always more stories to write and sometimes it's the matter of what is the one that needs to be written right now if something is happening a bill is happening a lawsuit is happening something like that and that sort of sets the agenda for you of what you need to write now in terms of I mean I would love to talk with you afterwards and find out what exactly that context was when I decide so I get a lot of research not just in election stuff but just a lot of vulnerability research that comes to me and I have to decide what's important and when it's not it's a matter of knowing what's been out there already is this new is this two enterprise that readers the general readers isn't going to be interested in it is this something that I can actually extract information for a reader to understand if it's too technical I'm going to say to them this would be great in a paper or take it to the tech press so for something like that it really is about a lot of stories aren't fully baked right so someone will come to me with a tip or a little bit of information but it's not quite a story yet and then I might get a little bit someplace else and so it can take a while for a story the story that I did this week about the voting machines online they came to me a year ago I was in the middle of writing a story for the New York Times that was really taking all of my energy and they weren't fully there yet they'd done some scans they found systems online but they weren't very far along and it really took a while for that story to bake until it was finally ready to run turns out the timing was great because everyone is interested in non-elections that's another thing about elections this stuff goes in cycles and the media and the public aren't interested in it in non-election years we've had waves I mean like I said I've been covering it since 2003 there are waves of interest peaks in between 2004, 2006 between 2006 and 2010 drops off because we got paper ballots in 2004 a lot of places got paper ballots everyone thought problem solved so it goes in waves so it's partly about can you get an editor interested in it now? can you get the readers interested in it? is it the right time for the story? I promise that gentlemen everybody was convinced that the earth was flat and then we found out oh hey no this weirdo in the back office he was right all along so what do you guys do in that situation and that's happened a lot recently did anybody not hear that question just wanna know if I need to repeat it okay so the difference is that you use the scientific method so the way that we found out that the earth was not flat is that people tested it and they actually didn't fall off the face of the earth when they sailed over the edge of human vision and similarly with cyber security, election security if enough people do research on a system and find that a vulnerability exists and other people who have no links to those people do the same research and find the same thing there's now consensus building around the notion that there's a vulnerability in that system and so it's not so much how many people are saying this is it 51% it's that 51% how did they arrive at that conclusion and can I understand the method that they took to get to that point so that I'm not blindly saying that there's consensus I actually understand okay they actually tested it in this way and I feel comfortable asserting to my readers that this consensus exists and that it is not some kind of flat earth type of situation right yes so at a certain point this gets to like you know how do we know that the mechanics of the space shuttle are what they are how do we know that the ways that people are testing our food for illnesses and viruses are what they are I mean at a certain point you know you can't test everything yourself right you can't do that in life you have to go through life understanding that you know the folks who are making who are paving roads and designing roadways are doing that based on the best knowledge about you know what kind of material should be used for car wheels to drive on you can't test that yourself before you get in a car for us it's about looking at the past so for example you know we've mentioned Matt Blaise before Matt Blaise's research tends to be right when he claims something it tends to be true and the people that he's claiming it about the vendor the local community or whatever they tend to eventually admit this and so when I'm citing a Matt Blaise research paper I filter in my mind the fact that he has turned out to be right in the past that does not mean that I'm gonna uncritically report whatever he says but it means that you need to look at track records that is the only way short of testing everything for yourself to determine what is safe to tell your readers is the consensus you have to look at track records there's really nothing else that I as a non-expert can do and that's difficult sometimes because not everybody has a long track record and sometimes you have to take a chance you have to talk to people who do have track records and say is this guy bullshitting or not and that's how you get to the point of being able to confidently tell your readers that something's happening there was something interesting in the early years of the election integrity movement there was a couple of people in the late 90s who were talking about paperless voting machines and trying to deter Congress and everyone else against them and no one was listening and I don't know, is David Jefferson here in the room? He was here before so David Jefferson, a computer scientist Livermore, a national lab he didn't agree with them he thought computers were fine and he'll admit this now and he's one of the leading integrity people so the consensus wasn't even there in among the computer science people that there was a problem with voting machines people thought that they were okay and they couldn't understand and it was a woman, right? So it was this fringe little woman who kind of got dismissed that she's trying to sound the warning and no one really paid attention and they kind of laugh at it now and they're sorry that they did it and now they're fully on the bandwagon and even more but the consensus sometimes changes and it takes time to catch up with it I mean computer science, you know we believe air gap systems were secure for a while there, right? And that, oops, stucks net Oops. So I mean, you know no one really needed stucks net to imagine the scenario of USB sticks and all that but I mean computer security what we understand common knowledge it changes over time. Ma'am? But it dropped off and all of a sudden I'm here at about a 10-year-old that hacks up which is just garbage when you read the story so honestly I'm going to be honest with you and it's upsetting you think that's funny because I never saw anybody from your... Ma'am could you phrase that as a question please? But quicker. What was checking each other? Well we have a group chat where we all share each other's stories and sometimes people will say, hey you missed something I mean seriously like, you know if I write something that's not that misses something I trust that eventually when Kim sees the story she will tell me that there's something missing because she is better at this than me and has been doing this longer than me and we all sort of try to check each other in that way but things are going to slip through the cracks in every single system and that is not a comforting answer and it's not a satisfying answer it's not something we should accept but it is the reality that everybody makes mistakes and that's true no matter how important your job is to democracy or to whatever system you're in. Do it publicly. Yes, that is very important we do issue corrections when we find that something is wrong and this idea of things being cyclical and us putting a correction out but the news cycle has already moved on there's no way to solve that, that's human nature people are going to be interested in whatever thing has just happened we can't do anything to prevent that but we can raise the visibility of our corrections when we put them out if in fact we need a correction. I want to validate what you're saying you're not imagining it yes it's a problem it's a problem in journalism it's not considered a good etiquette to call out other media outlets who are getting something wrong journalists who are getting something wrong if you know the journalists you can reach out to them I have done that, they ignore me even other journalists ignore me media outlets ignore me I'll point out an error or something it won't get fixed there are national publications that are notorious for not correcting stories even when everyone on Twitter is pointing out an error it's a problem it's a problem that journalism isn't self-aware it doesn't have that self-reflectiveness the profession aside from that though there is something that we can do and Kevin was kind of talking about that when you see a hole in what's being covered when you see that there's something even in your own organization that needs to shift or whatever you can do that when you do see media misinterpreting stories you can tweet and say actually and these guys do it a lot and say well no they actually got that wrong so we are out there very publicly sort of I don't want to say policing but we do have an expertise in an area we don't have an expertise in every area but when we do see poor reporting or that they've missed something or something like that we do, we do say it quite publicly on Twitter it does, I get that, I get that but it's better than not doing anything and the thing is is that if people are reading it and then you tweet and you say well oh actually they missed this this is actually what happened then people start tweeting that instead of the other thing so I do see shifts I get it, it's a problem Twitter is a problem Facebook is a problem for that reason that the false story gets out there and it's too hard to pull back I also have a pet peeve about stories not getting corrected there's a story out today there's a story that's still up from 2013 a major error story everyone knows it's an error every journalist knows it an error and the publication still has not put a correction on it so it's a problem so I'm actually going to just stop the Q&A and just kind of do one last question in closing remarks now so we can stay on schedule I apologize I'm sure you can find these guys around the village if you have a pressing question but I kind of wanted to ask for your closing remarks any thoughts of something you wanted to add as we wrap up we've got about five minutes to do so and if you just need a prompt just kind of what are your thoughts going into 2020 how are you planning on covering what are you looking out for and maybe I'll start with Eric So I think the big unaddressed issue is oversight of vendors and Senator Wyden has really been doing a lot of work and letter writing on this states and counties are over time learning at least that they need to improve election security even if they don't know exactly what that means it is the vendors that have a lot of power in this space and that have so far largely been able to avoid any kind of regulation when I mention that there are probably people in this room who work on the VVSG but when I mention the VVSG there's some laughter in some corners about the adequacy of that to protect what is critical infrastructure can anybody imagine if the telecom or manufacturing or energy sectors managed by DHS and other agencies were only governed by the equivalent of the VVSG and states could put other things on there if they wanted to it would be absurd we would all laugh but that is the current situation in election security vendors are not required to report cyber security incidents they're not required to let independent researchers test their products under anything short of an extremely onerous NDA and the reason for that is because nobody is telling them that they can't do that and as a journalist what I'm gonna be looking for and lead up to 2020 and afterwards is what changes ESNS has said that it wants to only sell paper ballots it's still selling paperless machines and for voters with disabilities but it's at least recognizing that it has to say something about this what are the other vendors gonna say what is Congress gonna do will there be a requirement to create some kind of vulnerability disclosure program that's what I'm looking for I wanna reiterate that we often don't know what we don't know and often rely on people like you who see things on the ground that need to be brought to the public's attention if you have a tip to bring it to somebody it's not a guarantee that it will make it into the news and you might not even know that it just helps our coverage in general if you alert us to something and I wanna say again if you do have something as a source if it's something sensitive you do have the power you can come to one of us we all have our signal numbers for the public come to us say I only wanna speak like in these terms and set the terms for discussion you have that power and we rely on you I guess I would say going into 2020 I feel encouraged and discouraged because DHS is touting how much work they've done in the last three years and election districts are talking about how much progress they've made in the last three years and then I publish a story this week that says all those voting machines are still on the internet so I don't know what they're doing or what they've been doing for the last three years I feel encouraged that there's a lot of tension on all of this I feel encouraged that DHS is involved that there is experts, security, knowledge trickling down now to the states and to the counties I mean Congress, one of the stops here is Congress giving states more money I feel like there's at least a willfulness in the states now more at the state level than maybe the county levels to actually listen and engage in this but they can't do much without money they have to be able to hire someone who's gonna monitor those logs and install them correctly and get secure processes in place and get someone in the office who's actually there saying no, do not put that USB stick in the voting machine so we have a long way to go but there's a tension on it and that's good Well I think we have nothing more to add I'd like to thank our panelists and thank you all for joining us