 Please welcome Joseph Marks and our panelists. I don't think there's going to be, it's a panel, so I don't think they're going to, yeah. Otherwise, it's like right in their eyes. It is the, um... Yes, three, four panelists. Absolutely. Have you spoken to the moderator? Uh, is this for the next panel? Yes. The one coming up right now. Okay. Wait, who are you talking to? Uh, I don't see that person on this list. Are you sure that's the right, you're at the right time? I'll sit there. This is Rita Wayne, Trevor and Alex. So what's going on? I don't know. Sorry, maybe Jacqueline? Yep, what's up? He's supposed to be serving in for our panelists. I'm looking forward to Rita. Oh, where is he? And actually, while we set up, I was remiss. I'm supposed to announce we have a new panel that will be held at 6 p.m. Let me pull up the name. So at 6 p.m., we'll also, we'll have a new panel that's called The Devil Went Down to Georgia, Did He Steal Souls? Which is the Georgia's electronic voting saga. It's basically an update on the case down in Georgia right now. So it's going to be a really interesting panel. I encourage you all to join us on it. Just a minute. I just introduced you while you were coming up here. Maybe it was a little premature. I didn't introduce your panels, so I left that to you. Otherwise I'd be like, yeah, I'm all done. So it's all yours now. Hello. So it looks like people are still coming in. I'm Joe Marks, cybersecurity reporter for The Washington Post. And I write the cybersecurity 202. And we have the stage for a really long time today. So what we're doing is splitting us up into three sections. We have on stage right now some government folks. We're going to talk maybe 25 minutes, take a little break. We're going to bring up some industry folks, talk 25 minutes, take a break, and then... Hello? Is this good? Okay. We're going to talk with industry folks for 25 minutes, take a little break, and then come back with everyone. And I'm not sure how we'll make the seats work. And we'll talk a little bit more with everyone. And hopefully that'll be primarily audience questions. So be prepared to ask all the questions you can. So we have not met in person today yet. So I'm going to start naming names and then raise your hand. We have Alex Joves, who is regional director for the Cybersecurity Information Security Administration at DHS, covering a chunk of the Midwest. We have Rita Guss, Chief Information Officer for the California Secretaries of State's Office. We have Wayne Thorley, Deputy Secretary of Elections right here in Nevada, and Trevor Timmons, the CIO of the Colorado Secretary of State's Office. So the big thing we wanted to talk about all throughout today, and especially on this panel, is are we ready for 2020? So I thought it would be good to start by getting a feel for the room. So how many people here by show of hands think that just in terms of the security of the ballots themselves, not the deep fakes, not the bots and everything else and the influence operations, think that 2020 will be more secure than 2016? Okay. I see one of these. How many think it will not be significantly more secure? Okay, we're going 50-50 in this room. Okay, and then second question, and again, we're just talking about the security of the ballot itself. How many believe it will be as secure as it needs to be to have high confidence in the outcome in 2020? The panel in the room are divided. And how many think it will not be as secure as it needs to be? Did we have high confidence all throughout the panel here? I didn't look over. Okay. So, let's start with, because you have sort of country-wide but certainly a larger remit, several states. Alex, tell me why it will be as secure as it needs to be and tell me about what you've been doing since 2016. Yeah. We just have to pass this one. Okay. Okay, you can hear me okay. Okay, so, you know, I'll talk about the work that we've done again since 2016 at DHS Now SZA, right? So, again, you've heard, and if you were in the room before me, you've heard both our NCATS team, you've heard Katie Trimble from our continuous disclosure team. You've heard probably a lot of different SZA folks talking about some of our services there. So, let me just highlight what we've done again since 2016 when DHS named the election sector critical infrastructure. So, number one, we really have put this into three buckets. One, information sharing. So, we'll get more into specifics of that, but that is anywhere from, one, the establishment of the EISAC, that's the Elections Infrastructure Information Sharing Analysis Center. Sorry, that's a really long title, the EISAC, but essentially, think about that as your situation room, your 24-7 threat info mitigation, and again, we have states and locals on that information sharing. We also can ramp that up for election days and we're able to set up an election day situation room where again, our state and local officials, again, who run elections, let me just start there first. Elections are run by state and local jurisdictions. We, the federal government, in particular, are here to support the jurisdictions and help with our cybersecurity and physical security resources for that. So, one again, information sharing, that's just one part of that and really our state and local partners can talk more about that. Two, the technical assessments and their services. So, you heard again the panel before us talking about the left of boom services, right? So, these are the services again. We do this across the 16 critical infrastructure sectors. So, traditionally right, I talk about the lifeline sectors, water, energy, transportation, communications, but now we're looking at election infrastructure as well. So, those same technical assessments, you heard Jason talk about the left of boom ones, those are those cyber hygiene, vulnerability scanning, cyber assessments, enterprise assessments that we can do on site, all the way to that red team, the phishing campaigns. A lot of those resources, again, are now open to our election partners. And then you're right of boom, right? He talked about when you do have an incident, we're able to bring in those government resources, what we call our Hunt and Incident Response Teams to help them identify those and mitigate those threats. Again, supporting our state and local partners. So, that's that technical assessment. And then the third piece I'll talk about is just the training and the outreach, right? So, I've heard some questions here. Hey, how do you guys get the information out? So, one, we are out there across the country and again, I cover six states in the Midwest getting both these services and this information out that's available, right? So, I think the statistic now is that we've got 50 states involved. We have 1800, actually it's 1900 now, local jurisdictions, so it keeps going up, involved with our services. And again, that's both with the outreach of that, we do tabletop exercises, it's a national tabletop, the vote, all the way to state-specific and county-specific exercises. So, again, I've seen it over the past three years that I've been involved since 2016. Now, it's really robust. The other thing I'll talk about here is that's actually expanded as well and that might be the other question you're going to get to. So, as we go into 2020, that's expanded now to obviously at the state jurisdiction as much of the local jurisdictions, because again, we're talking about 8,800 local election jurisdictions to get to, but also to the private vendors and then finally to, again, to candidates on both of all parties. So, we're working with, I think that we're at a dozen presidential campaigns that have, again, received a briefing about our services and, again, can also take those, including the RNC and the DNC. And so, Rita, tell me about your work with the California Secretary of State's office and to be clear, you don't cover voting machines, right? But you do cover all these back-end systems which, if you guys have been reading your report at the Senate Intelligence Committee report, have been major targets. So, can you tell us what you've been doing since 2016? So, yes, I'm the Chief Information Officer for the California Secretary of State. My role is to do technology oversight for our infrastructure, data center, endpoint support and security operations. Our election team is responsible for policy and working with the counties in running their election systems. Our responsibility is more on the voter registration and websites and anything that support the Secretary of State, not just elections, but we also do business filings, archives, political campaign reforms and all the other functions that belong under the Secretary of State. So, when we were racing, when I was racing my hand that says that 2020 is much more secure than 2016, I'm talking about California. Since 2016, I've run five elections already. Since I started in June of 2016, just right after the 2016 primary general elections and we have done so much between 2016 and now we have done so much in the difference. We have created different programs. Two programs was created just before the 2018 general midterm elections. We have created two programs that is our election cybersecurity office. That is probably one of the few or even the start of an office that is very specific on voter education, county clerks and staff training, sharing of the information with different partners such as MSISAC, federal partners, state and local government as well as we also have a team of individuals whose primary purpose is to monitor misinformation in the internet. So we created that office within three months. We got the budget, we created the office and it was up and running for the 2018 elections. The second office that we have also created is an office of risk management. That office is probably one of the few California State Department that has a risk office which is the information security officers are under and it is very specific on information security. That risk office was created two months before the elections. So now that we have all these programs in place we are in a better position. We also have implemented initiatives that is very specific on misinformation and disinformation campaign. So we created a website for a one-stop website for voters to go to election information. We created that within five days from planning to implementation. Now for you guys here in the private industry you would say well five days for a website that's normal but for government agency that is not normal. Typically it is weeks or even months or even years but we created things like this within five days. Also we also created the first paid social media campaign to combat misinformation. That was also created probably less than three weeks. So within that timeframe we were able to implement this and actually as a CIO for Secretary of State we have a very big challenge. With any IT programs or IT initiatives when we do our planning with elections things doesn't stop. We have to finish everything that our initiative is for elections. So for example if you're implementing a tax filing system like that you can say oh okay we missed our deadline we can postpone it or maybe implement it the next time. With elections once you plan it for that election you have to finish it. We have to be able to finish it. So that's part of our challenge but we are in a better position now because we have implemented a bunch and a lot of successful initiatives that went well in the 2018 election. So Wayne and Trevor please tell me what you've done since 2016 to make elections more secure but also read ahead an interesting caveat. Are you speaking, when you said more secure are you speaking for your states or country wide? I don't know what other states are doing but I'm sure like with Trevor here he's pretty much in a better position too. Sure. Thank you and again thanks for the 202. Love it. If you're an election official whether you're a tech person or an election official subscribe to that, they're fabulous. Sorry, they're fabulous. So I want to talk about a couple things. So we have done in Colorado, again I'll contrast 2016 to 2018 to 2020, it's really an evolution. I mean 2016, some folks in summer 2016 when DHS and the FBI started sending out information about some of the activity that they had seen some people were caught unawares, right? I think many members of the public were probably caught unawares with that information that oh my goodness, stuff is happening. The reality is in most of the election offices in terms of managing state voter registration databases and election night reporting systems in many states there has been awareness of the risks of those and comparing where we were in 2016 that's before the critical infrastructure designation of election systems and so that's what I raised my hand. Are we in a better position approaching 2020 than we were in 2016? Absolutely. Today in our state I think many states, maybe all states, we've got a pretty good relationship with our DHS region staff and with DHS headquarters. You know they won't tell you, the DHS folks that were up here earlier, they won't tell you who has done a risk and vulnerability assessment who has brought in that hunt team to look for evidence of compromise that is latent, that is in a system, ready to be exercised. They won't do that because it's victim notification, they want to keep that information private so that they can encourage other people to take advantage of those services without fear that it's going to get out and say well what did they find? What did they see? But Colorado, we've done that. The nine services that he was talking about in terms of DHS and working with the EISAC in terms of adding those services to your portfolio to secure those systems, we've done those and we're continuing to do those because this is something that is never going to end. We are never going to be able to say we're done. Solve security, don't worry about it. Some of the other things we do is really reaching out to those local election officials. Incident response, planning and training. I'll give credit to the folks at the Belfer Center out of Harvard. They actually reached out to election officials from the technology side and kind of the business process and policy side to develop kind of a TTX, a tabletop exercise in a box. So they brought, in the second iteration of that, they brought in over 100 election officials from across the country to actually do this role-playing of a tabletop exercise to think about the things that could happen and how you would respond to them. DHS, this summer they did the second of the annual exercises where they brought in local and state and federal partners on the government side and on the private sector side to actually participate in these exercises. You know what, it is fabulous to have a clerk and recorder from a small county respond to a question about, hey, so one of your candidates just posted something on Twitter alleging that there's some abuse of the election system going on. How do you respond? No, it's even better because when you do this in a role-playing exercise, you will have a camera and a microphone in front of this local election official. We brought in technical people, cybersecurity people, elections policy people, public information officers from local governments to participate in this role-playing exercise. We had a statewide exercise in Colorado in September of last year, two years ago. And just to prepare for what could happen, we'll be doing it again in January because there's turnover at that local level. Being aware of the risks that are out there and understanding what resources are available to react and respond when something does happen, I mean, that's part of the battle. It's not all of the battle, but that's part of the battle. I just want to rush through because we're running a little short of time. Go ahead, sir. So, I too am very confident that specifically for the state of Nevada, and welcome everybody to Nevada in Las Vegas, my hometown, glad you all could be here, that the 2020 election will be much more secure than the 2016 election. I can confidently speak for the rest of my colleagues in other states, too, because we've been at conferences, other trainings than the national TTX, and I know my colleagues in other states that administer elections are taking the threat to elections very seriously, and I've taken a lot of steps forward in that regard. So while I'm extremely confident that for Nevada, the answer to that question is yes, I'm really confident, maybe not to the extremely confident level, but really confident the answer is yes for other states, too, because I know that my colleagues in other states are taking this issue seriously. You know, my role in Nevada as deputy secretary of state is to, you know, I oversee the administration of all elections in the state of Nevada from federal elections down to local elections. I also advise the secretary on matters of policy and law. I think I'm the only non-tech person on this panel. My background's in economics. I'm a numbers person, but so it's been a lot of an education campaign surrounding this issue, both for people in the state level and also getting it down to the local level. Here in Nevada and Clark County, right here where we are at, one of the largest voting jurisdictions in the country. We have over a million registered voters, and it's like in the top 15 largest counties in the country. We also, on the other end of the spectrum, have Esmeralda County, which is like three hours northwest of here if you want to get on the 95 and start heading north, and there's like 200 registered voters there, more cows than people. And so there's a huge difference between the amount of resources that are available for the counties and the both financial resources, but human resources too. So the technical expertise, some of the counties contract out to their IT support services because they just don't have anybody in the county to do that. Okay, I'll jump in quick. I want to draw a couple of different threads together here. As Wayne was just talking about, there's a varying amount of resources that different counties have. You all said you're very confident in your own states, and yet Politico had a report a couple of weeks ago looking at counties in states that did not have paper ballots in 2016. By 150, they're still going to be using non-paper ballots in 2020. A separate report out of Motherboard found I think three dozen situations in which voting machines were connected to the internet that should not be. None of them are in California. None of the paper states were in California, Nevada, or Colorado. So one, do we have the wrong voting officials up here? And two, what needs to be done about this? Senator Wyden, when he was here yesterday, left by urging everyone to go call your member of Congress, demand election security legislation, demand mandates, and more money for elections. Is that what we need? Why should people have confidence in the entirety of the voting system, not just these three high-performing states? So I'll take a stab at that first. Election officials across the country, we're not waiting for Congress to take action on the issue of election security. We're moving forward. If we're going to wait for Congress, 20 years or more before Congress takes action on this. So we don't have our heads in the sand and we don't feel like we're stuck and not being able to move forward without permission or some sort of action from Congress. Nevada, we've been very proactive in reaching out to our state legislature and securing funding, but also getting laws passed to enhance the security of elections in Nevada. Just this last legislative session that adjourned just in June of this year we were able to get legislation passed that mandates risk limiting audits which is a type of post-election audit so that we can have a high degree of confidence that the reported outcome of the elections is accurate. So we're going to start working on that. We have mandatory information security training for all of our county election officials now thanks to legislation that was adopted because we're seeing those things that CISA mentioned earlier, these common issues pop up as we go out and meet with the counties and one of those is phishing emails. So we're running a phishing email campaign right now in our state sending out fake phishing emails and then seeing who clicks on the links in there and if they do then they get a little call from our office saying hey you got to take this remediation training. So we're doing that. Do you need more federal money? I'm not going to say no to federal money. You don't look a gift horse in the mouth. We would absolutely love more federal money but we've been very resourceful through our state division of emergency management which gets a grant from FEMA every year, Homeland Security Grant. They sub-granted out to state entities. We've gotten a quarter of a million dollars through that program over the last two years to enhance the security of our voter registration database. So we're not waiting around for Congress to get their act together. Anyone else want to take a quick stab at whether we should be more concerned about the states and counties not represented here? I think the answer is yes, we should. I've got a little different perspective on this. So states and local governments, they need to stand up to their responsibilities to do the right thing, take the right approaches, look at best practices and implement them. And I do favor more federal funding. There are some states that have paperless voting machines and it's a problem and it takes money, it takes training, it takes resourcing to actually address that problem. What I fear is that with an influx of federal money that there will be a perception that a one-time infusion to address today's problems would be enough and it will not. This is something that we are in for, you could say the long haul. I can say as long as I'm a voter and I think we need to look at it that way and look at it as a national mission at the federal, at the state, at the local level and at the community level to actually address that. So I just want to jump ahead to another question because we're running short on time here. So we are here at DEF CON, Voting Village last year found vulnerabilities in numerous systems and there was some conflict with the election administrators. The National Association of Secretaries of State criticized it for not having realistic conditions, called it a pseudo environment which in no way replicates state election systems, networks or physical security, said it misrepresented the actual security of the election. Is that still true? Are things getting better this time? What's the relationship like between you guys and these guys? I think it is better. I think if you look around the conference this year you'll see more elected officials, geeks like me who support those processes than you've seen in years past and I expect that to continue to grow. I think having the conversation actually establishing the relationship and the trust amongst the partners, we need to do that before we can all get together and start to trustfully, purposefully move together and address some of those issues. We need to engage. That's why I'm here. I'll agree with that. It took us a while to build this relationship of trust with DHS when they first started reaching out to us in 2016. I remember getting a phone call from someone at DHS and being like, who are you? They actually had called the governor's office first and on their side they weren't sure who administered elections and so it took a while to build that trust and speak each other's language and I think we're working on that here too with the hacker community where we're building that relationship of trust. Trevor's exactly right. There are more people that work in elections here than there has been in the past two DEF CON conferences. I think that will continue to grow and with that will come that trust as we learn each other's language and learn what your motivations are and then you all learn what our constraints are then I think we can really have a productive relationship. Rita and Alex, are we working with realistic conditions now and are our hackers and election administrators speaking the same language or are there still a gulf? Well, for California we have a very complex... it's not a single, simple, centralized event. It is a very complex of multiple... an ecosystem of multiple technologies and processes including non-digital. So it's important that people understand how election processes are working. It's not a one-size-fit all. Every county is different, every state is different. So when they say, hey, someone gets scanned everybody should first do their due diligence and providing the right information because at first when DHS says, hey, California you get scanned, of course everybody gets scanned, by the way but you know, Russian gets scanned your network it wasn't even our network that they were referring to meaning the Secretary of State network. So that was the first time in 2016 but the relationship has gotten better, a lot better like the Secretary said yesterday. And I'll just wrap it up actually more as an answer to your last two questions. One, you know, it very much and what we found and as I started again states and local jurisdictions run elections. They make, again the rule when we have the recommendations it's up to them and their legislatures to implement that and again we obviously have our federal government resources that are available to support that and enhance that. I will tell you that that's not the only solution though, right? And that's why we're here, that's why every year it's growing the partnership part of that is so key. So a couple of the states again that you may have mentioned in your reports, again we're trying to, one, get both the information out there that there's a threat here, right? So that's better than it was. Second, we're trying to build resiliency measures that's there, right? So again if it's just paper list ballots right now what is the resiliency measures that can be set you know, short of the funding whether you get that through your state legislature or not. And then again for mitigation and incident responses again state and local jurisdictions are doing what they need to do to again address these threats. Now it's not just the federal government that's providing this you're getting this from third party vendors from industry. Again, state and local jurisdictions are working directly with that to fulfill a lot of their needs. They're also working again with the community here to get those needs set. So it really is, I mean I think it's important to emphasize it's a partnership. I know they say it's a whole of government effort it's a whole of election sector effort on that. So that's really grown. Thank you very much folks. We're running short so we'll take a little break and we'll be back with some folks from industry. Alright, we're back if everyone's ready. So here we have public sector folks starting with, sorry. It's Saturday, it's okay. Starting with Alissa Starzak with the policy at Cloudflare. We also have Josh Benelow, senior cartographer at Microsoft Research and Jay Kaplan co-founder and CEO of Cynac. All of these guys are doing something, all these companies are doing something to help secure elections. I wanted to start though by just put them on the spot and do the show of hands again. Do you agree will we be secure enough as curious we need to be highly confident in the results of the 2020 election? Okay. They're more with the crowd. Alissa, tell me what Cloudflare is doing. We'll shoot in for like one minute. So if you don't know us, we're a security and performance company. We provide security to websites and anything connected to the internet. We actually have a set of free services that's available to state and local election officials and lots of states use us. I think we launched in December of 2017 and the time of the last election in half of the states had some jurisdiction on us. Josh? It's going to be hard to be that quick. I would love to be able to tell you I've got a way of securing elections and we're done. Great. I don't know how to do that for even a simple application and voting is not a simple application. Secret ballot voting is even a harder application and with the challenges that we have, we heard about with Miranda County, you said, in Nevada with 200 voters being able to withstand an attack from a nation state in Russia. I don't think it's reasonable for us to be able to claim we have any hope of being able to actually secure an election system. The one thing we actually can do is we can enable detectability of tampering of any kind, any and all tampering in an election system and I'm not just talking about internal tampering, external tampering, not just internal, even tampering that hasn't been dreamed of yet is all detectable by technology that we can build today and that's what Microsoft is working with partners to deploy technology that enables any tampering to be detected and I'd be happy to talk more about how but I don't have an opportunity to do it but it's going to be freely available open source, anybody can use it, we're working with vendors, we really want to get it out there. So for those of you not familiar with CINAC we are a platform that enables big enterprises and government institutions to engage with a crowd source community of white hat hackers in a very controlled and trusted environment and back in late 2017 we announced an initiative where we were going to allocate a million dollars towards free services for any state or local entity that wanted to engage in CINAC's services or products specifically related to anything that could be accessed remotely. We weren't coming on site but for voter registration systems, for any voting machines that we can connect to remotely and for any of the reporting interfaces we were going to perform testing for free. This has been incredibly challenging because offering something for free is actually a lot harder than offering something that people pay for as we realize pretty quickly and we can talk about what some of those challenges were. We are engaged with several different states at this point and some local entities and are seeing some great results but not nearly as much of a pickup as we had anticipated. Speak about that please and other people can jump into but how tough is it for a company that has a product that it believes can help make elections more secure to actually get that in place at the state and local level and why? I think one of the challenges that we experienced was that there were just so many different organizations involved in every single one of these disparate systems and so while we can go engage with the secretary of state's office the reality is they are utilizing a number of different vendors for their voter registration systems for the voting machines and pretty much that entire flow from initial registration all the way to results and so our challenge was who is the right person or company or entity to actually go to to offer our services for free because they sometimes they're just like the connective tissue and so we we just had a lot of challenges figuring that piece out but once we started to realize that if we actually got the SOS of a certain state or we got even a local municipality to engage on a paid basis it was just a lot easier to kind of once you're going through that contracting process to engage with the right party so we actually now work with one of the largest cities in the country on a project and we've seen pretty amazing results that actually are now forcing them to switch to paper ballots ahead of the 2020 election so we'd love to talk more about kind of paper versus not and some of that uptake so yes it's an incredible challenge to get things out there we are partnering with most of the large election vendors, most of the small election vendors have agreed to partner in this project that's called election guard just to be clear, Microsoft has no intention of building and deploying election systems it would be a bad idea, we don't want to do that but we're building this open source toolkit that we're sharing with vendors and showing them how to use it there's been a lot of uptake and a lot of interest there so I'm very encouraged and optimistic that that will happen the next step is partnering with jurisdictions getting some pilots going getting some adoption, getting some trial and one of the big hurdles there is regulatory the federal regulations, even though they're voluntary most states adopt them in some form, which is a good thing but they're not very flexible and what we're doing is something that's very different very new very unusual and doesn't fit in with the current process what we're doing is a process which enables validation of actual elections and the regulatory process is for validation of election equipment and it's like a round peg and square hole trying to say, well, but you don't need to check this because it's externally validated, the regulations do it anyway. Do we need to move beyond the voluntary voting system guidelines which turn into regulations at state is that not agile enough for it's certainly not agile enough it is very valuable so I don't want to say we should abandon it anyway it has its role but especially with a system like this where external verifiability of an election is possible you don't need to verify the equipment in which the election is run to verify the accuracy of an election then it should change the way the standards are written so that standards can concentrate more on some of the things like is the font large enough is there enough contrast some of the basics we need to get to and some basic security things that need to be there the standards are very good at checklist things and checklist things are not good at security Alissa, do you disagree? No, I don't disagree I think that there are different issues though and I think that we should talk about them differently because I think what that is about is about wholesale change and how we deal with elections and so maybe it's not surprising that there are some pretty significant regulatory challenges I think that we have a separate problem which I think Jay alluded to which is that we have an incredibly disparate election system which is great, it's decentralized that's good from a potential problem standpoint but it's also a challenge to the extent that you're talking about services that are potentially available it's hard to connect with everybody and frankly from the private industry side we're offering the service from our standpoint as a free service as a corporate social responsibility project but people don't believe it because people are generally skeptical and that's a really hard place to be too because every time you come in it's part of a social responsibility team it's not a big team and they're like you should engage with every single person first and then when you get to know them then maybe they'll adopt your services and when you're talking about the number of election jurisdictions in the United States that's not feasible and it's hard to explain that from a private industry side but it's true and so I think in some ways we do need to think about ways where people are aware of what services are available for them in the short term of things that they have gaps for now they can assess what tools they need that's the role of election officials and that's the role of the public sector but the private sector can help fill those gaps as necessary. So, Alyssa, if you could do two things today be national election czar change two things to make 2020 more secure, what would they be? You know, I actually think a lot the reason I raised my hand to your first question but maybe not your last question is because I think a lot of things have been done I think SZA has done a phenomenal job of trying to think through things and to look at vulnerabilities and I think that they if you look at some of the things that they did they actually tried to make the assessment step by step of what should you do and where are your gaps I think that the thing that we haven't quite done yet is the actual assessment against those products and that's hard because again that's resource specific or resource intensive and we don't necessarily have the resources to do it but I think what you want is every election jurisdiction which sounds like an answer, right? It sounds like some of the mandates we've been talking about in legislation, you have to have paper ballots you have to have post election audits are those good candidates? Definitely Josh, two things? Yeah, so those are easy for me because it's all about post election auditing so this public verifiable technology that I'm talking about, it's called end-to-end verifiability should be done everywhere risk limiting auditing should be done every contest, every election they complement each other beautifully we should do both in every single election I think we have more of a I don't think security is actually the problem in election security I think it's actually confidence voter confidence more than anything and I think having instilling that confidence in those who are voting we're not doing a great job at it Of course it should be justified and so if there's something I could do very quickly I would basically force every single technology environment end-to-end whether it's a voting system whether it's a registration system whether it's reporting the results and open it up to a vulnerability disclosure program and allow every hacker in the world to try to find every vulnerability out there it's the only way we can do it we're not going to be able to sign enough contracts to get all of this done especially in time for 2020 there's an amazing community of hackers out there look at everyone at DEF CON who are patriotic who want to provide help and assistance and want to instill that confidence to those who don't understand cyber security and I think it's something that's doable I think it's hard but getting everyone on board but if there was appropriate legislation in place I think we could make it happen So on that confidence question moving back to the 2018 voting village report and the National Association of Secretaries of State's critique of it Are we communicating this wrong? Why do hackers and election officials disagree about what's an important vulnerability? I can You want to try it first? I'll do it first So security thrives in sunshine that's really what it's all about Various industries have learned this in a different way it's a difficult challenge to understand but yes we need sunshine on the systems and naturally vendors in an industry often come in not understanding this we need security we can't show anybody anything because that would lead to insecurity we need to convey the importance of openness in achieving security in Las Vegas the gaming industry has learned that they're completely open about vulnerabilities the biohacking village over here the medical industry has now learned this many industries have learned this lesson unfortunately the voting industry has not learned this and basically insecurity thrives in the shadows I want to listen to the answer but incorporate this you said the voting industry hasn't learned it should the major voting systems vendors be here and be mimicking the biohacking village yes they should be I think some of them are to some extent but they should be here contributing their equipment talking listening more openly yes I think the other thing that's changed from last year is I think what people weren't thinking about on the cybersecurity side last year was the fact that the campaign that did happen was intended to undermine confidence so the goal was to undermine confidence and when you start talking about vulnerabilities you have to think about how you talk about them it's not just that they're vulnerabilities oh my god that's horrible it's that we are looking for vulnerabilities on purpose so we can fix them and I think we all know that from anyone who's in the anyone who's worked in IT anyone who's done anything in cyber knows that vulnerabilities are a fact of life it's just that then the response to it it's the fact that the next step is mitigating we should be talking about it transparent about it but then we need to talk about the fact that we can mitigate it I just add to that we currently engage on both open vulnerability disclosure programs as well as closed bug bounty programs for the largest organizations the world whether they're big financial institutions big government organizations even the department of defense the internal revenue service etc if these organizations, these conserved entities can embrace engaging with the global hacker community I can't these voting machine manufacturers I don't know why it's so challenging and difficult but I know we were working on several projects kind of just on our own and as soon as they got wind of it we started getting letters in the mail and that's not how it should be there should be way more openness I love your point of like shining sunshine on these systems it should just be out there we're using them and ultimately we have confidence in these systems and if the cyber security industry doesn't have faith how is anyone else supposed to so Josh, Microsoft has been I think working with some of these vendors in order to get an election guard hooked on to them is the industry changing and is the tech industry helping to change voting systems I think we're getting there it's a slow process it's a difficult process there are some vendors that we've talked to but vendors aren't ready to say we'll just disclose all of our sources and all of our designs yet I would like them to I hope they're moving in that direction one thing we talked a little bit about mandates audits, paper and so forth is it your sense as you talk with jurisdictions do you need more money I'll say yes absolutely when we wrote the National Academy of Sciences report on this we looked a lot at the entire infrastructure and not only do they need more money they need a reliable source of money a continual reliable source of money these fits every 10 years here's some money, quick spend it build a reliable infrastructure but yes elections are administered at the local level in the US in almost all states county level, sometimes even municipal levels election equipment and security is competing with potholes and roads and other things and it's very hard we need a good reliable stream of money to build a secure system infrastructure I'd love to ask the panel that went first when they come back on stage if the federal government was able to put out kind of a baseline image or system that they built that they made accessible for you to procure and even provided some funding and subsidized that development in partnership with private industry in partnership with the security industry like how would you feel about that would that be a good thing a bad thing, do you think having this separation makes sense or should we be moving to a day where we have one reliable end-to-end system for voting that the states can take advantage of I'd be really curious to get your thoughts on that I don't think this is going to pass Mitch McConnell's senate probably not so we're here at DEF CON you've talked positively about the voting village what more can the hacker community do for the security of elections going 2020 and beyond I think exposing vulnerabilities is a good thing I think the hacker community should be engaged we need to find better ways for the hacker community to engage directly with election officials and vendors so that we can do responsible disclosure right now there just aren't mechanisms to do that and we certainly need to be able to do that but don't stop doing what you're doing absolutely look for vulnerabilities and I would hope the manufacturers and vendors will make their systems available and open so that there's a more cooperative relationship I'll just say it's hard right now because without an actual structured responsible disclosure policy from these organizations if you are doing this research is it illegal or is it not and I think that's the issue I think people are the hackers in the room the hackers at the conference are worried about even working on these systems in fear that they're going to be prosecuted so we need to get rid of that stigma before we can actually appropriately engage this community so there needs to be more policy that says this is okay it will actually happen and be successful the DMCA exceptions don't get you there there isn't sufficient confidence in them you don't want to think well I probably won't go to jail for doing this you want to have confidence so we talked a little bit in the last panel about the distinction between voting machines and back end systems we talk a lot about voting machines we have no evidence that voting machines are actually penetrated in 2016 we do know that the databases were in Illinois in another state can you talk about that and whether the enough focus is being put on these broader systems and if the public understands that certainly the broader systems are important my focus my specialty is the casting and counting of votes but that's not to say that the registration systems aren't important and all of the other systems around elections are accessed in various ways there are so many issues it's a very complicated field these are all important questions I worry that we'll say well we didn't see any attacks on voting equipment in 2016 so we don't have to worry about that that's all good because we know there are vulnerabilities there and we have to give that equal importance with all the other aspects I actually want to take a completely different take on that because I think one of the challenges is that the voting issue the actual changing of votes is something that's kind of a big headline everyone's scared that the votes were changed and I think that the problem again this goes back to the point of what we saw in the last election cycle if you're trying to undermine confidence it's actually the back end systems that are even more problematic because you might not know if they change votes but the back end system where local election officers website is flashing something that it shouldn't be flashing that's not going to be building confidence in exactly the system that may be just fine that may not have votes changed and I think one of the reasons we actually got involved in this space as a company is because we thought that's what we do those are the types of systems that we actually protect and it seemed to us that as a company that was in this space there was something good here and we could offer the services for free that companies pay for and so the notion that that is something that you can help with was really important to us I was just going to add I think it's the voting registration systems that are the ones that are typically connected online and so those are the ones that we really should be most worried about and if you really think about scenarios if a bad actor was able to suddenly change the address of everyone that had an absentee ballot where it went to some other location and you know votes were not counted it would cause chaos right and so you know I think we have to look at end to end there's not if there's one hole you lose total confidence in the whole system how scared should we be I'm not going to say this on stage but if you want catch me later I'll tell you about an interesting voter registration system issue you can't do that come on there is a vulnerability that I recently discovered having to do with a misunderstanding of the value of randomness in driver's licenses that Washington state is suffering it's been a mess I don't want to put it on this is one of many little stories that keep poking holes in our confidence I talked earlier about Kim's editor story and mother board about three dozen system back end systems that were connected to the internet that should not have been how scared should we be by that and all of these other little stories that could affect a county here and a county there across the country I'd say scared I bet you there's way more than that you know I think just think about how many different counties are setting these systems up and who is setting them up it's scary but we want you to have confidence in the system but very seriously actually on that point I think there's a messaging point I think we have to be very careful when the option is only scared or not scared that's not good enough the reason everyone's here and the question of what can hackers do it's help fix it so that it's not that everyone is scared it's that we're thinking about it together and that we're actually fixing it together and then that helps build confidence and the more people who are involved who are trying to fix it the more confidence there is we're running out of time here to close out Alyssa you talked a lot about confidence and it cuts two different ways there's the one there's no evidence that any votes were changed in 2016 decent chance will be able to say the same in 2020 and so a lot of people will stop caring on the one hand on the other hand there are all of these little stories and all of these vulnerabilities which will make people care and increasingly there's a concern that the amount you care depends on whether or not your candidate got elected or not so is there I say as a journalist who writes about this stuff is there a communication problem and how should we be communicating differently about this yeah I do think there's a communication problem I think this goes back to the point of transparency to talk about the fact that our systems are vulnerable and then that's how you build resilience to it right it's the notion it has to be that people understand that vulnerability is a part of things but again the goal is to fix it and I think as journalists it's not it's a system like any other of course there are vulnerabilities and of course there are going to be circumstances where things could go wrong the point is that shouldn't undermine confidence in the entire system if there are mechanisms in place to do incident response if there are mechanisms in place to mitigate and I think that's actually really important from a journalist perspective because I think unfortunately some of the communication that you get is the big headline is the thing that actually undermines confidence confidence needs to be earned we should have election systems that are publicly verifiable so that any voters can check the accuracy of the counting of the votes we should have systems in place that are entirely open so that people can see what's going on we shouldn't just say oh don't scare people I've heard that too many times yes I want confidence undermining confidence is a big problem in elections but we make it easy to undermine confidence when we have everything in shadows so that nobody can tell we need things to be open and publicly verifiable to elicit confidence in elections I think until we get there at the very least it's pretty confidence building if you start actually talking about the vulnerabilities that were found and posting them somewhere and then saying that we fixed the problems I think that's a good thing and I think we're always so squeamish and scared to admit that we had vulnerabilities and to talk about what we did to actually remediate those issues and I think other localities can learn from the vulnerabilities that exist elsewhere that are discovered and see is absolutely key thank you very much we're going to hang out 4 minutes or so and then somehow get 8 people on stage and then be ready for all of your questions so please prepare them alright folks it's 3 o'clock okay so now we have 8 people up on the stage and we've decided to do a public-private partnership and alternate public and private my goal, you've heard enough from me my goal is for you guys to ask all the questions from this point on so what have you got I see in the back we don't have a mic so we're in the center in the back and we don't have a mic so please project and I'll repeat questions too everyone got that okay some of that is pointed a little bit at me I don't have a good answer but yeah it's a real problem we put election officials in a terrible terrible position where we say oh there has been a vulnerability discovered in your certified system here is a patch but if you install the patch your system will lose a certification this is insane we need to have a better way of making what are called de minimis changes so that patches can be installed small updates can be made so that we don't put election administrators in that terrible position and I'll jump in and say that the group that works on those they are today voluntary voting system guidelines and one V should be removed someday but the group that works on that they're actually moving along that path to where it's componentized certification and de minimis changes so you can actually make adjustments through the life cycle of the equipment 15 years ago and assuming that it's still fine today questions? in the back there so it's a little bit more difficult in application than that we have a good example here on the stage where Colorado does almost all their voting through the mail and in Nevada we do almost all of our voting in person and specifically early voting in person and doesn't lend itself easily to a kind of one size fits all voting system but not say that those problems can't be overcome but it's not as easy as just here's the national system that we all use and know and that's what we use every jurisdiction since the election is so decentralized every jurisdiction has adopted a little bit of different approach to elections and then have developed systems that complement that approach but it's a little bit tricky of a prospect to have a national solution if you don't mind I'd like to invite people to think in your mind what a streetcar looks like in the US there's sort of this sense of it's got a sort of bulbous rounded front there's sort of something that's a streetcar it used to not be that way it used to be that every city every jurisdiction had its own streetcar it was expensive there were safety problems and there was a presidential commission that was formed to standardize on this now it wasn't a requirement but here is a standard and probably in the voting systems industry maybe did not one but maybe several standards and it's not just one company anybody can build to these standards but once we have maybe three or four standardized designs we could actually get economies of scale that are much better than what we have today and that might be a much, much better way to go Jay, you sort of first pose this question do you have a position on it? Yeah, absolutely I actually think all voting should be done online why do we still do it in person and through them out that's ridiculous I think there needs to be an easier way to vote that is deemed, you know, I shouldn't say 100% secure but as secure as possibly can be that's completely open maybe even open source that everyone can look at and everyone can vet and validate the security of that If you said it's as secure as possible that's not consistent with what we're going on with well, hey, if we're able to transact online online banking, you know what? Hands up for online voting I got see they told me to be controversial up here so I think there's a certain level of risk that people are willing to accept for online transactions I don't know what that number is but I don't think the public is willing to accept that level for voting at this point Here's the critical difference right now if you shop online and you get the wrong thing you can tell if your bank account is double billed you can tell and you can fix it you can fix these things if you vote online and your vote's changed you can even know we can actually we can do that but we're not there yet and there are many other online threats, we can build a system that mitigates some of these threats but there are some online threats that we just don't have solutions to the first of which is client malware what if you're voting on from home on a compromise device that makes everything look great and you can tell we can make it so that you can check elsewhere but most voters won't and that's one of the problems targeted denial of service is another problem there are numerous problems that we just don't have good solutions to it's just premature to go to voting online audience questions one up front here sure I'll take that so the Nevada Democratic Presidential Caucus which will take place on February 22nd next year will include what they're calling a tele caucus or virtual caucus matter of fact I met with the state party officials earlier this week on Tuesday to talk about this issue state and local election officials have nothing to do with the party run caucuses it's their private organization running their own nominating process and we don't provide any support to that process at all I mean we'll help them if they need but it's not an official election I wouldn't advocate for voting over telephone right now but they have they have gone the route of being trying to be as inclusive as possible sometimes security and accessibility are at odds maybe all the time there are a lot of times they're at odds with each other so trying to find that balance and they've kind of come down I was not involved in those decisions but come down on the side of the accessibility component and I think sacrificing some of the security there any other thoughts on that okay right in the front over there the question was about how to mitigate influence campaigns yes so what we have if you go to our votechurer.sos.ca.gov that is our one stop a website regarding election information we also do a lot of campaigns not just the digital but printed campaigns and we have a group of people a team of people that is very specific on misinformation we have like I mentioned earlier we do have a team of individuals that monitor the internet and then also basically call the social media if there's we found stuff that's down or not so our election cybersecurity team is responsible for that effective communication is a key and also information sharing with our DHS partners with our state and other local governments also between our state like Colorado, California we always look at each other we look at what they've done and not reinvent the wheel so information sharing is super important but if you need more information I can take it off the line votechurer.sos.ca.gov and I'll add on that from the CISA side so obviously that was one of the prongs that we saw in 2016 and we wanted to as we talked about again some of the services that we provide both on the proactive side and the incident response side we have a whole separate looking at the information operations so you may have seen in the lead up to DEF CON and Black Hat we've put out and again supporting our state and local jurisdictions to help them as they put out information on countering foreign influence a campaign called the war on pineapple so we're using a non divisive at least what we think do you like pineapple on your pizza and start to again start that conversation to say hey this is how again adversaries are sowing that type of discontent these are the things that you can do about it and again using again our state and local officials whether they're doing it on their own or again using some of the material that we have out there to get that information out then separately again on election day I think we talked about the election day situation room that we run again in partnership with the EII SAC and again with the 50 states the state and local jurisdictions if we're seeing items again on the information operations whether it's a Twitter post we had at the last campaign those social media companies side by side with us so again that state could bring that up in the situation room and like that again they're getting attention both from those social media companies at that I won't do a pineapple pizza poll but you're all welcome to tweet your preferences to all of the CISA Twitter accounts I've got a question right there in the center great question so that number one that is what we're doing with both the multi-state EISAC and the elections infrastructure EISAC right so again that is your public-private partnership that's funded through DHS CISA but again working with again our industry partners all our state and local partners whether that's information sharing and making sure these again the services that we have to provide are out there also talk about separately we have what we call when we look at critical infrastructure we have and again as I talked about earlier 16 that we consider critical infrastructure the way we work with these communities through what we call government coordinating councils and sector coordinating councils right so I always think of them as a pseudo board of advisors looking at best practices so that is set so I can tell you the government coordinating council has 27 folks on it those are from state and local jurisdictions and then the sector coordinating councils are industry partners right and again they're meeting anywhere from bi-weekly to weekly obviously the NAS the NAS said all our partners are there so when there's best practices out there right I think there's goals that the government coordinating council has said they want for 2020 including you know full paperless you know balloting but by then I mean those are you know some of the entities that at least on the CISA side we're supporting our state and local partners with questions right there everyone's still hearing all these questions right thank you everyone by the way for asking real questions and being pretty succinct about it so I can take your question in reverse order so first of all I think that there needs to be when we're procuring these systems and we have these manufacturers building them for us I think the onus is really on them to patch the security issues as part of their contract in perpetuity like if there is a vulnerability that exists there shouldn't be a question of okay who's gonna pay for the fix like they created the problem they need to fix it so to your first question about how do you kind of do bug bounty on systems that are this sensitive you know you don't want to necessarily expose them to Russian you know actors because if they find the issues and we don't and they don't disclose them like that can be a problem so I think there are definitely ways of doing this I mean it's kind of what we the genesis of our company at CINAC we kind of I don't mean this to be like a marketing pitch but we can do this in a highly controlled manner where we vet all the resources they can be US citizens only they can work all from our infrastructure we kind of can log and audit and monitor everything that they're doing but still get the benefit of more of that crowdsource type of approach and so I think there are definitely ways of doing this in a highly controlled and trusted environment and then maybe you take this in phases and eventually it becomes more open open vulnerability disclosure program but scope is very complex because it's so broad right you're talking about I mean we talked about all the different spirit systems that are out there and the different types of voting systems that are out there so I think it's a huge challenge but would require a lot of a lot of partnership you know kind of unilaterally across almost every single state yeah so I spent some time in the biohacking village yesterday as did some of the other kind of election official type people that are here at the conference you know we spoke to some of the folks who've been around the village since its inception and I think it actually might be a really useful model because I believe that as it started you had Jay who hacked his own insulin pump and tried to report it to the manufacturer and got shut down they would not talk and now today you know you've got the manufacturers bringing their stuff in and they're putting it in there and they're working with the security researchers to find those issues so they can then address them I think in several of us that were on that tour that's actually a really useful model because today it's a pretty antagonistic kind of arms length relationship voting machine vendors in the security community and it needs to change and I think learning from some models that have been adopted in other industries I think would serve as well any questions? Got one there No I mean I can't really speak to what DNI is going to do there what I will say again it is a whole of government effort which you've heard me say so obviously CISA has its part working with our state and local election partners and industry but obviously the intelligence community, the IC the DOD again the whole part of the federal government here to support our state and local partners and this is just a chunk of it a lot of what the election ISAC is sending out through DHS comes from the intelligence community right? That's absolutely right and again a pitch so the EISAC sits on our cyber center floor so again that same information that's being shared with our intelligence agencies that are there sitting on the floor as well so the information shared and filtered out again through the EISAC sometimes again there's so much coming in and make it more specific so the election officials can use that Questions? One right there What if the voting machine companies go out of business and it got five more years? The consensus seems to be yes I think it's a perfect example of why maybe there should be some kind of what you were saying before just a few standards that everyone should follow Other questions? One thought on that actually I think we often talk in if you think about procurement of IT you think about life cycles and there's no reason that voting equipment shouldn't have the same thing and I think that's really the point it's not that you have it into perpetuity it's that you have a life cycle for it that is part of the contract so that's not my expertise at all but that's just my two cents Questions? One right there Alex, can you give us the 30 second Albert Sensor Yes Devices that are deployed on networks to actually provide signaling back to headquarters for assimilation aggregation and then further analysis so they can warn other people so if they see something bad happen over here in some state that has an Albert Sensor they can actually inform everyone else to be aware of the same have their eyebrows up and all that sort of thing so I think Albert devices are a useful thing for collecting information about attack patterns attack sources and that sort of thing Albert's don't do anything other than give you some situational awareness on things you may be seeing it's really valuable in terms of information sharing and situational awareness it's not the answer I mean you need to have defense in depth there need to be a lot of other things in play in addition to Albert sensors in order to diminish the value of knowing what other potential targets are seeing to equip yourself better but it's not the only answer Alex, you're nodding at that This is why for me it's so important to have external verifiability this is why I keep talking about public verifiability, black box verifiability sometimes it's called end to end verifiability but the idea is that you should be able to check the results of an election be confident of the outcome without having to trust that the equipment is good you can do it black box don't worry about the equipment you can still check the election and I'll just wrap that up with of course and as we talked about layers of security it's just one of the resources Albert sensors that we deploy out there to the state and local partners to support them together, the red team pen testing so we can go on and on even working with industry and what they bring so I think it is one sense but it's a layered defense and if there is something that can be found let us know and then we'll take a look at that Questions, one right there so the question is about an insider threat basically right we can detect that too you can detect that too and yes any tampering whatsoever an election can be detected insider tampering outsider tampering any kind of tampering is detectable well there is technology that will make this detectable we're building that technology out now and sharing it with vendors hoping to get them to adopt this yes it doesn't exist in the currently deployed technology we have to get administrators to use it and get it out there if it's in some places but not others there's going to be a lot of pressure on those places that are not using it to do it Anyone in government want to talk about this insider threat corrupt election administrator question I mean I'll I'll talk about it from what we're doing from the state and Nevada perspective it absolutely is a threat the insider threat we spent a lot of time talking about the nation state factor and what the nation state can do but we're just as concerned about the insider threat I'm like the least techy person probably in this whole room so I don't want to be preaching to you all about how this works but you know we've kind of used the the standard of like lowest level accessibility you know and so we started working on our voter registration database where there's like user access levels where not everyone has the same levels that you need for your job and then tiering that up so that we can mitigate some of the risks that come with the insider threats but it's just one more thing that election officials need to be concerned about So time for probably one more maybe two Sorry, actually before we take a question I'd like to take this opportunity to plug I know that I'm excited that everybody's here interested about election but if you want to do your civic duty help the state we have a lot of IT and security open positions and we suggest you know you don't have to go with pay go work for the state that will help, help us a lot because right now we have positions and we have funding but if we don't have you in the state government then we can't do much so that's my plug Ok Alright Last question in the back Oh yeah, so the question was the Kim Zetter's mother board story found three dozen instances of wireless connections that shouldn't have been there how can you maintain confidence if those have been online for three years despite everything DHS has done is that fair? Ok Well and I think you know I might have answered that earlier on the first panel there but you know as I've said again since we started this in 2017 you've seen the steps that have been taken right working with our state and local jurisdictions across both cyber and physical services so but the work is not done I think and again those stories and these vulnerabilities and again officials that are here this year and I'm sure we'll grow next year and everything you've heard on this panel it's just going to grow and we'll be stronger again as an election community for that so I think you know in summary without you know running through everything that we talked about over these last 90 minutes we're definitely further along than we were and there's more work to be done and that's why we're here Anyone else? Thank you Alright alright Last question in the back Thank you very much Thank you for the wonderful questions