 Good morning. My name is Maurice Turner as you know the election may be over, but its shadow still remains For the last year election security has been a leading concern for governments and constituents Pitt practitioners have been right in the thick of it working to staunch disinformation Protect voting infrastructure and to make a path to make the polls into the polls easier Today our speakers are going to discuss the role of Pitt and election security What we've learned during this election cycle and how we can make use of the information in the future Again, my name is Maurice Turner. I'm an election security expert I've recently been working at the election assistance commission But I've also had robles in the nonprofit sector as well. So we're going to jump right into this Let's start off with our first panelist Robin Carnahan. She's a fellow at the Beak Center for social impact and innovation at Georgetown University She's the former Secretary of State of Missouri and founder didn't led the state and local practice at 18f a Tech consultancy inside the US government general services administration She's a nationally recognized government technology leader and in 2017 was named one of the federal government's top women in tech Her lifelong passion for improving how people experience and value their government Inspires her to work helping governments get the digital tools They need to deliver digital services to people that are better faster and cheaper as Secretary of State Carnahan co-chaired both the elections and securities Committee of the National Association of Secretary of the state and today serves on a number of corporate and nonprofit boards Including the National Democratic Institute for international affairs where she co-chairs the technology committee and has led and participated in international election observer delegations in more than 20 countries We also have Bruce Snire Bruce Snire is an internationally renowned Security technologist called a security guru by the economist He is the author of 14 books including the New York Times bestseller Data and Goliath the hidden battles to collect your data and control your world as well as hundreds of articles essays and academic papers Is influential newsletter cryptogram and blog Snire on security are read by over 250,000 people Snire is a fellow at the Berkman Center for Internet and Society at Harvard University A fellow at the Bell for Center at Harvard's Kennedy School of Government and a board member of the electronic frontier Foundation He is also a special advisor to IBM security and the chief technology officer of resilience Next let me introduce Rich DeMilo He's the chair of the School of Cybersecurity and Privacy and the Charlotte B and Roger C. Warren professor of computing at Georgia Tech He was formerly the John P. Imley Dean of computing Physicians he has held prior to George joining Georgia Tech include chief technology officer for Hewlett Packard Vice president of computing research for Bell communications research director of the computer research division for the National Science Foundation and Director of the test soft software test and evaluation project for the office of the Secretary of Defense He has also helped faculty positions at the University of Wisconsin Purdue University and the University of Padua, Italy His research includes over 100 articles books and patents in software and computer engineering Cryptography cybersecurity and theoretical computer science in 1982 He wrote the first policy for testing software intensive systems for the US Department of Defense DeMilo and his collaborators launched and developed the field of program mutation for hardware for software testing He's a co-inventor of differential fault cryptalysis and holds what is believed to be the only patent on breaking public key crypto systems He currently works in the area of election and voting system security His work has been cited in court cases including 2019 and 2020 federal court decisions declaring the unconstitutional use of paperless voting machines He has served as a foreign election observer for the Carter Center and is a member of the state of Michigan election security commission He has also served on boards of public and private cybersecurity and privacy companies including RSA security and secure works He has served on many nonprofit and philanthropic boards including exploratorium and the campus community partnership foundation He's a fellow of both the Association of Computing Machinery and the American Association of the Advancement of Sciences Jake Braun is also joining us today He is executive director of the University of Chicago Harris School of Public Policy's cyber policy initiative where he works at the center of politics technology and national security to advance the field of cyber policy He is the author of democracy in danger how hackers and activists expose fatal flaws in the election system And has co-authored two award-winning and seminal works on the election infrastructure cyber vulnerabilities In addition to his role at Harris, Mr. Braun is co-founder of the DEF CON voting machine hacking village in the president circle on the Chicago Council on Global Affairs and a strategic advisor to the Department of Homeland Security and the Pentagon on cyber security Prior to joining University of Chicago Harris School of Public Policy faculty Mr. Braun was appointed by President Obama as White House liaison to the Department of Homeland Security and Served on the presidential transition team for the Obama administration as deputy director for the National Security Agency's review He holds an MA in international relations from Troy State University an MA in education from National Louis University in Chicago and a BA in philosophy from Loyola University of Chicago Our panelists have quite a depth of experience in this field and we're going to jump right into What does it mean for elections in 2020 in America? How are they different from what we've seen in the international democracies around the world? And how do they work Robin? Well, thanks very much and thanks to everybody for joining us today It's been just over a week since the biggest election turnout we've had in in uh decades In the middle of a global pandemic. So it's a rather remarkable place that we find ourselves in This was set up to be a hotly contested election even before the pandemic And if you think about how elections are run, they're kind of a miracle In and of themselves because you have, you know, more or less 150 million people wanting to do the same thing on the same day And if you think about it like a business, it's as if all your customers might show up on one day And you know, they won't but you have to be prepared in case they do and You're having basically volunteers or part-time workers that are the ones who are treating those customers everybody Wants to understand the rules but average age and is is 72 of poll workers in our country They have very limited training Um, and the rules are changing all the time and the machines are changing Voters want to get in and out in 10 minutes ballots are long And the press wants to get results 30 minutes after polls closed So if you think about all of that in a normal election, it's a little crazy to think about And then you layer a pandemic on top of that and it makes it really mind blowing that we got through this as well as we did Because we needed to have both safe and secure elections in the middle of that pandemic Which meant people wanted to vote in new ways So rather than just on a tuesday People wanted to be able to vote at their homes or people wanted to be able to vote prior to election day So they didn't have to stand in long lines and be subject to to uh getting infected so That meant for election officials huge logistical challenges of basically running two types of elections one in person Both before election day and on election day and the other as a vote by mail election Some of the states four or five states out west have been doing vote by mail for a long time It's taken many election cycles for them to get comfortable with that But almost every state this year moved to it immediately And this was huge because it meant in some places, you know, 3000 percent increases in dealing with mail ballots Something that they just didn't have a lot of experience doing or the machinery or the staff to handle so Um Combined that with all the last minute lawsuits and all the stresses Uh, I think we should be super proud of both the voters that turned out in record numbers But the heroics, I would say of our election officials Who really pulled off a safe and secure election in the middle of a very unusual time So those election officials have had a lot more work to do this year compared to previous cycles What are some of the tools and training that they needed to have in order to be able to accommodate all these new ways of voting with the increase in voter turnout Yeah, so um it Election election officials know what they're doing, right? They they know how to deal with voters They know how to run their elections But what they were forced to do this time is recruit a lot of new poll workers, right? Because if you have the normal people who are working at the polls in a vulnerable category They were not going to show up and so that meant recruiting a lot of new people training those new people And getting them in spaces that were going to be safe both for the poll workers and the voters So that was on the in-person side And then on the vote by mail side It was just an awful lot of new process about how as a voter do I apply to get my ballot? How is that ballot tracked to make sure that it gets to the right person? How is it tracked when it's back? How does a voter know that it's been returned properly? How do we had a lot of conversation this year about the post office? And and and whether whether ballots there were going to be delayed or not So a lot of election officials set up drop boxes to receive those ballots Not only was that a thing that they could have more control over It didn't cost money for the voter to put that ballot in the mail. So Yeah, there were just an awful lot of new logistical challenges that Election officials were up against and as I said, I think I think they did a great job So the expansion of ways that people can vote Seems like they were popular in that we had a record turnout using this variety of ways to vote What are some of the concerns you have about if some of those expanded voting options stick around That they might become it might introduce more risk into our electoral process And what can we do to help mitigate some of that risk? Right. There's a lot of question this year about whether it was safe To vote by mail whether that was going to introduce fraud into the process Look voting by mail has happened for for years and years We've we've been voting in america for more than 250 years We've been voting by mail for many years and in some states they voted 100 by mail For decades. So we know that it's safe. It does require making sure your voter rolls are up to date And so that's that's something that we need to invest more Time and money in to make sure that those lists are as clean as they can be So when we know who's on the voter list, so I'd say that's the number one thing But number two Having it be easy for people to get those ballots, right? In some states there were still requirements that you have either a witness Or a notary Also validate when you do a vote by mail ballot There's very inconsistent rules across the country in some places It's convenient and easy and safe and in other places. It's very difficult And I think one of the big challenges going forward is to bring some more consistency for americans across the country To make sure that they have these options. They clearly like them And that it doesn't have to depend on which state you're in whether it's easy or hard to vote By the way, can I just tell you that This is one of my big pet peeves. Do you all now have any Any idea why we vote on a tuesday in november? A bunch of election geeks. You should be able to answer this question I got nobody you got nothing. All right, so Here's the story tuesday in november was set up a long long time ago because it was convenient Why was a tuesday convenient? Well, it was an agrarian economy And during an agrarian economy, you don't want to have it during planting season or growing season or harvest season So in most parts of the country november was a good time. It was after all of that So november made sense and tuesday made sense because people moved around in a horse and buggies And you didn't want to make people travel on sunday, which was church day and Wednesday was market day So tuesday was convenient because people could leave on monday. It might take all day to get to the county seat to vote So they set it on a tuesday It's obviously not convenient for people anymore and we ought to be thinking of other ways to make it convenient for folks to vote Well, the first step in voting is typically going to be registering the vote And now so many states have moved to online voter registration And so having a database with that much information that is so critical to the process is obviously Attempting target the tempting piece of election infrastructure that needs to be protected Bruce, can you give us some of your thoughts on what does it mean to be able to protect That kind of data but also make it accessible so that the election officials can actually have access to it And the voters can actually have convenient access to be able to register And registrations is an interesting process Many countries don't have it when you become voting age or automatically eligible to vote Registration itself is a holdover And to uh to suppress votes That it's another barrier that people have to overcome Uh, certainly disproportionately affects those who are poor don't have time And right it is a database of people who can vote You know us we have a lot of of databases of citizens. You can think of state databases of People who can drive uh national databases social security databases other databases for benefits So it is one of those databases and yes, it's vulnerable right the opm database was was hacked by the chinese a few years ago We know that russians have Infiltrated all seems not to have done any modifications to voter registration databases in the us Up there's really a lot of good work done by latanya sweeney and others. So she's professor at harvard looking at the security of those databases from changes So one of the ways they can be attacked is Someone can go in and change your address your party affiliation which will which will affect primaries And we're trying to balance convenience people want to update their address online Want to update the party affiliation With with security and that and there are risks there So far in this year, we were worried that there would be a tax against voter registration databases Which could result in chaos or you go to a state and the voter registration databases don't work So many states have paper backups and other manual systems to try to uh be ready for those kinds of attacks So it is Something we have to secure It is certainly a risk. It's a worry when I think of the risks of voting registration databases is one of the major risks Uh, we were successful. Perhaps we were lucky This year, I think we'll learn in the coming weeks months and maybe years if anything was attempted and afforded But yes, you know when we have a A database of people Who have a privilege There's always the risk of that database will be hacked by people who who want to abuse the privilege or deny the privilege or just so Discord and confusion in who has that privilege So what are some of the ways that either local election officials or other organizations can help Strengthen that infrastructure so that they're better protected against attacks Against the voter registration databases or the voting machines themselves You know, it there are things you can do but honestly a voting official in akron, ohio against the government of russia is not a fair fight You know, so we are we do have this very distributed election system in the united states, right 51 Individual legal jurisdictions local jurisdictions where the rules are different Where the defenses are all local and we're expecting them to defend themselves against nation states So we can talk about computer security and things you can do But really the best thing we can do is have some sort of national standards some sort of national systems Because the threats are national the distributed local nature of our elections was really well suited to the threats of the mid 1800s And you know, it is secure largely against those threats. That's why we don't see the kind of retail voting fraud That's often talked about people going in and voting twice voting for other people And that kind of stuff Tends not to happen when it happens. It's very local. It doesn't change anything. It's discovered It's it's the major threats. It's the nation state threats the computer threats Against the machines against the systems against the infrastructure And there, you know, we are seeing, uh In the united states government us cyber command In 2018 they did something offensive to thwart an election Uh fraud attempt by russia. We don't know the details, but it seems to have involved going into the networks and computers In the russian organization that was planning whatever they did and and shutting things down And that's the kind of defenses we need and you're not going to have a local official do that I remember sharing a stage recently With the person in charge of a state election talking about the briefings she got From her state's national guard on phishing attacks And that's well and good, but again, not a fair fight Yeah, can I just jump in and I want to just double down on what bruce said it's it when you have things that are national security threats done by nation state actors you can't expect Some local election official to to be up against that And it makes total sense to have a national response. The good news is that after the the threats that we saw in 2016 homeland security and Cyber command really did step in and begin to support state election officials and local election officials in very new ways That had never happened before They get credit for doing that. I want to point out one other thing and that is that when we think about elections and the ability for Bad actors to get in and change things You know, the easy way to do that is through the internet, right? So it's the it's the it's the features of the election that are connected to the internet That make it convenient for voters when it's connected to the internet But also makes it convenient for bad guys and that tends to be a voter registration databases And election night reporting type sites the the voting equipment itself The machines that you vote on and how those get tallied are not connected to the internet They are very distributed and it's a very, you know Arduous and step by step process to tally those votes But that's not something that's that is connected to the internet and therefore has a very different threat profile than those things that are Well, it certainly sounds like we're adding a lot of technology To the election process, but we're not necessarily adding a lot of Cyber security staff to that. So jake, can you share with us? Some of your thoughts on what it means to be able to support those state and local election officials specifically when it comes to cyber security Sir, hi, um, and thanks for having me. Um, and by the way, let me uh, first just uh Um shamelessly, uh, you know, thank our our hosts for doing this and for what you guys do to support academia um A little bit off topic from your question, but just to address You know the news from yesterday Uh, you know We of course had the president come out and make this a completely asinine claim about the machines being hacked and changing all these votes And then, you know, unfortunately we had I think dhs come out and make You know and nearly as equally asinine claim that this is the most secure election in u.s history When as you're saying there's all this technology that's added You know, obviously the databases can be hacked the websites can be hacked We had elections 100 years ago before there were databases and websites You know and and presumably things were actually at least more cyber secure back then and so, uh you know We we then of course have the far right who immediately although they've been saying election security stuff is fake news for the last four years Agree with the president and the far left to do a 180 That have been saying, you know election security is is uh, um, you know All swiss cheese and immediately agree with homeland security just because they disagree with the president Um, and I think what's really important is what academia does and what you guys are supporting Which is we take a long view or we try to we try and say, okay Let's gather the facts and let's um and let's do some research and see. Okay. Is it more secure? Is it not what do we need to do to make things more secure? Or where are our holes and so on and so forth? And so, you know, I do think that academia has a serious role to play here And I think, you know, you're guys in support of academia in this conversation That is not driven by a media cycle is is Appreciated and and important Um, so to to get to your into your actual question Um, uh, you know one of the biggest challenges that election officials have Um is workforce and it's and it's not because of them. It's everybody has this this problem in cyber security. There's um, the last estimates I saw said about 500 000 person shortage in um Cyber jobs that are unfilled across all industries in the united states about One or about 3.5 million globally shortage of of uh cyber professionals to open positions and so you have uh, you know, somebody in Alamaki county, Iowa competing with jp morgan and google and amazon and nsa and and cybercom for cyber professionals and It's incredibly hard to compete for a whole host of reasons. Uh, so what we did is recruited Um, uh, several hundred in fact, we had to actually shut down recruitment after a certain point because there was such interest um of cyber experts Uh, and got funding for multiple organizations Um, so that we could do background checks on them and a whole host of other things and provide kind of pro bono cyber security support to election officials Um, uh to to help, um, you know Do some of the just basic cyber hygiene uh that needs to be done in in so many of these systems um, and so for example um, you know, we had One election official and we take, you know, their their privacy and and and, uh NDAs and stuff very seriously So I won't say who or where but we had one election official who came to us and said just basically look You know, the state controls a lot of my infrastructure and the county controls a lot of the rest of it But I know I should be doing more cyber security. You know, I just don't know what that is. Can you tell me? um, and so we um connected her with some folks Who were able to go through like real basic? um cyber security, uh um Protocols that she could put in place and then we had somebody else who was um, really trying to look at um addressing um, secretary carnaghan's concern about uh voter registration databases And so he was looking to move his into a more secure cloud than where it was put blockchain Um on the database we can debate the merits of blockchain and so on but I think it's more secure than what he had before Um, and so we had some folks who helped him do that in the most secure way possible um, and and so, you know, and then kind of everything in between so you have the the people who are super Uh tech savvy trying to do, you know, really interesting things Down to somebody who's just saying well, can you just tell me what what security I should be I'm doing at the most basic level so I so I can try and do that and um You know, we were really really happy with the results and and look forward to to Expanding to more um jurisdictions in the future Because there's this this election, uh, sorry the cyber workforce problem is not getting solved anytime soon um in any industry not just elections, uh, the banks And uh military will continue to struggle with this for decades as well Everybody else is trying to hire cyber professionals. And so, you know, we're just trying to to um Help deal with it for for election officials as as we've done Well, that's certainly an interesting way to help address that workforce issue And of course that starts at the other end of the pipeline How do you get people interested in cyber security, but also interested in doing something for the public good? I think this is where uh rich could probably have some information for us But how do you actually recruit students to be able to work on these very interesting problems? Please give us an example of that Sure. Thanks. Um, and thank you for the invitation. It is everyone here today. Um, so we we just started A school at georgia tech um to to meet the the demand that students Have for for training in careers in in cyber security and you know, some of this is driven is driven by Driven by digitalization that takes place in in the commercial sector in the government sector We submitted a proposal to uh To for a bit un project this year to do covid safe secure polling places and and kind of made a call to to students to To participate in in that we had to shut off the call after we got 30 students We just had no way of handling the size of the of the of the turnout that we got and so so what What that tells me is that is that there there is there is a there is a There is kind of a student-led movement out there to look at at At societal impact of technology and and to do that in an election year particularly this election year Just just blossomed into Into a project that that was much bigger and much more much more widespread than we than we when we thought You know, we had we had some some support for the for the students, but to be honest that Most of the students that are in the georgia tech safe and secure elections project Are doing it either for credit or for for the Experience and and I think that that's a resource to be to be relied on I've seen over the last Last 10 years or so the evolution of students even at a very very nuts and bolts play Nuts and bolts place like georgia tech Students becoming more and more aware of social justice as an issue more and more aware of equity As an issue and and this all just came to a head in in 2020 Well, georgia has certainly seen its share of changes in the way that it actually holds its elections Can you explain a little bit about how elections in georgia specifically are different this year? And what are maybe some of the the challenges or concerns that you're seeing on the ground as a result of those changes? So this is this is not the fight that I thought I was going to be having in in georgia when um when we started Looking at at the new generation of ballot marking devices that that georgia was considering back in 2018 We we thought we were going to be fighting a battle over hand mark paper ballots versus machine mark paper ballots and and voter verification of of ballots and and We've been fighting that battle for for for two years. It's just grown pretty contentious here in the in the state and lo and behold The world changed And and the things that we thought were going to be important turned out not to be so so important I will tell you that that that about half of the ballots cast in georgia This election cycle were hand mark paper ballots And and it had nothing to do with with with with our well Maybe had a little bit to do with with with our with our advocacy for for safe and secure election technology But it really had to do with with with the way that the assumptions broke in this in this election We started calling voting in the state omni channel voting Um 18 percent of the votes In this election cycle were cast on november 3rd 18 percent Which means what which means that that that early voting kind of a mix of of of in person voting at at polling centers and And hand mark paper ballots the absentee ballots in the in the state Made up the bulk of the of the way the votes were were cast We were planning we were planning on a on a on a A massive turnout In atlanta After the the june primary, you know, there were new york times articles showing these long lines People waiting in line eight hours ten hours to to that's that's what we thought we were going to be going to be facing Um Meanwhile meanwhile voters looked at that and said hmm. I can see five other ways To spend my time then then waiting in line for for ten hours and they took it and they took it We had we had the private sector stepping up the the The atlanta hawks made state farm arena Available for early voting that turned out to be a really really popular Choice for for a lot of a lot of voters We had We had drop boxes. I think I think Robin mentioned the appearance of drop boxes. Those were really really popular So as people began began to hear that the mail would not be necessarily The safest way to cast their vote they drove 15 minutes to the nearest drop box and and use And use that So so all the models that we built and we spent we spent a lot of time modeling voter turnout and and wait times All the models that we built for the 200 polling places in fulton county, which was our our focus area Resulted in really really good software tools that we will make available to to election officials But but it turned out not to be not to be where the action was this election cycle And and I think I think the the idea that that the scenarios that you plan on um Based on historical turnouts historical voter patterns historical vulnerabilities Needs to be reexamined because because we're not we're not seeing traditional patterns For anytime in the in the future. I mean we we will have a january 5th runoff here in In georgia and heaven knows what the what the situation is going to be like for for that But it won't be anything like what we've seen in in prior elections for this election cycle And then and then as you look forward we're going to have to make some conclusions and make some some reasonable predictions about where to put our resources going forward Well one option that's been noticeably absent Is internet voting so people using their own mobile device or computer to be able to cast the ballot obviously in georgia at least there was a lot of Voting that didn't happen on election day and it seems like we are going to have The impacts of covet 19 felt for at least another election cycle so people may be wary of showing up in person So is it now time to invest in internet voting? What does that look like? What are some other risks and how can we mitigate those risks? Were you asking? Yeah I'll just say no I'll just say no hell no don't even think about it. That's idiotic my god run away fast And we get a just just just in case just in case it's not clear. This is a really bad idea All right jaker are you the lone hold out here or are you going to jump on the bandwagon? No, I think I mean The challenging issue is there's like, you know military and someone who vote over email and stuff like that which is Horribly insecure and there should probably be a better option figured out for folks like that that are already voting online um but I agree with the group that there are serious security concerns in terms of doing this widespread But we should find a more secure option for those that are already doing it So this is actually important to talk about a little bit, right? I mean, so you asked a bunch of security experts that all told you not that let's work on it They're a trade-off that told you don't even consider it And this is not obvious to the average person because they'll ask why Why can we bank online? Why can we do all these things online securely? We can't vote So I want to take a second to explain The problem is the anonymity of the ballot That if if there if we didn't have voter vote anonymity voting online would be trivial That it would be like banking online And you could audit it if there are problems you could undo it Or you can make it public you can check to make sure it's accurate But the the need for ballot secrecy Means you have to have a clean break between whether you're allowed to vote and your vote And because we can't even begin to trust the security of the objects you're voting on The phones the computers the software We can't make it secure and even worse if there is a problem we could never know Because we can't audit in the same way you could audit banking Where all names are known all transactions are recorded. There is a database of what happened So we can't do internet voting You can't do voting by facebook It just isn't secure in ways that could be catastrophic and can't be detected If I can just add to bruce's comments So the the idea that that you should be able to vote using your your mobile phone or Logging into a web browser is based on a false premise It's based on the premise that that banking is secure It's based on the premise that that that these things that we think are secure are secure But my goodness you pick up any newspaper in any any medium-sized city in the country And you're bound to find an article about some ransomware attack On a on a hospital or an individual or or or something that has caused people to To lose something of value to them because they conducted that transaction On online and and it's not that these are anecdotal stories The banking industry the financial services industry builds loss into its business model the financial services industry Subsidizes the losses in online transactions Just like they used to subsidize the loss in in in bad checks that were that that were passed You don't have that luxury in in in elections There's there's no one that's going to subsidize your lost votes If it's five percent of the votes that that are are damaged In online online transactions those damaged votes are gone. There's nothing that you can do to To to recover them. So I I think I think one of the things that we can do It is start the discussion in an honest plane And and and and recognize that there is an inherent risk to vote to conduct any transaction on online and and and then If you want to discuss from that point on What might you do For voting you're starting from a point where It's an inherently risky activity Well, I think it's fair to say that No election is run perfectly you know, there are always cases of Interesting things or unusual things happening and that's what makes election officials, you know some of some of our greatest public workers is that They have to have backup plans for their backup plans because Things do happen. They have to expect that something will happen Whether it's a broken water pipe or it's a missing box or you know, the power goes out And so I think what we've seen in the past is that whenever there's any sort of an auditor recount There are some mistakes that are uncovered. So no election process is perfect But we do accept some of that imperfection because overall it does work very well so What level of imperfection are we willing to accept and moving to perhaps having some ballots come in over the internet It's less imperfection. So There are two types of failures. There's the the Failure to find that a recount this ballot was done wrong. This ballot was done wrong Right, they sort of the retail level the individual level But there are also systemic failures Right, you get to the end of the day and the internet says it's zero to zero or a million to zero or negative a million to negative a thousand You could have these complete systemic failures where everything went wrong So the computers failed differently than normal things Right, they all work perfectly to one day when none of them do And that's the kind of failures you're worried about when we say internet voting is insecure It's not that this person's ballot will be recorded incorrectly. It's that everybody's will be recorded correctly Right ransomware against the state Okay, can I just I want to just jump in there on on two points one is We talk about the banking system and how convenient it is for us to do everything online nowadays They do invest a ton of money In in all kinds of systems and security even imperfect as it is It is very clear that we don't invest that level of money in our democracy We got 400 million dollars last year out of the federal government for election security And then maybe I think another 300 million this year A lot of it came too late to be able to really be implemented And so one of the best things that we can do is have some dedicated funding stream To maintain our election systems and decide that it's worth it And not have this be this political football that always runs up to right before the elections And then you've had three months before the elections. You tried to get millions of dollars out the door That's not how you run a good election. You had it takes planning It takes investing in in your infrastructure and your people And last minute funding doesn't help that Where should that investment come from and who should that investment go to? You know, you you mentioned the several hundred millions of dollars coming from the federal government And then the other end of that is that we've seen a couple of smaller companies like democracy live or votes You know looking into their own version of internet voting We've also seen some investment from microsoft through their end to end verifiability research project Now, is this something where the money should be coming from? state local federal government and going into the private sector or perhaps even into universities to continue this research to get to a point where we can have a high level of confidence in internet voting Can I want to say one other thing about this whole who who ought to own? election infrastructure topic um I have strong feelings about this When you have a one of government's missions is to be able to run elections And outsourcing that mission to a bunch of private companies who may or may not Share all the same incentives is Super problematic We saw this year in some of these states that You know, there were problems in their primaries If many of them use the same election vendor, they're going to have similar problems in other states And without a lot of extra money They are locked into whatever this problem is and have their hands tied in their ability to to resolve it. So I'm for I'm for government owning their mission. Uh, and that means when it comes to elections Being the owners and being responsible for maintaining the systems that run those elections Yeah, and if I could just go ahead bruce. Sorry I'll start great quick. I mean the the market of internet machines the companies that make machines It's kind of a screwy market. There's not a lot of motivation to make them better to make them more secure and and uh, I mean, this is Right having this in private hands is causing us a lot of problems that that seemed to be needless Yeah, and I think I mean, you know piggybacking on what secretary carnehan said and also what bruce said earlier about It's not a fair fight between a nation state and uh um a county clerk you know as secretary carnehan said, you know, government should own its mission and the mission of um national security Is that of the national security agencies in the in the u.s. Government now? I think local election administrators can still and should still administer the election However, it is the federal government's role to fund And um help execute our national security and this has become a national security problem And therefore the federal government has to step up and fund the security. There's no um, you know tax base out there that For these local jurisdictions that will ever enable them to spend the money they need to do Anything get close to the security that we need and by the way a lot of this Requires as I forget who was saying if bruce or rich Deterrence where cybercom and ns. They need to go in and and make the price of attacking our infrastructure so high Um that the bad guys don't do it And and by the way, I think that that needs to go even a step further where the democracies of the world come together and say that any bad Nation or any nation state that attacks any democracy is going to face repercussions from a whole host of multilateral organizations as well as um You know potentially the u.s. Government or nato or or whoever um, and so Unless we're doing this stuff at this national level Again, not saying the federal government needs to come in and start running elections, you know At the local level, uh, then then we're just not taking this seriously So so there's a there's a history to rely on here. Um uh, the supercomputer market, uh, is a really is a really interesting analogy to to election so supercomputers were decided in the late 70s To be to be critical to national security. Um, there's not enough money spent nationwide Uh on supercomputers to be able to innovate at the level that you that you need to do it So the federal government invested in in r&d To keep the u.s. Super computer technology at the forefront of of um of um what could be done, uh, and and and There's no there's no corresponding Will or or initiative on um on the technology that supports democratic processes elections in in uh, in particular The entire i mean if you if you if you believe the estimates the the entire um nationwide market for For voting technologies 300 million dollars a year That's a really small number. Uh, and if you if you just did the the the the the net present value of a of a research proposal Into that into that marketplace you simply don't get A big enough group Funded on on r&d money to be able to innovate. Uh, and so so having the federal government step up with with a trusted source Uh of investment Regardless of who it goes to uh, whether it goes to to private companies for or industry I think is the only realistic pathway Pathway to get there otherwise you're going to continue to see what you what you've seen in the in the in the past um niche companies um manufacturing um components That have have low margins that that don't have any particular innovation uh incorporated in in them and and and because because they're the kinds of companies They they are pretty pretty inscrutable to to public to public questioning and and It's not that that there's not a way out of it, but but but you do need to have I think a sympathetic sympathetic federal source That's willing to listen to the arguments for why this is the right thing to do for the country So one group of voters, uh, typically overlooked, but it's still rather sizable would be the accessibility community And this is an area where we've seen uh digital technologies over the past 20 years Really make great advances to help people live their lives independently and privately Which is something that the health america vote act says should be the case in voting So i'd like to hear a little bit more about how accessibility technologies tie into The modern day voting process and what that means for security Bruce you've been muted which in november is an embarrassment Uh, I I can start it's I think that that is important problem we had after 2000 is that uh accessibility became a dominant A requirement and gave us very insecure Touchscreen voting machines. I mean there are ways to build voting machines that are both secure with a voter via voter verifiable people ballot and also Are accessible and provide options for a variety of different abilities for voters But you have to do that and again this speaks to the voting industry Which didn't have the incentive To build in both and after 2000 we actually made things worse By putting in all of these touchscreen machines and they are slowly being phased out in different states With machines that are more secure like have a paper ballot And also have more uh a diverse set of capabilities for people with different abilities to vote independently I I I think I think realistically speaking though Bruce the the amount of r&d that's gone into accessibility In this in this industry is pretty small In the studies the studies are are are mainly vendor Funded funded studies. So so you kind of have to set those set those aside the the What they tell you about accessibility flies in the face of of of other studies other other usability Usability studies and and this is one of the areas where where I think rather than then create a one-size-fits-all solution That promises more that it can deliver to To disabled voters Starting with with almost a blank sheet of paper and building up accessibility models That are really useful and and and make use of what's known about about technology and incorporates the best cognitive science Is probably the thing that we should be looking at how long do you think it would take Before an investment like that under those criteria would be able to make an impact and Actually show up You know in front of voters for an election Well, they're they're there already Disabled voters particularly voters that that that have that have visual disabilities Carry their own assistive devices with them And and and to a large extent the the the people that that that go to vote In in polling places either come equipped with with surreptitious devices that they use to to To figure out what's on the ballot or with people that they trust that can help them that can help them along in the In the process, it's not it's not it's not frankly speaking the major the major problem Wheelchair accessibility to polling places is still is still an issue You know, I I I went to You know 25 or 30 different polling places in Fulton county that promised Promised wheelchair access and turned out not to be the case Uh, and and you know the height adjustment that's supposed to take place on ballot marking devices to allow voters Who are in wheelchairs to be able to access? The the screen are either not functional or or the the polling place workers don't know how to Don't know how to activate the the equipment. So I think you have to take look the whole portfolio of what have a Promise versus versus what ADA? Let's says you should you should consider and and find a way to Mel the two of those two of those together rather than retrofit an existing technology to to something that may or may not be an issue so I'm going to switch gears a little bit and Move us over to a discussion on disinformation. That was a major concern over the past couple of years ever since People were able to recognize the influence that Facebook and other platforms have on our not only our everyday conversation But especially our political conversation What did you all see during this cycle in terms of attempts at disinformation? And if you could share what you think might have been the effectiveness of those disinformation campaigns Well, so I'm happy to start. Um, so I guess there's two things that I was far more concerned about I mean not even Uh, not even the same stratosphere as you know random tweets or whatever saying You know this side of the other thing um We're we're two instances one, um here in illinois where someone put up a um fake Facebook page that looked exactly like the cook county board of elections facebook page Um, this is actually reported by NBC Um, fortunately the county caught it and got facebook to take it down before they did anything with it But we know this is something that for example russia did or not The type of thing that russia did in the ukraine in 2014 where they hacked the actual the real government website To change the election results Fortunately, the ukrainians took a website down But then russian media had announced their candidate one when in fact he hadn't and caused all types of chaos So this attack that I think secretary carnan mentioned at the beginning on these these You know internet-facing Websites and so on that are the actual official government websites that announce the the unofficial results tonight of election Or for that matter polling places times to vote methods by which you're supposed to turn in your absentee ballot All that type of stuff um having those attacked I think um is is what i'm most concerned about and what we saw in the cook county instance But also um in the iowa caucuses. There was another even I think um, maybe more insidious aspect of this so Um, as we may remember the election night Reporting app crashed the party says it wasn't attacked and so we'll take them at their word. However Um The precinct workers for the iowa caucuses moved to the backup as they were supposed to well The backup was the phone and they were supposed to call in the results to the iowa democratic party headquarters um But lo and behold none of us knew this at the time on for chan Um, somebody posted hey Um, how about everybody call in the iowa democratic party headquarters? In this time frame and jam their phone lines up by the way if you get through give them fake election results And and so I mean this is about as low grade and unattributable of a hack as you can as you can do But we know that those phone lines were jammed all night the iowa democratic party actually even rolled their phone capacity over to the the dnc The national democratic party phone Center and then that was filled up with calls And and then we saw, you know, what ensued was three days of Basic chaos of the first nominating contest of the 2020 cycle and these attacks Um, that are essentially Disinformation attacks on reporting election results I think are were the two most disconcerting to me if that becomes a trend I think that that's really scary for For how our democracy can and will unfold in the future, especially because they're just so hard to protect I mean protecting a website is is nearly impossible and and these reporting apps and so on are are, uh, you know They're incredibly vulnerable Can can I just I've got I've got to like have one point here about this iowa caucus conversation Which is, uh, just reminding everyone that the caucuses are not elections They are party activities run by political parties. So The secretary of state and election officials are not involved in party caucuses And so none of the normal, uh election based security measures and processes are used in a caucus So I just I want to remind people of that just because it's you know, we we tend to conflate things when they sort of seem the same But a caucus process run by independent private political parties is very different than uh an official election So Totally true. And by the way, we should totally we should get rid of caucuses. They're terrible That being said with the exception of this year at least the democratic caucuses in iowa have essentially decided the nominee In every election since carter, um with the exception of this year. So but yes, they they're they are run by the parties. That's true You bring up an interesting point about Non-governmental entities having such an impact in our electoral process And that brings me to content moderation on social platforms. So think of twitter facebook google snapchat We've all seen how messages come through these platforms and whether they're Coming from legitimate sources and then are amplified by malicious factors or Their providence is suspect from the beginning They seem to be having some impact those messages seem to be gaining some traction And so as a result, we've seen An increased number of these platforms labeling such messages and even pulling down messages What has been your impression? over the past Month or two have seen that increased activity of platforms labeling content as either being suspect or coming out right and saying This is false. This goes against our policies and actually pulling those messages down or even Taking down the accounts themselves Well For one sec and like as somebody who's been on the ballot A bunch of times and been involved in a lot of campaigns Dirty tricks and lies are are not new right? This is a thing that we've had throughout our history In and happens in every country What is different about this is the scale and the speed in which those things can move and the anonymity and for me I say it's about time That these platforms take some responsibility for the things that they put up And and begin to moderate these things I we can have debates all day long about Whether there should be that much power But there there are ways that they can figure out When there are bots that are driving things or that there are fake news websites that have been put up That are based on you know Things that have already been used by the BBC or something else I was taught I was taught in 10 minutes how to put up for 20 bucks a fake news website By by someone and I'm not even a technologist These platforms know that they could they could do better at this and I say Thanks for getting started and we need to do some more And the research really is you know, Finn is a bad word. We don't really know how effective any of this is The the best work has come out of the Berkman Center at Harvard Looking at 2016 and found that this was a lot of misinformation a lot of things on twitter and facebook That seems to have actually changed no actual voters So we see this we are outraged by a lot of it. We don't know what's effective We don't know what is working our core doctor out had believes that Facebook doesn't turn people into racists turn racists into voters And that is another way of looking at it So we see these fake news these fake websites Misinformation lies. It's true. This is not new Speed scale and scope are very different But we don't know if it makes a difference I'm actually all in favor of platforms taking more responsibility for the things they publish But we really need a lot more research On what works what doesn't what's real? What's a moral panic? And it's easy to to to guess which is what most of us are doing I think one of the complicating factors has been that That sources that you would assume in normal times would be trusted Turn out to be not trusted not trusted sources. So so if you're inclined not to trust the government agencies tweets Or government officials tweets Um That may affect you but it's not going to affect people that don't know that that that's not a trusted trusted source Um, and that just seems to be a Not a technological problem. Not not something that we can that we can deal with By by adding another layer of security to to the social media platform and and It's a societal issue. I don't know. I don't know What we can add to the to the discussion there other than other than to to hold public officials to a level Of ethical standards that's consistent with their job Well, the national association or secretary of the state launched a trusted info 2020 campaign to help get across to The american public that their best source of trusted information is in fact their local election officials I did note that over the past couple of weeks that election officials were very active Any time that they heard about any misinformation or even just questions from voters Going around they were quick to put up truthful sources And and contradict those misinformation messages and even contact the social media platforms to get those messages Pulled down like the example that jake gave where if there's a fake page up if the platforms don't pull down themselves And they're notified in an effort to get that down as quickly as possible. We've also seen the department of homeland security Launch a rumor control page, which has been referenced a lot over the past week or so Even to the point of getting some senior officials in some hot water apparently and this is something that's happening now where You have different parts of government saying different things about the elections And so, you know, how do people Get a better understanding of where accurate information should come from when it seems as though Different levels of government and even different officials within government Are saying different things about the election and the results Well, that's not a that's not a purely election related question. That is a how are we as a society going to decide A common set of facts It's it's a broader question about the media and the the role of the media and actual journalists versus you know, citizen with opinions on the internet and The ability to easily differentiate something that is Journalistic content And something that is not journalistic content is something that you know, we've we've been able to do in other medium You know, if if a television ad comes on When we're watching the news We can see that it's a television ad we can see that someone is paying to persuade me For a thing because we present it in a different way Likewise in a newspaper or a magazine advertisements are presented in ways that are different One of the problems that we see with so much on social media is that the presentation between the news And the thing people are trying to sell or persuade you on is not very visibly different So it's just a little harder to change. So I think that's just a broader question Societally about how we how we want to deal with this Now I would take social out of the description media in general. I I think deserves that kind of of scrutiny local reporting That runs a follow of local officials Is is sometimes modulated And and and that's not that's not good for for transparency or the local public broadcasting station feels that it's It's funding stream is threatened if they delve too deeply into Into some sensitive sensitive matter. That's not good. That's not good for for transparency And there's broader questions to ask of how we as society get trusted information But the census is supposed to be a trusted source of information that's used for a variety of different different government programs and there is mistrust of that mistrust of the courts mistrust of the election systems It's really very hard to build these trusted systems when so many things are untrustworthy And I see my video seems to be down. It looks good from my end. So I don't know what's going on Well, maybe there's some election hackers who Don't want people to hear what you have to say. Yeah, a little bit too late, isn't it? But okay They're attacking me. They're not attacking the important stuff. So that's good So is this a case like we heard jake say where it's almost like national security where it becomes a federal level issue because it does impact the ability to actually run elections nationwide and actually, uh, you know Make sure that we have a firm foundation on which our democracy should be built Or is it something that should be better addressed at the local level? Because it is those local election officials who have a higher level of trust From their own voters I'll jump in. I mean, I think it's both. I think the local election officials You know should continue to administer the elections as they have But in terms of providing security um, you know, both in terms of cyber hygiene And best practices and so on At the very least in terms of training funding Etc. Etc. And then on top of that what needs to be done globally in terms of cyber deterrence Um, you know pulling together again The democracies of the world to kind of stand together on this and so on and so forth that that needs to be done at the federal level In my view So even though we haven't actually finished the 2020 election like for us to look forward Rich mentioned that there's going to be a runoff in georgia in just a few weeks But then we're also going to continue to have elections in 21 and 22 And then looking forward to the next presidential election in 2024 So what are some things that we should? Expect what happened on the election security front in order to make sure that we're no longer repeating some of those mistakes from 2016 or even from 2020 So after every election, it's quite common for election officials to get together and do a bit of an after action Reporter retro about what worked and what didn't and so I expect to see lots of interesting Uh new ideas and approaches come out of that. I will say that Uh in particular we touched on this earlier People are going to want to vote in new ways. I think that uh this Clearly opened up new avenues For for voters, uh, and they're going to want to do that. And so I think on a legislative level There's going to be a push to try to open that up more clearly I would say uh the other thing that makes a lot of sense to me because we've seen so many differences in state laws and whether it's about voting hours or days or recounts or Observer rules or whatever it is some consistency That is in uniformity about access to the polls. I think is really important We we get our information nationally now instead of From from local sources. It's just very confusing For for voters and even for election officials to know what the rules are from one place to the next And there's in many cases not a lot of reason for it Other than just history. And so I do think that it's time that we had some sort of baseline rights And accessibility rules for voters. Um, and now would be a good time to start that Yeah, and I think, um You know this I I I agree. I don't think we're putting this genie back in the bottle on You know, everybody voting on one day, you know And potentially standing in line for hours and hours now that people have seen the convenience of Of early voting voting absentee and so on. I think they're just going to demand it In the future And then also, I think as rich said before and I think this really gets overlooked a lot You know the the Voting machine companies, they just don't have the margins to ever do the r&d. That's necessary to really rethink and come up with a really You know, definitively different Voting technology for the future and you know, I think that if we if we can get a bill in the next congress Um and with the next administration, you know, hopefully they'll put real money, you know hundreds of millions or billions of dollars into You know really Try and you know give some of the best and brightest minds in the country Um Capital to really do some r&d and try and rethink the technology behind how we vote Because there there must be a better way than what we're doing now But it needs the r&d money to ever really come to fruition, I think And by the way, I just put in a plug for if we do that r&d and the taxpayers pay for it It would be great if the taxpayers owned it Or that it was otherwise open source Rather than being turned into some sort of, you know proprietary system That is hard to change or access going forward I I'm teaching class actually in election security right now at harvard kennedy school One of the questions that that we discussed is how much Democracy do we want and are we willing to pay for it as are we as a society? willing to pay for Accurate elections other countries have a permanent bureaucracy in charge of safe and accurate and secure elections elections canada for example the uk does australia does japan does We don't have that kind of national bureaucracy like we have for our census And sure, I mean all of these suggestions. I think are important and necessary But are we willing to pay for it? I think what we want are perfectly accurate elections fast and free Which isn't possible So we have to decide this is worth it If we just that level of investment then how do we actually attract the talent that's necessary to the public sector? Rather than having those folks go to some of the larger companies that were mentioned before Well, I'm sort of embarrassed as a technologist. I'm sort of embarrassed to point this out But but we tend to be attracted by shiny objects Uh, and and it's it's true that that a lot of the western democracies use technology to To conduct their elections, but a lot don't And and and there's there are there a lot of low-tech elections where the outcome of the election Is not a subject of debate because people filled out Ballots by hand and they put them in ballot boxes and they counted them in community community Events and they were recorded in public In public places and and I just I I think that that there's a point in this discussion where you have to say Have we used appropriate technology for the task that we're trying to Trying to carry out and it's not always the case as someone who used to sell Computer equipment to to to customers who had to be convinced that they needed it It's not always the case that you need the level of technology that we're trying to jam into this into this process And and and that seems to be that seems to be an ongoing an ongoing conversation that has that has varying levels of participation depending on Where you are in the country and who you're talking to One place where it seems like the introduction of technology can be helpful is in the area of post-election audits With this idea of a risk limiting audit such that It's something less than a full recount, but can give a relatively high level of assurance that the outcome is confirmed Do you think that rla's? Have a place in elections moving forward I'll weigh in on that for for just a second. I think they're a great idea It's another way it's another sort of belt and suspenders to to make sure things are working right And it gives people confidence That that the results are what what we say they are so the more we can do those things the better and I think they've they've come up with a smart process on these risk limiting audits I agree the answer is yes and the way to think of it is is an audit that size to the margin of victory Right. So if there's a huge margin of victory You only have to audit a few ballots to ensure that there wasn't any Kind of malfeasance because you have to change a lot of ballots to change the To change the results if the margins very close you have to do a larger audit Because you only have to change a few ballots. So detecting it will mean you have to audit more And so this ties the audit automatically to the margin of victory. You don't have to Do the Sue for a recount that happens automatically and statistics determines how we do it and it's an it is an excellent idea I I I agree Bruce. I I think I think the the thing that you have to keep in mind though is is that The math is non-trivial And and explaining what a risk limiting audit is To even someone who's knowledgeable Who's supposed to know what it is is a difficult task And and it's it's easy. It's easy to mischaracterize any old audit process as a risk limiting Audit if you just kind of roll some 10-sided dice every once in a while and and Have people make marks on on on ballot manifest And and we don't we don't want that so so you're coming back to who's who's the trusted source? On this is important. I mean nuclear technology is complicated, too But we have the nuclear regulatory commission That that that we have pretty good confidence in in their their technical chops In that area and we sort of agree agree to have them think the deep the deep thoughts Who who is the technical? authority on risk limiting audits Maybe maybe we need a federal agency To to to take on that challenge. I don't I don't know but but it is it's not a slam dunk I think to to to get people to buy into complicated models like RLA's and and and come away with a lot of confidence that that the election has been correctly decided another trend in elections that is Seemingly complicated on the outside, but tends to make sense if you understand it and you've seen it in practice This idea of ranked choice voting is something that is gaining popularity In some smaller jurisdictions, but not quite at the state level. We've seen mass issues in Alaska to feed initiatives Whereas it seems to be working smoothly in Maine So do you see a role for ranked choice voting that might help eliminate some of these issues of runoffs and get a better sense of The will of the public at a single moment of time Rather than having additional elections Down the road to help solve for those close races I live in Minneapolis in Minneapolis. We do ranked choice voting for uh for city elections And yes, I think it is it is a really interesting way of capturing the will of the voters Potentially breaking the two-party system. It's really outside This seminar really has nothing to do with with with voting security or election security But in terms of capturing the will of the voters it is a more It is it is a more robust and expansive way of doing that It's a little hard to explain. We had some trouble in Minneapolis explaining how it worked But now it's working just fine. Mains seem to have done pretty well with it as well Who did the explaining in Minneapolis first? It was done by the election officials by the press You know because because we had to have a ballot initiative To get ranked choice voting so when it was being voted on There was a lot of explaining by the media Of what it is. What it is. You're voting how it would work What it does And it did pass which is kind of neat to see Was there vocal opposition? There was I don't remember the details Yeah, I'm in I'm in st. Louis and this year there was an initiative on I don't even know what they called it, but basically you could vote for everyone you liked So you could have multiple votes or you could just vote for one And I'm still a little unclear how that all is going to work But it passed and so what this suggests is that the voters are looking for sort of new and interesting approaches to having their voices heard and That's one of the there. We've talked a lot about the downsides of having such a Disaggregated and federated system of elections But one of the great upsides is that we can have these experiments and see how they work out for voters and And adopt them or not going forward and that's exciting I think that's called approval voting where you we vote for everyone you you you like Yes, it's another way of doing voting voting theory is super interesting If people want us to go to going on that rabbit hole So it seems to me all of these local experiments are are adding complexity In a way that will ultimately end up into something that Should attract more voters But as we know complexity is going to be the enemy of security And so I think trying to strike that balance is something that remains Very difficult both for practitioners, but also for policymakers because ultimately It's going to be those elected officials who are going to be setting the rules for how people vote And I think it's incumbent upon the election officials to be able to Explain to the voters and to the policymakers the rules of voting so that way we can reduce the amount of confusion and hopefully Contesting a vote further down the road. And I think that's where Technologists can come in the folks that understand How the technology works and how it's supposed to be implemented to make sure that we can have Safe secure and accountable elections We're coming up to the end of time And so I wanted to go ahead and give each panelist about a minute to wrap up And leave you with something that they feel is critically important. Let's go ahead and start with robin Well, thanks again for bringing us together here. I I just hope people remember Um, like the elections don't just happen. Um, they we take them for granted But there's an awful lot that goes into it And we've touched on a little bit here about Whether we think it's worth it What what is our democracy worth for us? What are we willing to invest in that? And what are the trade-offs to having the technology the convenience that technology can bring? But the risks that that brings as well. Those are not technology questions. Those are policy questions And I think it's it's a worthy conversation for all of us as Citizens to be thinking about how we want our government to work The constitution talks about the just power of government coming from the consent of the governed This is how we get the consent of the people To set up our system and we need to be rethinking some of those things And so I'm excited to have Uh, a new generation of people interested in this kind of public interest technology And to be able to take up the cudgel and and and and join us going forward It's exciting. It's fun. Uh, and it needs your energy. So please join us and Jake um Yeah, so a couple things one of I'd be remiss if I didn't shamelessly plug my book Democracy in Danger when we talk about a bunch of these issues So please go get it. Um, but secondly, I think you know, we see You know just over a week now after the election, you know, we find ourselves with You know competing ridiculousness of the president claiming that all the machines were hacked and then homeland security saying that everything is the most secure You know election we've ever had And I think that What we really need Now that we're going to have a new administration and um And many new members of congress and so on is to to really now get serious about this and from multiple perspectives um in the coming uh in the coming administration Which is one really focusing on The r&d that can be done and should be done. I think and funded by the federal government To come up with the next generation of voting equipment that I agree a thousand percent with secretary karnic And absolutely should be open source and publicly owned And by the way may be available to the world not just our own democracy um And secondly, we need to get very serious about deterrence and pulling together the democracies of the world to let everybody Everybody else who would attack ours or any other democracy? or their own for that matter um that that this is not okay and You know what we've done in the last four years. I think it's certainly been progress with our question But there's so much more that can be done. I think now that we're we can turn a page We should we should do that and really step up our game category So i'm encouraged by that by the discussion a little daunted but but but also also encouraged Oftentimes discussions like this tend to drill down into a particular piece of technology, whether it's it's online voter registration systems or voting machines or rla's um securities an end to end problem and and and keeping keeping yourself aware of the context and and and things like Things like the the political environment in which in which all this stuff has to fit is equally Is equally important. So, uh, you know, I hope that we can keep in mind that that the ultimate goal is safe and secure elections and individual pieces may be faulting that may be Maybe insecure Other industries have managed to go through this kind of transition and I think I think elections can follow that same path Thank you bruce I want to put a plug in for national standards and national systems That uh, you'll often hear the argument that we are secure because we have 51 different election systems and that's a standard argument in computer security balancing a distributed system versus a centralized system Distributed tends to be better when you're as secure as the average centralized sense be better when you're secure as the weakest We're in a world where it is secure as the weakest and centralized system standards will serve us a lot better when the threats are so large How do you do that in the united states? Probably looking at the same way you do, uh, a national speed limit The federal government had no ability to set a speed limit. So they tied The state's ability to set a speed limit with receiving funds for national funds for highways We could do the same thing if the state wants to get All this funding for election security safety reliability machine systems They have to adhere to federal standards And I think we should look into doing that because the problems are national. They're not state level Thank you bruce I want to thank everyone who's joined us today and especially to thank uh the staff of pit you in as well as the IEEE for putting this together And I hope you all are encouraged And excited about getting into this field of election security because it's something that can certainly use your help And it's a great problem to try to solve