 Awesome, so a couple of things right before we jump into it. One, please keep your cell phones on silent, just as a courtesy for the speakers. There are mics throughout the room, so hopefully they should be able to pick up everything you said, if there isn't, we'll repeat the questions. And please keep your masks on at all times. With that, I'll kick it off to Yan, who can introduce the panel, looking forward to this discussion. All right, welcome, everybody. Oh, good. I'm sorry? What? Oh, the Chatham House Rules. What are the Chatham House Rules? OK, OK. No rules here. No, maybe that's not a... All right, so we are here to talk about kind of emerging topics in cyber policy. Now, I'm Yan, Yan Shorteshvili, some might know me as Zardis, and we'll go and introduce ourselves. And the general idea, I'm a professor at Arizona State University, when we say emerging topics in BLA in a class, it means we're just going to teach anything, right? It's just a catch-all for whatever. So hopefully that won't be this. We'll be a little more structured about actual security policy topics. But why don't we start with introductions by my co-panelists, and we'll go from there. All right, confusingly, I'm also Yan, so you only have to remember one last name here. So I used to work at the EFF with Kurt over there doing the technology side of policy. So we made things like Let's Encrypt. We helped out with the tour projects, et cetera. And now I run security app Brave Software, which is a browser company. Good afternoon. My name is Luis Eduardo. My day trade is I run the product security for a large networking company. And my other job here is to run the network team. So that's all I do. I'm Bill Woodcock. I'm the executive director at Packard Clearing House. And we do internet critical infrastructure and the policy around that. Quad nine in the house. Yeah, quad nine also. That's like our consumer facing. I think that's about 130 CCTLDs. Yeah, yeah, 150. It's over 150 now. Basically half the world CCTLDs. Hey, everybody. My name is Kurt Oppsall. I am the deputy executive director and general counsel at the Electronic Frontier Foundation. EFF has been working on tech policy issues for founded in 1990. I work a lot with this community through our coders rights project, trying to provide counsel on legal issues to security researchers about doing their research and about the issues surrounding disclosing. And especially if the vendor is upset or there are people who want that information not disclosed. So we try and fight for things like free speech and privacy and innovation policy and try and making the world a better place through good law and policy. And technology too, John Welpeth. And I'm Jeff Moss, founder of DEF CON. And I'm really happy to see people here. This is our attempt to really try to increase the engagement between the technical community and policy-minded individuals. And I hope it moves us past the sort of tourist phase of us kind of cursorily looking at each other, moving into a more substantial way in which we can interact and help each other. And what really galvanized my thoughts around this area was the success our vote hacking machine village had in influencing not just policy in the United States around voting machines, but globally. And it turned out that our simple little report impacted that space in one weekend more than a decade of a lot of activist groups trying to raise awareness. And it turns out that you can tell people that plate is hot, but until they touch it and burn their finger, it doesn't count. And so by us, in one weekend, hacking and successfully compromising these voting machines was more important than a decade of people saying out loud that these machines are vulnerable. And so I think there's a lot of what we're trying to do is speak truth to power and provide a third party kind of more neutral perspective. Because I don't think policy gets that. I think policy gets a lot of trade associations and self-interest. And civil society doesn't have a big voice, and that's what we're trying to do. Cool. So from that perspective, I'm going to do my best moderating moderation experience. I was in a couple of rooms over. I ran CTF for four years, and that mostly involves screaming at teams that are misbehaving. So I'll try to do that as little as possible. I'm a very loud person by nature. But let's kind of start things off. So Jeff just said the voting machine village. And that was a great example of something very practical that informed policy down the line once it came out. And so in some sense, it would be great if one of the outcomes of this roundtable discussion were ideas of potentially future villages that we can do to then inform people down the line. So I'll start with one thing that actually I got administrative pressure for the other day, unless you have an idea on direction to go initially. No, I was just going to say the reason of the success of the voting village was completely random. I would like to take credit for it being such a success. But until the year before 2015, there was no exception in the DMCA for good-faith research or security research in critical infrastructure and voting infrastructure. And that's why people weren't hacking this stuff. It was against the law. And so the Library of Congress put in this exemption for the DMCA. So now it's legally possible. But you can't get the machines. There's not enough machines. They're not available. They're only sold under contract. The whole system is designed to make sure nobody ever figures out how these ancient machines work. But there was a storm, much like in Las Vegas. The storm collapsed the roof of a warehouse where a county was storing their voting equipment. The insurance company totaled the equipment. And because they were a green insurance company, they sent out the zero-value items to get recycled at a re-PC company. That electronic recycler is like, well, they still work. I'll sell them for $200. And just start selling them on eBay. So now the law has changed, and they're available. The manufacturer catches on that this is happening, and goes to them and says, you have to return them all to us. And he says, great, OK, $200. How many do you want? They're like, no, no. There are machines. You have to return them to us. He's like, well, I've never signed those agreements. I got them legally from the insurance company. No. So well, then you have to destroy them, because they have proprietary parts. He's like, OK, $200 an hour. How many do you want me to destroy? He's like, no, we don't want to pay for it. And he says, well, then we're selling. So because the insurance, the voting companies didn't want to pay for the machines, they got out. And we had our success. And so I think since then, they recognized the importance of that exception under the DMCA, and they want to expand that. So that was a very tangible law changes. Machines available. Look at this outcome. And I'd like to see those kind of exceptions expanded, if possible. These in other countries, you can't do this, right? Well, I think an interesting event that I very nearly witnessed in my first DEF CON, but I stepped out of the room and my brother witnessed it instead, was Dimitri Skliarov being arrested basically on stage after disclosing something Adobe didn't want disclosed. This doesn't happen anymore, because as a society, we've grown up and accepted vulnerability research and proper disclosure channels. And when you went to pull the parachute and tweaked the bug, that was interesting. I'm not to hijack every conversation. But that was interesting because Dimitri was an employee for a company, Elkomsoft, in Russia. In Russia, there was a law that allowed you to make a personal backup of your software for your own safety. And so they made tools to allow Russians to make that backup, right? To crack a lot of protections, PDF files, I think, were the thing that Adobe didn't like. So they were in some kind of argument. And Adobe is like, well, we're in Russia and it's legal here, and Adobe didn't like that. So when their employee came into the States, they got the FBI to arrest an employee as a proxy for the company. And then all these protests started. And Adobe went to the FBI and said, oh, OK, we don't want to prosecute anymore. And the federal government said, well, that's not your choice. That's a state issue now. You started it, but. And so now Adobe flips around, and now they're on his side. And they're the ones that created the whole mess, because the PR disaster was so great. And so I'm sure you have stories like that, but the policy outcomes can kind of end up being very bizarre. Seems like what we're all homing in on here is the difference between public interest and private interest and the government's role in which side of that fight the government comes down on. And I think many of us would really like to see the government acting more in the public interest more of the time and less in the pocket of some private interest, like Adobe or whoever. John Deere, there are a lot of different kinds of areas in which this comes up. The rectory pair stuff is a lot of it. A lot of it is around misuse of intellectual property law to keep people from advancing the state of the art. Something Curt and I worked on what, three years ago now, was there was a private equity firm that tried to do a hostile takeover of the entire dot org domain so that they could charge every non-profit in the world the maximum possible amount of money for their domain name, which the Red Cross was pretty scared by that, given the size of their budget. And the government didn't really have anything to say about that and to make sure that people are paying attention to the public's interest. So I wanted to add into this, how could we go forward? So we've seen a couple of these barriers. So the DMCA has been a barrier. They have a rulemaking process every three years where you can try and get some exemptions from the DMCA. And that is a process which has made a number of good strides forward. But it still is a restrictive process. The DMCA impedes a number of possible research paths. We would like to see that expanded. But I think also one of the things coming out of the voting villages is that the public got interested and then the government got interested in having it happen. And I think this could be expanded to other areas. And I see some of this like the aerospace village here and the car hacking village bringing some things in. We've had some trouble in the past with aerospace where they don't want people hacking on their planes. Certainly not a live plane. But I think it would be great to get these industries involved willing to open up their systems to have security research, do it so that there's some rules around it. So it's a plane, for example, that is not in flight but rather parked and safely done. And then that research would be published in a forum like this. Give them a time to fix the flaw before they go out there. But have that kind of cooperation with people who are otherwise hesitant or might like the voting industry for a while is basically threatening people if they went ahead and did this research, flip that around. And I think corporations need to change on this, but also I think it would be great to get policy support from all around, from government levels to see there's a good thing. We want to make the environment in which it can happen. So how do we make that flip happen? I guess Adobe flipped due to public pressure. Do we have the bandwidth as a DEF CON community to public pressure? And the normal trajectory has always been researcher finds flaw in access control system, physical lock system, right? What have we got? Medical implantable medical devices, whatever it is, manufacturing people freak out. Threat and lawsuits are successful or not, right? They get the Boston students to withdraw their access control talk on the subway system. Are they, you know, physical HID got some talks pulled on access control stuff, right? And then EFF gets involved and the media starts reporting and pretty soon the manufacturers realize they can't keep doing this. Like the cats out of the bag, people know their stuff. And it's like a forced maturation and it's painful. And they didn't all grow up at once. It started with like network and operating system companies. But I remember when it was access control and then it was, and then now it's been aviation and cars and like it's so painful but they all follow the same trajectory, it seems. And at some point we're going to run out of industries, I hope, and we'll all find out. There's a first contact issue to solve. Like when an industry has not previously been in contact with the security world, they're always nervous at first contact. So maybe one way is to think about what are the first contacts and how to make that run smoothly? Because we've seen just so many injuries where, yeah, it goes badly at first. And this oftentimes comes in an industry that previously did not think about computer security. In fact, many of them didn't have something that had much computers in it like 20, 30 years ago but now everything has a computer in it. So many of them are connected to devices and then they're getting treated like a computer company as they should because that's where their security problems are but they're like PR and their lawyers are not yet ready for it. So to try and like smooth out those first contacts so it doesn't go into that bad threading lawsuit phase and gets into the good cooperative phase or at least minimize the time in the middle. In the U.S. particularly, the liability insurers are a huge actor and they're pretty unpredictable and figuring out how to get liability insurers to take the approach you want is really powerful. I mean, one of the problems that one of the huge problems we have right now is ransomware and the reason we have the ransomware problem is because liability insurers are seeing fit to pay ransoms, right? And every ransom that gets paid pays for 100 more attacks. So it's spiraling out of control because the liability insurers went the wrong way on that but with stuff like vulnerabilities you could see that going the right way a lot more easily that an insurer having been informed of a liability in a product that they're insuring the manufacturer of against liability, you can imagine them saying, oh, well actually you the manufacturer or insured party needs to clean this up, right? Not just sweep it under the rug. I would love to see it if there was an insurance program that would try to reduce the risks that they saw in their interest. Oh, we have a question down here, yeah. It's around people. Oh, maybe just say hi to who you are or say hi to who you are. I have personally chased down industry sectors to try to socialize exactly the patterns you just described before it happened to them. So with some partial success, so just an immediate response one idea could be, and I think this has maybe worked a little bit, at least in the industrial control systems, IT space, a little bit of success here. I don't know who does this, right? This community, an actual somewhat planned campaign, go to the medical device sector, go to auto, go to aviation, go to their conferences, don't expect them to come here, find their trade industries, any channel you have, and multi-year concerted campaigns that introduce these sorts of things. So maybe the first time through, they're not completely surprised by it, but I've many times seen what you both described as the reaction. I didn't know we used computers and our things and then that response. There's a paper I've partially read, I think it's called, blessed are the lawyers for they shall inherit cyber security, it's a great title. Well, here's the point, right? It gets to the liability question and I've seen a little bit of this as well, right? So vendor has a vulnerability where they get compromised and there's their in-house people is there, the IR law firms legal is there, insurance company's interested. If no one else sorts this out, maybe eventually our civil courts might end up defining it maybe slowly over time. Hey, company X, you made an honest mistake, but the best nation state now on the planet dot you, you're doing all the good things. That was an accident, you're off the hook. Or I knew about it, public vulnerability, it was packed two months ago, I didn't get to it, there was a high track for it and I didn't match my stuff and I got caught. You lose. And honestly, my opinion after 20 years is if it costs enough to lose out, that will make a change. Lots of techniques we could do today, we don't because the motivation and the price model is not, you get punished enough for screwing up cyber security really badly. If there's a way to help define that minimum liability of mine, I'd love to work on that or do something else next. So, one thing you said that was really interesting is this increasing the cost of the failure and it makes a lot of sense. And I think you can fail cyber security from one, I mean in many, many ways, right? One of the ways you can fail is just not patching, but another way you can fail is never, I mean it's related, but never looking at your systems from an adversarial context. And so, if you're talking about insurances, something popped into my head when people always tell me like when you replace your roof, you tell your insurance agent and your insurance will go down. I've never experienced this, I've told my insurance agent, he's like great to hear. But maybe it's something like that when you do a pen test and I guess now that I say it, this already does happen with insurances, but maybe what we can actually do is when you receive a disclosed vulnerability in your product from the community to encourage this sort of community involvement to bring it back to how to be closed that shock of first contact, the liability insurer actually says, hey, actually if you receive a properly disclosed vulnerability, that's a good thing for you. Maybe that can like reduce your risk and lower costs. Yeah. We're seeing, sorry, a season from a legal perspective, we're ending up with a lot of the privacy controls are starting to drive with the liability insurance does. So GPR is a prime example of that because it carries both the physical liability and the appropriate liability with it. And that's where we see a lot of the changes starting to take place with regards to policies around privacy, not necessarily in post-sac because they don't understand it, but they do understand privacy. So a lot of drive and a lot of focus in that area because of these large data breaches like Adobe, LinkedIn, and others is where we're seeing that traction take place. But with that in mind, it all starts with how do you close the door? Yeah. I appreciate what Art was saying about the difference between you getting popped by a nation state that there's basically nothing you can do about it and not patching something for months and also putting in places, things like bug devices. And it was saying, what we want to actually incentivize is having good practices, having the most well defended system. And one of the things about where problems occur is it's not distributed necessarily according to how your practices are. Sometimes you have the best practices but you're also hiding out your target and you get popped by a very difficult to defend against. Sometimes you have the worst practices and just you happen to get popped. And so like, how do you tie it so you're incentivizing people to take the practices due to if a patch comes out, you take care of it immediately. You open yourself to finding out that information and act on it. But focus it on incentivizing doing the things that would defend with less, like whether you actually got popped is an interesting aspect of it. But it's not necessarily meaning you have bad practices or a good practice. Well, that sort of ratio of risk and cost is exactly what insurance companies in theory are good at dealing with. I mean, I've got a manufacturing facility and the liability insurer for that does an annual audit. Like they said, the next birth who comes around and says, I mean, the last one like said that we had a broken window that was, you know, window pane that was adjacent to a window patch, right? And yeah, okay, I guess none of our guys had noticed that, right? But that was, you know, on a long laundry list of little things that ought to be fixed up. And that was someone, the insurance company employed, the insurance company sent them. It was just part of the insurance company's business. Right, but they're not thinking that way in cyber yet. They're thinking that way still in physical realm, but not in cyber risk. To bring it back a little bit, one of the things that we've seen that's been successful at Black Hat and DEF CON was in the early days, US cert, when they were still around in their old form, we would get, a researcher would say, I've disclosed to the medical manufacturer, they don't believe that this is a problem. It's really a problem. And I don't wanna dump it publicly. Can you talk to the manufacturer, US cert? Study my results, go and talk to the manufacturer. They might believe the message if it comes from the government. And so there is this partnership role in which maybe the government can validate the finding of the crazy researcher and then deliver that message to them. And it's different when US cert's calling the CFO than Rando. And so I think there are some tangible outreach style programs that would increase the credibility of the government, increase the accessibility by the private sector. Right. So it might be then kind of to bring it back, back to that first contact scenario. It might be worth preemptively identifying sectors that haven't had their kind of moment in cybersecurity and preemptively reaching out to the big actors and saying, hey, let's build a channel so that when this inevitably happens and it'll happen to you eventually, there'll be a less shocked response. We'll be ready to go. Awesome. Hey, look, we're already... Oh, oh, oh. Boom. Drinks on me. There's this story about the voting village. I'll go back to that because it seems relevant. When we wanted to try to grow the voting village community, we figured we had to reach out to the municipalities that actually are running all the election officials. There was no list of election officials. There was no list of what all the counties were and who to contact to. So one of the people, Jake, who helped form the voting village with his own money from his company and five employees, they tracked down the 8,000, or no, it was, I think it was 6,000 or so, but we made 8,000 contacts, phone calls. We physically mailed every single one of them in the country a piece of mail explaining what's going on at the voting village. You're invited, you can come, everything. Like that wasn't being done by the government, it wasn't being done by anybody. None of these groups that had been around for 15 years that were champion voter transparency had done that either. It's like Jake with his budget, right? And that brought about two dozen officials and that was sort of the beginning. Well now there's just a different, everybody's jumping on it, but I mean, it was like a storm that got us voting machines and Jake with some budget started the whole community off. Now it probably would have gotten there a couple years later maybe, but I think there's a lot of ripe opportunities like that where it's just somebody with some initiative and 10 grand and some phone bankers. Yeah I wonder some feedback if we are too loud. I wonder if there's something there going back to how, well what's crazy to me is the whole voting village was a convergence of like two random events that enabled it. But like remember ATM hacking was a thing for a while because none of the ATM manufacturers had thought about security. Well we don't see ATM hacking talks much anymore. It's kind of been. Success. Yeah, mostly figured out or people have moved on to something else. So yeah, I would be interested in the first contact principles, but I mean as a policy connection between the two, I think it's more about building, remember when we eons ago there was like a cyber storm exercise between the civil society and the people who run the internet and government. And the most important thing is building relationships and building trust because if there's a big problem you want to pick up the phone and know you're talking to someone and people generally at least in our communities don't trust organizations, we trust individuals. And so I'm not calling the FBI, I'm calling somebody I know specifically in the FBI that I trust to not get me sideways. So half of this is I bet building relationships so when something goes wrong, right? So part of the conversation could be how do you build relationships? And how do you get the government to recognize some consistency in relationships that are important? When I lived in D.C. I spent three years trying to build relationships at all these different agencies, almost like internet governance related topics. And I swear every six months the person I just spent 5.9 months getting up to speed moved on to another, got rotated. And I'd spend all my time just trying to figure out well who's in charge now and then get on their calendar and then get up to unspeed on the issues and they just do that for a couple of years and you're burned out. And so where Woody has always been Woody he's been doing that forever. There's a consistency on one side but the other side it just seems like there's a forever turn. And that makes it hard. Everyone in Washington, D.C. is perpetually 23 years old. It's like September on the internet. One of the things Jeff just said, I mean the original topic of this round table was going to be how to give advice to government. And you know governments periodically ask for advice because they're all 23 years old and they're just out of school and they want to know what their job should be and so like they ask industry, hey what should we be doing? And industry always says, you know there was this thing, it was called cyber storm. It was great, you should do it again. And they're always like la la la la, we can't hear you, that was when we were in kindergarten. And for some reason, like this just goes around and around and around. Every time government asks for advice in the U.S. the private sector says cyber storm, that was great, we should do it again. And government says, you know we'd really like to have some advice from you. I run into this as well. There's this feeling that, hey we already did that and we stopped doing it. So why would we start doing it again? Right and that feeling is that there's examples like this all over the place in the research field. There used to be this awesome program called Cyber Fast Track where DARPA would give small awards to, very quickly, to small teams to do cool stuff. And then it stopped and it was awesome and everyone wants it back and it's like, but why would they do it again? Right, they already did it, they stopped. I wonder how we would, I mean that could be like a government agency standing up and say, hey we should look back at things that worked and for whatever reason stopped being done. The part of it is that government tends to reward novelty, right? Like if you're a politician, you gotta propose something new and get support for something new and get momentum for that and then you're on to the next thing. Whereas if you're in operations, you need continuity, right? You wanna know that if we got something going that we don't have to constantly tend it that it will have some momentum and it's not just gonna fall apart the moment you look away from it. So, I mean that's a fundamental difference between the private sector and government in that government doesn't really reward spending that money again next year because if you spend that money again next year then that money is not available next year for something flashy and new that you can get elected on. But I think people have continuity. I think citizens like continuity. So, I mean, this- I wanted to bring up one other thing. When we talk about government, we're not just talking the US government and there's a tendency to think myopically but these policy issues are global in nature and the problems that we face pretty much every government faces and I like to say that internet problems are global problems and so we're going to have to talk to the other governments of the world if we want to solve some of them and that means talking to China in Russia whether we like it or not. So, you probably need to have mechanisms and ways of communicating to other, more so China because they're a major manufacturer of the technology. So, if you're gonna try to fix a problem it's probably gonna be potentially to manufacture that's probably gonna be a Chinese company. So, we should keep that in mind that a lot of times I've seen it where there's ideas in the states that don't work very well or don't get much traction and then Singapore does it like a labeling requirement for software and then it's like, oh good, somebody else did it first mover, now we can talk about it, now we can do it and so it's almost like venue shopping as hackers. Fine, we'll talk to people in the European Parliament, wherever it takes to move the ball forward and I remember talking to a member of European Parliament and I explained what we did this voting village and it just happened, we had the report and he said, oh my God, I would love to have that in Europe because what I get is a report from a trade group and I get a report from like a special interest and there is no civil society counter balancing report. Now, I want to not do what's in these two, I want to do this third thing with a little bit but I can't point to a third thing, I can only point to these two things and if you could give me a scrap of paper with a scroll on it, anything I could point to that would make my job much easier. So I'm thinking that just sometimes trying to get a five page report out that says experts in the room think X, we think that's trivial and not in depth, that might be hugely valuable to other governments or other legislators. It's interesting, even if we produce it here, I think we have one, two and then we over there. Oh yes, I was gonna say, it's interesting that outcomes that we produce here in the context of this community or the US government, et cetera, et cetera can kind of spread and have impact beyond and maybe bootstrap those conversations because what you could probably do is then leverage that connection and that interest of that PM in Europe to start up some sort of a long-term communication with them, super interesting. How will we work with the community to engage with consultation? So part of my day job is exactly that, with the great associations and provide the commentary. But often these associations, even if you look right now in the US and the Brazilian side, which is a very important piece of legislation that is likely to have point-of-vote in this particular requirement, how can we make that accessible to the community and maybe create a workshop that really talks about how to rent out and provide the talking points, how to be engaged, so the voice of the community is also running these consultations, especially because to the point made, you might have a precedent in one jurisdiction that will then not serve as a benchmark for our relations to that space. Interesting, yeah, it's, I think as hackers we're often not very good at organization, that's something that takes a lot of organization that, I think, but it's a great idea, yeah. We have a question down there? Yes. No, no, no. So actually on the one, on the color lights which you just said, it's, it's a safe place. I think to that part of our organization there is really, really, really good to be a, a training for generation now, it's being a really, really good, and I think it is like this, in Brazil it's being really, it takes time, it takes, it's not so, I understand, it's impossible. And although, probably, the mindset should be actually, it should be a system that can improve from what's in it. And I think that this, basically, I'm very robust in the society, present side of the organization, or the, honestly, the society that is maybe, that is developing right now. We are actually moving forward. There are so other channels, whether the industry is not as, as present, and therefore, like, I forget this before, it's less common, and there might be other access, because obviously, I'm very, very, very disheartened, and very, very happy with the OSCEs. Generally, I think what, through and at the end, it's kind of on technocratic policy topics, which definitely, maybe through the law, like after five or six years, it does that taxation, or the depreciation of it, and it takes a year to do it. So, I'm wondering whether, this level of taxation is difficult, but there is now a lot of, thanks to the society, there's now a lot of, in the back place, what you're going to make that, in the back, in the back position, it's one of the functions as well, and it's, it could be just, just, sorry. It could be, I don't know, a future that is a combination, and that is like, as you said, the first time relationship, at least the OSCE, the first time relationship during the European Commission, that's my seat, and you're all on this one, so, yeah. I don't know if the electronic, on the other hand, has a big plan, the strategy of how the policy of the environment outside the US, and whether, yeah, it's originally founded by Trudeau, the Crypto Village, to be able to lead to plan on this side, this side of the European Commission, and open a program for the best-intending of all different countries, because, you mentioned the industry reports as well, but, yeah, the industry reports not only anything for best practices for some reason, but, unfortunately, that's not what we're trying to do, and I don't want to, I think, be useless with that report. So, yeah, I think we can get together and probably get a lot more like a on-matter on behalf of these shouldn't-be-accidental that I'm all in for. What's in there? Well, one more. John, number one, pointed out, was identifying where we might do policy where it's more ripe with, say, villages or something, and we've tried this in the past, we've talked to different government committees and said, what would be useful for you? And they'd say, okay, AI and automotive are really coming up in the next Congress, we want to, you know, those areas would be great. Okay, so we go to the AI village, and the auto village, and we say, hey, let's try to write a report. And you quickly realize, they don't know how to write reports. Right? And so then we're like, aha, here's an opportunity, okay, so DEF CON, we can go find people from the policy world to help work with them, to help write reports that then can get released. And so the first time we tried with the AI village report was a disaster, we never released it. But we're thinking, maybe one of the things we can do is help, like the community can still come up with these ideas, but if policy people say, you know, next week or next year in Europe, it's going to be all about cryptography. Oh, great. Okay, let's do a work, you know, consensus, what are the experts at DEF CON think about this and we'll find think tanks or we'll scare up some money, we'll get some policy people. I think that's probably more realistic at these early stages than starting a new center of something. Yeah, I think she's been waiting the longest. Yeah. So valuable was because when it was timely too, it was just out of curiosity, you know, hacker curiosity. I've only been coming to DEF CON since 2016, but it seemed to me that in 2017, when that voting machine bill started, the community changed a little bit. We got in people from the news, you know, from politicians, more politicians who are interested in voting machines. So, and you built community through the badge. There are a lot of people here I think who are suspicious of organizations. Maybe deservedly so, but you've got community through the badge and you've got a world, a global, I don't know how many countries are represented here today, but I think like that at 111. So, there's already an international pool of intellectual capital, property, whatever, smart people who think outside the box, who like to break things, but also maybe have a heart for making the world better. So, and I think that's a very valuable culture at DEF CON. And I think, you know, you've created it and I think you'll keep it, but I think that couldn't underline maybe the way policy gets thought about. It's really actual planet and then fixed planet. Yeah, yeah, yeah, exactly. No, I think the nice thing about doing it at DEF CON is hacking is kind of, it's actually a weird thing to say now that we have the rise of nation states, waging, cyber war functionally against each other, but hacking is such a fundamentally international thing. I was once at a conference in Shanghai and I said, I was giving the opening talk and I said, you know, packets don't care about national boundaries. There are certain people at that conference got very nervous. And I said that, but it's kind of, kind of true, right? Like for the, you know, the community at least. Just to kind of mix and bring these points together. So, you know, why is it so stupid to do sports, right? You do have a version of this, but you work for whatever reason, but it doesn't make you a little bit of a mess. You all have to agree with that. And I think there, while there are a lot of new suspicious groups and organizations, there's not much to talk about, right? In order to change the system, you must be participants in this, and that's just the way it is. And the reality is that there's a number of us that are working in sizes, coming from roles in the shape of future policy methods of practice. And, you know, a lot of my is, we set out ways to build trust and relationships as Jeff pointed out, right? Big part of the aim is, if you don't have those, you know, personal contact relationships, you know, it's hard to even build all of those and reflect the message better. And to me, what I think about areas where we can improve and optimize this, is that they've got like ISA, but other large communities who are science, you know, they have ISA themselves, right? They have other, you know, life, you know, people like that. But I'm not sure. There's other groups, but we need to create something that allows for the message of these types of activities. And to your point, I've got a lot of interest for next year. A lot of interest, you're repeating A-I law, right? There's the Cyberslayer's law. Okay, we know what that's like. Spend a little money, put that in this next year's topic, right? And let's, let's promote energy and trust the corporation people, humans, others, all want to participate in that and help make that better, because everybody wants that answer, right? And I think if we get it out to the right people, I mean, to your point, plan ahead, think about what we need to get, what's coming, we go where we literally want to, and then make sure there's actually communities that actually respond to it, and everything can go on like, like, we'd love to see these kinds of communities also be like, hang out with us, or wherever, you know, just inter-represent A-I anytime. This is the perfect time to say, track two at six o'clock, there's a policy reception with free beer, but don't tell anybody that, because it's supposed to be for people who are interested in policy, not the people who are not in this room kind of thing, but it's after the last talk, track two and the secure signal messaging app are supporting it, because they're getting more interested in this stuff. So tell your policy buddies, track two, six o'clock, and more community building with alcohol. And I wanted to riff on something you were saying. Working within the system, you can make some effective change. I get that, but I think there's also a bunch of people who are mistrustful of organizations, and they need some help interfacing with that. And I think both is one of the things that is to be good for the community, and people have tried to do this in the policy track and doing some, is get people engaged individually system, they can talk to their representative, they can get engaged directly, but I think also as you know, riffing off on, you know, saying about like the A-I report was not perfect the first time around, is also helping people in the community who can do the research, you can find the stuff, who can wanna make the world, but they're not sure how to like coalesce this and present that in a good form, have that interface, that like, we want the technologists to be able to give their opinions, but make it effective, and sometimes there's a little bit of an art to that. So having some ways of either getting communication between these groups, but investing a little bit, and I think a lot of people here in the policy space can be part of that solution, getting involved with the community groups and helping them get involved in making reports and making a difference in a system that they may not be familiar with and maybe even miss-dress for life. So I think we have a lot of subject matter expertise in this spread out. I think the FF is a perfect example. We once did a research paper, we broke a bunch of digital rights management automatically, right? You might remember because we worked with you, because when we tried to publish it, the conference refused to publish it until the EFF kind of signed off that this is all good. And through you, we worked with all of the vendors of the DRM that we broke, right? So the EFF is one body like this, but the EFF kind of has a more scope mission. And sounds to me increasingly like more and more what we need is a some sort of organization that makes connections. Like a policy routing. At the top. At the top. Exactly. Like we get them all in a room with free beer maybe. This was kind of what CPSR said. We're gonna have this part of the solution. Yeah. We're gonna get everybody outside and introduce the community to a political conversation. You're gonna say something, Bill? I was just gonna say that's kind of what CPSR was, but like what happened with CPSR? I don't really even know the story. You feel like a key conference that everyone went to and sort of faded away. Yeah. Computer professionals for social responsibility. And they would like in the early days, they would make sure that like the ergonomic seats were made and employers had to provide, you know, monitors, you know, like they were worried about social responsibility. Yeah. And they kind of disappeared. There was another one, too. There was EFF made three with the third to arrive on the scene. And I can't remember with the very first one, Epic. Hey, Epic's still around, but they're very small. Yeah. Epic, Epic. What did I, electronic privacy? Information center. Information center. How do we get more light shine onto them? Do we say like, does the government agency made an official recommendation that people from around government and industry go to these places or? I mean, what should ask the government people? How do you find your civil society counterparts? I mean, I'm assuming industry is constantly bombarding you with freebies and we don't look and sound like what you normally see is industry. So how do you even find us here? This is the problem. I'm not familiar with the rules of the table. I'm the state of the EFF, EFF, EFF. And you're exactly right. They are constantly bombarded by somebody trying to sell you something. And so as a consequence, they develop a shell around them, which is a newer to the effects of somebody that actually trying to come. And so you need to trust your third party in order for that to be, that does not have another agenda. EFF, as an example, I've been coming to the conference one year because I thought it was important that I did the right thing. That I gave people that were trying to do the right thing and figure out the best program. So there needs to be a different level of outreach. And I'm willing, in my new role, to help them. So I'm a cyber practicing for a contract with the Sierra. And so these things are interesting to me. Having been on the other side and now trying to do the right thing, but we need an independent entity that is able to host a conference, even gather birds with feathers. Something as simple as that. In major centers to bring together folks from the community, folks from the industry, and the government. Hosted by somebody like you like that. Not to put you on the spot. Good. Do you feel pressure? We're all part of it. I think EFF alone is just part of a larger system that we need to help develop to get these things. We do a lot of work in the US, more so overseas, but there's still a lot of interfacing that can be done. And I think that's why I say this group bringing together here is actually a really important part of this. It's really tricky when you're dealing with government to retain that neutral, trusted third party kind of role and still get work done on important issues. That there's a balance to be struck. The problem is that, I mean, for the EFF, I think the problem is that the EFF, because of who it's donors and members and so forth are, is called on to take a strong position on critical issues. They're very active. And so then in DC, that gets viewed as, oh, they're at one end of a spectrum and we need a balanced approach. So let's go and ask the Koch brothers what they think. Or whatever. I agree with this. And the reason I agree with that is because of my interaction with them and how they work. I understand there's an adverse relationship to certain aspects of the government within the EFF. But that's OK, because that necessary depolarity need that balance. And so if not them, who? Because I used to work for the legislature in California. And I decried, regularly, how can I get you to listen when I'm trying to come inside? My interest is yours. You're not viewing me as a trusted third party. And at the same time, how else are you going to listen? What are the most valuable things that your security community is just factual, like technical analysis, like reviewing code, finding vulnerabilities, and invoting machines? What are like, I turn a phrase we use, and instead of speak truth to power, speak tech to power, to be able to explain how the technology worked. Absolutely, EFF has policy positions, and we try to promote them. And in some cases, we're in opposition on a government position. But the tech is still the truth. Of these, just going back to the voting machine example, because I think it's a really good one, it's literally finding the vulnerabilities, proving up what had been said before, but providing some evidence to it. And a lot of people who were doing it came from the academic community. They also knew how to write things up in a way that maybe not everybody in the community does. But getting that information, so even also if the government receives factual information and buys into that, they make a policy, you might disagree on the policy choices, but at least it's being based on how the technology works. And some of the worst policies or trouble ones is policymakers are doing it based on a misunderstanding of how the technology works. I really struggle with getting the two communities to work together. So in the last year and a half, my department has put out three or four or five consultations on potentially new cybersecurity laws for the UK. And most of the responses that we get from the consultation are by saying, you know, how do you know what we do? Is this right? Is this wrong? We haven't had a response, I think, that has been said to do is wrong, which means the wrong people are responding to us wrong. We get lots of industries saying this is the best thing we're doing, this is great. We get people saying, you should do more, you should do less, and no one's ever going to have to tap to our grave. I think part of that is because a lot of this stuff is new. And so anything to do with space can sometimes do a good starting point. But we're definitely not getting to the right people. The department comes to the department so it's my department's first kind of worry is how do we reach out to those groups that are not that we don't need to connect to. I think Black County Europe will be the next one. We do. And it's really a moment-based relationship. One of the gaps I think we also have in the UK is that there's an active community which we get involved in based on informal relationships. We respond to a certain amount of it all. We really start on Twitter because that's the most aggressive thing we get. And it's very difficult to follow up on that. But there's also a gap. I think both in the US and in the UK, but please, President, run along with the US side. It can think that that. I think there's a lot of stuff on national defence and national security, and then there's a lot of the economic side of things. So, unfortunately, not for a CV. Cyber security, necessarily, between the two, there's a national security issue in many ways. It's a really big economic problem. And it's the economic problem where you have to weigh out the risks of the cyber risks that are inherently using digital services with more benefits than the rest of the country. And in the UK and I think for the Europe as well, we're all looking at how to call the cyber security versus inflation, and how's that going to come at least. And there's not a space to have that conversation. And I think, I don't think there's only two groups that we're missing here. So I think if anyone can, if we can do more of this kind of thing, we need to be together at the meeting. I'd love to hear what you're doing, but also because if we're not doing it, we're not just going to go on and on. So also next set of energy issues that are going to be had, that are going to be at risk of a malicious activity that we need to get into to protect people. We're now at Best Friends. I think we're at the last of the year. I understand. We will be at the end of the world while we're at the end of the world to talk about some of that stuff. We are maybe a little bit of an issue about the role of the people. What about in our journey on the cyber security and the challenge that we have is, similar to what I'm saying, explaining to our state of performance services or pay-in-law services why cyber security is a relevant component of the word DJI. All the programs that you're funding, all the work that you're doing, all the issues that you have, unless that person has an existing background in space, it's really hard to do this. And one group can mention it's National Security and Second Army Group. It's also civil society is a touch on rank. It's journalism. It's race organizations. Those two kinds of states that, at least from the foreign policy side of the US government, that's now catching up on the cyber-saturday and the year after these conversations that we're writing into this. Finding ways to go from the community, but that also internally is why they're in a struggle. It's trying to explain in a slightly related piece that people are really passionate about. They're all a detective, and they're all actors, and they're all global husband and wife. It's a little bit of an interesting work. The last part that I was going to say is so much of the advertising on the cyber-proceedings like around the CISOs and the crime stuff, which is a little kind of subtle, but I think management people, they don't find it weird, but that's like 10% of the organization. Another 90% is trying to do a lot of work and compare all of their own pictures that are out there. So, we keep trying back, oh, you had your kind of question? Yeah. Yeah. I'm a guard for all of us, and we'll now work on the DOD. One of the problems we had to do is that a lot of the time from expertise and knowledge related to the day-to-day guys is that the work level. And when you're in the DOD, you're working with the intelligence community, and you're trying to engage with someone from civil society, there are so many hurdles to getting into debt for security reasons. When you're concerned about insider threats, when you're concerned about flight choices, doing illegal things, et cetera, that makes it really painful for someone to try to engage. On top of that, you pay budgets, right? And so, I had to work very hard to get the budget to come out this week to a conference in Vegas for, you know, five days, six days, or whatever, that cost us like $2,000, $3,000, $4,000. That's not going to happen real for a little bit of an image. So, it's kind of another barrier that we face to engaging in that, particularly out here. If you do more of that in DC, it's probably a little bit better, but for tenants, there's some thoughts there. Next slide, please. We follow up on the U.S. chain. One of the challenges we have is the consultations, and it's a very long term. And it requires a very consistent case of an incompetence, and there's a distance between them. So, it won't be, definitely an EU government legislation consultation, the scope of any school, or the Brazilian staff that depends on the jurisdiction. And there's also a lot of information out there. So, one of the things that needs to be useful sometimes for the trade association is either a lot of fleets or some kind of beans from the court, the most significant consultation that out there obviously this is based on the organization. So, I think one of the challenges, you know, the community, one of the agents specific deliverables that they use mostly after the practice, all of the year or two, and then by sharing this information that is addressable through consultation and carrying that with all states with that and by the states that come up with a chess deck, then using the community through the GHS. You know, there's information, a lot of public consultations that go on Twitter, and this is done in front of the staff. This is just a great example of how they end the GHS as well as see some of the law on that how they call it, how it's meant to be a deal. So, this is kind of a more creative way to get feedback, but maybe one of the quantities of ideas is in case you've been trying to refer to our associations and other policy experts that you need to know about the knowledge of the kind of news item, the way that hopefully that just for the community, telling them about it's out there and through the consultations, as well as something that is more appropriate to aid us in local communities that bears out the policy experts with the community to actually look at this as a local response. So, that's a little bit of what we've aged within a natural but with the opportunity to kind of address that. But also, maybe start small and have like a discord category, right? Because I think you mentioned budgets, right? And, you know, having in DC will fix one aspect of it, but maybe not others, right? Someone over here mentioned that it's difficult to make these connections beyond these conferences. Maybe Jeff mentioned something along those lines, but maybe if we have a online community that's kind of, let's say, semi-vetted, but, you know, fairly open to have these sort of talks much more frequently without the need to spend $6,000 to, you know, come to probably the worst city on the planet. It's, I don't know, do you think that would be useful or is there like a, what's the downside there? I found it really hard to build community online. On the DEF CON forums where we've tried for years to build this community on aerospace and satellite hacking, the people are out there, the content is out there, but nobody wants to sort of curate it unless the space has some sort of curated content, people don't really come back. And so, it's like you need a budget, you need somebody, an intern, somebody whose job is to gather all the satellite hacking articles and papers and YouTube videos. You have to create a critical mass, and then two years later, enough people might understand we tried this with farm hacking, which turned into sort of right to repair. And you get a couple of people who know what they're talking about. They post a couple of things to get low engagement and they never come back. It's not just like you start an online community. I've tried that many times. You need to have a poor group of people that are posting all the time to cross, I mean, it's a job. And who want to get together in person to argue about it, alongside meetings that they're going to anyway, periodically. I think, I mean, essentially that's what DEF CON kind of is. There's a whole lot of side meetings that don't happen in the same place at the same time. But building on that and providing continuity through between meetings is, I think, the part that needs to kind of power that or a group needs to power it. I nominate Luis. Again? Yeah, so it's good. I just had a thought. You came here with a good job. You have a part of the leader on these public-routed partnerships. They create programs that work very well. You ask not so much. We're trying to do better. But that construct is really, because in fact, we can't just create groups and we're going to show up and we're going to grow up right off of that. And so you have to figure out how to build that trust. And so there is so much automatic and so this also goes against people's grants and then you have to be like, well, stay and run around and expose all this stuff. But the reality is is that if you want to participate, in order to win, you've got to play. And so creating a public-private trust kind of relationship is going to be really important. It's not whether that's a dot word or a community that does it or whether it's a government that does it. You've got to do it because you've got to have some amount of evidence, some amount of evaluation and just have some random group. That's the that's the online is great if you've already met the people in person. Right? And I think Bill and I were joking that one of the best trust-building relationship exercises out of Cyberstorm would probably just be the beer and pizza phase of Cyberstorm and all the computers and pretending you're running the internet and whatever. That's fine too but really the valuable part was actually the beer and pizza part. Starting to run out of time so let's try to figure out what's our sort of key takeaways. I just wanted to ask something like we're talking about possibly meeting or promoting meetings and stuff like that. Maybe there's no answer for this but if there are other meetings that are not related to cybersecurity is there a way to have a cybersecurity breakout for these like different meetings that might or might not exist because people that's a way to kind of start getting into the conversation right? Maybe say what are the cyber security implications for a certain thing or for a segment or for an industry or something like that. That's one possibly one way to start that. Wasn't trying to So an interesting thing on that and I think that goes back into a lot of what we talked about is in order to organize that I guess the effective way to do it would be you identify the trade group meeting of X, Y and Z industries or whatever groups and then you go and reach out to them maybe do a breakout you know and the collaboration with DEF CON for example Village now in its I don't know X year incarnation the first few years was nothing but trying to convince industry that we're safe and we're not going to burn you down and now I think Bo you can maybe mention this like we're finally getting actual manufacturer representation but that was because of years of prospecting to build those relationships right and so absolutely and they also this one I'm also going to their events the aviation aerospace events and this has also worked a lot on the aviation aerospace Village and other things and part of that was bringing them in early into the conversation so it wasn't just a bunch of hackers that are out in the room there's a bunch of hackers and a bunch of really bold really brave people from industry from government who were a lot of times doing it in their personal capacity professional capacity wouldn't let them do it and you know that's I think things are changing now in a lot of places for the better where companies are starting to say oh yeah you can go and have those conversations you can do that that's cool but not all at the same time so some industries are farther along than others I think one way to family release service if we have a we identify a bunch of outreach conferences that we can go to and they're already there they can't escape right and we just kind of set up in a room and say hey let's talk about security of X I guess that could be a concrete next step from like a DEF CON perspective sort of like the DEF CON track aviation world exactly yeah we did that once for what the Tribeca Film Festival around storytellers and directors and actors and everything but yeah there's precedent you could travel you know we could have a DEF CON on the road so that's one common theme that to summarize that kept coming up is engaging with these other other areas the other common theme is maintaining relationships between different between people at different organizations across like government industry government government and kind of citizen groups that seems like a very hard problem that maybe that's where we maybe we use DEF CON outreach DEF CON goes and breaks into other conferences who are doing the industry to security connection like an online I don't know maybe like I see what you're saying about maintaining the online community but there's got to be a solution to the person-to-person connections I've got to go to Jen's today yeah cool thanks for joining us DT thank you everyone don't forget tell your policy friends tonight six o'clock track two see you there it doesn't cover it's a secret it's a secret and otherwise everyone would be there for beer alright and the organization of all of the talks in our public conferences especially the tutorial that you suggest to our colleagues in the department of industry and then when you raise those talks and raise those awareness for regular requirements on the infrastructure to have those talks and I think yeah and that's very different than building out the so it's also security community to security community that we got to connect anything we're missing in terms of the outtakes other than inter-connectivity I mean from both like there might be something very simple people whose employers don't want them to go there just to say I'm probably very sure that's not very acceptable and I think that's very different from the one more easy way and one more day of space you know I found that it was a very proper community to talk about how community support we just heard about the talk that that is going through a case that describes the communication as something of a change to be really inspiring for community to keep that communication and who they have to take any place and I guess that's part of the point of the policy track in general that we can enable these sort of things at DEF CON cool we got the 10 minute marker why don't we go around and see if anyone has any closing remarks on the panel or in the audience let's do audience first so then we can get the real closing remarks and I think my institutions are like a bit selfish on our the whole they are they are they are they are they are they are you might do a like to go down from him a Stevens or not? I have to know are you Peter Stevens or not? You're not Peter Stevens. Interesting. Are you sure it's you and not him? I'm joking, I'm joking. Cool. Yeah that's a great idea. I think that's what that's probably one more actionable routes forward. Are there closing remarks? Audience? Just to say, OECD was mentioned a few times in motion. Cool. All right. Well, panelists, do we have any closing remarks? All right. Oh, I heard that. Perfect. Well, not really as much of a closing march so thank you everybody for this conversation. I think, you know, brainstorm a number of ideas here and I think we're trying to find a path forward to get better engagement, you know, minimize the problems with first contact scenarios, get the hacking community more able to be engaged and I hope we can keep moving forward and make this happen. If you've got like a business card and want to be notified when, if we create a deaf and like policy discord channel or whatever, drop it off and we'll do our best. Or yeah, respond to my Twitter. Awesome. And remember to like and subscribe. All right, thank you everybody. Thank you everybody.