 we've got Stuart Scott talking about preserving supply side vulnerability disclosure. So please give him a warm welcome. Hey, everybody. Can you guys hear me fine? Awesome. My name is Stuart Scott. I'm an assistant director with the Atlantic Council's Cyber Statecraft Initiative, and we work kind of at the intersection of tech policy and cybersecurity. In the past couple of years, we've had a really great opportunity to work a lot on software supply chain security and do a lot of kind of translation from the policy community to the tech community and the other direction. And part of how we're trying to frame our thinking is looking at sources of risk and sources of security. And one that we recently kind of honed in on as a source of security has been the kind of pipeline of external vulnerability research from researchers and researching entities to private companies and open source libraries. So I'm sure many of you know the bug bounty platform market has been growing and is projected to hit some pretty crazy numbers in the next couple of years. It's a pretty direct manifestation of the many eyes theory of security, right? More people looking at code, better chance of finding and fixing vulnerabilities before someone exploits them. And we've also seen some increasing government adoption. Okay. So governments setting up their own kind of programs and giving some more clarity to the kind of legal protections and guardrails around researchers. And the whole process is generally called something like CVD, coordinated vulnerability disclosure. And we'll kind of refer to it by that throughout, but it varies between organizations. And we had a really interesting opportunity to look at some anonymized bug bounty data. It came up in congressional testimony in February. And it got us thinking about kind of concentrations of this source of security. And the big question we asked, you know, part of what we do as policy folks is try to figure out good policy. But another part of what we do is try to figure out bad policy and how to avoid it. So we started asking ourselves what kinds of policies could disrupt this supply of security, the supply of research, the coming from the community. And we had a really excellent opportunity to dive into that question and look at it with some data after the log 4J disclosure and kind of the saga around that. So at the front end, and this is a quote from Carnegie Mellon's CVD, All You Need to Know Guide. Kind of describes CVD as picking somewhere between finding a vulnerability and never telling anyone and finding a vulnerability and immediately telling the world. So trying to find that optimal path of information sharing that gets you to fix bugs as quickly as possible without kind of revealing exploits before patches are ready. So the general kind of timeline around the log 4J disclosures is a useful example of how that kind of information management happens. So research at Alibaba Cloud found the vulnerability and just directly emailed the Apache kind of maintainers of that library in November 24, 2021. And then followed up as he noticed that the vulnerability was starting to make its way more publicly into some cybersecurity and fora, which kind of prompts the Apache need to patch more quickly and to get people in a position to mitigate and to remediate. And so within a couple of days of that second follow-up email, Apache starts pushing out their patches and different kind of government organizations start notifying everyone. So that's a good management process, right? You tell people who need to know and can fix the problem. And then once that information starts becoming more public, you tell everyone so they can kind of prepare and do their side. But then we saw something interesting. On the 22nd of December, the MIIT, one of China's kind of ministries of this is industry and information technology sanctioned Alibaba for how they handled that disclosure process. They were sanctioned for not telling the MIIT quick enough about the vulnerability. And there were two kind of legal mechanisms that might have been part of that sanction, which involved removing Alibaba from an information sharing forum. One was that as part of the information sharing forum, they had to share information and they didn't do that. And the other that we wanted to look at was there's a law in the books in China in effect since September 2021 requiring companies that find vulnerabilities to immediately tell product owners and then within two days to also tell the MIIT. So we talked to a couple sources and it's unclear which mechanism was kind of the force behind the sanction. But pretty clearly the letter of that law is violated, right? So you can see the little orange boxes. Network providers have to within two days tell the MIIT of vulnerabilities they find. And this to our mind is a really convenient case study for testing this idea of security supply from vulnerability research. So a lot of the reporting focuses on what happens to vulnerabilities given to government early, but that's not what we're focused on here, right? China is a huge producer of this type of research so it's very easy in theory for us to tell if some kind of macro movement happens there. The law is pretty clear and we've got some good dates to test before and after. And the kind of mechanism for chilling effect is also pretty clear, right? If you're a private company operating in China, you have to kind of consider your internal policies receiving that information. And if you're a researching entity, you might want to wait on disclosing and even on performing research until there's more legal clarity because the law itself is pretty big. You know, it definitely applies to product providers. It's unclear to what extent it covers researchers who might be vaguely affiliated with a company, might be sponsored by a company, might work for a company, but are working on their own free time. And to be clear, this is not just focusing on China, right? We see similar issues going on in other countries. It's just not as clear of a case study for us to analyze, right? So in Germany, a researcher found a vulnerability in an application for election polling and was threatened with legal action even though they follow the right channels. And something similar happened in Missouri. I've seen a couple of those. F-12 is not a crime. T-shirts are round, right? Guy looked at a website, figured out he could pull social security numbers out of it and was threatened by the kind of state government. But we've also seen government kind of embrace this approach to security more. DOJ declining to prosecute good faith research, whatever that might be. And Anisa and SZA each pushing for government agencies and member states in the EU to develop their own explicit processes which is a better idea of what to do. But this is the main question, right? Did the RMSB have a supply shock effect in the Chinese research contributions? And if so, what did that look like? Did it trickle into the broader ecosystem? And are there other shocks that are worth kind of looking at? You know, along the way we got to look at just some really cool information about how that ecosystem works, where different companies are kind of pulling in research. But the basic selection of data we looked at were the acknowledgments in CVEs and security updates from Microsoft, Apple, F5, VMware and Red Hat. Give us a nice selection of different products, some consumer facing things, some very in the racks type things and with Microsoft kind of a broad swath of all that. As well as some proprietary and open source code bases and a nice spread of different kind of internal processes for how companies report this kind of information. And so we're going to take that data, match it against a pretty simple timeline, right? The first RMSB kind of public mention is July 2020. Law is passed in July 2021. And then takes effect September 2021. And if we have a recent enough data, we can even look at kind of post first maybe enforcement even though we can't really confirm that it was enforced at that point. They don't exist in a vacuum as events of course, right? So the summers of 2020 and 2021 both have a lot of other cyber activity going on in China. They have increased sanctioning of Chinese companies by U.S. companies. And they also have a couple other regulations being passed that might interfere. So even if we see an effect that lines up with our time, we do have to be careful about kind of assigning causation where it might just be correlation. But our basic process is pretty simple. Pull out all that data, either some scraping text or just direct downloads and huge things to those companies that make that easy to do. And then parse out a CVE, a relevant date for when that kind of disclosure happened. And then the entity credited. And then we had a kind of list of companies and a list of individuals. We just focused on the companies both for kind of concern for the individual's privacy and also just for ease of access. It's much harder to tag an individual, it might be a Twitter handle or a GitHub account to a location. But for a company it's pretty straightforward and it gives us a good proxy for the legal context they operate in. And then we just batch those by month, clean out the data and analyze it. And these are kind of the five companies again that went into that process. And there are a lot of issues with that data set. We eventually got it to work, we got that data stored, how that's organized. So Apple, for example, will tell you about CVEs that it remediates and updates. And those updates might predate the date that the CVE is kind of publicly acknowledged. So there's some back work going on. Other companies organized by CVE, some companies organized by a little bit of a blend of like affected product lines. And there's tons of miscodings and typos and different organizational practices. But in general we get a good sense of where things are happening. So we're going to talk a bit about the Microsoft and the Red Hat data. We had some interesting findings. In the Apple and VMware data there was no supply shock effect. But we'll show the Apple data a little bit just to give a sense of what that absence that no finding looks like. And for the F5 data we didn't find any even Chinese affiliated entities participating in that contribution ecosystem. So here's a quick look at the kind of proportional breakdown by month of contributions to vulnerability disclosure in the system. So looking at the left part of the graph we get a pretty good idea of a general story throughout our data set. Early on the U.S. is kind of the dominant contributor. Then it's other countries, IT sectors mature. It's other kind of relationships are built between the kind of security center at Microsoft and all those researchers. And as they develop we see more participation internationally. So this orange line shows increasing Chinese contributions but it's true of pretty much all the other countries. But what's really interesting is the steep drop lines up as you can see on July 2020, so one of our key dates and interrupts a pretty clear trend. So to get a bit more of a close look at that we look not just at portions but the raw counts of contributions and the green line at the top is just the total number of contributions. It's a little more of a nuanced story there. So as you can see the drop is a bit of a return to the mean for the Chinese contribution count but it takes place in the context of overall more contributions happening at that time and not a corresponding drop in that total amount. So the trend lines are a little less dramatic but that steep decline is still there at one of our key dates. It is kind of our most compelling key date. It's when a lot more sanction activity happened between the US and China and it's also when a lot of those kind of new Chinese cybersecurity regulations were drafted and started circulating but it is also a little harder to assign money to any single one of those laws because of that kind of flurry of activity. The other data set we wanted to show off was the red hat data and you'll have to forgive the very spiky stuff on the left here but it went a lot farther back to about 1996 and early on again very US dominated and then more countries start to participate. We see kind of more general representation but again we find an interesting decline here but not at a date we immediately had a good explanation for. So this is between February 2017, steep decline of the proportion of Chinese contributions to this research ecosystem and again a kind of disruption of a trend line that's kind of then resumed afterwards but at a decreased rate and if you look at the contribution counts rather than the portions, same story again it is worth noting too that because as an open source ecosystem there's a lot more individual work going on, there's a much bigger gap between the total contributions and the by country contributions but again same story and then kind of maintained throughout as a decreased rate so we had the chance to talk with some other folks and they pointed out that around this time frame red hat had entered an information sharing agreement I believe with Huawei so there's something going on there we don't know the exact mechanism or if that's even the relevant event if there is one but it is worth kind of looking into and then just for final context here are the Apple data sets and there was no drop off here but it is nice to note that kind of again very prevalent ecosystem and then as other countries kind of tap in more broad representation and here again are the kind of total counts rather than the proportions. It's very spiky data too, there's some seasonality I think every quarter and then every other month there'd be kind of an increased amount of security updates going out but to take away from those very very crowded graphs our main finding was that on the one hand there isn't a clear kind of ecosystem wide supply shock occurring because of the RMSV yet although it's really hard to judge the exact timelines going on but there's definitely potential for one given that kind of very steep decline in Microsoft that seems easily explained and also that less well explained but still present drop in the red hat data so our main kind of recommendation to government and policymakers is to not take the kind of supply of external research for granted and to find ways to kind of invest in that community either proactively avoiding supply shocks in general just to improve kind of the volume and the spread of research they're getting so we figured about three kind of different avenues to approach that for government and industry one is just for governments and partners of governments to better harmonize their kind of CBD processes and guardrails by providing clarity for researchers and not having to force them to worry about jurisdiction, right a researcher in Europe working on an American product but perhaps finding the vulnerability while they're somewhere else is useful and it doesn't have to be too explicit that it kind of limits the scope of researchers and their ability to look at things in an interesting and different way another great tool is just supporting kind of vulnerability discovery tools takes a lot of the burden off of researchers and gives them kind of new angles to plug into that work without having to start from scratch and then finally we wanted to take a pass at trying to incentivize work in areas where either some kind of supply shock might have happened or where there's simply not a lot of research going on in general if there's some product type or ecosystem where there's not a lot of vulnerabilities being found, any vulnerabilities found there have more security value and government and industry can shape incentives and bug bounty rewards to reflect that kind of increased value but to quickly sum up our work we wanted to start thinking about supply side vulnerability research start thinking about it as a community that contributes to overall security and figure out can bad policy kind of shock that system and avoid it and we found that our best case study so far doesn't provide an overwhelming abundance of evidence for that supply shock but it is kind of compelling and a little bit troubling and fortunately there are things that governments and companies can do to kind of protect that community and that doesn't depend on there being necessarily a clear case for a supply shock because it's a good investment to make in security and finally this is still very much a working project for us we've got some open-ended research questions about which companies stopped contributing to the Microsoft ecosystem for example so we'd love kind of any feedback any kind of experience you all have had contributing to work from companies bug bounty programs are directly happy to answer some questions and our emails are up here if you'd like to reach out and you can check out the full kind of draft report on our website but thanks so much for having us yeah sure we have a few minutes if you have any questions if not happy to give everyone some time back also cannot see out there so awesome thanks so much