 Keep it all right It's a little bright we did not plan effectively we good We do have slides They're pretty cool Hi everybody and welcome to I'm getting a little feedback. That makes me sad Sad sad better Welcome to preparing for zero day. We have an exciting panel here talking about Managing vulnerabilities and open source Let's introduce our panel first off. Why isn't and introduce yourself and talk about the perspective of the persona? She's gonna be talking from today. Yes, absolutely. Yeah, thanks for being cool with the Yeah, so my name is Amber Tosio. I'm one of the leads in Google's open source programs office and Kind of how we operate one of the things we do We're a small team that really helps all of Google work in open source. So we're kind of frequently teaching folks This is how things go, you know, here's how you be a good contributor and that does include vulnerability disclosure And so that's really kind of the first that maintainer perspective is perspective. I'll be speaking from on this panel Excellent next up is my friend art when introduce yourselves. Sure. Those that don't know you. Thanks. Thanks, Grobe art mania. I'm at the search coordination center the search dot org way back in like 1988 One of the things I spent a lot of time on is coordinated vulnerability disclosure sort of stuff. We're talking about in the panel today. I Think my persona or role is sort of probably third-party coordinator. So I'm not the researcher nor the vendor where this middle party that Tries to help sometimes helps. Maybe sometimes slows things down, but that's the role that I'll be excellent and myself, I'm probe I Work with a lot of open SSF working groups with the forum for incident response and security teams I've had the opportunity to work with these fine people both professionally and in the foundation for quite some time I'll be representing the supplier persona I spent about seven almost seven years working with Red Hat product security, which is one of the largest open source Product security teams around so we got to deal with a lot of these opportunities Some major exciting things that you probably all had to work with patches and whatnot So before we start I want to talk about a little bit of terminology. So we're all kind of saying the same thing First off we have bug a bug is an error or a flaw in computer software that causes Incorrect or unexpected results or unattended behavior So this is typically something that you do a bug review file a report with the developer They evaluate it they fix the bug they move on this generally tends to be functional related Then you have a vulnerability vulnerability is a weakness in software hardware or an online service that can be exploited and has security implications so a Developer might think of this as a security bug or a bomb a lot of different ways They might refer to it. This is predominantly what we're talking about as opposed to a feature problem We have CVE, which I think now stands for CVE They changed it But basically that's a unique number given to identify a Specific security flaw in a piece of software or hardware and the great thing about CVE is as a consumer of goods and services For multiple vendors that CVE represents a specific problem so you can understand what that problem is how it works what it does and You will have multiple vendors that potentially are affected by the same CVE. So if you have a Dell server and an IBM server and their CVE 2022 1 2 3 4 and they potentially both devices could have be affected by the same flaw We have a threat which is a potential cause of an incident that may result in harm And then we have the always awesome and much loved embargo Which is a period of time where a security flaw is known privately prior to a deadline After which the details become known to the public and this is a thing that is used both in open and closed source Generally, it's done where a reporter discloses to a maintainer They agree upon a period of time that maintainer has to work on a fix To correct the issue and then once it's fixed normally you go out and have public disclosure A lot of different opinions and implementations have embargo, but generically it's designed to help protect the end users of the software hardware All right, so Being my I'll begin my duties as kind of master ceremonies. So I want to talk to my friends here From your perspectives, what actually is a zero-day? Sure, I can I can say one thing a colleague of mine Looked into this at one point during one of the heated periods of discussion on the internet and I he found you know He being a true cataloger. He found 11 or 12 definitions and looked them all and sort of did some analysis of it but Fundamentally, it's a surprise and and the surprise is subjective as to who is surprised But the concern we would you normally have is the maintainer supplier vendor developer gets surprised by usually a public disclosure So it's off to the races Vendors had no lead time suppliers had no lead time. They're racing to figure out what the problem is analyze it work on a fix through a patch Whoever posted Depends on what happens there Someone who might want to attack it is off to the races attacking and it's a giant mess and there's a lack of authoritative confirmed information. So, you know Twitter is ablaze with Discussion and everyone's running around chasing stuff Very expensive response when you have a zero-day Anything to add in I think you hit all the the big points, you know yeah, previously unknown and Frequently with you know, we are when it is released without a mitigation and I guess the one thing I would highlight is if you are Responding to a vulnerability report You may be actively working on it in private under an embargo But that doesn't mean that there aren't people that also have this potentially discover this issue potentially might be exploiting it And the format we're going to go through is I'm going to ask a question of the panel will have a conversation The audience has any questions along the way Feel free to raise your hand and shout it out or repeat it and try to address it And we'll also a lot time at the end specifically for the audience or the cyber audience Dylan to be able to answer questions my friend Dylan might be watching see I got a little build here I'll get out of the way just some more information about Zero days And I'll I will figure out a way to post this up to the sked so that anyone that's interested can review the slides We have a lot of annotations and links for people to look at if they are Vulnerability CVD enthusiasts All right, so my next question and I'll start with Anne on this one How do projects share vulnerability information? Is it always the same they do it differently? Yeah, but how do we share an open source? Yes, so I think you know kind of a Misconception is that there's only one model of vulnerability disclosure vulnerability disclosure is just a process for Documenting and communicating and sharing information about a vulnerability and it quickly gets very philosophical I think everybody is acting from You know what they believe to be I'm protecting the user I'm doing what's going to do that best, but there's wide disagreement about how that should be done Am I getting feedback? No, everybody's good. Okay, so I think actually have a little chart Yeah, you can kind of put things on a sliding scale where you know full disclosure the intent being I'm going to share This as widely as possible as quickly as possible not do something like Share the information privately beforehand because I think that the sooner the users have this information The sooner they can take action and that is what's important On the opposite side of the scale Private disclosure is coming from the idea that you know if we widely share this information an attacker could use it and Exploit it before the mitigations are applied. I believe that is not giving a lot of credit to attackers who are very Sophisticated and we have to remind ourselves that our attackers are very sophisticated So if a researcher found it, you know to be conservative We should be operating with the model that they're potentially it has already been found by an attacker who also has that information in The middle is pulling a little bit from both sides is coordinated disclosure Which as a working group in the open SSF is really our recommendation to open source projects for the most part I know if What do I say there's some very strong feels around it. Yes. Yes, and I think we were assured that this room is Folks are here from more than just the kernel other projects involved. I know the current has its own process Generally for most projects our recommendation is coordinated disclosure. Do we have an quite all we do have another Yeah So the thought being in coordinated disclosure that the best way we can protect users is Recognizing that the security researcher has once the project maintainer has once the project user has once We're gonna balance all those but also with a lot of respect given to the researchers findings This is this is theirs. And so we're gonna have some kind of mutual negotiations about things like timeline The maintainers might say we prefer to disclose on a 90-day timeline researcher might say I'd really actually like 15 For these reasons okay 15 it is if the project hasn't patched and disclosed by that 15 Those 15 days researchers free to you know share their finding how they see fit So it's kind of a balancing of that model things can get a little bit complicated Which is why for open source projects we developed this guide to CVD If this is kind of new information for you I would recommend finding time before there's a very an active vulnerability in your project to read through the guide and Then when you do have that there's a runbook if you need that kind of reminder checklist of steps things to do And we also made a lot of communication templates that are in a directory in this get hub repo and and from a From an ecosystem perspective You'll hear oh there is no the open source and Operates the open source way every project and community is unique. They have their own tooling practices communication kind of acceptable norms, but in General when you're looking at open source software and Ann and I just had a presentation where we had some interesting statistics about how many projects have dependencies and dependencies upon dependencies and the understanding is no software Exists in an island and there's very often a community of potentially collaborators contributors to your project that might depend on your software or Some major downstream ecosystems potentially that need to have the time and ability to potentially Take and ingest the first-hand patches and maybe make alterations in backport Or just potentially get it deployed to a wide fleet because ultimately the goal of CVD Is to ensure that no end user of the software or hardware is put more at risk than any other group so everyone has opportunity to apply the fixes at the same time so that no one group can be attacked more than another and that's Why you will see some passionate conversations around Have in maintaining privacy around issues at least until an idea of how an issue is being going to be fixed Anything more to add aren't Couple of thoughts the one I can remember most easily is and you sort of cover this There's not one single open-source way to do things, but from from my role where You know in one sense we're agnostic to what software has the vulnerability it can be very proprietary It can be very open it can be a mix The open-source community generalization Actually means the word open pretty seriously and sometimes the projects or the The software's policy is file it in our public GitHub. It's a zero embargo. We don't care We'll get to when we get to it and Saying they don't care isn't quite the right right phrase But they have a very open process and they may even document this and the answer is file a GitHub issue You can say security if you want, but it's just going to be dealt with in the open Which you know zero embargo is Sort of a self-zero day situation, but again if that's the maintainers and the and the projects policy that Might be okay, and whether or not I think it's okay if that's how they're going to work on it We still want this fixed so you may you may go with a zero zero embargo, and then I remembered Yes, I think I think anus is your point right we I spent a lot of my time Deepen embargoes and counting down days and thinking how secure we're being Sometimes lose sight of the fact that Probably somebody else found this just hasn't been talking about it loudly And sometimes after the fact we look back and see evidence that indeed someone else didn't know about it while it was under embargo, so There are certainly classes of threat actor that are sharp enough to have discovered things and in the big picture They do So your embargo might not be as secret as you think it is I try to keep that in mind On the other hand The the whole goal of the embargo is a risk reduction right somewhere in that middle of the line is an optimal It's not perfect If you can keep the exploit sort of off the public View and off of Twitter and off of Metasploit for a few weeks that actually limits a certain class of attacker Who's just kind of grab public stuff? You know plenty of ransomware that is costing people money just uses a bunch of known already patched vulnerabilities Which is a whole different problem, but there is some risk reduction in that embargo. We think or I think Just that yes, some attackers are gonna always sort of know better, but if you can cut off Maybe the 80% The old term used to be script kitty, but I don't know what you talk about what you say these days Right general-purpose background noise crimeware rough class of adversary The embargo might help with that and if nothing else And I work with the security vulnerability Security disclosures working group within the open SSF and who created this guide and our big advice to maintainers and communities is It doesn't matter what your thoughts are around how you want to handle Security issues the most important thing is you talk about it and agree upon it as a project and as a community And please document that so that consumers understand what your community's rules are and so that reporters know Because you may have a reporter that's just interested in trying to get this is altruistic And just want something fixed and reports it to a community that potentially might zero day itself So it's important that you know if you have policies that a bug is a bug is a bug and we're just gonna fix it in The open just make sure you have that documented so that as people are trying to work with you and report responsibly That they understand kind of how that your community works. Oh, we have this So we have we put together some interesting good ideas around CVD and some Things to look out for so for example Generally within the industry especially if you're working with a Commercial entity that has any kind of planned release schedule. You'll typically see Requests for embargoes in the the 14 30 60 or 90 day windows so if you're a reporter coming in and Asking a project to fix something you'll very typically see these types of things So just understand what works best for your community and your project We strongly recommend that as Vulnerabilities are reported that you get a CVE identifier so that your downstream and actual end users can understand What that issue is and what they need to do to fix it and there's a lot of different ways you can get that Your project potentially is a CVE. It's a an entity allowed to give out CVE identifier so you could contact It's is it still miter The CVE program would be the proper term You contact the CVE program directly through a web form to request it could take about five minutes or so to Get a CVE now on a lot of different way or maybe you have some third party that does your CVE Assignments for your project a lot of different ways But it's a recognized good practice that if there is a security vulnerability You need to find some way to identify that so the consumers of your software can understand what they need you to fix it It's also strongly encouraged that you publish some type of advisory. It doesn't need to be a long 20-page treatise on the state of global security, but you need to Inform your downstream that this is a bug. This is the CVE This is the patch version or if there's other tech other things that need done to correct that flaw So you have that documented you give it in some type of format preferably machine readable like a CSA for oval or something like that and It is very typical After a researcher reports of vulnerability to you depending on the severity of it that they're probably going to either do a tweet do a blog potentially Get their PhD around it or they might present it a conference So these are very typical things in the research community. So just be prepared and understand If your project needs to do any additional Preparations around this type of thing. Do you want to talk about the bad stuff and sure and I'll just add one thing on the The conference but you know, I think sometimes it can surprise people at security researchers when we're talking about timeline negotiation They say no no or I really need to hit the state and You're going as a maintainer wet who cares about a conference if you're a security researcher Depending on you know your career path that is actually very a very important place to get your work recognized and credited So sometimes that strikes folks is odd, but in that career path. That's a really important thing So things that you know, you kind of should kind of raise your heckles a little bit Asking you for money and this is outside of you know, you're not in a bug bounty. Some people call this a big bounty There should not be any sort of extortion or bribery if you are not participating in a documented Vulnerability disclosure program that has cash rewards if it's not part of a bug bounty where this is in scope somebody asking you for money should Yeah, a little a little sketchy probably should not engage and just prepare yourself if this is really real There might be a zero day against your project Similarly, if you know, you're asking for can you you know You're as part of going through the recreation of the mobility yourself to make sure and verify it if they're asking for privileged access To your systems or infrastructure to do this also sketchy If you get something I say no, no, just just run this I swear this is what I did Don't do that Yeah, yeah hard hard pass on that one Any sort of you know, NDA or legal agreement and again I know some bug bounty programs have NDAs if people choose to participate in that But if you haven't set that up as the maintainer you shouldn't be signing anything You know to get this from the researcher And this is where we know again if we go back to that model This one's got a little caveat So researcher dropping zero day without basic things being met like really trying to attempt to contact you You know if you're going back and forth about well We weren't able to recreate this and they say no, no I swear it is and you kind of have that conversation And if they don't do those things and they drop it anyways that can be a little odd However, if they are someone a researcher who really abides by the full disclosure model and that's how they operate Then that would not be unusual. So that's the one where that's a little bit of a caveat Anything else good or bad you observed art? Yeah, just a bit on on the you know researcher trying earnestly Yeah to earnest attempt to reach the the project Very often reaching an open-source project is shockingly much easier Maybe not shockingly than reaching a Proprietary or closed-source vendor or supplier So that's a huge plus nonetheless In this policy that you have and to have documented already about your desired embargoes Please include sort of how you want to receive the issue even when we deal with an open-source project It's often we might post something publicly and say hey, we have a security report. How would you like it? How would you like to receive it? Please just save that trouble and just say, you know use github security advisories or PGP adhere or send it to our bug bounty Or post drop it if you can just make that part of that initial document a reasonably professional mature researcher a reporter kind of find that and they'll just follow it All right, let's move on to our next question. We've mentioned a couple of these terms Could I ask the panel to explain the difference between? CVD VDP and bug bounty Who'd like to start you got squinty. I'll uh, I'm enthusiasm or no, I'm squinting not fear I'm squinting so the CVD VDP difference is the most squinty to me, so um, I will my opinion right so CVD right coordinating the disclosure is sort of a general thing It's probably the most umbrella of the three terms. I guess I would say I Treat a VDP as a official very official project that someone is operating VDP typically does not have Money compensation involved There might be a you know hall of fame web page or stickers or something or thanks afterwards But nonetheless VDP is good because you someone's writing a scope and saying whether it's whether it's my software products or my systems or in the case of the Department of Defense runs a VDP for The DOD and I figure what their scope is but it keeps increasing a little bit every time I checked There were more of an operator running running a VDP and to step back CVD stands for Coordinated vulnerability disclosure and I agree with art It's a description generally of a set of good practices VDP is a vulnerability disclosure program Or policy it depends I go with program but sure yeah, so that there is some organization around Collecting triaging and responding to I guess I would say that it's desirable at a VDP Practices CVD or encourages it although they can't control the actions of others involved and then briefly bug bounty There's an official program and it might be a commonly an outsource this right because there's a international payment involved possibly And you want to outsource the payment part for sure Right the bugs of this nature will receive this level of compensation And there are some more terms the researchers probably agreeing to going into it in order to be rewarded compensated for for the finding Interesting entirely interesting topic bug bounty at least always but these days again, so my version of the differences Hmm We'll see but I was gonna say Normally bug bounty is associated with a commercial entity, so a corporation that provides goods and services to people that pay money for it Open source projects can and some do elect kind of outsource their vulnerability management to these third parties Otherwise open source projects might be Contacted by these organizations saying we have a researcher that found a problem in your project Click this link to log into our system to learn more and that's again It's a reality that projects don't necessarily have to have their own bug bounty But it is a reality you may encounter that as you're working in your community and releasing your software I would say for maintainers, you know I like to think about these in terms of kind of this maturity Timeline and you are not ready for a bug bounty program if you don't have a smooth process And I think you know with the advent of these bug bounty vendors It's very common and easy for maintainers to go. Oh, we need a vulnerability solution. I'm gonna Google something Hey, this program is here. Let's sign up for it. And you know, and we think about the difference of the ethos VDP Program is kind of the you know, if you see something Please let us know versus a bug bounty is I'm going to dangle cash and actively incentivize you to go look for a particular set of Vulnerabilities that are in scope. It might actually be leaving out things because they're lower paying but do affect your project those are very different kind of takes to that and I think there's umpteen examples of Projects signing up for bug bounty programs not realizing that now they are turning a fire hose on on themselves for potentially You know a lot of the same report false reports You're not ready for that if you don't have a smooth practice process Absolutely All right, let's see if we have another slide We don't We cut it. All right. So our next question is From your perspectives what are some common challenges you see in the Coordinated vulnerability disclosure and this could be an for example or you see you recognize a bad practice So who would like to start on this? I Can go with it. We're just general challenges trying to do a nice. Yeah. Yeah, we've talked a little bit about timelines But I think that is a frequent challenge is that people who maintainers of projects misunderstand the dynamic with a researcher By choosing to contact to you even attempting to contact you They've already done you a favor the first answer is always thank you No matter what happens after that the first thing is always should be thank you And then when we get into timeline, you know, sometimes I've seen projects say oh, well We we're just not ready to sort this out in seven days 15 days We always like 90 it's like well, that's that's not your choice If the researchers agrees to that then excellent you get your 90 if they don't they don't and that's just kind of the dynamic of this So that's a common challenge. I see I'll also note while you're talking about timelines Generally the researcher controls the timeline, but that doesn't mean that the the maintainer or the project is completely Helpless you do have the ability to negotiate or you can say, you know We're planning our next major release We're going to include this fix in that or you know Maybe you're super agile when you can deploy a fix in a couple hours and you know the timeline might not be an option Or if holding it for that conference is Like a testing problem if you write the fix and then you have a lot of builds in between It's challenging to continually do regression testing on this patch you're holding so just it feel if you're contacted by a researcher You don't necessarily have to accept the date. They say as Gospel you have some negotiation you have some you have the ability to push back and help Massage and coordinate something that is amicable to your project as well That's a great pressure and that you know if you're in this negotiation explain why that you have your change in timeline Just saying we're not we can't move that fast as doesn't usually is not sufficient But if it's something like we're working on this patch And we we're gonna miss this deadline because we just don't think this fully mitigates the issue a lot of researchers are sympathetic to that and you know happy to work with you and extend it because you're actively trying to fix this couple a couple of points I know of at least one open source project where They're the ones sort of driving the timeline so if I'm a researcher or my role as coordinator I forgot and I come and say 45 days, which is our very soft, but starting point at certs. He see This vendor says nope. It'll be out tomorrow They fix aggressively and quickly and that's their more or less known policy. So The the fundamental piece to keep in mind is anyone with knowledge of the not yet Public vulnerability has one leverage point, which is the power to publish And that can be the researcher that can be one of the projects if it's a multi-vendor multi-supplier issue system Yeah, it can be one of the three reporter researchers who found it. So that is the one leverage point Typically people don't get too Fighty about I'm gonna I'm gonna drop this if you don't act But that is the implication and to your points right it often reasonable people can negotiate and discuss things And if there's good faith and reasons for things that goes a long way Radio silence does not go a long way and we'll get you published on What other opportunities have you seen during coordination if you don't have an idea? I have one in my head Oh, yeah, sure. I'm one of you know mistake or challenge is the researcher might be really interested in patching this I think sometimes the maintainers assume that the researcher has submitted this to me and Then they're out. They're done and maybe that's how they want to operate But I always encourage maintainers to ask the researcher Would you like to be involved in the patching of this? That's you know our next step Because they might actually have some ideas They might have you know already kind of start tinkering of here's what would mitigate this and if in open source Nobody can really say no to help But you know, it's make just ask I've seen maintainers accidentally offend researchers by not even asking because they just assumed it would be bothering them and I Kind of go on the you know Give all the options to to the researcher I had to write this down so I would not forget it from at least seven minutes previous, but Right right on this topic Even if the researcher or a porter maybe isn't going to write the patch Ask them to test the fix great great time saver And even if you're under if it's the you know the zero-day Time pressure issue we started off with right public disclosure no maintainer prior knowledge off to the races Under that pressure you might write a quick patch, and it might be complete might not be complete might have a side effect Hard to do but even under stress under time pressure Test fixes and the researcher knows something. Hopefully they gave you a proof of concept or steps to reproduce if they didn't That's also a sign. They should be doing that to be professional Asking the test and again they're in their involvement now. It's a team thing. You're working together way better dynamic than Silence or we're fighting Say for my experience probably the single biggest complication of a smooth disclosure is Lack of communication So for example a project may be May not have a security policy published they might not have a web page or a mailing list or any way is very hard for the researcher to contact them and So as the researcher is going around and they might go to a third-party coordinator like art they might contact a bug bounty program, but if your Project doesn't have a means with which you can communicate with the public or at least say Questions to this address or it we were in discord or slack or wherever your community lives That complicates things and then once you've once as communication has been established It's in the project's best interest to be as communicative as possible Saying yes researcher. We got your report. We're going to triage it. It's going to take us Three days three hours Five days, whatever it is. We will contact you again on this date So being as communicative as possible because sometimes the researcher finds things that are Very corner case very esoteric hard to reproduce But don't don't feel bad if you you know as long as you're communicating with them You are less likely having that zero-day dropped on you saying you know what we can't reproduce this Can you give us an example configuration? We need more information or are you know the guy the guy or gal that works on that subsystem has these questions So just feel free to be communicative and always state, you know our next I will talk with you again on this day or time or within this time span and follow up if you don't do that that again encourages the researchers may be negative perceptions of Slacker developers and they may proceed to Publicly disclose which we don't want we want want the project to have as much ability to manage this This remediation as possible Any other kind of gotchas that you've seen or heard of I don't know if we have a slide on this, but so far We haven't strictly stuck to this but we have this you know, there's a reporter or researcher and there's a project And you hinted as you touched this earlier, but often these days. There's not one project Multiple projects and those projects have components used by proprietary vendors and It's not one embargoed in a good discussion. It's whatever one to the end forget the math on how many parties you have involved It's not a good number That gets messy very very quickly I'll venture to say there's not a good known solution to that problem. We honestly state of the art practices we Try to one of our roles is a multi-party Coordination when a third party is useful, right? So it's not one project or one researcher or one of the vendors trying to run the show potentially there's Third party might be funded to do this and paid to it their job professionally and that actually takes some of the stress off Well, who's who's picking are you biased towards your own choices? You can blame the third party Or for any choices or mistakes or anything like that, but multi-party Is a big mess we They can be done reasonably successfully but it's sort of a Manual process at this point every time So yeah, many many projects involved and these are these are supply chain is right a dependent component others use it More downstream Everyone has to figure out if they're affected by you know log for J upstream or Open SSL upstream or pick your pick your popular library of choice and that's I I lost my train of thought. Oh, let's spend a lot. I hope I didn't break the flow is a lot of slow No, okay our next question. Let's talk about some lessons learned what what practices have we seen that? Work within Whether it's closed or preferably open source. So what have you seen that works that other projects might be able to learn from? Well, you stole my favorite about communication Because you know, I think that that is just the biggest thing is learning the that researchers are not scary You know said it before I was saying again that they're contacting you. They want to see a good outcome I think sometimes people who don't work in security get nervous the all the stereotypes and kind of miss about who works in security They want a good outcome. They want to work with you. So communication asking questions sharing information All all that is good From my perspective And we've I've seen it with evidence over the years as a project matures. They have more capabilities Typically carving off part of your team or having people do double-duty where you have dedicated people that do the security work of the project Security team so to speak That is helpful because it can offload some of the work of maybe the main line maintainers as opposed to contributors Having your project be able to portion route some of the work to these security experts that can handle some of the the nuances of Getting a CVE writing the advisory coordinating the testing of the fixes kind of offloading that off of the main developers That's a good practice and that definitely can help and that's you know We've seen this with colonel and Kubernetes and open stack and Apache and and and there's a lot of really good examples And not everybody has a foundation Backing them you might be a one or two person project But there's also a lot of good publicly available resources like what we're producing through the open SSF There's a lot of security experts within the open source community that can help you can lean on if you need assistance What good have you seen are there anything else? I mean we've covered some of this really If on the maintainer if you can just do a couple of basic things how you have your desired reporting path published a security.txt or just the Security.md. What's the right? You know one of the both of them maybe They can say the same thing anyway Don't have them say different things. Well Sure You would not publish one policy not to the conflict. Yes Actually that is the thing to yeah for some reason you do have two policies. I've seen them go Wow, all right. Yeah out of sync, but anyway, it's a new honor As a maintainer if you want security reports to come in a certain way Just say so and publish it and make sure that's Findable you'll just save a lot a lot of back-and-forth email round trips right off the bat by providing someone a front door and then you just said this but If reporters coming to you the researchers coming to you That already is a big signal that they're kind of trying to help because they could just sit on it. They could sell it to Try to try to shop a bug bounty they could try to shop to a more offensive sort of broker depending on the nature of the vulnerability They could do nothing with it, which is all those are worse outcomes than them telling you So that's a pretty strong signal. They want to do something Helpful and they might not want to do much more. They might want to hand it off and drop. That's fine But it's worth asking and offering that offer of help and that sharing is I mean that's that's the human You want to help I would like your help Great sharing is caring. Just caring. I think your slide said it wasn't earlier But it is Let me see. So we have some real quick Recommendations and then we'll get on the if any audience has any questions But first off have a security policy. That's kind of our the single most impactful thing you can do it costs you Electrons on your source code repository and that's it Documenting your expectations your process and how to communicate with you Being contactable having a security at Project email address is very helpful. That's very typical in the vendor community So most commercial vendors will have security at Intel security at Microsoft as being contactable. So if your project can emulate that that's a very well known convention Again, you're gonna be casting a wider net collecting things Because researchers might not do their due diligence and find you on you get lab get lab hub Or whatever, you know mail form you guys lurk on Remember feedback is a gift and first mention this they their intention almost always is to make things better They might have other things on their agenda like it's going to get their them their PhD Or they get a conference talk or they get to put another CVE notch on their belt But the first thing is they're trying to help they found a problem. They're saying something Communicate along the way ask questions if you don't understand and Just to kind of help the overall security profile of your project depending on what your source code Management repository is there's a lot of really great tools that help you manage the security your dependencies That do vulnerability scanning that do static analysis fuzzing There's a lot of really good tools that are completely automatable and a lot of them Also interact with your developer ID ease so they could be right there helping provide advice while you're typing code And we have How we're ready for questions so does anyone have any questions or comments sir So that while art is thinking because he has the most data I'll rephrase the question so we show displayed a spectrum of different styles of Disclosing vulnerabilities and the gentleman asked is there any evidence to show How any of those are performing is one? Observably better through data than another and you know search CC does a lot of this work So he might have the easiest access to data. Yeah The data isn't like a lot of security data and no Sometimes sort of you know art we have our own cases, but that's a very small sort of subset And it's probably very biased in the way we get we get things reported to us I keep a I have a printout somewhere of a paper from a while back from some CMU professors and they have a nice curve and it's sort of It's based on some data at the time this would have been CDE and it from from this is 15 years old probably at least They have sort of a utility greater good societal good curve and you know never reporting the bug is sort of an infinite badness and Dropping the zero-day is pretty bad and you know somewhere They sort of thought that our 45-day number was good at the time But that may have been a data bias problem is is a sweet spot, right? I'm not familiar with anything that really nails this down and I think one of the big factors is There are so many different sort of kinds of software and software systems, so if you have a room full of Industrial control and operational technology operational tech people They don't want to change their working stuff because bad things happen and they have they may have hardware replacement lifetime cycles for updating software they would like to have Years for disclosure embargoes right and then open BSD will fix it as fast as Theo wants to and that is the answer and you can Live with that or not like I love it. No, so They're totally different kinds of software there may not be a one answer, but I have this almost permanently etched in my head curve where There's a you know on a line there, you know closer towards the faster release You know a quarter way into that line. There's some optimum Not very scientific, but it's it's there somewhere. I think and I think the labels we chose were kind of Arbitrary colloquialisms and when you're going through a disclosure at the end of it. You don't go back and mark this. Oh, this was you know Totally coordinated or this was you know totally private and so there hasn't been a lot of record-keeping on that meticulous But I think it'd be an interesting area of study. Yeah, sure you could cut it up. I Yeah, I Think it's very interesting. Mr. Wheeler. Do you have something to add to this? I'm sorry. Dr. Wheeler Do you have anything to add to this? Yes, we agree. It's a good idea. We'll take some notes. Thank you Go ahead David So pause for a second So David mentioned that there is a tool called the open SSF Best practices badge that developers are able to showcase how goodly they do security and one of the ways that they You don't earn that badge is by having it be difficult or not able to contact you for security issues And then what was your second point, sir? Test fixes even under time pressure But it eat that you need to do better than what does test mean, right? How many people or do they know the tech very well? Yeah, breaking breaking patching the exploit breaking the exploits a common thing and the the closed-source world makes this mistake all the time Well, it's not an open-source specific thing so and I'll note and like the log for JK specifically while downstream it was Frustrating that the first fix wasn't a full fix and within 24 hours a second patch was released that remediated the totality of the initial issue I Think that's pretty amazing from an open-source perspective We should be proud how reactive we can be when we need to be and we recognize that the first fix wasn't Comprehensive and they went back and did additional work without being asked or waiting for someone to find it again But yes that and that is something that could be measurable how many regressions There were from an initial patch, but you know, I think as I said testing testing testing testing Testing with a mind towards right that attacker point of view of I'm gonna just you know make a small change and still bypass your fix Who's that you go ahead straight in the back. Yeah, so to rephrase your statement Generally when you're working with a researcher They have much more if thought about the issue much more than the maintainer developer has the developer is gonna Get the problem react to it try to fix it but ideally the researcher has a lot more knowledge potentially and can provide a lot more feedback and Directly to that point and and I the working group were in on the open SSF The vulnerability disclosures working group We're working on a new coordinated vulnerability disclosure guide and it's focused on the finder persona So we are giving Recommended good practice to researchers and I'd be glad to specifically get that added in that you know You collaborate with the maintainer provide help test test test I'd be glad if you want to provide me some more a definitive wording I'd be glad to get that put in the guide gentlemen Right here the way no sure Have a virtual question. Oh, is it from Dylan it is it is Okay, have you seen examples of reporters or partners with CVD being excluded or ignored for disclosing in bad faith Is there a reputational cost that comes due for bad behavior? Thanks, lady open source is an amazing community where you're able to Produce work and your peers get to provide you feedback on it and hard work and good quality code gets rewarded in the same vein if you are a bad partner and if you leak information that's shared to you or you Immediately disclose early that there potentially are repercussions because this is an ecosystem we live in communities and need to be respectful of your neighbors in that community and Yes, in the existence of the open source there have been some people that have damaged their reputation and their inclusion in in helping fix some of these things because of their Unprofessional and uncomunity like behavior, so it exists. It's very definitely the rarity You know normally when people get included in trying to help fix these kind of things people are very professional They react pretty quickly, but you know sometimes we're all human and sometimes Some humans are nicer than others But I think showing the the humanness is an important thing that works well You know when mistakes are made like this this is this is a process between a lot of people and Parties and mistakes will be made Reputational damage has real outcomes. I can tell you the story of a project That you know they were nervous about something and As we were talking about you know afraid to ask the bother the maintainer or excuse me bother the researcher They were afraid to bother the researcher and kind of fell into that radio silent Mode just trying to do what they thought was the right thing and the researcher was very frustrated like what's going on what's going on and You know the maintainer wrote back eventually and said I'm I'm sorry. I didn't mean to be radio silent for so long I thought I was bothering you here's where things are at We can't figure out how to patch this and they worked together But in that interaction the researcher said you know Thank you for saying that because I was planning to never report to you again because prior to this this sucked and so I think just you know sharing those things and Repairing those relationships is really important because it that reporter was not going to come back And if you ever are involved in some type of private disclosure Provide feedback how your this worked well for me this didn't work for our community and you know in the future We're going to change our rules So just provide feedback to everyone involved especially You know if you see areas for opportunity if you're involved in a very big coordination Provide that feedback so the next time we can do it better for everybody for the next group of folks that potentially are impacted Small window Man my straight man So those of you that were here for open ssf day or have been in any of the open ssf talks We released a 10-point mobilization plan where we are proposing back to the ecosystem Here are 10 specific areas we can Concentrate our on and help improve the overall security posture of all open source communities in the ecosystem so Specific to your question and the question was is there going to be a platform with which to help ease some of these communication and coordination problems and that issues dealt with a stream 5 which is Lovingly titled the OSS cert security incident response team So myself and members of the community and it's open to anyone that's interested in helping collaborate on it We're going to try to focus in on these specific issues and one of the first things we're looking at is the cert CCs Vince platform which was just recently open sourced and that's a tool that a lot of commercial vendors today use To coordinate these complex multi-parter disclosures and we think that potentially this could be a good platform for anyone an open source to use Because it allows you to allows the researcher to report to a maintainer and then the maintainer to include other parties They feel need to be involved in that conversation and kind of help set the rules in tone And then you could include experts like from this new OSS cert to potentially help with advisories or negotiation of a timeline if that's needed or You know just trying to track down additional people that might need might be needed to help Worn or maybe subject matter expertise to write a patch potentially so we are working on this So if anyone's interested, that's the open source Mobilization plan and stream 5 and we start work We start group meetings the first week of July and we hope to actually start to take action on this before the end of This calendar year Any additional questions or comments Sir So the question was how how do we feel if there's a vulnerability and it's discovered that it's being actively Exploited during the embargo period Embargo is over. Yep And it's off to the races. Yeah, and ideally people have enough warning that they've prepared at least some type of advisory or statement We are aware of this problem and we're actively working on a fix So there will be times and if it is ever noted that it's publicly abused the embargo is off and everyone is kind of Hopefully able to inform their constituents of what their plan is and potentially a but an ETA for a fix might be Sometimes Yeah, and the statement there was even if there is not a fix available in the embargo breaks We always should try to provide additional advice for compensating controls firewalls any kind of IDS type rules anything to help any alternative mitigations until that actual software fix is done Excellent feedback. Thank you Other questions All right, so we've I've had this up here blinding and this is how you can get ahold of us here at the open SSF A lot of different ways. We have a lot of different working groups The mobilization plan has 10 work streams that will eventually be incorporated in those working groups But you can get ahold of us on github The web page is open SSF org We have a YouTube channel where you can watch hours and hours of working group meetings It's great. We have slack channel and all of our working group meetings the slack channel is all open So if you're interested in contributing to try to solve these problems or you have ideas or suggestions or comments This is all open to the public. We invite you to come join the conversation with us. So with that Thank you very much. We have a disclosure guide. We talked about there's some other really great Resources, I'll make try to make this available through sked if I can figure out how to get a PDF on my phone And that's it. So thank you very much. Have a great day Thank you