 Thanks Beijing hackfest reminder to register Todd I don't know what the registration is I don't think I've registered I know I'm going but So we want to do that and then again, you know as we usually say, you know Feel free to go in and add things to the agenda and well We obviously we run these as sort of an un-conference When we get there and see who's there and who wants to discuss what but it helps to put things out there And sort of help to spark some discussion Bad certification update we now have Sawtooth and fabric at 100% and a Roja very close at 97 and I know Dave is is working with the teams there And so we have two discussions the first is For Dave to review his revised draft process for security handling CVs and And then there's the top-level versus sub project discussion. I don't know if Brian is on I Don't think so I don't think he is so if he's not on then I guess we're gonna defer that which means we just have the one discussion Dave All right, I'm trying to see if I can share my screen If not, you guys have I have to pull up the draft on your own Sorry, I might I was delayed three hours getting out of New York last night. We didn't take off until almost midnight eastern time Which means that I landed about three hours ago So Dave oddly it will not allow me to give you presenter rights I've not seen that before so I don't know Yeah I'm I'm calling it what I'm accessing it from my phone here, but I was trying to do it for my laptop as well So there now I'm in says you don't Posting a link to the draft in chat Yes Hold on a second. Why am I a game reporter twice? You're echoing Something has to be muted There, okay It says now that it's waiting to view your screen, so maybe you can share now Yeah, except I'm on a Chromebook and this isn't We'll get the link yeah Links in chat now. All right. Thank you. Sorry Dave. I know. All right, so This ties back to the earlier point that the CII badge as in process for our three main projects our graduated projects Part of the security section for getting your Co-infrastructure initiative badge was that you needed to have a documented process for handling security bugs and The teams that are at a hundred percent I have advised them to just point back to This document and the wiki that say we are we have a process or you know, we are going to follow the process And then I told them that as long as we got this process into place before we went to 1.0 that everything would be fine so this The security bug proposal is nothing new that this is nothing I invented Generally the philosophy behind this is that we just wanted to all agree on One way to cooperate on the security bugs It doesn't make sense for all of us to have our own committees our own groups of people handling security bugs Mostly because I wanted to spread the load out One of the CII badge proposal requirements is that each team identifies one engineer that has secure programming experience and Would I think that's all it says secure programming experience I want to leverage that list of engineers to build a committee that will review the Confidential security bug reports the plan being that as they come in We would all get together. We would triage them anything we thought was We would probably investigate them and anything we thought was a real Serious security flaw we would then pull in other engineers from the effective project on the need to Know basis and work with them Quietly and quickly to come up with a fix for them Then the next round of once we have a fix that we all agree on then We would reach out to any vulnerable large installs and give them a heads up so that we could deploy the fix before it's announced publicly and then Once we have all that in place then we will go through the normal CVE process announcing the The disclose or yet disclosing the vulnerability as well as cutting a new release that includes the fix as you can see most of this document is Basically out the Apache projects vulnerability handy procedure handling procedure. I went through all of theirs and I am I'm exceedingly happy with how it is. It's essentially the same as what we did at Mozilla and it's also pretty much the same. We did at linen lab. So I have lots of experience with these processes I just really wanted to put it in front of the technical steering committee because at some point the TSC is going to have to approve this The key highlights here are that we will have team members from each of the major projects Bug reporting is confidential. I have we have already set up the security email The security at hyper ledger org email reporting I'm also working with rye to enable security groups In Jira so Security groups in Jira allow The broader community to report a Jira and to flag it as a security critical bug and when it's flagged as a security critical bug it is kept confidential The only people that can access it would be the members of this security triage group and I'm working on a solution for github for the teams that are on github and I'm still looking at All of the projects to see to make sure that they're either on github or Jira. So Let me see. Let me distill it down to the points of discussion one There is I would like to have a standing Committee of security oriented engineers where we meet on a regular basis To triage incoming security bug reports. So that's gonna be TSC report approval to I've already put into place or I'm putting into place confidential bug reporting and handling and Three this document here is going to be the Document of record for how we are going to handle things which includes Which includes the process for responsible disclosure of vulnerabilities so Now that I'm at that point, I guess I'll open it up for discussion So this is Chris You're you're looking for just one or a minimum of one from each project a Minimum of one Ideally, I'd like to keep this committee as small as possible. It hopefully we're not getting a lot of security bug reports Really only It all really needs to be three or four, right? I mean, right I was just thinking that you might want to have a minimum of two because people go out they go on vacation Or they're not reachable or something critical Yeah, yeah, okay Sure, I guess we could have a primary and an alternate. I I don't want it to be a big Yeah, no, I Understand and you got eight projects now and more coming. I'm just suggesting that there be some sort of a phone tree at least to That's actually really good feedback. I'm that's actually a good idea I think we should do that we can at each of the teams can identify a primary and an alternate and The idea is that we would meet on a regular basis, but obviously if a report comes in that looks really scary I will I could potentially call a meeting on a short notice And also, I don't want to be the leader of this meeting. I will be a part of the meeting but I Will I would like that the security triage group to elect elect somebody to be The moderator the person who calls the meetings and runs it And we will also take Extensive notes as we go through the the bug triage process, right? So it'll probably be recorded in the confidential bug handling system for full transparency once it's disclosed Okay, and part had a comment part Yeah, sure. I was just curious what exactly we were defining as security bugs because for something like an OS, right? It's relatively straightforward. What's a security bug and what's not for a piece of blockchain software? It's not necessarily so clear It's something that Impacts certain aspects of the performance might be might be, you know disastrous for some applications Do you have kind of a guideline as to what you might consider a security bug and and what wouldn't be one and my Question there was a bug in a consensus algorithm. Would that be a security bug or not? Um Well, I think the rough definition is that any bug that allows some non or non-trusted actor or you know I guess even a trusted actor in a permission network to Adversely affect that functioning of the network that would be considered a security bug So yeah bug in a consensus algorithm that's preventing it from coming to consensus for sure would be a security bug now the the triage committee would look at these and decide whether it's a critical security vulnerability or not and For something like a consensus algorithm problem You know, it's really up to the committee to decide it could be something where it's like, oh well We're we're not able to get our throughput that we want because there's a problem in the algorithm So maybe we'll just kick this out as a critical bug and and let the project handle it But if there's a bug in the consensus algorithm and allows people to like DDoS the entire network and prevent it from functioning at all I would consider that to be a critical security bug and so we would We would deal with that quickly and Confidentially because we wouldn't want to reveal the ability to shut down and establish to credential network Okay, sure. I see what you're saying. Thanks. Yeah I mean in just a few if you define things really liberally then kind of any bug on a blockchain practically Can be a security bug But if we have people who are following them very quickly and kind of measuring the impact then we shouldn't be overwhelmed by Security bugs at least that's the hope Yeah, I think it'll be pretty clear. I think the vast majority of these bugs are going to be bugs that are remote code executions, you know, they're gonna be attacks on the IPC layer You know remote access attacks those kinds of things But you know, we will deal deal with bugs on Things like consensus algorithms and even smart contracts as well I think this is why we have a committee and this is why we meet on a regular basis And as we move forward, we're gonna get good at being able to spot what is it isn't a security bug It's hard to define it right now. Obviously, right, but it's general rule of thumb is is there It does does this bug allow the network to be shut down or to be adversely affected by somebody Who hasn't been trusted with the ability to do that, right? Would it be helpful to look at what Ethereum and Bitcoin have done in response to those in order to get an idea about what? Kind of appropriate response to algorithmic bugs would be Yeah, that's actually a good idea Yeah, so the Ethereum one and specifically right the Dow was a really interesting test case for smart contract bugs, right, so I Guess we could go through I I could go through it and I can put in references to that in the notes To help give broader definition or a narrow definition to what a security bug is Okay, it just seems like you know building on somebody else's experience specifically with with the kind of problems We'd see with the blockchain might be useful It's Marta here. Also, I think that we are very early stage in general with blockchain and We haven't seen attacks so we will just learn as we go It's it's hard to define what a blockchain bug is a lot of it will be just implementation problems with As we move forward, but we'll just respond bugs because they we will see attacks so Putting definitions today will be hard because well, some of them are easily defined. We know that There is a potential for DDoS or potential for 51% attack But some of them will be just well, we'll just have to face it and as Brian said yesterday We will be what we will know that we're doing a good job once people will start attacking us because that means that we're interesting Yeah, I agree with that. I think largely what we're gonna see is implementation errors at first And then we'll later on be become experts on smart contract bugs. I think as we start finding more and more of those To augment this process I we are also putting into place security scans security audits and continuous fuzzing For the various projects to try to get out ahead of any implementation errors The reason it Roja is only at 97% with their CI batch is because they are a memory unsafe language C++ And I'm working closely with them to get American fuzzy lock in place which is Which is notorious for finding Bugs, you know implementation bugs in memory unsafe languages If you look at the trophy case for American fuzzy lock, it's things like Libf. So the Apache web server the Linux kernel like previously undiscovered Implementation flaws that had been in the code bases for many many years. So That's the fuzzer that I've been going with with them and I'm exploring options for Sawtooth and fabric as well because there are Python and go adapters for American fuzzy lock as well So hopefully we're I mean we're taking a proactive stance on security here we're trying to get out ahead of it and It's about time we start standing up the the committee to deal with Security bugs because we're gonna start finding them from the fuzzing to so Yeah, are you gonna be able to deal with Or will the committee deal with like a SSL bug that that's not in our code per se but can be used to To do something malicious to us But we have to address, you know, we'll sort of like a gray area, right? Yeah, so the answer to that is absolutely if we find a security vulnerability and we trace it out to An external library like open SSL We would immediately contact the security team on open SSL and we would work closely together with them we would make the report to them and then the way it's always happened in open source is when you have an external security report that person becomes sort of The primary resource for investigation. It's like okay walk us through the you know, how you did it, you know walk us through How you can trigger the flaw that kind of stuff So we would work very closely with that external team security Team to do it now not all open so open source libraries have security teams So in those cases, we would become the de facto security team for that open source library So we would probably try to fix the solution or fix the problem and then it feed it into their community in a responsible way Right, but I was actually thinking of the other case if like, you know, somebody else reports outside of Hyperledger, but you know open SSL bug is reported You know, does the committee meet to figure out the impact? Yes, yes, that's the answer to so yeah if we're affected by something. It's not responsibly disclosed for sure We will have to come up with a response on a quick notice but most of these foundational libraries like as open SSL have a Confidential security reporting process for responsible disclosure and it's likely that we would be what you know It was likely would be included in the confidential reporting because I part of this committee would be to reach out to those core pieces That we do rely on and let them know that we are a major install of their software So if they get Something that we should know about, you know that they're trying to handle responsibly let us know so that we can respond With that sorry, this has been Borough we have an obvious source of our teams but in two and the consensus engine is built by tenement Would that work in two directions that we would also report to tenement? If bugs would be found there. Yes, or how would you envision that relationship as to be structured? I mean, we have a close relationship, but there's probably some value in thinking of how that Well should be structured security response teams are kind of like a It's hard, you know for lack of a better term like a secret society They tend to be very small and we all tend to know each other And so it's very much like you want to reach out to the security team that Manages major pieces that you rely on and vice versa offer yourself up Like, you know, if there's gonna be a major install of fabric somewhere and they have in-house security say hey We'd like to get our security teams in touch and to establish some kind of you know understanding that will let each other know Security's gotten me such a big deal in the last ten years that this is not new right most security teams all eventually start to know each other and we have Secure back channels between all of us This is just us forming our committee and then joining that broader community that already exists I know from my experience at Mozilla We had back channels from our security teams to you know, probably two dozen other groups both libraries and Users like the Tor browser project for instance was the one I was most heavily involved in we traded Security vulnerabilities back and forth, right? So yeah This is us forming our committee and this is us You know stepping up and joining the broader community that exists so that we can be part of the responsible disclosure process On the question of how do we classify security vulnerabilities, you know, are there any other Examples where threat models or threat frameworks are put in place to kind of analyze You know More proactively, you know, what part of the system would be vulnerable Now this is Conversation we've had, you know hot and others You know, how do we kind of get a little bit more proactive from a security analysis? perspective given that you know For distributed ledgers, that's a very important in enterprise applications. That's going to be a very important consideration So to answer that question, that's what I was hoping to get out of as a consequence of the outside security audits that are under way with some of the projects or at least will be shortly a lot of outside security auditing firms are very good at doing Threat modeling and doing analysis on Identifying the pieces that are probably the most vulnerable or the you know pieces that need the most scrutiny That's not necessarily the security committee's job, although curating those As we go forward would be part of that And it would actually fall more under the secure programmer the secure Programming expert that each team has identified. I think that role for each of the teams would be curating the Threat model and doing more analysis on the code base. I mean, that's what it means to be a secure programming expert That's part of that job. So to bootstrap us into having done the analysis I was going to rely on the outside security auditing teams and to work closely with them to come up with good threat models for these applications and then moving forward just having the expert on each team help curate it and I believe the core infrastructure initiative badge requires Some major analysis and review for every major release. So Going through the process again, not necessarily with an outside security audit But going through the threat model and and making sure that you know all the new stuff that has been rewritten gets a full Security review on it before it goes out the door Does that answer your question? Yeah, that does, you know, I think it'll be good to kind of see whether we can capture the learnings from the threat modeling That can be shared between projects as well, right You know, maybe Longer term we can have a You know a common framework for the threat models for Distributed ledgers perhaps, but you know, it'd be good to kind of figure out how we can Consolidate the learnings if you will. Yeah, I'm the I don't there's I don't think that there's any reason to keep the threat modeling secret so I see all of this as being part of the public documentation and I guess the Security triage group could facilitate that but the way I view it is there's two sides to this, right? It's like we're kind of like the fire department. There are fire inspectors, right? That would be the people doing the security audits and the experts for each of the teams Doing the analysis work And then there are the EMTs and the first responders the people who go to a fire and try to put the fire out that would be the security triage Group, okay, so they're they're very distinct Very distinct set of responsibilities the triage group is really just a an emergency response team And it's kind of like a volunteer fire department Whereas the other group that takes a lot more work are the fire inspectors, right? People going through the code making sure we're not introducing you bucks Got it Yeah And the secrecy levels are different, right because of the different posture they take So I mean any other questions because I don't I don't intend to Call for a vote today, but in the very near future probably next week or the week after I'd like to put this to the committee for approval both to create a standing security triage group and to adopt this as our process for handling security vulnerabilities What is outside the scope of this is Mandating the threat modeling because the CII badge requirement mandates that as part of the security review I don't think this is that's within the scope of this proposal But it is related so It's good that we had the discussion here. I'd encourage people to comment in the the google docs Because over the next week or so we can shape the languaging of it before it goes to the committee for approval But that's it. I mean if nobody else had any questions, like thanks for your time That's some sleep So do we have I guess we don't have a quorum right Todd? No, no, we're we're quite shy from quorum Okay, so And what I would suggest since I didn't hear I mean, I think there was some good feedback there Dave I think there's maybe a couple of nits that you can do to tweak the proposal for And having a backup and so forth and for sure you know defining what You know what we mean by a security defect um, yep, yeah precisely and then Since next week is also a holiday week in the u.s. Todd I would suggest that we Dave that you just send it out and Todd you just call for a vote by email just so that we can Get this moving. Okay for next week Yeah Okay So I just I'm worried that you know because of the hall. I think that's why we're shy this week as well but you know, we have a holiday in the u.s. Coming up on Monday and um Many people will be you know going to the cape or whatever for their yeah Memorial Day weekend and I So we may we may be shy next week as well as all I'm saying so I would just suggest that we just do this by my by mail as opposed to waiting for us to get quorum on Okay, sounds good. I'll make the tweaks. I'll put in some more references um of all the things people had suggested and Yeah, it'll hit the mailing list soon awesome. Thanks Dave for all the effort and um give every actually one thing to mark that you're joined the call and and um I think might be worthwhile to give yourself an intro give the the the tsc and others on the call an intro Was it for me? Yes, yes for you Marta. Oh, sorry. Yeah, so well, I guess I'm at some of you or most of you during Consensus, uh, but just to give give a quick intro. Uh, my name is Marta Picarska. I'm the director of ecosystem for hyper ledger I was announced on monday, but I've been working with hyper ledgers for the last three weeks uh, brian hired me to do three things Though pretty big ones For uh, first and foremost, I'm responsible for Managing and building relationships with our existing members and reaching out to new members. What that means is that? I would like to launch a program where I help our existing members to better navigate our projects our ecosystem and help them um figure out and uh, understand how they can uh Get the best The experience with hyper ledger projects and uh other members So I'll be reaching out to all of our members to Have a quick chat with them. Uh, understand what were what was the motivation? uh to join hyper ledger and understand how can uh, we We can get uh involved Uh in what they're doing and how can they get involved? In what hyper ledger is doing uh looking at you know building connections between members building connections Uh with the projects and introducing them to the projects is quite often We see that they only you know about one or two out of all eight projects and uh, I'll be also responsible for Overlooking the working groups that we're uh launching. So we already have the identity working group the health care working group we are working uh to build and soon announce the Uh insurance working group and that that is the model that we will be pushing forward to kind of Make sure that members are working together on new projects Second thing is I'm responsible for building relationship with our community Uh, so the open source developers Uh organizing meetups making sure that people all around the world understand that we are very proud of Our community and we know that uh hyper ledger would be very different if it wasn't for the volunteers that developed for us um And uh this foundation of open source And then the third thing that I'm doing is I'm Brian's the substitute and backup I'm humble back up when it comes to going to conferences and giving talks So yeah, and I'm based in Cambridge, UK So if you ever would like to meet me just uh reach out on merlin at lenticfoundation.org and um, I would be very happy to meet you Meet you in London or Cambridge great Oh and I have background in security and privacy If you ever wonder if I can code. Yes, I can code and I understand security and privacy So when I sometimes back up Dave on his comments and security, that's why Sign her up Dave I I can actually vouch for that Because uh martin I did work very closely together Back when I worked for mozilla and she was um a phd student at tue berlin working for dutch telecom doing Research in privacy. So we've known each other for a few years and I I plus one everything. She just asserted Thanks and welcome Marta Everyone else Give you 24 minutes back So thank you. I am already asleep. Bye guys Cheers