 Hello everyone. Jonathan will be unable to make the meeting today, so I have stepped in. We'll give everyone a few more minutes. Okay. So for those of you that are just joining us, Jonathan will be unable to make today's meeting, so I am going to step in and assist. So bear with me. We've been making really good progress on the content for the paper. I would like to point out a couple of things worth noting and I'll drop them in the channel later. For items that are specific recommendations, they should be captured under a recommendation header with an assurance category and a risk category. It looks like we've started to settle on a format for what that looks like, so I'll make sure that's included. So we've got a lot of really good content in here. There's a couple of areas that do still need to be cleaned up, but overall we're on the right track. So according to our schedule, by next Friday, we should be finessing the content and we should have most if not all of the commentary cleared out. So any active discussions need to be resolved associated with comments, any suggestions that were placed in the document need to be reviewed and accepted. So over the course of the next week, we'd like to focus on kind of finalizing the organization, making sure that we have all the content in the right spot, clearing up any potential confusion in the content as it's written, and providing any further clarifications if a discussion were to highlight, for instance, that something isn't exactly clear or we're focusing too heavy on web applications, for instance. Which is actually the area that I've been checking out this morning. So I will type all of that up and drop it in the channel, so everybody has notes for it. Does anybody have any items for open discussion? Emily, are you asking which controversies do we want to bring up first? Yes, that is what I am asking. Which controversies can we knock out on the call? Yeah, maybe I will jump in. I think I was going back and forth with Faisal. Maybe there is some misunderstanding from my side. Why Pat is better than the SSH case, right? Like, so from my understanding, there is both pros and cons for both authentication mechanisms. And I don't really get why the assurance is a bit more over for the SSH. That's why I mentioned that some threats related to personal access token, right? So, yeah, so there is a, we mentioned about implementation or rotation policy for SSH key. But why, you know, I don't understand why we don't need some kind of rotation policy for Pat, right? So Pat is, from my, from my understanding, it's a shared secret and then it can have more security risk compared to the SSH key. So if he won't rotate it. So that's why, you know, there was a going back and forth with that particular one. So if Faisal can help me to understand that. Yeah, Vinod, thank you for your question. I tried to clarify this thing on the comments as well. So, so when we are talking about Pat, right, we have to understand that it is a token, but it, the automated agents do not maintain that token. This is just a one-time token that is used to basically use OAuth mechanism to get, get other tokens. And then those tokens are basically used to kind of communicate back and forth with the Git repositories. What I am, when we say in this section, let me show you just a second. Sorry to jump in Faisal there, but from my understanding, the GitHub or Bitpocket personal access token is not an OAuth token. No, it is used to get an OAuth token. It itself is used. It is not maintained by the agent. So agent will not keep it, right? It is just used to get an OAuth token and then OAuth token is used to basically get, get any request, right? Or any, any kind of operation, get, get related operation is then performed by that OAuth token. The personal access token itself is not maintained by the, by the agent or the runner, okay? But again, the point that I'm trying to emphasize is that we are recommending policies, right? So if you have any policy, right, you are saying, okay, this is the comment, right? If you have any policy that will allow us to address shared secret concern, then we should include it. I am, I am with you totally, right? That we should have a policy if you know of any policy, any example, let's include it, right? But we cannot include that shared secret, right? We are not identifying concerns. The focus of the paper is around best practices, so we are recommending policies. And then those policies help us to address something. So what I'm asking you here in these comments is, what is the policy that you want me to include that will address shared secret? Yeah, okay, so I'm, I'm with you. Yeah, I completely get your point. I said, I only have that particular issue when we recommend one or another life. So I was commenting where it was a, for a personal access token should be preferred or as I said, when we make such a statement, right? So I think we should have that level of assurance why we are doing that. Yes. One of the challenge I see with the personal access token is the rotation part, whether how you can programmatically or automatically rotate a secret that that can break a built pipeline or CICD things. When you change it, if the board side is not saying, you can break the thing. Yeah. So OAuth works this way. OAuth has two tokens. Okay. Once you have received the OAuth token, you also get a refresh token. Okay. So there are two tokens. Okay. Normally for every communication, you will use your OAuth token. Okay. But before the OAuth token expires, you will use the refresh token to get a new token. You never again use the patch to get it. If you have a continuously running CICD pipeline, of course, now can there be out of band processes that you do not use refresh token? Yes, there can be. Right. But you have to understand that's why I'm, I'm sorry, my understanding or the personal access token is similar to OAuth in GitHub or BitPocket. For OAuth, there is a client identity. There is a client authentication in order to get a refresh token, right? Like there is an initial authentication when you get an access token and refresh token. Thanod and Fusall. Okay. Before this turns into one of those terrible like city council meetings that happened on Zoom, can we move this to like a thread inside a slack? And I think it needs to be a thread inside a slack because these, I will say these Google dot comments are not. They're not easy to follow. If you can just do a thread inside of Slack, if you write this down, we don't have to keep thinking or remembering what was said on the call. Nobody has to watch a recording either. I think I actually do think we need these for each controversial item. And I and there is a strategy element here for something that we're going to recommend. Are we going to like vote on it? What is this a democracy white paper or? So the way that I would consider it being done is when we have a case when there are nuances associated with one and another that are typically conditional with their implementation, that we discuss those conditions about when it could potentially be used. So this is I don't want this to turn into who's got the best idea argument because a lot of that gets gets very heated. So we it is our responsibility to ensure that we are presenting both cases and allow the reader to make an informed decision. So realistically, if that means adjusting the recommendation to be you need to use something and here's two options. And here are the conditions under which you might use this. And here's the conditions when you might use a different one. But I would also state that just for the record, personal access tokens in this context need to be explicitly clear as to how we're intending them to be used. Because pets can be used in other applications in different environments and different cloud service providers. So it's one of those items that we need to be very clear in what we're talking about. But I agree, it should be moved to a thread in Slack. I think that's if you could, Vinod, at Fizal, like, let's let's let's hash it out there. Everybody weigh in. So so just to end this discussion, right? So if you look at the recommendations right now, we have recommended both SSH keys and also personal access token. It's not like we have completely disregarded SSH keys. We are just saying that for developers, this will be make more sense right now. And for CI CD pipelines or agents, access token make more sense. And then in this paragraph, if you read it, we have given our arguments, right? And you have to read those arguments that why we are preferring it over SSH, right? Shared concern is not the only thing. There is expiration attached to personal access token and they are randomly generated. OK, so there are these things which basically gives Pat more ability to operate more in our least privileged model. OK, and this is why this recommendation is being both of them are being given. So I have not taken any side, right? But I just not can randomly say, well, there's shared secret there. I think I understand it. I think the key thing getting back to to Emily's idea of Fizal is that it's more it's more about making a clear recommendation. If we put everything out there, it's not recommend we're not taking a side. We're not recommending anything. We are taking a side. CSED by plan agents. OK, I was rewriting it differently. Just make a copy, make the edits, and we can wait one of the other and how we do it reframing. Yeah, I mean, OK, yeah, I mean, I think I think ultimately one of the things we got to get to is like getting the gist. So Mike is on the call. Mike Enzer is on the call. You were you were involved with the software supply paper from Google. I'm massively curious how you figured out what what made it into the paper in terms of recommendations versus what fell on the floor of the editing room. Sure, I think a lot of it was passing around. So I wrote the paper pretty much the entire thing. So what I ended up doing is like I had a lot of things additionally that didn't make it. What I did is I originally just put everything together and sort of passing it around to the rest of colleagues and even some outside of the company and had them kind of vet it. Like, do you also believe in these things too? And so whenever I had a lot of the controversy, those are the things I ended up pulling out because even in this scenario here, like, I don't believe that or at least I'm not entirely sure that like access tokens are always the same between all the source repositories. What happens when you don't use the get repository and stuff like that? You're going to have differences. So I tend to like, I tend to pull those things out and I didn't use them as as a or I didn't put those in as a hard recommendation many types. So I kind of thought like hit the 80 20 rule where, you know, it's if it's if it's everybody, you know, largely said, yeah, that's totally makes sense and I left it in. But if it was a very obscure, I generally either deemphasizer or pulled out because I don't because you don't have to make a recommendation. I'm like, that's the thing. You only put in if it's really strong, right? Yeah. And it's like, you can pull it back out. Right. I mean, from my perspective, the entire point of this paper is that I want to know very succinctly what I should be doing. That's it. And so if I see six equivalent options, that's not helping me. That's just making me more confused. I think we're doing here, right? We're taking a secret and we're tying it to an identity. So that is what we need to talk about rather than the specific method and how we do it, right? We can say, hey, these are the implementation methods of doing that, but this is what we're doing. We're identifying that workload to pull that get repository, right? Or we're saying, hey, this person has the ability to do that, right? So we're giving them a secret to do. So we're tying secret to identity. There's a lot of different ways to do that depending on where your risk level is. Yes. So Rich, I agree with your point. I definitely will continue discussing the slide even though I'm not that active in slide. So yeah, so in my opinion, there won't be always one thing for all the problems, right? Like there won't be a silver bullet for all the things. So maybe there can be a situation we have to leave it out one or two or three. But definitely we shouldn't give six, seven, 10 options that make people confused. But at least there might be pros and cons for each and everything. I don't think we should go in details. I agree with Faisal that we shouldn't mention all the key management, all the problems and all the threats and all the countermeasures. I think this paper will go too much. But instead of that, maybe we can give them two or three options depending upon organization, risk appetite, whatever options they can use, right? Like there is a risk elements to everything. Sounds good. So that'll be moved over. I kind of heard there as well that, you know, you basically, I'm sorry, I didn't see who was speaking. I'm still a little bit new to you, but I like what you said there where it's like, you know, we're tying a profile to a secret or an identity to a secret, something like that. So like having that explanation before you try to say, here's the recommendations. It's more like you're setting up the principle and the interface for what you're trying to describe. And then the implementations, you can't have multiple of them after that because you've described what that problem was and then here are some ways you can solve it. I like that approach as well. So rather than just straight into recommendations, sorry. No, actually, that was something that we discussed previously is the formatting of the different sections, which I'll repost that in the Zoom or in the Slack channel that way everybody is aware of what we're looking for. So the discussion on Pat and SSH, please move that to the Slack channel, thread it that way it doesn't blow up the feed. We need to get that resolved before March 12th. So if that means providing dual recommendations or moving the recommendation to a higher level and discussing those conditional options, let's go ahead and make that happen. Any other markets of debate? Emily, to take further what Richard had said, and I think this is going to apply to any debate, ultimately, whatever recommendations we make need to be able to stand on their own in front of a security review team, that none of us are going to be there to defend the argument. Correct. And it will be the justification of like, hey, this council of people who have cut their teeth and this stuff, we cannot do the service to people who are trying to put software into places and saying, we're going to do it this way. And on even take the identity thing, sure, like map identities, identities to secrets call right on in that. But even the order of like, well, which one do you do first? Because if you're just distributed in tokens to anything that's been identified, you're back in square zero. So how do you test for identity in first place? If you put secrets first, you're going about it wrong. So we should be arriving to that level of detail and we can even like, write down like the thought process of getting a bet. So I think ultimately reason will stand up or pop up from the document if we capture like with that fidelity. And don't be afraid to add footnotes and refer to blog posts where others have waged war on the internet about those particular topic areas. Can I can I ask that for the controversies? If I think we should thread all the controversies and I think we should label them, I'll find some sort of cute emoji to make that happen. And I think all of us should weigh in one of the things I think we really should do it. And, Andres, you had a good point about this. It should pass our sniff test first and then the the real sense of satisfaction should come if outside security folks read our recommendations and don't go what they're talking about. Because that's that's I think ultimately the goal is to be concise. Am I wrong on that? Should we I kind of crowdsource some of I have a and a big DevOps slack that I'm part of and I go to some of my my close counterparts and like, you guys do this as anybody use Pats for for developers, you know, Pats for CSED tools, fine, but developers know. So just is that is that sound like something we should do? First, all of us should kind of agree that this is a sensible recommendation. And then eventually we we expose it to the greater that way when when it when it does get released, people aren't caught off guard. So so just to add to this point, Richard, so we have 200. So this is my point of view, right? We as a community are gathered here. OK, and we all have different backgrounds, right? You might be a developer in the open source community, right? And you act as a developer, right? I am more on the enterprise side, right? And the idea is that we take recommendations from each other and then incorporate in this document when we say that, well, I go to someplace or I have not seen this thing, right? For example, I have not seen in total deployment anywhere in enterprises. So I cannot go into that section and basically say, well, I have not seen it. Let's remove it. OK. If people have not used it, let's take the opportunity to combine feedback from all in this forum and give recommendations. So if people do not know about something, they know about it. If somebody has been using things wrong for 10 years, we cannot say, OK, great job, continue 11, 12th and 13th year on this one as well. So we have to combine opinion from both enterprise side and also from the development side and then give these recommendations. So I think in many of your comments, the comment is like, I have not seen it, right? Yeah. I mean, what I mean, what response I can give to it, right? So I mean, similarly, that's where we can provide evidence that people are doing this and people consider it best practice. Yes. And then, since that allow list, no, I can't find anywhere where somebody recommends that. No. So this is where I will tell you, right? This is where you will take my recommendation, right? Because I come from enterprise. People do set up project templates, build packs, and they do define using post pre hooks and post hooks in Git repositories. What what developers can come in? This is what I do and I'm telling you if this thing is not out there, I am telling you based on my experience in a similar way as a developer, you will tell me in development, I use SSH keys more and that's why we have given the recommendation for SSH keys for developers, right? But you when you say that I have not seen it or my community has not seen it, right? Then I have no argument against it, right? Because I'm like, well, I have not seen in total, for example. So does that mean it does not exist? Right, right. And those of us who work in the enterprise know that there's a whole variety of maturity levels. The question is, who do we want to offend more? You know, like, do we want people to be looking at it and saying, this doesn't this is just an added process that makes sense in the enterprise, but maybe not for my small open source project? And how do we parlay that? So I would make a recommendation that for instances of discussion or recommendation and justification within the paper where there is a particular enterprise viewpoint where the impact is significant for software producing shops. And that is our goal and objective that we need to make sure that we are clear in couching that and associating the assurance and the risk level. And if that means that we need to break the recommendation into two corresponding areas, we can certainly do that. While it is still in draft and then revisit upon a final community review, because ultimately we can have the best, most brilliant ideas documented within this paper and we can get it to a semifinal state. But if it does not hold up to a community review and we cannot objectively adjudicate those comments and suggestions from the community, we will have to revisit the topic area in question. So for the purposes of continuing to move forward both today and with the paper, if there is a viewpoint that is specific to enterprise or global level security for supply chain versus open source specific development or I'm just a lonely developer working on a fun project, we need to be clear and the recommendation, the justification and the associated assurance and risk levels. That's why we have that level of assurance and those risk categories so we can separate those line items. Yeah, I agree. Excellent. What's next? So I have to drop everyone. Yeah, so please just last thing, do check this assurance level and risk categories with Pat's as well. So yeah, and we'll follow the discussion up in Slack channel. Yeah, thank you. I have to drop. Yeah, sorry about that. Thank you. Thanks for all. Okay. Does anybody have any other highly contested areas? I think that can be potentially another one, but I haven't received anything so far yet, right? It's with the software bill of material. I'm not sure. It is also something, you know, may not be actively used in all the enterprises. So for now, we, I mean, at least in the document, I have put both SPDX and Cyclone DX, right? But I'm not really recommending one or another. So it's interesting that you bring that up. So I had a lovely conversation with Alan Freeman from NTIA who is heading up the SBOM effort concerning some of the topic areas and the content. So I have engaged with him and requested that he provide some insight and feedback. There are some organization-specific endorsements associated with one format over the other that we're going to dig into, and probably offline to figure out what's the best and most appropriate recommendation there. Because ultimately, whatever we identify in this paper is likely going to be what the industry is going to start looking towards and settling on. And we want to make sure that they're set up for success. Definitely, sure. So is Alan going to join Slack or Meetings or anything or just the Google Docs feedback? I'm working on that. Okay, that's fine, just thank you. Emily feels like the biggest controversy is, well, we want to push cloud-native security practices everywhere. But now we're talking about, well, here's where enterprises are at and we need to account for traditional ways of doing things. I think we need to call that out in scope. I'm not inclined to do the latter, honestly, because we're just perpetuating status quo and just reconfiguring existing pieces versus like, hey, guys, see the light. And I don't think anyone here is like a lowly developer working on a fun project. Everyone has like a ton of valuable, unique experiencing like a lot of like large scale deployments working with a bunch of other people. So we should make a determination on that. Well, I would defer that to the group's side as well. Yeah, and the community review will actually help resolve a lot of those items, especially if they're reading through it and something seems out of place. In some cases or in previous documents I've worked on, we've provided a disclaimer around a few line items where there might be considerable differences for an enterprise that's doing software production versus a shop that is fully cloud-native. So just something to consider. Yeah, and like, let's have the debate on paper, make a copy of the doc as if you're making a fork, make the edits you wanna see, and then share that back. It's often easier to come with someone like, hey, here's how we think we can frame this better rather than trying to get someone to rewrite what they already wrote that may be personally attached to. Okay, so moving back to the schedule. So a quick reminder, and again, I'll put this in the Slack channel that we should be working towards clearing out the commentary on the document as well as finalizing any last-minute content that needs to go in and smoothing it out or dishing it to make sure that it reads correctly that it's not an incomplete thought. Are there one or two individuals that would be interested in kind of going and doing that initial first pass and making sure that the recommendations are formatted correctly, that we actually have assurance and risk levels associated with them that the justification is fully formed. Don't everybody jump at once. What's the date on that, Emily, that you want that done by? March 12th, just that initial first pass. Not looking for you to spend two hours writing a justification on an area that's controversial to you, more just to make sure that we've got everything buttoned up and that we're not missing an assurance or a risk level or that we're missing a format error somewhere. Just a good first effort. March 12th. All right, I'm blocking off time now to do that, Emily. Thank you, Cole. Can I get one other person to help him out? Please, please, please. And if you could, Emily, just kind of slack me what those individual tests are so I can make sure I stay on track and I don't get distracted. Yep, I will. Awesome. You can jump in and help make a pass as well. Is there a style guideline out there to make sure that? Because I'm just new to this. I don't want to try to make recommendations that I'm not sure about either. Yep, it will be in the Slack channel so you all have something to go with. I'll help out with that. Awesome. Okay, is that everything for today's meeting? I want to try to give folks back 30 minutes if we can, but if not, we still have another 30 for more discussion. You know, I've been thinking about this and I think there's some themes that we need to pull out of this and kind of start thinking about the executive summary to start tying everything together. And I think the theme of verification is really important, right? How does everything tie back to verification? Verification that the software that the code came from where we know where we think it came from, the things were built away. We think that's all the guidance that we're kind of seeing coming from MITRE and some of these other places that are printed out reports. I think we should continue to follow that and really tie that all together. So each one of these points, right? If there's a way that we can tie that back into verification, let's work on doing that. I'd like to get some feedback on that, that terrain of thought. I'm not positive I'm correct, but I think I might be going along the right route. Yeah, I actually had a somewhat related thought when looking at the distribution sections yesterday, where we have this like separate call-out section for how to verify metadata. But it seems like that would almost make more sense when you're describing what the metadata is to kind of tie that whole end-to-end. You create it here, you verify it here process, or at the very least just like calling out, no matter what you generate or sign or whatever, make sure you verify all of those pieces. Like right now we verify left, right? We verify like with the CI CD process. I think we want to move that to the right as much as possible and verify as close to the execution of that workload as possible. And that's just kind of some of my thinking that I have going to that. So that could be like a theme of this paper and how do we do that? How do we have these tools in these recommendations? How does that help support that effort? So I'll tell you one of the things that I did. So I heavily leveraged our binary authorization concept, the idea that you have like these signed attestations. And so when you create any of your, so you do the verification in the pipeline and you run through some sort of rigor and then you create an attestation for that. As long as that artifact doesn't change, you can only, you know that you got the immutability and then your trust can stay in there. So then the verification is kind of a, the attestations or groupings of those are kind of the proxy for that at the execution time. And so I did the same thing. I just didn't do a whole series of them in the environment. I relied on the attestations and the buildup of that to transfer that trust. So then you're not trusting the signature, right? You're trusting the attestations. Correct. It's a zero trust principle. You wanna use the buzzwords. Correct. And then I essentially use, so that way you can have many of them. So it's both the signature of the software, it's a guarantee that it's the person who wrote it, it's all of your tests and automation, all that kind of stuff is bundled into these attestations. And then you can have a policy that allows you to, allows you to do a couple of extra things too. Like if you do need that break glass scenario, like like I just gotta get something out right now, you can create a quick policy in placement to replace that. Rather than going in and hitting every single tool to say, normally re-check the validation software, you can, it gives you a little bit easier of a point to make changes for. So. Right. And I think that's what we wanna do in a cloud native CI CD environment, right? Traditionally, right, we just rely that the pipelines configured the way that we configure it for best security practice, right? We don't do any validation, out-of-band validation of that pipeline in the artifacts and metadata from that to make sure that's correct, right? We're replacing trust in some administrator that configured a pipeline, which we all know, right? As good as we are doing this stuff, we always make mistakes. So I really wanna emphasize that in this white paper and it sounds like either some agreement is consensus on that. So if we can push for that, I think it would really bring the whole thing together and make it more cohesive. So yeah, I like it. In fact, that's actually one of the areas that I didn't get to address in my paper because I don't have a good answer for, and now that there's a really cool community, I can ask it at some point here, but it's the, I can easily get around things in the CI CD pipeline if I have access to it. I can literally just redefine something and override it and just bypass all the security things if I want. So we can't always rely that the CI CD pipeline is completely controlled by an administrator and all of that, you have to build in these checks as you go along the way. And that's where that verification is important and then just be able to capture it. Anything else? This is your time, folks. So I just want to review the schedule. Do we feel that between now and March 12th is enough time to start cleaning this up? It's not the final review for this group. It's more to get the content in an area and a layout that we can actually do a cohesive group review. If not, we can bump it out another week, that's fine. And I think we should maybe do a poll, see how many hours we have next week to work on it. And you know, people on this call and then kind of make a decision from there. So right now I'll start, I have six hours blocked out on Wednesday for that and zero the rest of the week. So that's what I can commit. Emily, I just want to add, right? There are some areas which are not still missing main items like distribution of S-Bomb. I don't know who took ownership of that like a couple of weeks before, but there are a few items like that. I think we need more content. And I don't know if someone else is interested to jump in and help with that content, maybe, you know. I plan on reaching out to Alan and company to get their engagement on those particular areas. I did reach out to some other people as well in the background, but you know, everybody's busy, right? So that's the problem. I also ask some other people, but I think there were a few other individuals joined the call like two, three weeks ago from your own channel or, you know, they told that they can help, but seems like they may be busy or they're not available at the moment. But, you know, if someone else is still interested in those areas, maybe if he can have some content that have things ready. So we're going to target for March 12th. And then during the March 12th call, if we feel like we need more time, we'll push everything out one week. Does that work? Yeah, yeah. All right, let's try to get that done. If there are areas during your reviews and commentary cleanup that are grossly lacking in content, go ahead and highlight them yellow. And I will make sure that I add that into the notes from today and drop that in the Slack channel. That way we can get that taken care of, right? Thank you. Speak now or forever hold your peace. Anything else? No? All right, everybody gets 23 minutes back. Congratulations. Make good use of that time working on the paper. Thank you all. Thank you. Thank you.