 Hey, Marina. How are you doing? I'm good. How are you? Okay. Let's go ahead and get started. I Put down some agenda items. I think the first few items shouldn't take too long So we can spend the bulk of the time Going through the document you pulled you wrote and then I can share some Comments there. So I think the the first thing that I wanted to cover was you know, we've been starting to discuss prototypes in other aspects of notary I Put together an outline for the key management components that we would want to prototype and the stages that we could go through them in so the Do you have the notes visible by any chance or would you like me to share my screen? So yeah, okay So the ones that I proposed are one first one like, you know, let's have keys available for signing For the client. So this would be addressing like, you know, can we have keys stored? On the device or in sort of like a cloud HSM So less about how we manage the keys are more about just kind of having access to the different areas that we expect Keys to be stored in so I think that's a first prototype A second one would look at key rotation And so this could be either through tough for any other mechanism that we agree on but essentially making sure that you know as these keys are expiring and I'm talking more about sort of the The signing keys and not necessarily the root but the signing keys time stamp keys any other keys that you want We want to have a prototype that rotates these and makes is available for signing The third component of the prototype that I saw was kind of having the on the deployment side Having the trust or being configurable And using that for signature source validation and I call out signature source validation here because I think the First version of the signature validation, which is where you're just looking at the signature that's presented and determining whether the artifact is Integrity has been compromised. I think that's something that can be handled outside of key management and then the key revocation or allow list denialist I think can be a separate prototype We may end up doing both at the same time. I think tough allows for both at the same time But I want to track these as separate prototypes So if we end up with an approach that takes parts of tough but has other components for a lawless denialist We can implement the separate prototypes So those are the four components that I think from a prototyping perspective Made sense to me, but I want to get your thoughts on it and see sort of like, you know, what we want to go back to the larger working group with Yeah, totally. I think for the from kind of the tough perspective anyway I think mostly the key management that we need is for the root keys because kind of the root manages keys for other roles Within kind of how it's designed, but I think there's still this big Like the kind of the original key distribution question for those root keys. I think there's definitely You know still still a question And so I think yeah, I guess my question is kind of like the scope of it And then I guess like the key rotation that could happen both within tough and within those those root key Situations and we can kind of discuss those cases Right Yeah, I think the what I want to kind of set here is what are the scope for each one of these prototypes and then like, you know We're we're gonna have I think two key conversations I think that are refed are like, how do we do key rotation and how do we do key a lawless denialist, right? Or signature a lawless denialist to be more specific and I think those conversations aside Which I think kind of like, you know bleeds in sort of like Whether we can use tough or tough needs modifications or we need to go back to the drawing board. I think Those are separate conversations, but in terms of like, you know, let's say even like let's say we agree on tough, right? Would these four components make sense as like, you know things we want to prototype to you um Well, I guess like my question be like because it Kind of depends a little bit on what tough is scoped for because with our idea of scoping tough more for the organization level A lot of the like developer key management would then be done within the organization and then kind of chained up to that root key But then for that root key, I think these questions do make sense like, you know, especially something like a trust store for root keys I think could be really valuable. Definitely something we should prototype and make sure that that works Um, I think the question of the you know, like the exact keys used for signing images might be kind of more within the scope of the tough prototype piece Mm-hmm of this, but I guess those can really be done in tandem I think like the tough prototype as well as the kind of root key management right Like I think the the the one that you've when you talk about like the signing key management I think that plays more into the keys used for signing piece and then when you talk about the root key management I think that plays more into the key rotation component, right? And from my understanding of tough those are things that you know, we can rather than prototyping like an entire end-to-end Tough based workflow. We could potentially do sort of like parts of tough And then you know because we know how the remaining architecture works We can say here's where we're gonna scope this prototype to and as a next step We're gonna address the next step of integration So I think logically or sequentially like do these different components then kind of like align with what you would envision for a prototype Yeah, exactly like for tough you could do you could implement kind of all the targets metadata pieces first Right the exact signing of images and then things like the timestamp and snapshot I know we've had some discussion about where exactly those should be Generated and things like that and that could be like into the piece of that prototype Okay All right, I want to keep the prototype scope very generic for now And then as we kind of align on the key rotation and signature a lot of this deny this conversations We can start adding in more around what we expect those implementations to look like So what I'll take a crack at is then based on these four components. I will put together a Pull request later today Take a look at that. I think we can iterate on it offline and then present that to the larger working group on Monday But this should be a small doc and I think this should be easy to kind of pull through Okay the next one I Wanted to go back and revisit the pull request that I have open and I think the The thing that I put there as a to-do for discussion was the key rotation component Let me go ahead and pull up that pull request Yeah Just the updated key management scenarios. Yes Yeah, lots of comments in here. Let's see I'm Are you still there? Sorry? Yes, sorry. I always struggle to kind of pull this Yeah, no worries document up finding the right place for it. Okay. There we go I think most of the comments we had Covered in the previous discussion But looking at the file itself The area where we're I'd like to get some more feedback is the to-do scenario, right? So I've listed out I Think six areas that we need to cover Let me actually share my screen. Give me one. Yeah, which section? Actually gonna just edit this document live so we have notes Okay, so I think this is the area that we want to refine further so We called out that we want to have a discussion between Having a mechanism for rotating a key versus not having a mechanism the the challenges here, I think like, you know, we have some Assumptions that are baked into purchase. We are used to Which I think need to get called out. So for example from, you know, the traditional X509 certificate based assumptions There are things like you have a CRL or OCSP endpoint that's defined in the certificate Which is something that we just kind of consider as part of the root key sort of definition, right? I think from the tough perspective that's slightly different in the sense that the Pointing to where the signature or the key repository is if you will Kind of gives you that information. So I think those are things we want to call out and I think the The the comment that I had more in here was oriented more that a Root key itself should not be Designating what the next route is, right? Like you shouldn't be having to sign off on an update with the existing root key But if there's like a shared repository of keys or some other trust mechanism that you have to validate where what the what a valid route is I think that's useful In in in getting your roots and things like that in place. Yeah, so Sorry Go ahead. I think that part of the reason the tough mechanism is what it is is that we we really expect there to be multiple root keys Preferably like three or four of them and when you do that key rotation that that root rotation You're you are technically rotating the keys, but you're most likely not rotating all over the keys You're saying okay, you know this one key Maybe needs to be updated and kind of we collectively are then updating That key and it just kind of makes it a little bit more transparent to the user just to have that all Yeah, I think like you know when I think about multiple routes when you have some when you have large Organizations like multiple routes make sense to me because you can have isolated roots You can potentially put the shoots in different HSM's give different people axes like that's where I think the multiple route really becomes a A good mechanism for addressing that But if you think about like the individual developer, right? If you're asking them to handle multiple routes, they're likely being stored in the same key store And in the event of a root compromise like I think we I would work with the assumption that all Roots would be compromised at the same time, right? I think that the that's part of why like root delegates to those targets metadata is which are given to the individual developers I think the idea there is that By making root a little bit bigger. You can you can do more of that kind of secure key management stuff Right, and I think that's the that's the that's the call out here is that in that model You're expecting that individual developers and and I'm using developers loosely here when I say individual Developer, I'm saying like a publisher is like a single developer, right? Right So like let's even call it a team like even in the in the in the aspect of like a team contributing software They may not necessarily have the same Key hygiene that like someone like a registry operator like Docker might have or like, you know Large enterprises publishing software might have right and I think when we talk about Multiple Roots, I think we want to also add in requirements that says The multiple Roots is a great use case as long as these additional things are in place, right? And calling out what those additional like separation of keys are and how they're managed I think that's where that that becomes useful calling that out and also saying like, you know in this scenario We're expecting either registries or some central root authority to kind of really manage the roots and issue more of the Delegates to developers that then need to sign with it So I think that's really what we need to call out in this root key rotation discussion is Who is taking advantage of these? So let me take take some notes here Yeah, I think that makes a lot of sense and if you have like a really small one organization registry situation Maybe that's a little bit different but for like the dockers or whatever of the world I feel like it makes sense just to kind of Allow them to do that kind of management I think Roots So who's handling Roots? Yeah, I guess yeah, do they have the infrastructure to store them in separate hardware and I think that you know whether or not we allow Root key rotation there. I think there are definitely going to be situations Where it won't be able to you know, we have to actually revoke the keys manually Right. Inevitable Hopefully rare, but you know Yeah, and I think we can further refine this by We want to call about providing mechanism for configuring and Rotating a root key right because those are two separate conversations So adding a mechanism to configure slash rotate keys I think pros for the rotation section that can figure Yeah And I think a call out here is consider whether this mechanism Needs to rely on existing Root or Roots Make the distinction Okay, and then not adding a mechanism. I think It goes pretty broadly. I think Right now We'll want to capture more data to make each one of these clear and really make it very Precise from the developers perspective and from the perspective of the different personas I think what it what what this means In terms, sorry go ahead. I was just agreeing. Sorry Yeah And then in terms of scenarios to consider because I think I found the scenarios to be very helpful and Kind of walking through like what different use cases would be So far I've got publisher loses key Needs to update key for you know maintenance reasons whether it's like, you know Your keys expiring or you need want to move to like a more secure key type And then sort of like some of the compromise scenarios anything else you think we should consider here Um, I guess I mean, I think it's not really. Yeah, I think that those are the main ones I think for the root key rotation that are that are kind of relevant And this expiration, but that's kind of I think tied into the need to update it for some other reason Right So this let me let's kind of like keep adding here includes Expiration that's probably actually the most common case here, right? So, okay All right So I think this is a good doc to start collaborating on What I will Hmm. I wonder what a good way to have like collaborative docs are have you had success in working in git or Have you used other mechanisms for like a shared doc? I've used both git and git No, sorry git and Google Docs They have pros and cons I Think that the git history is a little bit easier if you have a lot of people but Google Docs I think is a little faster edits. So depends on Okay, I Found like git to be useful for tracking comments but I've also found it very difficult to kind of like take someone and Take someone's requirements and edit What I'll do is I will create a doc In my repo for this one If you don't mind kind of like submitting pull requests against that And I'll merge them in as soon as they come in and then we can go through comments and stuff And then we can merge that into the main repo Once we once we're aligned and sort of like this doc has all the information That we feel is needed to present to the larger working group. Does that work? Yeah, that sounds good. Okay This one I'll likely get done by tomorrow and I think we'll probably need like a week to kind of get this in place. So Yeah And then let's go ahead and review your full request Oops, let's save these I'm gonna stop sharing my screen as I go through this Yes, this is basically just my like putting on paper some of the discussion we had and some other thoughts I had key revocation Kind of break it into those three big categories We kind of talked about the expiration lists and I called it like distribution of trespass keys I think you're allow a list denialist is kind of a similar idea Big options there. I think that the key expiration is like it's it's not sufficient, but I think it's a nice addition to either of the other Solutions so that you know with no other action keys don't last forever. I mean, there's just the long expiration times, but They I think they added a nice benefit there Yeah, I think these are good call outs What I think like to kind of make sure that we are Focusing on the right area here like key expiration like I think regardless of whatever Mechanism we go through right like all the whether it's like x509 inserts or whether it's tough. I think key expiration is Fairly well understood Where I don't think we have a lot of debate there, I think calling out what keyword key expiration is and what the expectations are I think are good because it's going to drive a conversation around Do we want to look at a time stamp authority because there are two implications of key expiration, right? one Was the key expired when you generated the signature or is the key expired when you validate the signature, right? If you do the latter, you don't need a timestamp authority But then that means you're on the hook to re-sign every artifact that you've ever signed With the key, which is a pretty daunting task And you can't actually you can't really do revocation with just expiration. It's not exactly revocation. It's like expiration. Yes And so expiration is just I think like a way of controlling What are the signatures that you need to track actively because you're essentially saying I have I'm I'm putting fervor to contract that says Up until this point, I'll give you more information on this artifact that I've signed or the keys that I've used to sign it Beyond that you're using it at your own risk because I'm no longer going to give you that information, right? I think that's how We've typically viewed key expiration so I I Think for a overarching doc key expiration makes sense and I would call out sort of like the Where the timestamp authority comes in and I think tough handles timestamp authority with the timestamp key And so just kind of calling out that you need a timestamp from a trusted source To show that when the when the signing was done that the key was still valid. Yeah Actually the timestamp, I think even more so for then for expiration is really important for both revocation and For both of these other scenarios because whether you're distributing a list of trusted keys or a list of untrusted keys You have to know that that list is current and you can't it can't be certified by the key itself It has to be kind of some external Yep, otherwise it's kind of meaningless. So that's really where the timestamp service comes in and I think we'll need that Kind of with either solution. Yeah, I think I would call that out for a separate doc and Because I think in key revocation What I what we want to What I'd like to kind of deep dive on is the allow list denialist model Which essentially says as long as something has passed through this expiration check And it fits with the trusted keys that have been distributed We still need a mechanism to track whether a signature is valid or not, right? And that's sort of like the dynamic updates that even when you've set expiration time Even when you said here's my trusted set of keys, you may still want to say I don't trust this for a multitude of reasons For example for tough. We're saying we're only trusting. Let's say the last X number of updates That's your lawless model from a revocations perspective You're saying I don't trust X Y and Z updates because they have some security issue I think both models work, but we just want to capture what the trade-offs of those approaches are Yeah, for sure. And I think that that's kind of my goal here is to kind of describe both of those models and What the trade-offs are I do think that mentioning time-stipping here is important because I think it's important to understand that You know a list of things that are revoked or a list of things that are valid is Like it has to have actually a short validity time. Otherwise, it can very easily be replayed and Lose a lot of its security properties Yep, I agree. I think I Would call out timestamp and I would say let's actually do a separate doc if you let's do a separate doc for this and I think we want to actually review this overall doc first. So to set context, I think this is a good overall doc and I would call this Actually, what is this like signature validity, right? Because you're you're touching on everything So this is Source expiration A lot less than I listed it pretty much covers everything which I think is a good overall doc Okay So key expiration, I think the distribution can be handled in the The key rotation mechanism doc because I think we want to set some context on how keys are being distributed So I think that my kind of my point here basically is that if you have a distribution mechanism that's updated on every like refresh cycle That can be used as a relocation mechanism Kind of like the way tough handles it where you read down the you know, for example root metadata And if something has been expired in there, you then get those new ones as part of your new allow list of keys Okay, I think that's a The That's a good point and I think we want to track that But I think we're in terms of thinking through sequentially. I think the first step we want to track is the Allow list and I list models, right? So if we break it down into What do we want to do right like in terms of sharing? Let's have that conversation first. So I think if you think Where do we put this? Yeah, and I guess it kind of the distribution is is relevant but it's definitely a separate question but because basically if you're doing the Allow list, you only need one distribution mechanism for for keys Whereas if you do the denied list, you now need two mechanisms one for distributing keys and one for distributing this relocation list um I think for a law list you may There are other considerations to take into place, right? So if you for example when we Right now when you're expecting the allow list to be shared by the registry There's a question of like, you know, once the artifact gets to multiple registries What are the requirements we're proposing for that allow list to be propagated, right? So let's take the scenario that I push a signed artifact to docker hub Someone else pulls some information someone else copies that that that artifact into some other registry Then now I am pushing My latest updates into tough. I'm pushing the allow list in sorry. I'm pushing the allow list into a Docker maintained sort of tough repository, right? Someone else then has to pull in that information into their registry and keep that Tough repository updated as well, right? So I think there's a couple of options there But I think that that's definitely the easiest the other would be that the registry kind of performs its own verification Um and trust it independently, but I think that the better solution is to definitely to to pull those lists across Right and if you start kind of looking at how this starts chaining down I think that's where the separate the separate mechanism allows you to kind of rely On a centralized distribution for trust Whereas the tough model is kind of going to a chain model Where you're essentially having the registries update information as artifacts are moved through so I think That one I think rather than look at distribution of trusted keys Well, here's a question, right? I think I think this one I think you're actually kind of putting on a fourth area here so Distribution of the the one is an allow denialist for key revocation And then the second one is a distribution of this allow and denialist Um, I think tough is merging together distribution of trusted keys as well as distribution of the list Which I think we can call out, but I think we want to look at those two separately um The reason I say that is that when when you need to rotate your keys is different from when you need to kind of update this allow or denialist um I I think that they need they they should have some areas of overlap when we look at it in the overlap In the overall document and we make a final decision But I think in terms of calling out the key revocation list the The the the separate the distribution of that list is this is probably closer in that doc So I think you're you're calling out a need for three docs here Um, does that sound reasonable? Um, maybe or maybe like a section to document or some such, but yeah, right different sections um, I think another um interesting question is what are is keys are trusted for what and I think that that's another um piece of the puzzle here because um Like you might say, you know, you might temporarily want this key to be trusted for this service and then say Oh, no, it's not trusted for that, but it's still trusted for this other service And I think making sure that that fits into the model I guess one option would be to require people to generate new keys, which is Which is fine, but I just think that that's a scenario that we should think about um, I think that should be covered in the uh key requirements doc or say the the the initial requirements doc in terms of um, how we are configuring trust in the deployment environment, right? Like I think we called out that we want to make sure that for um, whether you want to specify it for a registry for repository or for sort of like um, I forget uh, what the correct terminology is like the individual artifact or image you can you can go down and specify What key or what root you want to validate against, right? Yeah, but I think the question is like if you If a key is revoked for one role, is it universally revoked? Um, and like that that's just it's just a question of um Of scope of what vocation means kind of in this context How would you um Not revoke a key like what's sorry, I'm I'm a little bit confused and help me understand this Uh, where do you envision sort of like having like a partial revocation of the key? Oh, so like for example, so say, um, one key is you like one team develops both I'm trying to give an example project name, but yeah, so let's go with like the like Ubuntu stable and the Ubuntu like, um Nightly whatever. I don't know if they have a nightly, but you know Some project has a stable and a nightly Release and they might both be signed by one team to start with and then maybe Um, like you want to have some additional checks on the stable Build so you you revoke the other people like their access to Sign the stable and give it to you know, some other qa team or something Uh, that's where you're revoking access to use the key, right? You're revoking the key, but yeah access to it for that. Yeah, I think that gets, um I think that might fall outside the scope of like how um keys get They you know distributed and set up, um My expectation here was that we give a model and we give some best practices and guidance and let Uh, let publishers decide how they implement it, right Again, that could be done to just be okay like to just making new keys for new purposes. Yeah and and I think like if we start thinking about like um, like the big challenge here is going to be like, you know, how do we set this up for Um, open source projects, right? Because I think that's when when one where that we expect a lot of churn like Who owns a route there who owns sort of like individual keys? And I think this is where having like an example implementation That can be lifted and shifted. I think is is is is going to be helpful um, but I think like when you start looking at, uh, enterprises like let's say Uh, Microsoft or Ubuntu, uh, not necessarily Ubuntu, but like Microsoft like Um, where you know, you have where you have like designated employees and designated sort of like credentials Um, there's probably already a mechanism that you want to tie into for distributing sort of like keys and for key management So, um, I think there the question is well, how does Microsoft share that root information with the rest of the of of all the employers that are uh, consuming Microsoft software And so I think that's that's that's where we want to define the specifications for and then say well, you know If you want to use something like, uh Tough for this key management, here's how you would do it and here's what the information would need to Kind of be shared in so I think that's that's how it addressed that Yeah, like here's how you should set up those delegations so that you know, I'm using keys and and doing all of that. Yeah Yep. Okay. Um, I will go ahead and put some more detail feedback in for this talk then I think this is a good starter and this is this is doing well to kind of address the areas um, but where I would like to kind of see, uh, a little bit more is, uh, the pros and cons of the Allow list and I list and I think your key revocation list is a good starting point for that And then let's consider some scenarios, uh and see sort of like what that would mean Um, I think from a scenario perspective, we have concerns around You know as we're pushing out updates like what you know, are people moving to the latest update? Um Is having an allow list the right model for that or should we have signature expiry as like as a way of enforcing hygiene? or a combination of both? um, and then for Um, also for sort of like security updates like, you know, what do we consider insecure? Right? I think there's a there's a broad term in saying like in if software is old It's insecure versus like in if software has a known cv. It's it's in cure Um, and I think to a certain extent you're right on both But like cvs are more like a a burning like, you know fire versus sort of like older software You know if you have like your own cv scanning and things you may have more acceptable processes for deploying them So I think coming into sort of like a consensus on what we want a signature to convey and what we consider insecure Will help that so I think that's something I'd look for in this doc Okay, that makes a lot of sense. Okay. All right. Awesome. Uh, I think this is great. Um, I will get you the detailed feedback first, uh, and uh, then I'll go ahead and kind of create the The different prototypes doc And I'll share that with you. Uh, and then lastly I'll work on the outline doc for Key rotation and I think that can actually handle some of the distribution of trusted keys as well So, uh, you can point to that doc if you want over here as well Okay, that's good Great. Well, thanks for bringing the time and uh, yeah, this is uh, I'll see I think we may want to actually have this conversation on an ongoing basis. So, uh, is this a good time for you on a regular basis or? Yeah, yeah, I can make this work Okay, uh, I'm gonna go ahead and share this at the I shared this time, uh At the next meeting on monday And ask that we just kind of like, you know, have anyone that's interested in participating in some of these conversations and potentially picking up some of these work that we're identifying, um to kind of join us in this but Uh, yeah, this this this was great All right All right, you too. Bye. Bye