 Good day folks. I get somebody volunteer to make some noise. Make some noise. Thank you. Okay. My speaker is working. We have two way communication. All right. Let's get everybody to sign in. Today is the 15th. And Brandon, I just added some. Thoughts to the. Revoking revoking trust to an artifact. I think it's the problem with doing Monday mornings is we post stuff on Friday. We try to take the weekend off and then we're scrambling to get something in when we in between our. Eight a.m. Monday morning meeting and. Okay. Let's give the couple of minutes. Marina is here. We get some water and we'll, I'll be back in a minute. We have the stability of incoming people. I think we do. All right, Marina. So we have four things. I think yours is the most. Mike likely. Well, I don't know. Probably the first two. Well, maybe all three. My last one, which should go relatively quick. Let's try to do about 15 minutes each. And I'll just, that'll give us a little bit at the end for whatever conversation. Yeah, I think that the actual tech section I added is pretty short. So it should be, should be doable. Let me just get this ready for screen sharing really fast. Sorry. Everyone seeing the top notary design. Overview thing. Okay, cool. So the main thing I added was just this section here. Based on the discussion last week to kind of talk more about how this effort relates to the notary to goals stated in the. In the requirements repo. There's a link there. Yeah, so this goes over kind of which pieces of it. Are the reason we're doing this type of presentation and then we can go more into what the implementation is. So the first thing is the signatures attesting to the authenticity and or certification. I think this is similar to the existing design as well. With the difference that the tough metadata. Kind of provides information about who signed the image as well as who's authorized to sign an image. That's kind of a minor difference there. The other ones I think are a little bit bigger, like the key hierarchies and delegations was one of the stated requirements and tough very clearly through the tough, through the, sorry, the targets metadata provides that kind of hierarchy delegations that you really know who's trusted to sign each thing. And also that prevents some key sharing and some of those other things that we don't really want. Key revocation, including private and air graft, air graft registries. So this is just talking about the, you know, we've been talking about q application a lot. So this goes over that again. Briefly how tough uses the implicit explicit key revocation, which we, we have other conversations about, but yeah, that's kind of summarized there. And then the key acquisition must support users from hobbyist open source projects, large sector vendors. And that's kind of where our tough design has evolved to these different sort of different routes, whether it's the registry for larger registries, public registries with you, that makes more sense. Or whether it's for these organizations where you need some more separation of control, you can have the different routes for the different organizations when that separation is needed. I think kind of addresses that's that step. And as well as kind of addressing these, I kind of just add a note that is also, we also adapted the design to work with the other goals of the project. Tough doesn't specifically address those goals, but I think it can work well with other things that address those goals, like the moving between registries and other pieces like that. Like that's not why we're doing tough, but that's I think something that will work as well. Basically, do you have questions about this or kind of the, any other questions about like why we're using tough, piece of the design is important. I think the piece of it you're talking about the hierarchy of keys that can be used to sign makes a lot of sense, whether it's individuals or different nodes of a multi-node build server, right? Like, you know, you've got four different build servers and you want to know that things that came out of build server a or different from build server B and C. So if build server a winds up being compromised in some way, you can differentiate the services, the subservices versus people because we hope in most cases, these things aren't signed by people. So that hierarchy makes a lot of sense. You can see, hey, this all goes back. Yeah. Wabbit network signed it, but if you care about a level of detail, you know, because something happened, you can see that not only the Wabbit network signed it, but it was build server a to somebody that cares. That part makes a lot of sense. So the place that I was struggling a little bit was two parts. There's a larger constant updating thing, which we can talk about, but the multi-signature piece. I was trying to understand how you're proposing that, maybe that maybe I'm jumping down below. So I should let you go and see. That's fine. If you have questions there as well. If anyone has questions here, maybe we should do that first, but I'm happy to go over that as well. I'll give it a second and then I'll answer your question. Totally fine. If anyone else has questions here, and then I have to go into that as well. I had a comment here. I think, you know, it's great you're calling out the goals of the overall loader V2 project, but we'd also kind of come up with some requirements we'd agreed upon in the key management space. I think calling some of this out specifically would be helpful. Yeah, that makes sense. I think that the main reason I haven't yet is I think that document is still a little bit up in the air and I might be wrong, but I'm happy to add that here as well. Yeah, there's, there's I think the pull requests that we have out right now contains all the other requirements we have agreed upon. The three that are still up in the air are ones that I pulled out from that drop requirements. So I think that's a good place to start. The other three docs as we come to agreement there, so we'll need to come back and revise this stock to include those areas as well. Yeah, totally. That makes, yeah, I can, I'll go through the agreed upon ones this week and, you know, iterate as needed. I think that makes a lot of sense. And we can get them agreed. We should get a merge. I mean, that, that even this document, I realized a lot of the comments I made were stuff that was after it's merged and that's fine. But let's, it's really hard to track all the loose things that are PRs versus things are merged. So let's, let's say there's a group. Let's try to focus on things we could agree, get those committed. And it's not that we have to, they're etched in stone per se, we can go back and change it, but let's, it's really hard to reference something in a PR versus something that's been merged. So like Niaz, let's, let's try to get, and I, I don't know, I've been back, I've been trying to free up more time. But if you feel like the content is ready for review and ready to be merged and let's mark that and let's see if we can get that done. So let's see, back to your other question about the multi-signatures. So just to make sure I understand the question is, are you asking about like how tough, organizes signatures in the, in the targets metadata or you're asking about like how multiple different signatures, how you can require multiple signatures for a single artifact. It was a little bit of both. And I, I had a really good write up where I was quoting and commenting, quoting and commenting and the lovely browser internet failed me and lost it. So I was trying to reconstruct as much, but it was too frustrating to try to go back and get everything from detail. The, there was a reference, the way you were talking about, there's two parts, the way multiple signatures would get added. I was making, I wanted to make sure that we were maintaining this immutability aspect of a registry that if I add another signature, I'm not changing anything to do with the other content that's already there. Are they truly additive was the first part. And I'll come back to the, how many signatures should be considered trusted to, to move forward, but because I do that more as a policy. Yeah. And I guess I'll answer probably slightly different, but related questions. So like, so the way the targets metadata works is like each target's metadata is independent of the others. Well, mostly independent of the others. Like they do delegate to each other, but as far as uploading and being valid, they're independent of each other. Independent. I'll call them files for lack of a better word there, you know, independent things. And you could update one of those, we upload it and everything else would remain valid. As far as individual signatures on those target metadata files, I think that's a little bit trickier to answer because I think that the, you know, the targets metadata isn't valid really until it has a certain number of signatures attached to it. And I think this is one of the benefits of using the targets metadata instead of directly signing. The artifact is that you could, to update the signature, you just update the target metadata, which then points to the artifact. Not like, not like points in a registry sense, but it includes data about the artifact. You seem to be implying, like I view the number of signatures that must be associated with something to be considered valid, a policy decision. So when web network signs a piece of content, they don't know or even care what acne rocket says. It's like, they signed it and that's fine. In the acne rockets environment in dev, they might only need one signature to consider it valid to promote. And then in the next environment, they might need to, they might need a signature from the scanning piece of code and the functional test that says, okay, those two things passed, there's two signatures. So those get added to the image. And then when it goes to production, it says, what did it pass security and did it pass functional testing? Great. Okay. I think I mentioned the concern now. So basically the way this would, that's a particular situation would work is that you'd have, so basically you would encode that policy into a piece of targets metadata. And I think that in this design, we allow that to be either in the targets, metadata defined by the registry and mapped there or in some user to find one that just encodes this policy basically. And then the policy that's in that target metadata says, okay, I want signatures from rabbit networks and from acne rockets. I want, you know, three signatures from acne rockets, two signatures from our networks because that's, you know, that's our policy in that I want to make sure that those are the correct number of signatures. And then once like the client knows that policy, it goes and it tries to download the targets, metadata from web networks for this, which is basically the signature from web networks for this. And it looks at the number of signatures on that and make sure it matches their policy. Does the same thing for acne rockets. So those, those are two independent things and they don't, they don't have to be signing the same targets, metadata. They can sign two different pieces that point to the same artifact. Does that make sense? And I can make that a little loss. Yeah. I mean, it might, it might help to show like I've included in some of these where people have asked questions, I've included a document that shows like, here's what the manifest would look like for this particular case. Yeah. So I think a whiteboard helps a lot with this. So I can kind of draw it out too. Cause here's the thing. I really struggle that policy is embedded into an artifact. Like I think policy is at the same reason why I don't think vulnerabilities should be assumed to be in the artifact either, because things have, things have vulnerabilities that we discover after in most cases. So you want to have an added. It's an external entity that stores that information. Yeah. That wasn't clear. The policy isn't in the artifact. It's in a piece of targets, metadata, which can be defined by the client. You know, like any other configuration would be. Okay. So I don't have to push. I can push the individual signatures, the scanning software that scans it and found that it's whatever compliant, that pushes a signature to the registry. And then the functional test that says, yep, this thing passed. And it's been stamped a certified, stamped a certified. I can push that as a separate signature as well. And they're not, they're not even ordered. Those could happen in different orders. Yeah, totally. And then I could have a special policy at my deployment node, like through OPA. If you look at our scenarios diagram. Yeah. Somehow. So this targets metadata is not something that's pushed to the registry. So it's a little bit of both. And part of this is, is because part of the reason this is confusing is because of the adjustments we made. For the new review to version of tough. And I think maybe there's a better name for the. Client specified configuration thing, but that's. You know, so, um, So, but to be clear, I should like the basically what, you know, I envision right now is that the registry has kind of a default. Set of, um, configurations. And then the client can define their own. Um, you know, and I think that in some cases, like for prior interest use and stuff, the client will always define their own. They'll always have a configuration that, that does that. But if you imagine for something like Docker, it makes also makes sense to have a default. Public, um, configuration that says, okay, you know, this is the person. This is the signature you should use for this, um, artifact kind of thing. So Marina, I want to hit on a quick point. You said there when you were saying that you could pull the targets from say the acne rockets repo and then pull it from your local web networks, um, registry server repo, wherever you're thinking of, are you also envisioning as a potential that we could have all of these targets metadata in the same repository, but separate. And the reason I'm thinking of that is you might want to clone the upstream repository to something local behind the firewall in your own environment and have that constantly synced. And you want to know that the upstream acne rockets, uh, signing data is identical to what's upstream. And so you're copying that verbatim. And you don't want to touch that. Is that something that makes sense in this model? I think so. Um, so the, um, like the piece of tech that would probably be most useful there is this like idea of mapping metadata, which is really just a client configuration saying from which repository to pull things you can do for multiple repositories. Um, I don't know if I have anything in this document mapping that out at the moment, but I think this would be a server side thing. And so from the client side, it's going to see multiple targets in the same repository. Yeah. So then, um, so if the, if the configuration is set up like that, the client could then be configured just to pull it all locally, which I think in a lot of cases would make more sense because then, you know, you can control your own uptime and all that. So yeah. Okay. Yeah. And I think these are definitely really good examples that, um, I think some of them are covered, but I think calling that specifically would make a lot of sense. And so I will, um, either to hear or to the other document. Um, yeah, with like exact scenarios and walking through this. Yeah. And not a big deal, but something that's bit me on some of the other pictures we've got is in dark mode. These don't look quite as pretty. Oh yeah. The black, the transparency doesn't work so well. Yeah. Okay. I just been putting a white background behind them. Keep it simple. I'm the rebel. It was, it was a good, a good example. Thank you for being the rebel that was being vocal as opposed to the others that were just struggling. So it's an easy fix when you think about it. Yeah. So the other piece that I had some questions on is the, um, if you go down to your diagram, it's the Coke and Pepsi problem that we keep discussing that happened. That's what he actually, sorry, you're right. Is the, uh, This one. I think it's that one. Yeah. Where we have contractual obligations that we can't even acknowledge that content exists in a registry. So we can't make any assumptions around. Yeah. Did you have any questions with regards to. To. To companies or even to orgs within the same company. Like we've got to have complete isolation. Was there something about this that suggests that otherwise? Um, no. So I think that this example image is using a registry. Um, cause I think it makes a simpler example here, but I think that we have loosened. I think up the idea that the route should be only tied to a registry. multiple routes on the same registry. For example, a Coke and Pepsi could each have their own independent one. If there's like, if you really truly can't combine them, I think that there are, you know, some security gains you get for some security and simplicity gains really you get from combining them. But when it, you know, if and when that's not possible, I don't see anything particularly wrong with just hosting doing the exact same system, but splitting it. Coke and Pepsi get their own independent metadata files. Yeah. Yeah. In this case, Coke and Pepsi would definitely get their own. The question is within Coke and Pepsi that they want to have subgroups that don't want their information shared or acknowledged across them as well. Is the other problem and the general thing that I saw, because I saw this as a reference to air gapped environments. There's this assumption that when things are all together and you have full visibility of everything, it's the most secure. But if you start breaking it down to smaller units, it's considered not secure. And yet those customers are asking for more isolation because they want more security. So the idea that somehow is less secure in that is going in the opposite direction. Yeah, I think there's two things here. I think, first of all, we created the system with the snapshot Merkel trees where the only information would be hashes, which I. So secure hashes, which are not backwards traceable, as far as we know. Using like, you know, shot shot by 12 or your hashtag with them of choice. So that's part one. And then part two is that like the reason that the the snapshot and other aggregate pieces give you a larger measure of security is that. You can no more as you get better rollback protection, basically, you get a better view of what's currently valid on the registry. And that gives you a better view of. Yeah, when something goes wrong in some other package, you then get a better view of that. I think these snapshot Merkel trees also in some ways address that concern in that if you have auditors auditing the the Merkel tree and are in the contents of that and they detect something that goes wrong, you can then get a similar protection using that. I think it's a little bit more work to make sure that that auditability goes through. But I think maybe in some cases that makes sense, even for smaller projects that don't need the scalability of the Merkel tree, they can use it for you know, the auditability kind of using these. So basically using it kind of as a transparency log to get a similar snapshot protection. Yeah, I think but you keep on coming back to the the same piece of that there's this concept of a public registry where I'm trying to make sure I have the most updated version of a specific labeled thing. And somehow I would do that across multiple companies. This is the part where it's just we can't even begin with the concept. There's any kind of data sharing whatsoever of any type, even acknowledgement that other data exists across two companies or two entities within even the same company. I the piece that I and it comes back to the other piece of where they having to do a constant update. I think I'm just getting more and more and I wanted to make this more of a discussion, but I wanted to get to the other and I was going to write it up more and talking about the the expansion of registries when you think about it from a couple of public to all the private to all the words within that private on how many things are getting constantly updated. It's not just that computers couldn't technically scale. I just don't think I'm struggling with why it's the priority when there's only a few things, few important things that are you do want to know they have the latest tag. And it kind of goes to Brandon's tag sign in conversation. But I just want to I want to make sure we're keep on coming back to whether it be content moving into an air gap environment. It can't be less secure because it's going in air gap environment. It has to be more secure. And there can't be any assumptions that that any kind of data is shared between multiple entities. Yeah, I think that the air gap one makes sense. And I think that with basically a there's a couple of options for basically just extending the timestamp allowances, including, I think, one where you make sure that it's more recent than the previous one and within some window or whatever makes sense for that. I think that the shared data one, I feel like it. I think that I've addressed it by allowing to split the routes into different things or allowing the snapshot, Merkel trees, which only have against secure hashes of information on the registry. And this is it's secure hashes, by the way, of the metadata, but not of the actual artifacts themselves. And so you couldn't even use it to. Yeah, that's what Obama said about metadata, phone records. So I'm just somewhat playing at the game. But there's secure hashes of it. It's not even the metadata itself, right? We can not acknowledge the content even exists. But that's where I think the splitting of the routes allows for that as well. So really to condense it down to to that. And really at that point, if you split, if for a specific use case, you have to split the route down that much. You could almost even combine. I don't want to say this too quickly without thinking about it, but the snapshot and timestamp would both be very small. And you would basically just be an auditable thing, saying, this is what it is now. We can check this later and make sure that, you know, if something went when something goes wrong, we can see when, you know, exactly the point in time when it went wrong because of these snapshots. Yeah. You made a comment earlier that combining routes would make it more secure. I'm not quite sure I followed the reasoning behind that. Yeah. So when you have a route for just, for example, sake, a whole registry that contains lots of different targets, metadata, lots of different images, that basically that the tag starting piece becomes more secure because the you get that the snapshot metadata you'd when you wanted client downloads, the snapshot metadata, they check that, you know, that nothing in that snapshot file has been decreased. So in the in the simple case without the Merkel trees and it's similar with them just for simplicity. So if a client on day one downloads the snapshot metadata file, they download, you know, image A from company X and then the next day they download that same image from company X, but then cover up some image like D from company Y, any better company names. They that that image has been rolled back. So the version number has decreased from that client's view. They can actually see that rollback on everything that's within that snapshot file and not just the particular images that they've downloaded because otherwise all they could see is that the the image that they downloaded is newer. They can't see that the next image they're going to download has been rolled back as well. That's kind of the protection you get. And that also gives you a little bit more of that again, that audit ability that if everybody's looking at this one document, they can then see when things are wrong anywhere on the registry. I think this is still on the assumption that there's a few versions of a few products that you're trying to make sure it's the most up to date. In when we scale to customers trying to deploy software, they're building their own software. Every build is a unique ID. It's not considered the latest because the newest thing might not actually need the thing that works. So they they have to make it. They make explicit deployments around what version they deploy. Yeah, I think that this is basically this is important as long as the we are supporting the tag signing scenario. And maybe we should talk there about that scenario. But as long as you're supporting tag signing, you need to make sure that you have the most recent tag. And so you want to make sure you have the most recent metadata about those talks. And yeah, as you was trying to say something. Yeah, I'm not quite sure I followed the interdependency there. Right. Like having, I think, like a diagram or like a step by step, like that comparison would make sense. Because if you had individual routes, you would have individual snapshots, Merkel trees, right? Like I am I'm still failing to follow how the individual structure would be any different than sort of like having like a single single key. So I can maybe draw it out better again. I think diagrams help here. So I can work on that. I think it'll make this a little more clear. Yeah, if I can try to rephrase it, maybe you can say if I'm on topic or on topic with this one. Are you trying to say when you have a combined route with a lot of different things under it, with a bigger scope to your route that any compromise so that snapshot and timestamp stuff just has a bigger blast radius. And so therefore it has more visibility and more opportunities to get caught. So yeah, I think that's that's one of the big pieces, more opportunities to get caught. And also if you as a user are downloading from multiple from two, for example, different places on that registry, you get that kind of rollback protection for both combined, which is just a little bit stronger protection. I can try and figure out a more like approachable way to present this visit. I can share a paper as well, but I don't know if that's the best way to to share that. You know, and this probably a good kick off to Brandon's next one for tech signing. I think this kind of goes back to this again, the same thing that is, do people reference a fewer set of things that I just want to know is the latest the latest version of that label, that tag, I'll use tag label. And I don't, and I'm not, the majority of quantity of content is not dependent on that. The majority of content would actually be a unique label a unique tag, sorry I flipped using label. It is a unique tag. No, it just so happens that a lot of the things that people build from in the container space do come from these, what I call stable tags, right? It is node nine. And I do want to know what's the latest version of node nine. At the same token, the latest version of node nine is not necessarily the one that I want because the latest version of node nine just did a change that broke me. So I think we just have to be, I get the, I mean, you think about tough, the update framework wants to make sure that the latest version, you're definitely getting the latest version of something. But that is how SolarWinds got exploited. They said, hey, here's an update. And it was the latest version and it did come from SolarWinds. So it would have certified that the latest update to SolarWinds should be installed because it's the latest version. It came out of their build environment. Yeah. And I think we could get into the whole supply chain aspect of this as well. But I think that the bigger thing is that if a particular user, a particular system doesn't want the actual most recent, I think the registry ecosystem allows them to not do that, to pin it to a particular digest that they're downloading instead. I think that the problem is that not everybody does that. And so that everybody who doesn't do that needs a way to make sure that the thing that they're downloading is the correct thing. Yeah. I think it's definitely a user choice. If you don't want the latest Node 9 because the most recent version breaks your build, then that should be a choice of the user's making, not something that you're relying on and in the term of behavior of the notary fore. Right. In fact, your copy of Node 9, so what you could do is as you, because hopefully you're not dependent on Node 9 from some public registry, you're taking that public image, bringing it into your environment. So let's you take Node 9 for the first time and a test and it's fine. And now you put it in your registry and it's signed by the Node community, whatever that means, and Acme Rockets. So you're good. But now there's a new version of Node 9. And as you test it, it turns out it doesn't work. And you don't know why. It could be security fix. Could just be some funky piece of code of yours that depended on something they removed. So it's not that it's insecure, it's just incompatible. The older version of Node 9 still stays tagged as Node 9 in your environment. That signature should still be valid. The search should still be valid. And then when maybe either the community figures out themselves, they put a third version of Node 9 out that either resolves the problem or Acme Rockets fixes their problem because it could be on either side. Then they have a choice when to float forward to the newest version of Node 9. And all of those cases, all of those signatures and search are still valid. Yeah, I totally agree. I think that there is a difference between just signing the digest and when there is tag signing needed. I think that this system would work totally for signing the digest as well. But I think that the problem is that registries today support polling by tag. And as long as registries today support it, I feel like there needs to be a way to make that secure. No, we do need to sign signatures, which is again, back to Tia for Brandon. But you, Brandon, you were saying something. I was gonna say to your comment there of saying that you're gonna clone this into your internal repo. And so now you've got Node 9 signed by Company Co and not Node Co, or it's gonna be signed by both. When you're using this internally, I don't think a lot of companies are gonna be looking at this and saying, okay, all of our builds are now looking that it's signed by Node. I think they're gonna say, we internally attested this, we need to verify our internal signatures on this thing. And that's all we hear about. And so they've transitioned, so they might only import if it's signed by Node or Docker Hub. But once it's inside, they're only deploying because of Acme Rockets. Yes. And that's fair. That's fair. I think that could be done with the configuration again. I think it's definitely a real world thing where the new version doesn't work, but I think that that's down to company policy more so than whatever we do. All right, tag signing. Tag signing. Yeah, I think this is a nice transition. Okay. So yeah, I've got two of these. It was originally just one and Justin had a great comment. So I've broken into two, but I'll start with 48. And that's adding this scenario 7.1. And effectively, what I'm looking at is just to say, let's make sure that we are signing tags and that we can verify if a tag was reverted to an older version. And so the risk I'm thinking of is an attacker gets into the registry similar to 7.0. That's why I made this 7.1. And the attackers, they will modify some stuff. They don't necessarily have to modify the actual signed content. They can just revert a tag to a previous state of the tag. And if we aren't signing the tags and verifying if those are current signatures, we could say, let's grab a version of Alpine version three. And that could give us an Alpine version three that has a known CVE. And we would say that's perfectly fine because someone at some point signed it. And so that's where this scenario is saying, let's make that not possible by verifying that tag signature and verifying that's current. And I'm looking at doing this as something that the signers deciding when they want to not sign the old ones. Maybe that's something that goes into tough makes that decision for us. But at least for right now, I'm saying that this is up to the signers to say we no longer want this digest to be valid for tag version three. So I can let people read that if they need a few minutes to read this. But this was kind of the overall point here was just to make sure that we couldn't have some kind of reverting of that tag to go to a previous state. Has anybody read it? Cause maybe that is the 940 till 940 or 40 minutes after we read for people in different time zones. And what I can do while people are debating that one, I could also bring the other one that kind of came out of this if we want to. Do we want to spend more time on this one or do we want to look at the other one and look on both together? I can't read and listen at the same time. Yeah. And I've, you guys have, you had this out for a couple of days so we could have read it before. Yeah. That's right. If it's just me, then we should go forward. A lot of the points here are almost verbatim from above from the 7.0. I just changed a few of them made sense to change. So this is the one. So what we were talking about here is how do you differentiate an evil person versus a good person? So use our previous example of node nine has an update and it turned out it didn't just break acting rockets broke everybody. It's the left pad scenario for instance. So they go, oops. And they do want to know whether they roll back or roll forward is a different thing because you can argue both cases. But let's say that they did intentionally roll back. They want to go back to digest A. How do we differentiate the valid scenario where digest A gets rolled back versus the evil person that wanted to put between digest, I could digest. To me, this is always a roll forward scenario. And so you're going to roll forward to a node nine. And so in that, in that signing of the node, I got to pull the comments up now, don't I? In that signing of this, you're going to need to make sure that you're probably moving forward to a newer version of this tag signature. And so maybe you've got some revisions on those tag signatures saying, here's the newest version of that one. And it may point back to the old Shaw, but it's just going to be a newer revision of this tag reference. So this is, if I'm good, and I'm sorry, I didn't read the whole distinguished me, good and evil, I was just laughing at that. I'll read that. But the evil person can pull the digest A and do a push, sorry, let's rephrase. Obviously there's no anonymous push registries or there shouldn't be. So they have to at least get push access to the registry. So if they get push access to the registry, can they push digest A back to node nine? So in that case, like, okay, so node digest A was signed, but it was not, and it was signed as tag nine, but it's not currently. So then the question, what we haven't solved today was tag signing. We said, we're doing digest signing. So the evil, was it Alice's evil? Do I remember this right? There's some names of people. Anyway, the evil person got push rights, but they didn't get the private key. So now as long as we can say that that tag has not signed, then we know that it wasn't valid. If the valid person did want to roll it back, then they have the access to the private key as well, and they can sign that digest. So that's part of it without fully reading the RFC here. Jay, does that kinda cover that or is there a completely different thing? I don't wanna skip past your pointer here. Oh, it's a joke. Okay, sorry. Well, it was fun. So this is extending scenario seven where we were saying scenario seven is that the attacker pushed some non-signed content. And this is saying, well, they've got some signed content from the previous version that this vulnerable release of this code was out there. And so they just changed all the pointers to point back to that. And if we're not signed tags, they can do that. And then everybody starts trying that vulnerable code. Yeah, we've had a lot of discussions about this issue in tough and basically rolling forward is the thing that makes the most sense in the most number of cases. Because once you've recovered from the attack, you can then just sign a valid thing. Just call version three, version five now. And it'll be all right. Although there is, I think a very subtle distinctions we make about a, what we call a fast forward attack where an attacker pushes like a version number two in max. And so that no, no feature version could be considered valid. And I think this is another thing that tough gets you is protection against that attack as well of the availability to recover from that by rotating keys. But yeah, this is a really good scenario to think about. Yeah, to me, it's what is a tag signature. And so if it's a pointer, there's probably gonna be some kind of signed metadata there that says this tag points to this digest. And then something in there should just have a revision number and some kind of expiration time on it, other things like that. So there just needs to be some reference like that. And then you have the ability to increment that. Well, I think the thing we're really saying is we should sign a tag. We haven't addressed that yet. And in fact, I was gonna part of, I don't want to tease too much into it, but that's part of what I was gonna get to prototype two and prototype three. Like I think prototype, like prototype two where we're making progress on, we'll talk about this, but prototype three, I definitely want to bring in like how should we sign tags? Because not just how, but I mean, I guess it is how, but the point is while I do believe the roll forward concept in many ways, the question is, do I have to rebuild old content? Which I may not actually be able to rebuild the same way. So I think it's important to be able to say that in a registry that has this concept of tags which are movable and digest which aren't, that we need to be able to say that a tag can point to an older thing and it be considered valid and attestifiable. Attestable, is that a word? Verifiable. So I think that's the key part. I think I absolutely should be able to. That's to me, that's something that a tag is just a reference, it's just a pointer. And so you should absolutely be able to make a new pointer to an old digest. Or update an existing pointer to be fair. Like a tag is a pointer. You could argue a tag is a pointer. How do I make sure that that tag can be moved to things and it's verifiable? Yeah. Makes sense. All right, so being conscious of the time here and I know that we've got a few things going on. I'm gonna jump forward in case unless somebody wants to stop me real quick. To the point that- Jan, was yours for this topic or just in general today? No, it's more of a general question. I can ask those after the agenda is done. Okay, so I'm gonna move forward faster so that Jan gets his question in. All right, so adding a scenario for revoking signatures. This kind of has snuck in. There's one of my little denim items there and Justin properly pointed out that, hey, I should probably break this out. And so what it's saying is it will be nice. There's a way to revoke a signature. And right now we've got the concept of revoking a key out there so that a signer can be said that everything he signs, we don't wanna trust anymore. But we don't really have the concept of, hey, I pushed this old digest out there of node version nine, CV was out there, left pad, who knows what went on, something massively broke. And so I as a signer wanna revoke my trust or my signature on just that one digest without revoking everything else. And so that's what this is kind of getting into. Originally I was thinking about just in the concept of a tag, but if you kind of expand it out to overall scope, I just wanna break into a different scenario because we're all society digest. So that's- If it's that bad, why not just delete it? Say it again. If it's that bad, why not just delete it? So can you delete it when we have the ability to copy things? You know, you have the ability to mirror these things out there. And so is it ever really deleted at that point? So this is the feedback that I was putting there is the, just because you don't like it doesn't mean somebody else doesn't depend on it. It's somewhat kind of like the example we just talked about before. And I think we might answer that earlier when we were saying that this goes back to the ACME company when they bring this internal they're probably gonna resign as their own signature. And so as long as they're trusting their own signature, I think we're okay on this one. So I guess that part of the question I was, and I was rushing because I realized I was getting time to the minute. So I mean, I have to reread what I wrote. See if I became as crisp as what was in my head. But I think what I'm sort of wondering is this really a forwarding concept. Kind of like when you look up a part number that a manufacturer put out and sometime later they said, yeah, we don't not gonna make that part but this other part is a replaceable. I forget what that term is called. And I'm wondering is that something we should be adding as opposed to you're actually revoking a signature like revoking a key makes sense. Roking a signature would also, sorry, go ahead. Now I was gonna say, it really depends on what you mean by signing something because signing something to a lot of people means that you're just detesting this came from me at some point but then there gets into this other concept that you sign something you're saying this is secure. That's what a lot of people are trying to get out of the notary projects. They wanna know the stuff they're running is secure. And if you as a vendor that produce an artifact know that this isn't secure you should be able to tell the users that don't trust that anymore. I'm trying not to jump in from Niaz. Go ahead Niaz. Was that Niaz or is that Hank? Sorry, that was me. Yeah, as running like a container scanner obviously the things it's looking for over time changes. So if you wanna have it certified that something is free of every CVE it knows about that's only really good for one point in time. Like sort of as a use case here. And what Hank considers insecure I've just productized his product as Hank. Company A versus company A what the company B considers insecure and two companies that use companies B's security product are gonna be configured differently. So I struggle a little bit with this one as does a company wanna revoke as opposed to maybe be able to say, hey by the way you might wanna use this other thing instead but if you wanna use that it's still signed by us we still signed it. But I guess my point is more so like even within the company if you run it against your scanner and it gives you the thumbs up and signs your container image and you do that next week and it gives you the thumbs down do you just end up tacking on signatures or anti signatures at like regular intervals? How does that ever or does that ever interact? Do you just never worry about it? I'm provoking signatures. I was gonna say revoking signatures makes sense in the sense that it's a finer scalpel than revoking an entire key because the key could have been used to sign multiple artifacts and at the end of the day you're trying to say something I signed before is no longer valid whatever the reason for it may be. So the trade-off here comes in in terms of one information you are looking for from a revocation perspective. And this also kind of goes into the allow list denialist model from a tough perspective as well. Like what are you removing from an allow list versus are you publishing a signature to a denialist? The questions I had here are slightly different. So I'll just kind of wait until the individual signature one is addressed but I think that's a good requirement to have because the blast radius shrinks down significantly. I do want to touch on one thing though because what people want to get from Notary I think to someone implied by Hank's comment I think is there is a separation between here is the content that's in our registry and we can with all validity say it came from entity A, B or C. We can't say whether that content is secure or not. It's not really up. Like I'm not suggesting that Notary is trying to make any of those sayings. It's just saying that it did come from sorry integrity that was the word I was looking for. It's really an integrity stamp that says this content did come from this entity. Now if that entity got hacked and they're no longer valid or somebody the evil person stole entity A's identity then that should be that's the revocation kind of scenario. The idea that when and how and who determines a particular entity's security status I'm just suggesting is an external thing that can be compared to it. But I don't know if it ever gets attached to the thing. Maybe in a company you might want to put a flag that says hey by the way this thing is we don't consider this particular content usable in this company anymore. So that could be to your point of a blacklist. But I think we want to be careful that I think Notary is really around the integrity of something as opposed to its state of security at any one point. I think I generally agree but I think that even ignoring the security implications people make mistakes and they might sign something accidentally. And that doesn't mean that they're keen to be evoked in every single location but maybe they want to take back their mistake. They want to say oh yeah that was actually the dev version not the prod version of this thing that I pushed or whatever. Can you delete a signature or is that okay? Like if you can just delete the signature. Well, but if you delete the signature it's not caught. Oh sorry, it's such a suck. Now I was going to say Steve I think what your question here is who is the owner of the revocation action, right? And I think the owner is always the owner that has the route or like owns the route itself. So if for example, if a CVE is discovered it's still not the, I would assume it's not the registry that's revoking the signature. It's still, I think, or at least it was implied to me that the person that originally signed the artifact or owns the route that signed the artifact is responsible for revoking that signature unless I miss something here. Yeah, it's definitely the person that originally signed it has the choice to do that revocation. And one of the implications I threw here kind of answers Steve's question was I really want to make sure an attacker can't go back and take something you revoked and replay it and say that something that was untrusted by the original signers now trusted to a client that's receiving this. Is that the tag signing? It mirrors with a lot of the tag signing. But it's, you know, same idea. They get into the registry they're able to push something out there. They don't have access to your key but they have access to something you previously revoked. They shouldn't be able to pass that off as something you trust. Okay. So with a couple of minutes left let's do one thing and I'm saying this for myself and if others can do as well let's try to take the stuff that we feel is stable and get it merged this week. So let's make a conscious effort to go through the various PRs and see what subset we can agree and what we don't agree. Let's, for the purpose of getting something merged let's see if we could cull it down. And if we culled it down to one line out of 500 then obviously we didn't make progress but let's see if we can try to do that. And he has, I'll get to your questions here in the PR year a bit. Go for it. Oh, I see you want to do that offline you're saying? Yeah, yeah. Okay. So Steve. Was that your other one? Did you cover both of yours? I made it through both of them. Okay, great. I'll let me just do a quick prototype two three so we can get to Jen's pieces. People have been asking like how we're making progress on notary and we obviously got the status report out but people want to know where they can engage and Brandon was asking about some places on this last week too in addition to the OPA work that people have been asking about. So, and there's some new stuff coming up and we've always said we wanted this incremental model. So prototype two, what I was going to propose and I've got it partially written up but I'll get it posted. I'd like to finish up the discovery and acquisition of the associated content the linked artifact stuff that we've been doing because I feel like there's good progress being made there. It's generally useful. It doesn't solve tag signing yet but it solves the other elements everything from how do you associate an artifact with it? Sorry, a signature artifact with it to an S-bomb because I'm seeing a bunch of people ask about S-bomb progress. So I'd like to make sure we get that stabilized in a place that people can actually use because prototype one was a hack to prove the concept. Nobody should run prototype one in any kind of scalable form. Prototype two, I think we have the design that we can make that in a scalable form. So that's part of what I focus on two and I think it's a defined range of stuff that I think we're solid to build upon. What I'd like to do with prototype three is pick up on the conversation that Brandon's kind of made a good point to run the signing of a tag to make sure that it's, we know that basically you need two pieces of information. Not only do you need push to the registry but you need to lease some access to the private key and that's signed. And I'm trying to be really careful to avoid any assumption around implementation detail but I totally convinced that we need to solve that. The other one that came up is this concept of the originating signature is an interesting one also. We spent a lot of time talking about multiple signatures which I think is extremely important but there is a differentiation between the original signature of content. How do you know that Wabbit Networks was the original signature? Now you may not care about Wabbit, you may only care about Docker Hub certified something and eventually intact new rockets. But there's something to be said of who was the originating author of the content. And something we have to kind of think about similar to the way tag signing is not something to registry natively supports today. So we're gonna have to figure out how to make some more changes to it. I think I wanna go back and look at the OCI artifact manifest to see whether it be that or something else. I don't know yet but I wanna see how can we break this loop that the thing we're signing is a digest. So if you take the digest of the thing you're signing and then sign it, you can't update the digest because then the digest is different and you're constantly signing something that you have to change because you've signed it. So I wanna figure out how we can do that with some level of confidence that we can to some extent stick the originating signature as a tightly coupled to the artifact. It's not a link, it's always a link of some sort. Link is a loose word there. But the point is that they're not, you know that that was the originating signature as opposed to an associated signature after the fact. I think that the one thing I would add here as well is that I think that working on the tough prototype aspect as well. And I think maybe this is more like a prototype for but figuring out what that integration could look like and whether that's something that makes sense moving forward. I really think that getting it working will I think help answer a lot of questions and get this whole thing moving. So I'll just throw out there that that's another thing to be working on for anyone who's interested is getting tough integrated with some of the OCI artifact work that's been happening. I wanna actually try to get a conversation with the tough folks together of this concept of something has to be constantly updated is the part that really gives me the most angst. So I wanted to, and we got two minutes left and wanted to give Jan the moment. But yes, so I wanna figure out how to make progress on the expertise you guys bring. I'll put it that way. Jan was yours quick. Yeah, okay, thanks Steve. This is kind of a related to what you just told about the prototyping, but I would definitely like to know the end goal. What is the end goal for this V2 work? Is it gonna be part of that CNCF project and is that gonna be a new version coming out from there or are we end up into the totally new kind of a product there in here? No, absolutely. Great point, cause I know new people are joining and wondering like what the hell's going on with this thing? What is this? So if you look at the Notary Project Notary Project I purposely split that out from the other experiments and prototypes we're doing to be kind of like the website of what is the content that we're working on. And the latest conversations have helped me kind of form the thought process to integrity of content. So you Brandon, Sharon, I can't tell who it's. Oh yeah, thank you, perfect. I wanted to thank somebody, but I see the green bar at the top. So it's really around the integrity of content. Once if you ship a package and you put that safety label on top of it when it gets to the other end you wanna know if it was cracked and modified. That's really kind of the main thing that I think we're trying to shoot for here and maybe just making that statement and getting agreement on that I think will help part of our goals that we've talked about. And the idea is that if you look at the goals of the Notary Project it is around signing content that can move between registries to support the consuming public content mantra. You should not be, the world cannot be dependent on a single point of failure. Everybody has to have their own isolation. But the question is if you pull content into your environment how do you know it's the same content that you wanted? And when you generate content in your own environment whether you publish it or keep it internal how do you know it's still your content? So that's the goal of what we're trying to deliver and then the timeline is I am hopeful that we have a runnable solution by the summer that customers can use in their production environments. It's not to say it won't change but I think we'll get out of a prototyping phase and the output of prototype two should give us the beginnings of pieces that we could run in a production-ish workload with a set of stability. It won't be complete. It won't have prototype two won't have tag signing for instance. And well, basically we're gonna keep on making iterations to the point where we feel like we've gotten to the point where this is good enough for now and because no security is every finite we got to get to a point where we feel comfortable and then we'll build from there. Okay, that sounds good. And I also saw that non-goals is not the compatibility with notary V1. So let's say I would like to build something on top of notary V1 and when V2 is coming out it's not the compatibility. Correct. Okay, so it's kind of a anyway kind of a on its own. This V2 will be kind of its own type of a thing. Well, we're learning what we did with notary V1 and the fact that content can't move within a registry much less across registries was a fundamental stopper. And I think I put it in some documents somewhere that if we, oh, it was a PowerPoint presentation I did this last week. If notary two isn't easy enough to sign the content you have, then we failed. Like the result of notary two should be like this is no big deal. It doesn't matter there was a V1 there's because in practicality there's almost no substantive usage of notary V1. Okay. Like we use it, we implemented it in ACR and it sounds great and customers start to use it like, oh, this doesn't do anything I want. And it's not just we didn't do it as a marketing thing we wanted to figure out what we want to learn we put it out there. We realized like, wow, this really doesn't work. So we haven't heard anybody say that other than it sounds bad is it really a problem? Yeah, I don't see it as a problem. But yeah, I was more interested about the actual timing and if you say that this next prototyping is we are more close to be able to test and run it kind of a not the full production environment but we get more ideas there. And you were saying like in maybe you get something or we get something more by the summer or did I understood that right? Yeah, so I wanna be really careful to not make this an Azure thing by any fashion. That's why I keep on leading to other people where we are gonna get a very like overall has already gotten some versions out. It's just, it's not stable enough to use anything at scale. We will make some changes to notary V, sorry to Docker distribution, CNCF distribution which we have a fork of under the notary projects. So if you got a notary project slash distribution there is some changes that he's done there. And yeah, if you wanna pull up the PRs, yeah, I don't know if he merged it there go to the pull request. We probably should just merge it because I realized this is a little bit prototype refer metadata store. The second one, the bottom one. So that was prototype one. And that is the demo. If you go to the NV2 slash docs slash NV2 demo that explains how to instance that registry and do the end to end demo. Like you can actually do this end to end demo today. And so, Brand, if you go to NV2. This one. Yeah, and then go to docs and then the NV2 again. And then there's a demo script. There's the read me but then click on the demo script. That demo script will take you from end to end and get the experience working. Okay, yeah. And I pushed an image to make it easier so you don't have to build the image. So we'll get a prototype two version going and you'll be able to instance distribution locally. And then assuming it works the way we want it to work we'll, and I'm hoping others will do this also we'll start running a stamp of it in ACR in Azure. So the customers can start using in Azure. And I hope AWS and Google and Red Hat and Harbor and whoever else will do so as well so that we can continue to get the diversity of feedback to know that we actually got this where we think it's good enough. And prototype two won't be finally good enough but I think if we can get digest signing we've started to make some progress and Brandon's completely convinced me of the tag signing. So we'll get tax on it figure out as well. Okay, thank you. These help a lot. We are over. I have to run about others. But I want to make a conscious effort. Let's try to get these things instead of referring to things that are in PRs let's try to get some stable basis committed. Thanks folks.