 Hi, how's everyone doing today? Good, how are you? Hi, hello. All right. Marina, do you know if Justin is going to be joining today? I think he was planning to, but I don't know. Oh, I see, no. Hi, sorry for the delay. I had an audio problem and had to restart. Who else are we expecting in this meeting? I'm not sure. Let's give it a couple of minutes and see who else joins. I'm just going to go set up the notes. I think we can get started. I don't think anyone else is joining today. The, what I kind of wanted to get done today was go over the full request. And finalize that. And Mark, I think you've already taken a look at it. Justin, have you had a chance to review it and come up with some comments or feedback? Is this what I was looking at last week? Or I just want to make sure I'm talking about the same thing. Can you just post the link in chat so I can. Yep, I'll put the link in chat. It's the same one I added in some of the, the reasoning, like we discussed last week. So to clarify. So let me put this in chat. Thanks. I'm playing it up now. Yeah, I do have a couple of questions I have when I'm looking through this. So one thing. Okay, so you talk about the publisher and the deployer as different people to the. The employer is I assume the person who's actually just sort of consuming the containers. Like, if you know what I mean, like you're, yeah, okay. And the publisher is the person who's actually creating the containers. And then there's another party here that I think is implicit, which is sort of the person who's running the registry. Yeah, I don't know how relevant this is, but I'm just curious about one, a couple of things that come up. When I start to think through them and what they are supposed to be trusted to do and what they're supposed to not be trusted to do. More specifically, because I think in your system. You want the. The publisher, like someone who is a publisher, which I noticed, do you have like the publisher admin and builder and signer. You want them to do things like presumably control which, which images are trusted. Correct. So, what do you mean by which images are trusted right in the sense that there's two, there's two different definitions of trust here right. One, the employer saying here are the different publishers I trust, if anything is signed by them, I will trust that software. And then there's the other component of trust where a publisher is saying here are the, here are the containers or things that I have signed that I'm saying you can trust that having come for me, right. So, which one of those are you are you referring to here. I think they're both very related, because I think the first one deals with who you trust and the second one deals with, as I understand what you're saying, the second part deals with what the people who you trust say. Right. And so, in the end, I think the, like, I think the design, you're in part arguing for which I think is, which as I, as I understand it, and is that the publisher, like some set of people who are the publishers are the people that the deployer who has picked those publishers has enabled to decide what container should be installed when, when, you know, when the deployer goes to make a request. In some so you know, I don't know. Is there a sense where it's not true. I think. So from my perspective, I think we should split these scenarios a bit from two to two angles. So you have the consumer part where you would like to know okay what are the signed images and what are the keys which I trust. I think that's somewhere covered using these root keys which you can somehow wide list on your registry. But on the other hand, that's the topic we at Phillips are trying to resolve with not to review one today is how do we manage which developer or which CI job is able to sign an image at which point in time. So today today I have a team of five people. One guy leaves the company the other one moves to another department. In that case, we want to be have the team be able to self manage the delegations for the given target repositories. And we also want to have control on creating target keys and removing target keys and doing things like key rotation, and that's more or less a administrator kind of role. So let's say you start a new project. The first thing you will do is request for a target signing key to this administrator team. To create you this target key and on this target key, you can create your delegations or remove delegations or replace your key with with a new one. So that kind of things and for this we have this proof concept, Justin you you missed that as you weren't in that meeting where I gave the demo, but but it's available open source. So it comes with a with a small web interface to to manage these kind of things, but I think in not a review to as far I can see now in in the design. This is not not covered. Well I think actually if you look at the hybrid signing use case and then the signing by build machines or dedicated machines that does cover some of that right. So the administrator creating a root key and then delegating access for the entity that's essentially being the signer, whether that's a developer whether that's build machine to be able to actually get delegate keys that chain from the root. Yeah, that's something. Yeah, I think so maybe the we we could make them more explicitly or elaborate a bit more on what the persona does and what what it means to make that more explicit. I think that's that's I can I can take an elaborating kind of like who can take on those different roles but I still don't think we've answered Justin's initial question, which was around. How are we determining which artifacts we are going to trust overall right. I think Justin here the way I was envisioning it is that you know you're essentially like you call that the deployer is trusting some entity to say these artifacts are trusted these artifacts are okay to run. And then it is up to the entity that's being trusted in this case the publisher to kind of manage how they say here are here's here's the ones where we have signed and we we we want our sort of like consumers to say they can trust this. And then they have also mechanisms to say something that we have provided trust that before is no longer trusted right. I think at a very high level that's kind of what what we kind of envision but are there any gaps that you see there that that that's that you want to address. I mostly want, I think it would be helpful to be very explicit that this administrator party is not trusted like the registry party. And to be clear about that part of the process because actually I think the biggest concern that that we have overall is is that that a party that has control over registry can do things that are extremely damaging. And so, you know, I've heard some discussion and some, you know, like, oh, well, maybe the person goes in and clicks buttons in the registry but doesn't sign anything and this picks what this thing links to or something like that. And I think that's that obviously, you know, if what we're supposed to be having here is we're supposed to have the publishers and the people at the organization's actually deciding what gets installed and and what's trustworthy then that those are kind of very different models than one where oh well I just trust the registry and and, you know, I don't need to sign anything because it comes from the registry the registry is a good guy, which isn't isn't the model that Steve or anybody's proposing but I think there's flavors of that that creep in in places. I think that's that's also what I meant with we we should split more the role of the consumer and the sign also also in the admin part you have the admin part where you trust keys which are derived from a from a given route, or a given route where you can whitelist what what is the things we really trust. So that that could be a subset of images from different registries. And on the other hand you have the part where you are managing who is able to sign the images for a given repository. So the part of the goal here was to completely remove the role of the registry in that trust components. And I think that can be addressed by adding in a registry persona that says that here are the places where the registry owner if you will get involved what does their role mean what do they add and where are they required. The way this was written right now, the only place where registries are getting involved is in validating uploads to the registry and saying that, you know, when we're getting artifacts coming in type to a developer identity, you can validate and upload to the registry whether this actually was signed by the developer in question or not or the publisher in question or not. And even then the windows that container images are being pulled down by the deployer, they're essentially still able to revalidate that and that's happening independent of the registry. Right. So I think if I add in a registry persona and kind of show the end to end workflow of like you know a container being signed, getting uploaded and getting pulled down and deploy and show sort of like what role each persona has there. So I think that might address some of the concerns I'm carrying because I think that is missing in clarity from the scenarios right now. I think that would be interesting. I think they'd be helpful to see. I have a couple of other little questions but I don't want to kind of monopolize the meeting here but Okay, like root revocation, which you say compromise a compromise route should not be needed in process to designate itself as revoked. I don't really understand this well. I how does. Okay, so you've you describe a way this could be done which basically means you have to have a way to touch all of the clients. In like the second to last step. So basically, as I understand it what you're really saying here is generate a new key. Sign new metadata and go to all your clients and tell them to update their key. Yeah. The part of where this comes in from is in notary view one, one of the big concerns is if your root is customized that route can self then use to rotate to a new version of the route. So I think separating out like how you do route revocation from sort of like essentially other revocation was one of the areas that we wanted to address in the rabie to right with roots. It becomes tricky in the sense that there's really nothing else to sign off once your root is compromised and at that point it's really just a root rotation and saying no longer trust the older route is information you're trying to propagate at that point. So, so this is very interesting to me because it's almost like a features a bug and I can't quite figure this out. Why. So, what you're saying is that you don't want the ability to rotate route. And the reason why you don't want that ability is that the attacker can rotate the route. But if the attacker, like, okay, so I just I'm trying to walk through like a hypothetical situation here. That's okay. So let's say the. Need to have two things they both need to have the root key, and they need to have access to be able to serve content as as though they are like a trusted source like a registry or something. Does that makes sense so far. Okay, so they have those two things. Let's say they have those two things. Okay, so now you have two possible designs. Design one is is the original tough notary be one design where they then can rotate the route. Okay, so then they rotate it to a new thing. They've gone they've been able to compromise and do anything they want on on any of the parties that trust that root key for that specific repository, which in notary be one is a very local repository that only contains information. You know, like it's, it's, it's information for a small group of developers not like, like every repository that exists or something on Docker Hub. Okay, so, so that's scenario one so so they can take things over install it they want, but and you don't have a way to rotate to get rid of that or to change that. So now, let's suppose that we use this other design where you don't have this route, this ability to do rotation. Now if you don't have the ability to rotate route, but the attacker has the root keys. The attacker has taken over the registry. Then the attacker can still have the party control like control and install any software they want. And in both cases you still have to go to all the clients and touch them to recover from this. Yep, go ahead. No, I was just gonna say so I don't actually see what this, what benefit this gives you. So I think part of it is the requirement to have to go to the clients to enforce that right so with an automated route rotation, I think there's a, there's a potential that the root that's like if the root is being used to kind of say here's how we're rotating it. Then the attacker can change the route, right and then from that point forward, you're automatically locked up from rotating the route yourself because you no longer have access to the new route that the attacker used to kind of like the sign to do the rotation with right. Sorry. Sort of but I just want to ask a clarifying thing here I'm sorry to cut you off. But in, if you don't have the ability to rotate and you always have to touch the clients right. So, I don't want to touch the clients. Isn't the DC like I'm not I'm not sure I'm expressing my confusion well but I'm like, I think what's suggesting here is we have a different process to touch the client right, rather than having to go through the registry and rely on the route. I think what what I'm proposing here is that the clients have a mechanism to understand what the roots are, and this is not using the keys that are being used for signing the containers and images right. This would be kind of where I think the notary server morphs into. So if we think of notary server as being like a repository of keys that are trusted for different entities. Then you can essentially check the notary service to see what the latest set of trusted groups are. This way, if someone wants to, like, you know, say like here's a new set of roots that we want to use they can upload that to notary server. And you can your clients can check that to see what the new set of trusted groups are so that allows you to do book rotation without having to use the root as sort of like a as an authentication mechanism if you will. So, okay, so you talked about the notary server doing this so or this notary service server. I want to make sure I'm saying that's right. Does it does it matter which way. Yeah, I think this is this is this is something that we can look into more in the design, but essentially where I what I'm what I think comes out of this as a requirement is we need a component that is telling you what the latest roots to trust are for any publisher for a publisher without having to use the keys themselves as as mechanisms for authentication. But how do you know who that party is and why is that party trusted. Um, well, why would you like you have to know who you are getting software from in the first place right like when you're saying I trust this party, you know who it is that you're trusting right. Are you talking about like in a in like a tougher repository or you're talking about in just in general. I think more in general so like you know if we think about the concept of like the first thing which publishers do I trust. You would verify who that publisher is and you know where they're where potentially to get the key information from right. So this could be sort of like you know if you're saying like I trust let's say, I don't know let's say let's make up a company if I trust at me. Here's where I know acne is like keys are published from and here's where I'm going to go to get that key information because I trust that me as a publisher. Then anything that I see signed with that key I can verify that it changed from their room. Right, but but the reason why the CA system. Like does all this and works is is because effectively a bunch of rules and regulations were written around what it means to be a CA and how to do things. And the CA system because it has a lot of parties involved and has a lot of trust in those parties has actually been a pretty big example in many cases of sort of like a trust system gone awry. So, like, you know, the fact that you have 500 certificates like for different CA is all completely trusted by your browser is, is, you know, not a not a great situation, given all the did you know it or and other hacks that have happened in the past. But, but how would you. I like, I think I need to digest this, but to me it feels like you're taking. And you're, you're creating a new very trusted party that is like a third party. You're, I think the way you're talking about this is this is some. Is this a server that everybody sets up individually every individual party sets up a server that has these root keys. I think that's one where we. Yeah, we can dive more into sort of like how that is set up right like that could be done in a combination of ways for for publishers that have a strong notion of trust where it's kind of like you know hey like I absolutely want to make sure that no one else is able to kind of compromise me like have like you know like you know like if you have someone, you could set up this yourself and then this information yourself on the flip side like if you would want to kind of go down the CA model and say like you know I don't want to run the server myself you could potentially like you know designate a third party to establish and share that trust for you right. I think this really goes up to like each individual publisher determining what their trust boundary looks like. If they want to establish this trust themselves, they can set up the service themselves versus if they want to designate some other party to then this trust for them. They could use a third party to do that right, but I think both models would potentially work there. So based on this discussion I have some more questions. So, so let's say we have many notary servers and many of these notary servers could be cloud could be private ones, and let's say I would like to run a private server and in my close to network environment. I still want to be able to trust some of the images from Docker Hub based on trusting a route or trusting some other level of certificate. So, shouldn't there be also some something to administer how your server act changes the route certificate information with with other notary servers. I basically built a swarm of servers where this information can be retrieved from so my client only needs to connect with my particular notary server. And then just propagates to different servers to to retrieve the the information which is not stored locally so I'm more talking about a distributed model. So you see nowadays a lot of things are moving into these these distributed models where the information is not a one server but where it's just spread across the cloud or the internet. So that way you can also keep the ownership like okay who is owning what and and who is storing what that presents some risk in the sense that you have a distributed model of trust to where this information is propagating through whenever you're doing a rotation you also need to kind of make sure that it's going through in a reliable manner to all of the, all of the distributed posts and things like that right. So I think having a central server makes more sense in the sense in terms of making sure that if you've been in this in the regard to the curation or vacation. There's a central authority that you can essentially go and verify that information from for air gap environments right I think this is one of that. I have a section in the trust or configuration which we can expand on. The idea essentially would be you can periodically check. If you want and pull in that information, but you wouldn't get that sort of like you know immediate information if like a route revocation or something happened. No, it is so that like you know we could kind of come up with some kind of a notification mechanism that you could set up. And that tells you when to pull more information to your gap environments. But I don't think that the distributed model of trust itself it would end up becoming a lot more complicated to manage and make sure that all the all the information gets updated. And so in that situation you would need a secure trusted synchronization protocol or something like that. Not sure how that would look like but that that would be definitely something that that you would require but let's say if we stick with with with the central server. I foresee one once one issue there that with some of these big enterprises, even even in some of our own products or things which which are deployed at hospitals we simply are not allowed to do a network connection outside of the trust and network and stuff like that that's it sounds crazy but still 2020 this is the case. Let's say these these projects some of them they are running Docker containers, but the way they are running them it's it's crazy there. Instead of depending on an external registry. They make zip archives of the Docker image. They make checksums for those things they transfer those zip files, including the checksums to a system where someone manually verifies the checksums and then import the Docker images into the internal Docker registry. Simply because we are not allowed to push internally into their registry so those kind of use cases if if we would like to get those people on board that there needs to be some some kind of mechanism where you can still have your own server. You don't have a way to synchronize some some things. And this is a very extreme use case I know but I'm not even sure if we if you would even like to support this kind of things. But maybe that's something we should also give thought because at the moment with not to review one what what I see with most people is that they find it difficult, difficult to understand difficult to manage. And with all the constraints they are also getting from from clients like I just explained. But there are also use cases where it is of course it's not applicable where it's where it's a lot more straightforward and more cloud agnostic. So there is a work around that would currently work in that sense that you don't necessarily need to go to a server to get the keys. You can also configure the keys yourself right so in an air gap environment, one of the things you could do is you could look at what keys are trusted and configure them yourself. That is a at a point sort of trust right like you'd have to go back and update that at some point. I think in a true air gap environment where you don't have that network connectivity you're going to need some manual process to upload that. And I think that that's that at least the ability to kind of configure keys locally would give you that ability. Should we describe that as a as a persona who needs to be able to do those kind of things or is this really edge case. So I have a section for that in online 71 I will expand that to kind of show some of these because that was something that Steve talked about as well that we should kind of like clarify how air gap environments would build the trust and also the signing as well. Yeah, yeah. So, yeah, from my perspective, I think it also makes sense if we make a more specific splits on the on the personas, because now we have publisher admins, but I think you can even divide those admins on different use cases, and maybe also different steps in the process. So let's say someone managing who is able to sign images could potentially be someone with a different role then then someone who is managing the trust of certificates routes and things like that. I think making that a more explicit split probably also eases the discussion like who is doing what and in which scenario would they do these kind of actions. So the employer in this case would be the person that's responsible for doing that. Let me look at the, I think, probably another role that we add in here then is the trust or administrator. So this is necessarily like the same way a publisher split into an admin and a signer, the floor gets split into an admin who's configuring the trust and they develop a machine that's actually pulling the containers and running them. So, I think I can split deploy into two. So, if I'm carrying the feedback on the personas, this really should have five personas going forward. So an admin and a signer for publisher and for the employer similarly an admin and a employer, and then a role for the registry owner. Yeah, I think that that would allow bigger organizations to scale the work better and also to keep control on what is happening in the organization. Similarly for that scaling property, it would be nice if the different personas can like delegate pieces of their role to other parties, especially in like a very large organization. So like, you can have multiple people in charge of a particular persona, like it doesn't have to actually be a person or even a single key or anything. Yeah, correct. So in the POC we built today, we were now thinking to just add some role based access control with JWT tokens on the web interface part. But of course the best use case would be that all of this is managed within TuF itself. So it's not depending on this web overlay, which we now have as a POC, but that it just is embedded in the TuF framework on like who is able to add new delegations and who is able to remove them or who is able to rotate the certificates for a given target. If that could be embedded in TuF, in the signatures, then it's more watertight than having an overlay solution that builds this on top of it. I think those roles would make sense, but do we want to make that distinction now or do we want to work on a design and then figure out what the different roles are and come across that. I think what I was attempting to do right now was at a very high level capture what the different use cases are. So we can further find that into design and I think that design would spell out what the different activities are and then I think we can go more into like what the role distinctions there need to be. Yeah, definitely. That definitely makes sense. I think it's good to start with, you know, what's going to be done and then we can figure out how to make sure that all the right entities are trusted to do those things. Yeah. Make sense. Okay, so I think what I'm going to do next step is add in those additional personas. I had some more details that trust for configuration use cases, but just then I wanted to go back and I think we still run sort of like the root kind of management piece. We didn't quite really settle on an answer there. What are sort of like some of the additional concerns that we think we should debate. I feel like I still when I try to kind of reason through some of this, especially around around this like additional party for root revocation I still have a really hard time. I think it's just I need to go through the scenarios because as I kind of map this out in my mind. It feels like this is just adding another party. And I, you know, I'm going to probably go back through the recording for this and try to understand better where you're coming from but I still have like from just a security trust standpoint. It feels like this just makes this design weaker than having the root do the rotation, like like you're adding in another party. And I'll try to have an easier way to reason about it but like fundamentally the way I the way I still see it is is that the the trust like you're you're just adding a party that you're that you're relying on even more. And you already were in trouble if the root was compromised but now if this other party that serves as the central. Like, I don't know what you want to call them but the is that that's not the trust store, right, or is it. No, it isn't the trust or essentially is the employer configuring which keys or which entities they trust. So that's not the trust or I think what let's do this. I'm going to clean up some of the language here. If you want to take a look at it next week and then come up on Friday with sort of where you see sort of like the concerns being we can address this in the next meeting. Okay, maybe maybe it also makes sense if we add a small diagram on on how these things would connect on a high level to to get a better understanding to make sure that we are on the same page. I can do that I can throw in a diagram this weekend. So, Justin, I'll add in some of the comments we discussed today about the personas I'll add in some diagrams. If next week you can take a look and then I think we can have a debate around what the root management means and we can probably settle that next. And maybe another thing is, do we need to decide that basically in this phase how the management looks like or is it just enough to say okay we need some way of managing routes and whether that will be done via rotating or another way. That's probably also more in detail design right. I think where it sits is something we will want to cover now, because it does kind of I think play into like how roots are getting rotated how when you're doing key management where that's plugging into. And if we are essentially coming up with a requirement for publishers to manage a service. That's something I think we should close in with the larger audience as well and say we're okay moving forward with that. So I think one more settling. And then we can go back to the larger audience and see what what the consensus there is. Yep, I agree. Was there anything else I didn't have any other I think I've gotten some good feedback today as well for the next round of changes on this. Yeah, I still need some some time I'm sorry I didn't prepare well enough for this meeting I still need some time to digest this but I'll have more feedback next week. Okay, sounds good. Marco Marina. Sorry, can you repeat it. I was asking did you have any other comments. No, not from my side. I don't think so I think I need to also read over this more carefully excited for this but I should read over it and if I have anything else I'll just leave comments. I'll go ahead and publish another update by Monday morning, and then we can, if once you guys get a chance to review if you put in comments I can also start looking at it for the Friday meeting. But next Friday I'd like to close this and the Monday after just shares will be agreed upon with the larger audience, then we can start like focusing on the design aspect of this. Yeah, that's good. All right. Thank you. Thanks. Enjoy. Right.