 Hello, everyone. Welcome to Cloud Native Live. Very excited to have you all visiting us once again. I'm seeing this amazing program. I'm Annie. I'm a CNCF Ambassador. I'm here with amazing speaker and a guest as well, but a bit more on that later. Very excited to be here today. Just as always before these streams, please keep the code of conduct in mind, so that essentially be respectful within the chat, so that the CNCF code of conduct will be met and we will not face any issues there. So be respectful of our other watchers and seekers and so forth. Very happy to have you all here. As always, you can ask questions within the session and after the speech as the software has ended, very happy to receive those as well. But yeah, let's get started now. So we're here with Steve from Microsoft. Welcome. Thanks. Thanks for having me. It's great to be here. Yeah, perfect. So excited for your session today on Notary Versus 2. Versus 2, why don't you get started and show us what you've been working on? Well, thanks. Thanks for the opportunity to talk about all the great work that's been happening across a number of different efforts. One of the things that we started out in a while ago it was actually this effort started back in 2018 at a KubeCon event where we all realized we really needed to get re-engaged and solving the signing problem around content. And specifically, it was container images. Notary V1 was started in 2015. It was built at Docker. It came out as Docker Content Trust. And its goal was to sign the content in registries and that, you know, it did. But one of the things that there was a couple of problems with Notary V1 and Docker Content Trust, there was some usability issues that was very difficult to use, difficult to configure. If you had the flag turned on or off, you might actually get, you will get different content from a registry. There's trust and first use challenges. But the fundamental problem with it was it didn't support content promotion. If I've got an image in a public registry and I want to pull it into my private registry, it's great to validate the signature of the public endpoint. But when I pull it into my private registry, we wanted to be able to validate it there as well. And the problem was the signatures were tied to the location of the content. So that was the real fundamental problem that we needed to go out and solve. So that's kind of how the whole project started. I work at Azure. I work on our container registries that works on the private registries our customers use and the public registries that we support for Microsoft content. Microsoft is a software company, not just a cloud. And we need our software to run on AWS, on Google, on-prem, and this is like everything from Windows to SQL to .NET to Office App. So we recognize that we needed to solve this in a cross-cloud vendor-neutral way and not Azure-specific. That was a fundamental goal of what we had to do. So we got a bunch of people together and it was a who's who of registries and cloud providers. It was awesome. AWS folks hosted us at their location when we were all getting together on a regular basis. And we've been chugging at it since. It's interesting. One of the lot of the challenges that we've been facing is like, how do you sign content? And what is the platform capabilities that have to elevate from there? So that's been the challenges we faced on what exactly are the investments to make. NodeReview1 and DockerConnect NodeReview1 and DockerContentTrust required additional services to run on a registry. They're pretty complicated services. We built it for ACR. After a month or so, we were like, it's still not done and still a lot of work left. And we have some pretty, pretty really talented people that were working on it. And then when it was all done, we didn't support the core scenarios for content promotion. So when we engaged with NodeReview2, we set out a bunch of goals. And those goals were promotion across registries was fundamental. We wanted to make sure that the impact to registries was absolute minimal and not unique to signatures. We recognized that instead of just putting a thing that only supported signatures in a registry, what could we do to make it a generic capability to support other things as well? And then we didn't want to have to create yet another service. So we wound up realizing that if we improve registries to understand additional objects in a registry and they can be established a linkage between those, then we can meet all of our goals or they kept on going there. So one of the fundamental things that, for instance, that we came up with was all the Helm charts you have, all the pod spec, all the text files that you have, strings that reference your container image shouldn't have to change just to validate a signature. We didn't want anybody to have to change their existing infrastructure. The analogy we like to use as going into an airport and that workflow of getting in, getting approved and going on to the next step, you should be able to use your existing documents and there's an extra check. So one of the things that we designed around this was your reference doesn't change, but you can get information from your reference. So for instance, if you want to deploy a net monitor image, net monitor colon V1 or net monitor digest ABC123, that shouldn't change just because you want to validate a signature on it. So that innovated this concept of reference types. And if I can assign a signature to it, then why can't I assign a SBOM, a software bill of materials or a system bill of materials, or a scan result. So there's a lot of interesting things that kind of evolved from there. It might help if I actually show a little animation for how we think about this. Perfect. That sounds really good. And there's already people excited in the comments. They want to read the talks for this and they're sharing I think blog posts and everything. So very, very excited people already. Oh Lord, I want to read the docs. Yes. So I like that one. Okay. So I like comparing to systems that are tried, proven because there's been a lot of work put into those and they're kind of tried and tested. So one of the ones that I like using is the process of going to an airport, which is what we used to do and we're starting to do again in some aspects. And I was reminded internationally, TSA, what does that mean? So if I go to the airport, there's that person that keeps you from going into the secured area, a transportation security administration, something I'm actually not sure. In this case, it's not the timestamp authority. It's the person that says, you cannot enter the secured staging area of the airport yet. You have to prove who you are. So I woke up to the airport and I say, here's my pod spec. Here's my helm chart. I want to be boarded on that Kubernetes plane over there. And the agent goes, that's great, but who are you? It's great. You want to go there, but who are you? So I provide them some identity. Now the identity can't be a note from my parents. It's got to be an identity that can be validated and proven. Now the identity doesn't matter where I am. I'm Steve Lasker, regardless whether on the East Coast, the West Coast, Europe, wherever, my identity is independent of location. So the agent looks at that and says, okay, that's your identity. Let me, it's from one of the 50 states or one of a bunch of different countries if it's a passport. And they may or may not accept information from certain states that don't have real ID, for instance, just a detail here or certain countries that aren't considered trustworthy for whatever that means. But at some point, now the other interesting thing is I'm outside that red line. I'm handing my identity through the hole because I'm not allowed into the secure zone until I've proven that I'm worthy. That information to prove who I am is separable from me. I hand it through the window. That's really important from what I call it, you know, the Trojan horse attacks. If I had to go into the airport to give them ID, I'm like, hey, how you doing? You know, here's my ID. And by the way, I've got a gun pointing at you when I now own you. If I, sorry, you know, if an exploit is I can execute code and I need to get code on the machine. If the validation was I pull the content in and the signature is part of the content, then as it tries to run bad image from bad code or, you know, hey, this looks good, but it's signed by bad code. If the image is already on the machine and the validation says, no, you're not allowed, well, the image is on the machine. Now I've got code execution. So I want to be able to separate that. So fine. The TSA agent can see separate from me. I'm not in the airport yet. I might be a digital piece of identity. And once that person has decided it passes the test, you are identified as one of the entities that we trust, then they stamp it with another signature, right? You get that little blue dot on your boarding pass or if you gave them your, you know, your phone inside the system, they have stamped it that the TSA agent, the agent there says, you are allowed into the staging area. Now, interestingly enough, if I go into the staging area, I get scanned. They want to know what the exact current state on new information. Somebody just discovered that I might have liquid and that's something they should look for or solid things in my shoes or whatever that might be. So scan is done and, you know, I'm approved, so I'm allowed in, but I'm in the staging area now. I'm still didn't board the Kubernetes plane, right? I'm just in the staging area and I might have to go through some additional tests and verifications or whatever. Now, Agent 44 at the gate to the plane is going, whoa, whoa, whoa, you're not getting on the plane yet. You have to prove that you are allowed. Have you been approved to be in the staging area? Prove it. Again, I give them my pod spec. In this case, Agent 44 doesn't care about my external identity, doesn't care about my driver's license or passport or I was signed by Wabbit networks or anything like that. I'm in the staging area of a particular company. That Agent 99 signature is equivalent to my company's signature that, you know, proved it. So Agent 44, it's like, okay, you've been signed. I will sign you again to say you're worthy to go into the production plane and now off you go. So that's just kind of an interesting analogy that we use for content promotion that if I were to apply it to how would this work in our real software. So we have the small company, Wabbit networks that nobody's ever heard of, very intentional and Acme Rockets is our company that we work at. I want to deploy this image. Great. I was able to reach across the public internet. I got the image. I could even reach across the public internet and get the signature. That's all fine. I do have a policy manager that says, hey, content must be signed. Sure. Signed. Go deploy it. Signed by who? I don't know. There's no policy here. Of course we want to do some testing of signed by who. But the other thing is what, you know, how many of you have environments where you have a vNet that protects that environment? There's no public egress. You can't go off to evalsight.co or, you know, public registries. They want that locked down to only the places they trust from a network boundary. So if that node is trying to reach out to some public endpoint to get the software bill of materials, to get some scan results or source or some other artifact, you can't get access to it. So this was the kind of fundamental principle. It wasn't just the images and the signatures we want to travel with the artifacts that you're trying to use in your environment. We wanted that entire graph. So, and of course this isn't just one environment. You have multiple environments and within each of those, you're standing up a private registry that's accessible from within that vNet. And it might be a hosted registry that supports vNets. You might stand up, you know, an open source registry or a project or a product. It's all great. But that's why we want to make sure that all of these can support these capabilities. So the next one is do I bring the public content into each one of those private registries? Well, that's not really a scalable solution. We don't have every plane, you know, individually check every person, right? Everybody goes into the secured staging area. There it's approved. That's how it gets promoted. Companies stand up internal registries which has the approved content. And there's lots of public registries that people interact with. They, you know, I'm having fun here with some old company names or fictitious company names. But NVIDIA, Oracle, Microsoft, IBM, there's lots of public registries that are software companies that distribute their content from their registries. You know, you want to be able to bring that stuff in. So this is where it gets also interesting with multiple signatures. And we'll get into some demos, but I wanted to provide some context to this. So why do we need to support multiple signatures? We talked about promotion within the environment. Well, you may not ever know of Wabbit Networks as a software company. When you go to Home Depot or Lowe's or whatever the store is, the big box distributor, it's a redistributor. There's lots of products in that store. And you kind of give this trust because they're in that store that you trust. If that store is reselling that product, you kind of trust that that's a worthy product. So you may not know about Wabbit Networks. So it doesn't matter that they have a signature on the content. If it goes to Docker Hub, Docker Hub is a redistributor, will sign with another signature the content that they trust. This is now certified by Docker for whatever that means. So now as a company, I could say, well, I do trust Docker Hub. If it's certified content from them, I've established that trust and relationship, that's fine. Now I might not, you know, Spacely Sprockets may not become Docker certified content, you know, and so forth, but inside of Acme Rockets, I can configure who I trust. I don't trust Wabbit Networks, but I trust Docker Hub. I might trust, you know, Spacely Sprockets and some other ones so I could pull information in. But what you're doing is you can establish a trust policy that says these are the entities that trust. These are the states that I trust. These are the countries that I trust. Maybe countries I don't trust to do an exclusion policy, but I can make that list. And now when I'm pulling stuff in into the staging area, I can see if any of those are passed. One, it's in the staging area. I could stamp it with an Acme Rockets key. And now all internally, I don't need to worry about the policy of the day for what was externally pulled in. All I know is it was signed by Acme Rockets. It's the shared library content. That's approved. I'm going to work from that content. So that's really kind of the fundamental principles of what we wanted to support. And now it probably helps to show some real code on how this works. But were there any questions that before I jump in, there is, it looks like... Not so far. Quite often they come at the end, but please do everyone ask as we go along. I'll hop in and ask your questions. But yeah, our demo would be really great. So let's get to that. Okay. So I'm really happy to show that this is actually working with Azure Container Registry now. So we'll show the interactions. It's in dog food. It's not public just yet. We're working through the last details of it. But what you'll see here is, I saved as an environment variable, so I have to paste it every time. But there's the Wabbit networks that Azure CRIO, registry that we have. It's an authenticator registry, so we're working through all the auth stuff as well. If... So now what I want to do is, do a standard Docker build, Docker push. Nothing magic here. I'm now going to build this image and put it into the registry. And I want to sign it. Okay. If I want to sign something, one of the things we've been focused on is, what is it that our customers and users need for their production environments? The production environments are the critical environments that they don't really care about containers. That's the tech. They've got standard practices like Venus, like X509 certs for signing. That's what the crypto boards of those companies have approved. So what we're going to do is take an X509 cert. I hear I get a self-signed cert. This could be a CA cert issued publicly. It could be by a cloud provider. Doesn't matter. I'm just... That's what we're using here is I have a certificate that is going to be what I'm going to sign with. And now I can simply sign that image. No external, no other additional services. It's just the registry capabilities with a notation CLI and X509 cert, which is what companies are using today. I can see what signatures are on that image. And now I might want to verify that image. I want to see it. Does that image pass the test for the content I trust? Well, it's signed, but signed by who? Just because I just signed it here, fine. But this is like asking that agent at the airport to validate your identity, but they were never given a list of approved identities that they are allowed to trust. So notation, go to review two, the notation CLI has a trust, sorry, an opt-in trust model, which is secured by default model. So I have to tell the policy to trust the Wabbit network's key. So now if I try to verify... Well, actually, let me show you that policy. So we can see this was the signing keys that I did in the previous step. And here's the certs that we trust for verification. So now if I try to verify that image, it passes because it's not just signed, but it's signed by an entity that I've trust. I can configure that trust. Now here's where it starts to get really interesting. Great, it's signed. Great, that shouldn't be so hard. It starts to get interesting is, well, I want that signature to travel. I don't want to change the digest. How do we implement these changes on a system? So that we can promote this content. The ORAs CLI is... OCI registry as storage is how the project started. And it's basically a CLI for interacting with the registry. And this concept of being able to push reference types to a registry is an implementation through an ORAs artifact spec. It basically says, hey, I can push something to the registry and says this thing refers to this other one. The ORAs CLI knows how to ask the registry. Hey, what information do you have for the named reference that you were using before? We don't want to say you have to ask for something special. Your Helm chart says net monitor V1. It says net monitor digest 123456. Go ask the registry, what do you have associated with the net monitor image? And you can see there's an ORA V2 signature. And this digest... Steve, by the way, we are not seeing completely the bottom of the field because there's kind of like a bubble there. So if you... Yeah, perfect now. Perfect. It should work now, bro. Thank you. No worries. I always worry about being at the bottom of the screen regardless. So you can't see it on a stage, but people look at their screens. So it also can get trimmed. So apologies about that. So what we're seeing here is that graph of content, right? So that digest here is the signature. It's the manifest for the signature. And now I can go find the blobs. But wait, there's more. So let's... One of the things we said we wanted to do was we wanted to up-level the registries because we don't want to break existing workflows. Users have a way they look at the list of tags. They have code that runs based on the tags that are in the registry and they go and delete and manage lifecycle based on the tags in the registry. With this extensions to the registry, the tag listing doesn't change. It's still the V1 tag is the only thing I can see there. And the analogy, again, I like to think about is file systems. A registry is just a file system in the cloud. If I look at the list of files on my machine, I see the file names because that's what I asked for. That's the things I think about. If I want to see the attributes of a file, I ask it, please give me the attributes of the file. When I copy the file, those things travel with it, but I don't... My list of files isn't populated with noise. It's a horizontal inference, another pivot of information that can be displayed. I want to see that my net monitor image is signed, but I want to see it as a glyph on the portal that shows it. I don't want to see another tag that shows up in there. So that's part of the way we think about the breaking changes to registries. If I want to see, the registry does have the manifest, so I can go and ask... We have now fixed the bubble name tag, so if you want to go full size, you can do that again. We're iterating here on the go. No problem. So if I look, and this is the az... Sorry, the Azure Container Registry, or the Azure CLI, and we can see there is... Let's see, because the order can be different. Here's our net monitor image. You can see the tag is there, and there's a bunch of information, and whether it's writable and all that goo. And here's another digest that's the Aura's Artifact manifest for this net monitor... Sorry, the Notary V2 signature. So the content is there. We can manage it as content, but we've made sure that it's thought of as an attribute to the net monitor image. So just a bit of detail. All right, let's kind of pause for a second here, and I now have this Wabbit Network's net monitor image. That's the thing on this public registry, whatever. I want to bring it into the Acme Rockets environment. So a couple of things. Let's clear out... Let's pretend we're going to have what I call an ephemeral client where I've got a VM that shouldn't have anything on it, because we want to make sure we're not pulling in information. I think your important environment shouldn't have state from something else. If I look at the Docker images, the only thing I'm going to see is a local instance of CNCF registry. We'll use this later, so that's where it's showing that noise. Well, anyway. So for all kinds of purposes, this is a clean environment. And I'm going to clear out the notation configuration. So I'm going to clear out the policy. So now I have nothing configured. Now I want to create a certificate for... And I could pull this from Azure Key Vault. But the idea is that I want to have a cert that as I import information into my environment, I can stamp it with our Acme Rockets key. And this assumes that I've done the security scan. I've done the unit testing to make sure that update to the Debian image or the Net Monitor image or whatever is in compliance with the Acme Rockets policy. It's great that they put an update. But if an update is a change, is that change an accidental human thing that might break my environment or might be an exploit? Is it an evil person that made a change? Lots of things. We don't want to blindly pull in changes from public locations. And if we look at our configuration policy, I now have... I have no verification certs, but I can sign things with the Acme Rockets key. So if I try to verify, of course, it fails because I didn't say who I trust. If I sign it with the Acme Rockets key and I put the Acme Rockets key in my trusted certs path, let's go back and look at that again. Now I can verify the image. But I'm verifying it at this point, I'm at the boarding the plane. I want to verify that it's signed with the Acme Rockets key. And if we look at that content, we can see that the Net Monitor image, notice the indentation here. There is a hash of the artifact types. In this case, it's a Notary V2 signature. I now have two of them. And because I've configured my policy to only validate the Acme Rockets client, the Notary, the Notation client, knows to look for that key and pulls the content and figures that out. So that's how we can do multiple signatures and promote them. So, okay, so I start off with simple signatures. We want to be able to support other things as well. So let's create a really highly dense S-bomb that the software builds materials. It has a long list of all the packages and how the thing was built. And there's lots of great projects out there. I'm just going to create a simple JSON file that says the contents are good. All right, keep it simple. Now I want to push that S-bomb to the registry as well. When I set notation sign, you saw how simple it can be if I put it in the notation library. That notation library knows how to find things in the registry, attach them as a reference, because it's the notation CLI, it knows to set the artifact type and so forth. Or as this generic CLI, think of it as a file CLI for registries, can push content to a registry. In this case, I'm going to push it to the net monitor repo. Not by tag, just the net monitor repo. I'm going to say the artifact type is an S-bomb and set the reference, this target subject is that net monitor image. And then by the way, just take this .sbomb.json file and send it up and the type is JSON. If I had an S-bomb tool, all I'd have to say is S-bomb push, but because I'm using a generic tool, I have to put some extra information. So that's in the registry. And now I want to sign it. Notary V2 signed manifest by digest, so I need the digest of the S-bomb I just pushed in, so I can use the or as discover command. And now I can sign it, signing that digest. If I look at that list, I'm now seeing a richer graph. I've got our net monitor image. I've got two signatures. I've got another hash of S-bombs and that's the first S-bomb that I pushed. And by the way, here's the signature. Notice it's hanging off of it. I'm building this rich tree of information that because everything in the registry I want to be able to sign. But wait, there's more. Let's say I want to generate a scan result and I'm using SNCC, which comes with Docker CLI for Docker scan. And this is a little more meaningful. There is a bunch of goo in here. Not important. It does save it as a file. So I'm going to push that using or as again. I'm going to do the same thing to get the digest of the scan result. And I'm going to use notation to sign that digest. Again, I'm signing with the Aki rocket ski because that's what I configured. And if I look at or as discover again, my graph is getting richer. I've got a lot more information now that's in here. So now I've got this really great content and it's on a registry. It's not in several different services that I have to figure out how to configure access to it. Now, I don't necessarily need to take the whole graph. I can say, hey, I only want the Notary Vee to signatures because the registry has this refer, this new refers API. I can say, hey, give me of that entire graph, filter it to the Notary Vee suit signature type. And then I get back just the signatures. So this is kind of the things we want to support notation for Notary, sorry, we want to support Notary signatures for signing content, but we saw generic patterns that lift the entire ecosystem up. And rather than build yet another service, we wanted to build this capability into registries. So now there's one last piece that I just want to show from a pure demo perspective. And that is that promotion, right? So I've got this rich graph, this here, that's in the public registry. I want to promote that to my private registry. If I was using my file system and I had a PowerPoint file and I had a movie file that I did with some media program and I had some Golang libraries, if I want to copy that content from one directory to another, would I have to go fire up PowerPoint and Golang and VS code and your media file, whatever? Of course, that's silly. I want to use a file system API that says copy the stuff from this directory to this directory. And it shouldn't care. It just knows how files are stored on the file system, including those attributes, copy it across. So if we look at our private registry, we'll see that there's nothing in it. There's nothing up our sleeves. There's literally zero repositories in it. What I want to do is copy this, and I'm going to use a little early version of something. Because now I'm going to say, ORAS copy that public image to the private image with recursive. So go and travel down all of the references that we have in the registry. ORAS doesn't know around notation or S-bombs or SNCC results or your favorite thing you want to attach or all the signatures are signed to it. It just knows how to read that generic graph in a registry and poof, it just copied that whole content. And now if I look at my private registry, I see one net monitor image. And I see one tag listing for the whole thing. So that's kind of what we've been working on and why it might be taking a little longer because we feel like the investments that we need to make are for the long term. We really want to make sure that as people are signing their content and they're improving their platforms, they shouldn't have to stand up yet of a bunch of other services and there'll be more services that come out. But we should have to run another whole service that users and customers and operators have to manage. We just want to improve the existing services we already have. So I can see that rich graph. I can filter that graph. I can delete the net monitor image and all the stuff that's associated with it should go away with it. I might want to delete just the S-bomb. You can actually remove individual pieces on there because they're all separate individual objects that are stored in that graph. Anyway, so that's, I breathe, I'll pause and look and notes and whatever, but this is where we're at at this point on the project. We're really excited about it. The, what you're seeing here is a combination of the Notary V2 work, the Notation CLI to sign, sign, discover, verify artifacts in a registry. It doesn't matter what they are. We started with container images. We realized that we can do this generically. So let's sign everything. We didn't want to create yet another service. We didn't want to break existing workflows that people have for how they interact with registries. But we did believe registries should improve their capabilities to serve the users for these capabilities. So there is enhancements to registries. Here you're seeing the Azure Container Registry support that new or as artifact manifest. Most users shouldn't have to care. We're working with the various registry operators so they can add this capabilities. AWS is committed, Docker is committed. We saw the Zot project just recently started working on it as well. We're really hoping that we'll just improve the capabilities of registries because everything in CNCF is improving. Why shouldn't registries? That's been the main focus of our effort. Yeah, no worries. Maybe a few questions from my side at the moment. So we've covered quite a lot already. But as you probably know, supply chain security is a very popular therapy. So can you explain how notary to fit in this landscape? It's a bit more. Yeah, no, it's great. In fact, let me pull up a different slide. With everything, there's always a question of what's your focus? How much of the ocean are we going to boil with this effort? When I think about the supply chain, there's stages. There's the creation of content. There's the building of those binaries, building of the S-bombs, all of that information. And then there's the distribution. We tend not to ask our parents to compile and install from a Git repo. They tend to install apps from an app store or from various locations or our corporate rollout will rollout something. Or we click an install button from various distributors. And this is where I like the model of redistribution. I don't go typically to a farm and go to the cow and fill up my bottle from the cow. The milk manufacturers create that. They bottle it. They distribute it to QFC, Safeway, whatever the stores are near you. And you get that content. That content gets built and it gets distributed. It might get distributed on Docker Hub. It might get distributed on NVIDIA or MCR or whatever. You're going to promote it into your registry. We want to make sure that the redistribution of content is the part that we're focused on. As you're distributing it, the things that prove that I'm still Steve Lasker and wherever I'm traveling, that net monitor image is always signed by Wabbit Networks. Wherever it goes, the Kohler faucet is always from Kohler. That point when you consume it, you can prove that regardless of where it is at any one point in time, it's still from that entity. So that's the detachable signatures that we think about. There's a lot of work going on the supply chain part. And then what I call to the left, when I call, the term it's referred to is to the left of how things are built. Notary has nothing to do with how gets are signed or commits are signed or files or all that kind of detail. There's a lot of great efforts that are around that. The closest notary gets involved in that is when you're doing a Docker build, for instance, and your from statement references another image. We want to make sure that that consumption of another deployed, the distributed content, is who it says it is. So we're really focused on the distribution and validation part of the pipeline for who is it signed by entity you trust, not just an arbitrary entity. So that's kind of where we think of where notary fits in. Great. Do you have another slide that you wanted to go through? I think I saw something. Oh, there's lots of content. So I'm happy to answer any questions from folks if there's anything else. I could, there's always interesting things to talk about. For me, one of the new things that is coming up is, we've all seen this, Docker kind of wants to have taken the burden of the hit on this with we all want to pull from Docker Hub. And it turns out hosting all that content because container images are big is expensive. It's expensive. It's problematic because nobody ever wants anything to be deleted and so forth. So when you have, so what's been happening is same model. Like you don't go to the one store in the world to get milk. That milk is distributed across multiple locations. Well, so is the Debian image. Somebody's going to build that image, but we want to be able to get it from multiple locations. So that same image should be able to be distributed that signed, has an S-bomb, has some scan results, so on and so forth. And this is what we really trying to refer to is really teasing out identity from location. And we've been working with the SPDX and Cyclone DX community recently and Rose Judge from VMware did some great work around having a Pearl spec for OCI references to also decouple location from identity. So this is kind of a model we're trying to get to that the content will be distributed in multiple locations just like you can get milk from lots of locations. But you want to know who was that milk distributed by. So if there is a problem with it, we know which ones are the ones that we should recall. So that's another kind of important piece. Great. So I see that you talk a lot about S-bombs and scan results and other artifacts in their registry. So is notary just around image signing or how does it go from there? Yeah, great question. So basically they said the trying to carve out where we wanted to focus that no review to is all about signing things doesn't care what it is. In fact, you notice that my S-bomb was just a JSON file. I didn't worry about getting the details of Cyclone and SPDX and others. The is literally just signing anything in a registry and the idea is maybe we could use that same algorithms to sign stuff that's not necessarily stored in a registry. There's lots of other great projects that are taken on those things. So it's sign any artifact that's in a registry regardless of what it is. And the innovations that came from the notary requirements allow other things to be put into a registry that are also associated. So we're there to support the other efforts. We're not trying to overlap with those. So how about then the kind of the graph sign content? Can you elaborate on why that's important? So can I have S-bombs and signature information in other services or how do we go? Yeah, so we really like I said, this is one of those where if you look too narrow at a problem it seems overly simple. By working in Azure and working in the center with container registry and MCR for distributing public content, we see all of the complexities that users face. And we recognize that it wasn't just around signing. We had to deal with content promotion, with vNets and all those things. So the graph of content is the thing that we showed around how you can promote that so that you can verify that artifact in the location that you're operating. You shouldn't have to be able to say, well, I'm in my isolated vNet, right? I have no public egress, but now we need to validate the S-bombs valid. Well, where's the S-bom? Oh, it's hosted on this Microsoft service out in the cloud or it's hosted on some GitHub repository or something. Well, I can't get to it because my vNet is locked down. And yeah, we can go and ask for a hold we put in the vNet, but we had a customer describe this as putting holes in the submarine, right? You can't just have, it's only one hole in the submarine, it'll be fine, right? Any hole is problematic. So we wanted to make sure that you can promote that content. And that's kind of, and we're always looking for better words, naming is hard. We think of it as this rich graph of artifacts. It could be a long list of different signatures that get associated with it as it worked through a workflow or it could be a deep graph where I've got a signature, an S-bomb and a signature for the S-bomb. So that's kind of the way we've been thinking about it. Great. Yeah, and there's been a lot of good interactions in the chat, not so many questions yet so far or someone learned something new today and so forth. What do you have any questions? Now is the perfect time to ask them, everyone listening in. We do not have too much time anymore so now is the perfect time to start typing those out so that we have also the time to answer them. But is there anything else that you wanted to show us as far as the slides goes or the demo? Well, this was a last-minute scheduled thing so I appreciate the folks that did apply. There was another session that it wound up getting canceled so we took an opportunistic time of like, I think 20 hours notice. So thank you Bridget and Karen and for helping us get that coordinated. I'll just say for people watching the video later on, by all means reach out. There's Slack channels, there's the projects. Reach out to me directly then we can connect you to the various folks that have been working on the various projects. It's, this isn't an Azure thing. It's a community engagement. We've got a lot of great support and we're really excited for the work that could be done. It's, there are a number of different projects this impacts because we believe that this is a place that can innovate a number of areas. It's notary for the actual signing. It's the tough project which we're working on so we can support the rich capabilities of tough as this content is promoted. There's some challenges there that we're working through and there's Marina Moore from that's working at NYU has been really great in trying to figure out how we're gonna handle all this stuff. There's the work we did in Aura's artifacts to enable registry support not just notary signatures but S-bombs and other stuff. There's a CNCF distribution instance of this so you can test these rich capabilities without any specific cloud. And that CNCF distribution instance is the thing that runs many of the registry projects that are out there. So we're looking to finish that up so that all registries can take on these capabilities easily and we can get on to the next big problem. So we're definitely looking for more help more feedback, more support or just jump in. That's the beauty of the open source community is anybody can kind of jump in and help. Yeah, definitely. That's the best part. So definitely everyone jump in. Really great session by the way so far. A question from missing character. Any new public keystores? Okay, keystores. By keystores, I'm not exactly sure what we're referring to here. What Notary is currently focused on is we kind of work from the right to the left from production back. And what we heard very clearly was these are services that build and sign and distribute and validate content. Users are part of the critical part of the development workflow. By the time it comes distributed and consumed X509 certs was the standard that we wanted to focus on. So X509 certs. There's lots of great infrastructure there already. There's innovations we think we need to make. We wanted to build on the capabilities that already exist. So that's what we do today. What we might add in the future. There's lots of great opportunities for how we might add additional signing authorities, if you will. I think that's kind of what you're getting at. But the premise here for keystores and where those keys are managed, there's already great key management systems. Cloud providers have them. There's open source projects. That's what we're trying to leverage and not try to invent anything new. Perfect. And we got a confirmation from this character that it was exactly what they wanted. So wonderful. And maybe staying on this topic, a bit of info about the future there already. But do you have any kind of sneak peeks or ideas as far as like what will version pre-look like and what will the future hold for the project? Sorry, for which project? Pre-notary or like... Oh, pre-notary for us. Sorry. Yeah, I mean there's... Look, we have some work to finish here. The... A lot of interesting things are coming out of it. We're adding... We're starting to add search capabilities to a registry, which has been a long-standing missing thing. All registries have them, but everybody's done it separately. So there's no common way to get information out of a registry. If you have multiple S-bombs or multiple scan results that you pushed to a registry, which one is the current? How do you know Mondays versus Wednesdays? So we're thinking about how we could do ordering and stuff. We're thinking about how we can do search. The way that you can push the artifact manifest does not require blobs. So we're starting to have the ability that I can add other annotations to information in a registry. So I can start to get rich metadata. What images are deployed? What images should be deleted because they hit some kind of exploration policy? Who built that image? So there's interesting rich metadata scenarios that we'd like to be able to add to registries. And that's some of the stuff that we're thinking about. But we're definitely trying to make sure that we land incremental progress and not try to, like I said, boil the ocean there. There's lots of great stuff that can be done. Yeah, I'm looking forward to it then. Final call, I think, for questions from the audience. But I have to say that there was already been someone, I think, sort of saying great demo. Thanks a lot. I have to agree. Really great session already so far. Do you have any final notes that you want to add to the audience or so? Let us know what you think about it. Is it helping you? What are the gaps? Would you like to get involved? There's lots of great opportunities. We certainly have more problems than we have ability to solve at this point. So we're trying to carve them off. And if you want to help with some of the stuff we're already working on, great. If you have another problem you want to solve, there's a great question around, can I curl binaries out of a registry? So we've been thinking about how we could do that. So we're starting to get some ideas around it. But I don't have anybody that can actually work on it right now. So there's always great prayers. The more people that we have, the more things that we can work on. So that's always a call out. If you have some great ideas and you want to see those ideas added and improved, reach out. We'd love to figure out how we can help you help yourself and get people started on that. Perfect. That sounds really good. And I guess these are the links to get involved as well. So everyone take pictures and notes or you can obviously watch their own demand recording to get access to these again. So everyone can get involved. It's really great. So if there is no longer any more audience questions, I think we can start wrapping it up. Really great session. Amazing content. Great demo. Thank you so much for speaking with us, even though it was a no. Short notice. So very extra thankful for that as well. And thank you everyone and the audience as well for joining the latest episode of Cloud Native Lie. It was great to have Steve here for Microsoft talking about Notary V2. Really loved the audience interaction and everything here today. And don't forget to join us next week when we bring you the latest Cloud Native Code every Wednesday. And next week we will have actually a session on improved core to edge mobility and resilience for cloud native applications. So really looking forward to that as well. Thank you for joining us today and see you next week. Thank you.