 My name is Josh Dalitsky. I work at a company called Chain Guard. Come visit us in the booth area. We have socks and other things. Saje could not make it here today, but we have a fantastic video of him doing a fantastic demo. I encourage you to watch it while it's happening, and it's going to be a great presentation. We're going to talk today about OCI, which is something that I have found myself entangled in in the last several years. We're going to talk about this. How do you connect different things inside of a registry? If you're in this space or if you're in this talk right now, in this room, you probably know that people have been putting things into registries in questionable ways. That's an issue because tooling doesn't know how to work with that. It's really a two-part issue. One is what I just said. How do you get a list of these things that point to an image? I'll say the word S-bombs. How do you find out, basically, given an image, what are the S-bombs that point to it, signatures, or anything else? You should be able to attach any type of metadata without modifying the digest of an image, because that's what makes container registries secure and lets you know that nothing has been tampered with. Always use digests. Exactly, how are we going to get this list? The second part of that question is, it's implied of people are putting these things in here, but how do you actually do that? How are you supposed to do that in a way that has been blessed by the open container initiative? It's like the recommended approach because there's been a few different ways, and some are trying to change the system, and some of them are trying to work with the existing system, and certain registries don't have all the features, and you'll find flags on tools like Cosine that help you work around these issues. It's not a sustainable thing, so that's the problem, but it's a problem that has two parts. One is, how do we actually do that, but how do we actually do that with the community? For the people here who have worked in open source, you know that the harder part is working with people. Anyone can write an amazing API and an amazing tool, but how do you get people to agree on it? We broke down this talk into two parts with the speakers, so I'll be talking about number two, because to me that's actually the interesting thing here. If you want to step out and get a coffee and come back in about 10 minutes, you can see the code and that's when Saje will appear on the screen then. A lot of people are acutely aware of issues in OCI, but it really helps to paint a picture of why OCI is the way that it is. There's things to which I do not have the context, but there's several people in the OCI community who have been around for several years, from the beginning really, and they still attend the calls, and every time you bring up, well, what's going on here? They kind of all just kind of rub their heads, and it comes down to the reason OCI exists is so people wouldn't sue each other. It really is. I mean like Docker was doing things, CoreOS was doing things, and something happened where someone needed some legal something, and so then the OCI existed, and we codified all the things that Docker did to make it an open standard, and so it wasn't just Docker that gets to write the rules for how containers work. The tooling there at that time was Docker and Rocket, if you're familiar with it, if you remember Rocket. But yeah, 2015, and this predates really my involvement in any container-ing open source. It was formed. 2018, the registry, like the Docker registry, if you are familiar with that project, it's called Distribution. It's a CNCF project now. That was then turned into a spec in 2018, and that's a little bit after that is around the time that I got involved in this. So I kind of come into this, and I'm really confused why it's very slow, but it really helps to understand that it was kind of a thing put down, like a stake in the ground like, okay, we agree here, now don't do anything, but I want to do things. And also, this is a really important piece of the Kubernetes puzzle. It's kind of taken for granted that the way that a container is run, or referenced, and all these different things in Kubernetes, it just kind of works, and Amazon, and Google, and Microsoft, and now GitHub, and all these places have these registries, and it just works. It's really actually one of the pieces in technology that it really is ubiquitous and works well. So kind of a lot of the reasons why people don't like to move things is because, like, why would you break it, or why would you fix it if it ain't broken? And then also, there's competition. There's quite a bit of, and I don't think this is specific to OCI, but kind of, maybe. You know, it's like, I didn't come up with the solution, so, like, that's not really that great, and actually look at this thing that I did, and I have this blog post, you should read it, and that doesn't really help because, you know, we're trying to make changes that will, you know, help do these things, and there's, you know, supply chain security now, and we really got to, like, we really do have to do something, and you don't want the OCI to go away because it does have a good core set of things. The other thing is that even if you don't codify it, things end up doing this anyway. So, you know, these three tools, Scopio, ORS, and Cosine, in varying degrees and different ways, they have the ability to kind of either publish non-container things to a registry or attach them together, and then these things get used by ecosystem tools. So a great example is Homebrew. They actually, like, when you do brew install something from the official TAP or whatever it is called, it's actually going to GitHub's registry and downloading a foe image. So, like, this is already happening, and it's not, it's like kind of exploding, and we got to kind of figure out what's the proper way to do this. One note on Cosine, and I only bring it up because it was mentioned in the talk description, is that Cosine is like a fantastic tool and it has solved all these problems, but really it's doing a hack on the current OCI specs. And, of course, like, if the image on the right is like from a GitHub registry page where you've pushed a bunch of signatures, you end up with these bizarrely named, you know, tags, and tags were not really meant for this purpose, but it works, and it works well, but now when you list your tags, you have thousands of tags, and that's like, that's not, like, I'm not sure someone could look at that and say that that was the right approach to this. So, there's, if you want to go on GitHub and read the series of historical content of why this all happened, you can, I don't recommend it, but at some point in time there was conversations and discussions and suggestions, and the TOB of OCI decided, let's do a working group to finally fix this because there's camps over here that want to do this way, camps that want to do this way. So, with the help of Linux Foundation's legal team, they, you know, put the right words into the charter, which remember was really created to prevent people from suing each other, so like we want to still prevent that, and so they helped, you know, organize us to be able to do this, and then, so that was, let me see, from May to August this thing was open and people are asking, you know, should the wording be like this and should the wording be like this and it's still moving slow, and then finally in November, Steve, who's from Microsoft, suggests now that the thing has been merged to open up the working group, and that happened, that was in December, and then I started working at Chain Guard in January and working on this project. So, from there, we opened up a doodle poll and said, what are the good times for everybody? We found out Tuesdays at 1 p.m. or something was the time, and then we ended up on a Zoom call and we were kind of like, what do I do here now? And Lachlan, who's from Microsoft, had a fantastic idea where on the call we all sit down and we say, what do I think I'm here to do right now? And we wrote these down and we finally came to consensus on what we're here to do, and that was that first slide, which is propose how to describe inquiry relationships between objects stored in an OCI registry. So, okay, so we have that, and then now we have a lot of people with a lot of great ideas, a lot of people with fully complete ideas, some people with half-baked ideas. And you can't just get on Zoom calls and just talk, right, for one hour a week and expect for something to happen. So I had this idea, and the group was pretty enthusiastically supportive of this, which was create a markdown template for how I will propose how to do this. And you must, you're gonna submit a proposal and I'm not even gonna review the contents. I'm just gonna make sure that it meets this template so that when I'm visually looking between the different proposals, I know how to get the context for how they differ in these different ways. So short description, long description, links to existing blogs or anything that talks about this, and then at the bottom there you can kind of see this is what changes in the JSON and this is what changes in the HTTP API. So that worked out pretty well, and within a few weeks we had four proposals on the table, and they were all kind of similar with the exception of D, which was like, let's just do cosine. And people, like, obviously that was, it wasn't like, you know, it was a legitimate proposal, but obviously we formed the working group so that's not what we're gonna do. But cosine did do things in an interesting way, you know, it's working so maybe don't try to break it, but again, there's a reason we're here. After that we were kind of like, you know, there are parts of all of these that we like, and they're actually pretty similar, so let's combine them. And a big shout out to Brandon Mitchell, who first of all just did a lot in this group, but put together this, a combination of all of them kind of took the best of it. And then we were ready to go, and it was, we were celebrating, we were poppin' champagne, actually not yet, but so we decided we were gonna do E, and then all of a sudden, kind of out of the woods comes proposal F, which is sort of like, also don't do anything, but like, abuse OCI in a different way, and there was value to that, and it had support from the Docker team, and so we were kind of like in this, like kind of, we were kind of like, we thought we had it down, and then it got to this one, and we're all kind of looking around the room like, oh come on, now we're here again. So we came up with this idea where you will vote on, we're just gonna say, okay, E or F, and we'll just do democracy, and people thought that would be abused and make a bod, and I'd vote for it, it just never happened, people were pretty cooperative with this, and so we open up an issue and say, hey, put a thumbs up on this one, if you want E, thumbs up on this one for F, and E ended up having majority of support, so finally we got to that point, and from there, this is, it was kind of like, all right, we have the proposal, but how do we get this in a form that we can give it to OCI? One of the other things that we did is we created an organization called OCI Playground, no one has sued us for this yet, but basically we gave admin to anyone who was involved in the working group, and we just over the course of two months staged these PRs and did PRs in the fork to come up with what we would finally submit to the OCI main branch, and those were opened on August 9th, and I will cliff hang it, and you will wonder if that worked out or not. And now, Saje. Hello, everyone. I am Saje Anthony, an engineering manager on the Azure Container Registry Team and an OCI ImageSpec maintainer. Hold on, I want to do him justice, so let me figure out how to full screen. First of all, thank you to the community for making this happen. As Josh mentioned, the working group was tasked with finding a way in which we could store things related to image, like an S-bomb or a signature, and also defining a way in which we could consistently access these from different clients and services. We had gathered a lot of feedback, and it was the collaboration from the different community members that made this possible. Before I jump into the specification details, I would like to thank Brandon Mitchell, who had created the initial set of diagrams that were used during the calls to kind of like even talk through them, and I'm basing my content on that. And now let's jump into the specification details. Let's start with a tag. A tag or a reference to an image typically points to an image manifest or an image index. The index is a collection of manifests of different platforms, and whereas the image manifest itself is a definition of what an image is. It encapsulates things like layers and the config of that image. We needed a way in which we could define another type that would be used to store things like an S-bomb or a signature related to an image but not an image. And for that, we ended up specifying the artifact manifest. In the artifact manifest, there's a field called a subject that points to that specific image if you want to think about it, and that is what we refer to as a subject, which is the descriptor of the image to which it is related. There's an artifact type in that, and that field can be used by clients to kind of infer the contents of that artifact, which might be stored in its blobs collection and stored similar like layers. One more point I would like to mention is that the image also has a notion of a subject field now as part of the changes that enable a very similar way of storing these relations, and I'll talk to more about that in the fallback part of the specification. So now let's see this in action. First of all, we need to run a registry that is capable of understanding these new types and these new APIs. So in OCI Playground, we have an image that can be used to start playing with this. So I'll run this image on port 5000 and this image can be accessed from GHCR in the OCI Playground repo called registry. Let's copy an OCI image into this registry. I'm using a client called ORAS to do that. I have a test image here, and I copy it into our running registry with the following name. This image now, as you can see, has this digest which starts with 34B7, and to this image now I want to attach an artifact. You can use the OCI manifest on the registry API statically, but to show that I'm just using the client called ORAS that would use to create this new artifact manifest and attach a cat picture to this one for now. Use the attach command. Specify that I want to attach it to this image. Specify the artifact type and the file that I want here, and it's type. As you can see to this tag, we have now attached this manifest. Let's take a look at what this manifest is. I access this repository and say that I want to access this manifest of this specific digest. We see that this is an OCI manifest an artifact manifest, and it has the blobs that we defined and the subject of that specific 34B7 Hello World tag that we had used. Let's recap. We have an image, the artifact associated with the image, and we create a link using the OCI artifact manifest and we attach them using the subject property. We could go ahead and attach more and more of these to that specific image, but now the next interesting question is how do we actually access these things that are attached to that digest? For that, the distribution spec has a new API called the reference API. You can think of it as an API in which you can vary for relationships to that specific digest. You can ask it, give me all the reference for the image Hello World latest using its digest against this API. The API will give you a response of these are all the manifests that have been linked to that specific digest manifest. You can see that it follows the specification of the image index, and this is important because as you would see in the fallback the same similar specification or the index objects is used to link the objects as well. If you pay attention to it, it stores an image manifest and an artifact manifest and the response would actually enable us to link different types of artifacts together. Let's see the API in action now. I have the subject here which is my Hello World latest image. What I want to do is go ahead and query the registry. I access the V2 API which is the base of the registry and specify the repository. I say reference for that specific digest. You can see that I get an image index with the manifest of the OCR artifact and it has an artifact type that was specified here. Let's go ahead and attach another artifact to that. Same image, a different artifact type this time and let's go ahead and attach another text file to that. Now you have another manifest that was created. If I go ahead and query the reference API this time I see that there are two manifests and so as you keep attaching more and more content, this API will return you all the linked artifacts of that Hello World latest manifest. Now, for making sure that you don't get everything, the specification also supports introducing a filtering of that specific artifact type. You can ask it, show me only the signatures or show me only the artifacts that are of this specific type. And pagination is by a standard link header as well. Now let's get to how do you use this today. As you probably know, there aren't registries there that kind of support this, but our hope is that we will see more and more of them come out in the new future. But to be able to use a part of the specification as it is today, we have the fallback defined as well. A couple of things that we want to keep in mind is the fallback has to use an image manifest for achieving similar results. And as I spoke before the reference response is a dynamically generated index instead we depend on a statically client created index for this. Let's see that in action now. So for that, let's run a registry on the same port. And at this time I'm going to use one that does not have the OCI artifact support and I can use the registry image from Docker Hub because it's a standard distribution image. Let's go ahead and copy an OCI image as we did before. Keep in mind that this registry does not have support for the new OCI artifact manifest or the reference manifest. The fallback specification describes how we can use an image manifest to construct an artifact and push that to the registry. And it also talks about how you can use a digest tag that would describe a response very similar to the reference API. Now for this I'm going to be using a client called RedCTL that has support for this fallback part of the specification. So for what I can do is I can now push an artifact that has the subject which is the image that we just copied and the artifact type is as we did before and here let's go ahead and attach the cat picture. Now let's look at how the tags of this repository have changed. You can see that there are two tags, the latest and a digest tag. And if you look at the digest tag it's pretty much the digest of the image that we had copy. So let's now go ahead and view the manifest that is described by this digest tag. You can see that this digest tag stores an image index which is very similar to how the reference API stores it and it has a manifest which we have just created and it is an image manifest instead of an OCI artifact manifest. And let's go ahead and see the contents of what this artifact is here. As you can see this is an image manifest it has a config media type that maps to the artifact type that we specified. It's got the layers of the cat picture and the subject is the subject of the image that we had initially copied. So you can see that even though the registry does not have the support of the reference API, you can use a digest tag to formulate a manifest and it should be very similar to how the reference API should function. And this can be done all on the client side today. Just to see how this changes when you attach another manifest, let's do that. Let's use the same subject, a new artifact type and this time I'm going to attach the text file we attached similarly. And if we were to look at the tags, we see that we still only have the single digest tag. So each time an artifact is attached, it doesn't change the digest tag because the image manifest digest is the same. What we can do is look at the manifest that now what this digest tag points to. And now we can see that there are two manifests in this digest tag. So very similar to how the reference would actually return all the attached manifest, we can clearly see that the clients can formulate a very similar index and push them today. I think the community would benefit from others looking at the specification, giving us feedback. We're very grateful for all the contributors to making this happen and with more feedback, I hope that we can make this even better. I also hope that we see operators rolling this out so that the manifest API and the reference API can be used to attach more things and achieve more things in the supply chain security world as well. And it helps the community adopt the registry as a standard model of storage as well. And with that I want to thank everybody for the contributions that you've made and hope we see this improve over time. I would like to hand it back to you, Josh. Thank you. Thank you, Saje. So we left with the cliffhanger of we put months and months and months of effort into this but there was never kind of a guarantee that OCI was going to accept it and I'm happy to announce today that we did it. So our changes were merged. Yes. Let's hear it for OCI. Yeah, so our changes were accepted. And yeah, so basically all we have left to do is cut the 1.1 release. We're kind of working out a few we're changing words here and there. We want to get conformance tests to be to let you know which registry support this new stuff. We want other CNCF projects. So if you're a maintainer on one of these projects, please please talk with OCI. We somehow have our own Slack and we'd love to help you on board with that. And also just this was kind of it felt monumental to us even though we added just one API endpoint. So thank you to everyone. I know there's several of you in the room today and just it really means a lot to see this kind of all come together after many, many actually several years of kind of arguing online about this. And thank you for being here today and this link here is the demo that Saje gave. You can just run those forks of Aura and regctl in a container and you can try out the exact thing of the demo. And with that I strategically left no time for questions and thank you.