 Those ones got swiped up pretty quick this morning, huh? All right. That's my time queue. So welcome. Grab a seat, let's kick this off. Yeah, the title is definitely clickbait. So everybody that came here expecting me to completely trash registries and slam them. Sorry Phil, sorry Josh. I'm not gonna do that. I do love registries despite all the clickbait title but I'll get into why I'm going that direction with some of this stuff. So I mentioned earlier all the slides up online. So if I go too fast on something, don't worry, don't panic. You didn't miss a picture. It's all up online, you can download it later on. So who am I? Brandon Mitchell, that was on the first slide. So you probably got that already. Solutions Architect for Boxboat, just like the shirt says. We are now an IBM company. Got acquired, they pay us a bunch of money and boss said, hey, if you're gonna give us money, sure, why not? Also, Dr. Captain, and I got pulled into that after answering a whole bunch of questions over on Stack Overflows. So if you see B-Mitch over there answering questions on Stack Overflow, that's me. I didn't have to do pseudo over there, they just gave it to me. Also, I'm doing a lot of stuff on OCI which is why I see me up here on stage right now doing a bunch of this stuff. I've also, under the reason I got into OCI is I've written little command line tools to work with registries. So you'll see a little bit of that in this presentation and a bunch of stuff over on CNCF talking supply chain security. So I jump around a bunch of those working groups over in SSF working groups, projects, stuff like that. I'm around, some people are familiar with me. But let's talk container images in case anybody's not familiar with container image. And don't worry, we're not talking about just Dr. Bill would now put an image out there and what it looks like. I wanna talk about the nuts and bolts of this stuff, what it really looks like under the covers. And I'm gonna start with, there are two different kinds of blobs that are out there. And one of the blobs is the config blob. And this is something that should look fairly familiar if you've ever built an image and then later inspected the image. Because what it's gonna have is all the different, like the environment variables you've set, the user you're gonna run as, the entry point, the commands, any labels you attach to your image, all that goes in this config JSON. And that all gets shipped up to the registry as a blob. So that's one of them. And you might see up there that I'm querying this digest. I kind of cropped it on the image so you don't have to see some big long string that goes off the side of the slide. But there is this big long digest. If you're wondering where that comes from, the whole registry is a content addressable store. And so a content addressable store means is that if I ever query the actual raw blob here, and I say pipe that output, the actual byte stream output, through a shot to 256 checksum, I get back the same checksum. So basically when you push this to a registry, you just compute the checksum which are pushing up there and you know exactly what that content is. And so as a content addressable store is what that whole thing means. We're gonna see a lot of that today. We've also got the image layers. And so the image layers are the way that we can store all the files we're shipping with image, right? They're shipped as a tar and typically compressed with Gzip, there are other compression algorithms we're looking at. But again, same concept, another content addressable store. So this is all shipped up there. I picked an example from Alpine and you can see when I pipe it through tar you're just getting out from that whole bunch of stuff that points to busybox in case you didn't know, Alpine is based on top of busybox. So then we need to put those two things together. And it's not just one layer, you're gonna have a bunch of layers out there. And when we assemble them together, we assemble them in a manifest, another piece of JSON. And that JSON has a media type in there that says this is a container image manifest. And then it says this is the config digest. And it's not just a digest, there's a little bit of extra metadata around it. So we call that a descriptor. That's the media type, the size of it, the digest that we're doing, maybe a few other bits of data in there as well. And that all gets packed in a little bit of JSON that we call the descriptor. So that's the config descriptor is one of them. And there's a descriptor for each one of the file system layers. Take all this together. This is a bunch of JSON. You shift that up to a registry. This has a digest. This is the digest you would pull if you wanted to have a pinned image. So in case you ever say I want to run Alpine with the shot to check some after it, it's the digest on this JSON right here that you're running, that you're querying. And so the registries know how to work with this. And we can also tag this. So we can say this is version one or something like that. And we will be running things by tag. There's another way that you can push a manifest to a registry, which is to pack multiple manifests together. So we push that one manifest that might have been for AMD64. And if you wanted to push the ARM64, and you want to have them both referenced with the same tag, you will push a manifest list. So you got multiple manifests all together. And again, this is also content addressable as well. So this guy's own digest. You will point your tag to this thing. And then in a runtime, we'll pull it down. And it would say, well, I've got multiple manifests here. I need to query and figure out which one I want to run. And it's going to figure out just by looking for some other data in here. And so in this one, you can see, since we're multi-platform, it has the platform definition for each one of these manifests. So it knows how to pick out the one platform it wants to from it. And it will pull that one down, and it will run that one. So that's how the runtimes use it. But you can put other stuff in here if you really wanted to. And the runtime should hopefully ignore it if it doesn't know what to do with it. So graphically, we just saw a bunch of tags. Let's actually show a picture, because I think this makes more sense to people. We've got the tag out there that's mutable. So you can have the version one tag that keeps pointing to a different manifest every time someone pushes a new version of that thing. And they want to update the original v1 tag when v1.2 comes out. They can change that tag to point to a different manifest. And then we've got the index at the top. That's the manifest list. That's got the list of all the other manifests it's pointing to. And then each one of the platform-specific manifests is in there. And each one of those has to config in the layers that we're running. So this is what the registries are looking at right now. This is what all the little data structures you're pushing up to the registry you're doing. And this is the stuff that we're going to get into the important detail here that I want to stress is that registries care about the blue boxes, those are the manifest. Registries don't care what you throw in the green box. You can throw whatever you want to in there. While we were putting tar files in there, we were putting JSON chunks of code in there, you can put whatever the heck you want in there. And we'll see why that's important a little bit. All right. So we talked about what an image is. Let me do a quick detour on CI CD pipelines, because that's going to be the other big important chunk of this detail here. And so this is how we're going to build them. We're going to build them with a CI pipeline that goes through these normal steps. And we're going to do not just like a build ship run, as best as Docker has their cool slogan. And it's a great slogan. We really do things like linting, and we're doing the compile steps, and unit tests, and other checks in there before we finally build the image. And then we push the image up to a registry, and then hopefully we deploy it. That's kind of the standard CI pipeline, how we're getting this stuff out to production. Except this list is really growing. We've started with this, but I don't know if anybody ever heard the term S-bomb. That might have been floating around a little bit, right? A couple of people have heard that. So we're starting to throw a lot more stuff out there, in addition to just building our image. And so we're generating the S-bomb. We're doing attestations to say, this is all the steps we actually ran our CI pipeline, so that we have some verification of what actually took place there. We're doing vulnerability scanning. We're pushing up all kinds of other data. Maybe someone wants to push cat pictures. And oh, by the way, that's the drinking game for the day, because every time I say something about a cat picture, you're going to get a cat picture. So, and believe me, you can push cat picture up to registry. It's just another blob to it. So the big question is, should we be using a registry, right? That was the title of the talk. Don't use a registry for everything. Stop doing it. And it's not because you're filling my hard drive up with all your images. And I wanted you to stop. That was going to be like a quick five second TED talk. I was going to get to the beginning, but I skipped over that a little. Who knows if that was a funny joke or not. But the question is, should it go to registry? And the answer is, yeah, it should. But the question is, when should it go to the registry? It's my question. And maybe if they accept you in the cat pictures. You don't have to push a cat picture to the registry. But the question of when is because we're doing a CI pipeline, we've got everything in a CI pipeline today is based off of a file system. You build your application. You compile your binary. You run your linting. You generate your unit test report. All this stuff is on like a working directory within your CI pipeline today. And then suddenly you start shipping all the stuff out to a registry. And then you run your next step of generating the S-Bomb. And what does it have to do? Well, to generate the S-Bomb on a container, which it has to go out to the registry and pull those layers back down to scan them and to do all those other checks. And so what your pipeline starts to look like is this. Where we go out there and we generate our image. We push it out to the registry. And then we pull back the same image that we just had built locally. And we try to scan it. And then we go out and we pull the same image locally again so that we can do the S-Bomb generation. And then we push the signature up there. And I don't know if people can look at this and figure out some challenges here, but one is that we're doing a whole lot of network traffic we don't need to do. But think about the gap there between when we first push that image and when it's finally out there signed and ready to be deployed. There's a gap of a lot of potential issues, right? If the vulnerability scan says, no, this is bad, don't run it. We've already shipped it to the registry. And if we're trying to do something like only deploy sign code and we just push this out as the new version one tag, and then five steps later we finally sign it at the end of the pipeline. Well, what happens when someone in the middle of that tries to deploy this out to their production run time and all of a sudden the Ingress controller, or sorry, the Amish controller says, oh no, that's an unsigned image. Let me notify the SOC that we're just trying to run an unsigned image in our production infrastructure. That's not a great CI workflow either, right? So we got some issues. So how can we do this? Well, the CI pipeline is designed to work off a file system. So why don't we use a file system for this stuff? And some of you might be saying, well, great, I already do that. I've got the Docker engine. I'll just do a Docker and Docker build. Maybe not. I think there are a bunch of chain guard people out there that are probably saying use code or some other better tool out there. But if you're doing a Docker and Docker install and you've got the Docker engine there and you try to run that, well, you're only doing a single platform at that point because if you store the image local in the Docker engine, it's only one platform. And then to generate the S-bomb, you're still unloading it to a file system outside of the Docker engine into a tar file and extracting it and doing all this stuff. There are a lot of extra pieces and steps and that's only solved in one case. So let's talk about the better way to do that, which is there is an OCI image layout. So this is the first world premiere of this file system layout and no one else has ever seen this before. Kids, sorry, this has been around for a while, probably about five years now. But hopefully people are thinking, this is awesome, we can use this. And the reason you might be thinking that is when you first see the big chunk of this in here that your eyes probably jumped right to that big S-bomb or sorry, the digest section at the bottom, that's your content addressable store. So we've got a file system format defined by OCI with the content addressable store you can put right in a directory with the same kind of ways that we're querying stuff in a registry a day. So this is this now, it's been defined by OCI, it's been out there forever. And so we've got the content addressable store, we need two other pieces. One, these two other files at the top, the very first file you can see at the top of the screen there is saying this is my OCI layout so this is what you're expecting to see inside this directory. It's kind of like a little key there, it's got a version number, make sure that you know that this is the real thing that we're talking to, we're not confusing it with something else. The other piece there is really important, index.json. In case you're paying attention earlier, we were talking about the manifest list, also OCI calls that an index, same concept, it is a list of manifest. And so we just need a way to query from that list of manifest, the one manifest we're looking for out of that list. And if we do that in this case where we've got a little annotation there, we say this is the tag. And so if you go through this and you have multiple manifests, you can find your v1 tag, your latest tag, your v2 tag, your sbomb tag, whatever it is that's in there, all those can have different entries in this manifest list. And so now we have a way to represent what in registry land we call repository. Same thing you say, grow, grab, alpine, version one, version two, whatever from a place, all those things under the alpine entry, those are all different things in the same repository. And so you've got multiple tags in the same repository. We've got the way to do that on the file system right now with this layout. So I've seen this a fair bit. Bunch of places that you might see this in the wild are with places that are doing air gap environments. So if you're trying to do an air gap deploy between pulling this stuff out from a publicly accessible registry and you want to have this thing deployed into an air gap environment, you might use this format to store it into the file system and then ship it across the air gap and then reinstall it either into your local runtimes or into a registry behind that air gap. So it's a great way to do that. We also see that when we are packaging this stuff up as like a tar or TGZ, it's a great way to stream it if you have like an output from something. So a bunch of these commands out there will stream this as a single file. They just package it up as a tar or a GZip tar. And that's what OCI says, that's the valid way to ship it. The other cool thing about this is that we can use a Docker load command with the same file. It just needs one extra JSON in there, one extra bit of JSON and Docker can load the same file structure in there. It's basically compatible with each other. They're not identical, but they're compatible. So Docker can use that same blob store and you don't have to put the blobs there twice. So it's a nice little slick format. It's got some good advantages. What's gonna happen with our pipeline tomorrow is that we're gonna have basically these steps. Gonna change the order around a little bit. So I'm gonna do my normal get clone, my linting, but instead of doing the build and ship all the way to registry, I'm gonna just output it to an OCI layout and then I'm gonna do stuff like adding my annotations or attestations. I'll do annotations too, they're fun as well. Gonna add my S-bomb into there. I'm gonna do the vulnerability scanning, all this stuff, I'm gonna run on the local file system. And then when I'm all done and I said this looks good, we got check the box, ship that whole thing as one unit up to the registry at the very end. And so now whenever someone pulls it down, they're only gonna pull down the finished product and they're not gonna pull down something that's halfway finished or something that might have a vulnerability in there that you shouldn't have shipped in the first place. And so by doing that, I mentioned the magic word there, cat picture, so there you go, you get another cat picture. But the advantage of this, so let's talk about the feature. Well, I already mentioned that we're never gonna have an unscanned image that's been pushed up there where someone else could run it. And that's nice because when you're in a production environment, if you have somebody that's on one of your nodes in your cluster that's pulling down a blob from a registry, that's normal. Nobody even thinks a second thought of it. So if you've got those blobs up there that potentially have vulnerable data, someone could use that to get their way to get vulnerable vulnerabilities into your cluster that you don't want them to have. So we don't want them running this kind of stuff one way or another. We don't want them able to access it. Yeah, unsigned images, we're not gonna have those out there so you're not gonna be able to pull those by accident. And yeah, the CI system is gonna be a lot more efficient. So the previous way we saw the CI going back and forth over the network, now all this stuff is local on the file system. We keep all the steps local on the file system, and bury in, then we ship it out. So that's the goal, that's what we're trying to do. So you might be saying, this is awesome, sign me up. But let's do this, right? So if you're convinced I got the good news and I got the bad news. So the good news is that there's a lot of toy and sports this today. And so we got stuff out there, run times, pick your run time, container D, pod man. I'm not gonna pick a winner in this fight. They all can import this stuff. It just works. We've got a lot of the build tooling out there. Just about everybody has a way from the build tooling that they can output to this layout. And they can generate this for you. We've got a registry out there that uses this. Shouldn't really have to from a registry standpoint. It's usually out of their view, but there's an interesting example out there called Zot where it will take this as its backend for repository. And so you can query this as if it was a repository. So I'll show you an interesting use case so that can be handy in a bit. We've also got a couple registry clients. I mentioned one of the ones I was working on was Reg Client. There are a couple other competitors out there. From Google we got Crane. From Scopio comes out from the Red Hat side. We're all doing similar stuff. We're just giving you ways to access and work with registry. And so we can query all this stuff and move it back and forth between a registry and an OCI layout and then push it back again. This is how you work with that air gap. These are the kind of tools you use for working at that air gap environment. We've also got vulnerability scanners. So I'll pick on Gripe and Trivie right there where they'll work with the directory itself. They'll scan it. The challenge is they're only gonna scan a single image from there. Not even a multi-platform image. They're only looking for a single image. So it's limited support. Not perfect, but it's better than nothing. The other one that I didn't list there didn't call it their sneak. They're looking for tar file. So it works, not perfect, but it works. From the S-bomb side, both SIFT and Trivie, they'll generate from an OCI layout. So if you point them to one of these directories, they will do their S-bomb scan as long as it's a single image again, just like before. And so you can generate your S-bomb off of this stuff. And then for signing, I'm still gonna, I'm gonna have to work on the cosine team over there. They got some good stuff. I don't think we got support to do this directly from them signing it yet, but that's probably half on me to start working on a PR from them as well. But that was, you know, kind of the bad news. And I didn't even put a cat call out on the previous slide, but I am gonna call it out right now, which is that all this stuff is talking about how we can scan stuff that's in an OCI layout, but all this stuff can also be pushed to the registry as well, or into an OCI layout as well. So the output from that S-bomb. It'd be really cool to start shipping that alongside of our image, so that we're not trying to keep the two in sync from wherever they are, just pushing together. So the image and the S-bomb together being shipped. And then when someone deploys it out later on, they pull it down, they can pull down the S-bomb right next to their image. And so you can do that, because just like you can push any cat picture you want, you can also push any other data you want up to the registry as a blob, and put a little extra metadata around it so you can ship that around. So that's the good. I mentioned it with the bad. So the bad, Phil, we gotta do some work out here. We don't have a standard around this stuff. If you look at what reference is, and if you've never heard that term, if you've ever gotten the error message from Docker that said invalid reference format, it might be a few heads nodding, saying, yeah, I've run into that error before, I've mistyped a Docker run command in the past. That is basically saying this string translates into run this image somewhere. And how we translate that string to that image is not a standard. We have not standardized that in OCI, which would be a great place to standardize it. More importantly, it's kind of a convention that came out of Docker, and it got put in a distribution, and everybody's kind of adopted the same standard. So we all kind of agree that when you say alpine layers, you really mean Docker hub, and to go out and expand all that stuff, and to assume that layers is the default tag. We've kind of gotten that as convention. We're pretty good about that. We got no convention at all, and we referenced this stuff as an OCI layout. Everybody's picking a different example. Scopio went with an OCI colon, and I think they allow you to put a tag on the end of that. When I've looked at a couple of the tools coming out of, oh, who was it out here that I was thinking of? Angkor. Angkor has a couple of tools out there where they call OCI dash dur in the directory name. They don't let you specify which image in there, so you can't put a tag name in there or digest, so that's why we can't do the multi-platform from all the Angkor tools right now. We've also got the OCI dur colon slash slash, that came out of Reg class, so that's me. I was looking at URIs and saying this looks a lot like a URI, so that might be a good syntax for how to do this. And then just last week, I had to update this slide because Docker build kit started adding their own way to reference this stuff with an OCI dash layout and using the same URI syntax in there. So everybody seems to be picking a different way, so we need to probably standardize this. So that's not great. We can probably work with it. You're only building a CI pipeline for one tool, so you can probably work around this. We've also got a working group over in OCI is trying to figure out how you standardize attaching all this stuff to an image, and so you push your V1 image out to a registry, it'll be nice to have a standard way to say this is how you attach the S-bomb to that image. So OCI is working on it now. We've got a working group. We call it the reference type working group. I'd rather call it the refers working group because references is always already an overloaded term. Too late now, we already picked the name. I'm gonna lose that battle. But yeah, we're trying to figure out how we attach stuff to our container image. So just like we do signature stay with cosine, that's one of the several proposals we're looking at, the same kind of syntax that they push the signature up alongside your image. We're looking at, do we use that? Do we do some other formats in there? There are a couple of different competing options. So if you've got an opinion on that, now it's time to come over to that side and give us your thoughts and thumbs up and thumbs down of what you'd like to see. Also, a lot of these tools assume they're generating the OCI layout and that nobody else has touched it before or that they're consuming it and they don't have to do anything with it. They're not writing it. They're either creating it or reading it. They're not modifying it. And if you're ever getting into tooling, it needs to start figuring out how to modify one of these directories and assume that maybe there was already a V1 in there and you're adding a V2 in that same directory or maybe V2 was already there and now you're overwriting that manifest. We're probably gonna have to start worrying about things like garbage collection. That's an extra challenge that registries have already solved and anybody writing this tooling on the OCI layout is gonna have to think, oh, I need to do garbage collection now too. That's an extra problem they didn't think about earlier. So that's the bad. The ugly is that there are a lot of tools that don't even support this yet. And so I call it out cosine just because I really actually like them. So as much as I like them, I'm picking on just because I really wanna see them supported. So I'm gonna see if we can make that happen. But in addition to that, there are a bunch of tools that only support this if it's in a TAR or a TGZ. And so if you're doing a CI pipeline, you don't want to generate this thing and then TAR it up, ship it to a tool that then extracts the TAR into a temporary directory and does all this stuff. You've got the directory. Just read the directory. Why go through this extra bit of steps? So it'd be nice if everything could work with however this thing is packaged and not be picky about the input format there. And in addition to that, the big challenge that I've been pointing out is that a lot of stuff is looking at it, saying, well, I recognize if it's only one image in there, but not if we've got multiple tags, not if we're trying to pick out a single platform from a multi-platform image or heaven forbid, do an S-bomb scan on all the platforms that we just generated. And we're definitely not being able to pick out the digest and the multi-platform. So it'll be nice if they all support a syntax for that and so that would be another nice feature we should probably get out of this. So hope I didn't scare you away. Hope I didn't terrify you because I do at least have a workaround for people who want to try this today. And so if you want to try this today, Zata is a kind of a nice cool tool I mentioned earlier on where if I do my reg cuddle and I'm listing what is in this current OCI layout directory, you can see at the top of this slide here, I've got a latest tag and then a couple other digest tags in there, I call them digest, it's like a digest schema that we're using for this tag to reference an S-bomb and a scan result that are affiliated with this latest image. And if I want to do a cosine sign on this thing, I can spin up a Zoc container and I listen on a local port, so this is on local host, nobody's going to be accessing this from outside. I mount my volume into that, so my OCI layout is now mounted into the demo directory. And now I can do a cosine sign on local host against the demo repo and I'm doing a set sign on this thing. The end result is when I go query that OCI layout now, I've got a signature in there. And so I've created a signature in this OCI layout without having to upgrade cosine itself. So we just spun up a registry on the fly that was able to access this and make it accessible to the tools and don't know how to work with it. So there is a workaround, so never fear. So got plenty of time here, let's show some actual code. Let's show this stuff and I didn't want to actually flip screens around and try to get all this stuff fit on the slide and deal with me scrolling around. So static bits of code here. The initial, this is a GitHub action. So off of GitHub actions, start up, you know, the initial part of this is pretty generic, pretty standard where I'm just doing a checkout. I am doing in here the go install. I'm also pulling down my reg cuddle tooling, my build X and the cosine install. So I got my environment set up. This is nothing new, people have seen this before. But the next step in here, the test, again, nothing new on here. We're doing testing, linting, all the stuff that you should be doing in your CI today, right? This, hopefully I see heads nodding that you're doing this already a day and I'm not like shocking you if you should be linting your code. Good, okay. So then we get to the part that might be a little bit new to people, which is when I do my build from this case I'm using build X, instead of saying output this and push it up to a registry, it's got the option to output it into a OCI layout.tar. And so I'm just outputting this into local directory as a tar file. And I could, there was a trick in there where I could have this generate the annotation. I didn't realize that because it's an undocumented trick. So because it didn't have the label in there saying this is the latest tag, I was using a regcuddle to convert that into an OCI layout. They also put some read-only permission bits on things that I'm not a fan of. And so this kind of handles a couple of those other permissions and things like that. But it also doesn't assume that nobody else has created one of these OCI layout directories. So when you do the regcuddle import if there's already another thing in there and your variant this is also gonna handle the garbage collection and stuff like that. So a couple of features in there beyond just don't want to tar extract. But now we've got the OCI layout directory in our pipeline that we can access, all right? So then I do a couple of tweaks in here. I think I've heard people that have been interested in some of this stuff where I modify the image after I create it. And this is kind of like the branding going off on his own little world here because some of the stuff I'm working on, you can't set annotations on an image from a Docker build or from a Docker file. You can set labels, but you can't set annotations. And anybody that doesn't know that there's 22, the label is in the image config, that config JSON that we saw at the very beginning of this presentation. The annotation is in the manifest. So two different places. And the reason that it is important in those two different places is it's tooling that queries and looks for the annotation. If it's really looking for an annotation it's gonna look on the manifest. And if you haven't set it over there it's not gonna look for it and you're not gonna find it. So this way you can set it inside the manifest itself. The other thing I've been playing around with is how I can make images reproducible. And so I've been backdating my timestamps. So instead of saying all my images are timestamped from the current time right this second, I've backdated to the get commit time. I don't wanna backdate to 1970 because I don't wanna mess up my base image. I don't wanna change all the timestamps and all the stuff because not only am I changing the timestamp from my image history and from my config file there's a timestamp in there. I'm actually going through the file system layers and I'm backdating all the timestamps on the files in the file system. So a couple of tweaks there. This step right here was when I'm starting backdating timestamps is why I really wanted to play around with OCI layout because I really don't wanna go back and forth to a registry changing this stuff. I'd much rather have it all local on the file system before I ship this stuff up. So this is me just having fun. We had plenty of time for me to have fun so I didn't mind throwing this one in. But the one I think other people are really interested in is, okay let's generate the S-bomb. And so there is a way with, I'm picking on the anchor tools here but pick your S-bomb generator. Hopefully they support this stuff. Where they're processing as their input, the OCI dir colon OCI layout. The OCI layout is the directory name. The OCI dir is their little prefix that says this is an OCI layout file. So I do my S-bomb generation. I do my scan. And now all this stuff is happening local on the file system. We haven't gone out to a registry anymore. So now we're speeding up the pipeline. We're getting rid of all that network contention. And then at the very end, I'm starting to throw this up and I'm attaching this to my image now. Earlier you saw me doing a tag listing in there and you saw like the scan and the S-bomb result. Well if I want to attach it, I've got this big long crazy, you know a reg cuddle command there that's doing all this attached command. This is experimental. Don't trust this for a production use case just yet. We're still trying to figure out in OCI what we want it to look like. So this is one of the many proposals of how we could potentially do it. But if you don't use the refers tag there in the end and you actually give it a special tag you can do whatever you want to. That gets into normal just pushing up an artifact into a registry. And I'm capturing the digest here. So the output of this thing, I do grab the digest from that S-bomb when I push it up to the registry. And the reason I do that is when I come down to my scan step at the very bottom of the screen, you can see I push two scans or two signs in my cosine, two signatures in my cosine command. I sign the layers tag should be like a version number or something like that. And I sign the S-bomb digest. So now I've got assigned digest or assigned S-bombs. People can actually trust this S-bomb came from a trusted source, hopefully, now. And to do this, I had to use Zot. And so I spun Zot up in the pipeline. And I've got the Docker run command there. Just like we saw earlier, you just spin up the Docker run, runs the Zot image. I mount my volume in there. And there we go. I can now assign the stuff within my GitHub pipeline. Yeah, yeah, give out actions. This was all on the OCI layout. This is all local on the file system. So now I got to ship this up to a registry because eventually at the end, I actually do love registries despite the fancy title of the talk. And so this is the very end is when I say, okay, take this thing that is in the OCI layout and ship the whole thing, one unit, all the other digest tags, everything all together up to a registry and one big unit. And so this all goes at once. So now anybody that pulls this down sees it. As soon as they see the version one image, they also see the S-bomb. They also see all the signatures. We haven't separated any of those steps from each other. So that is what we've got for this stuff in conclusion. I'll give you a couple takeaways. I'd love to see one, I'd love to see OCI define what a reference is. So hopefully we can get that done. And the reason I put this up top is because if we can do this and define this sort of references, hopefully that can become a standard that all the tooling can start adapting to. And we can just have this as a standard syntax that all the tooling can expect. And then we can make this a little bit more portable across all the tools out there. And so once we get that, I'd love to see OCI layout much better supported across all the tools out there. And so not just able to read a single image out there, a single platform image, support the multi-platform images, support querying different tags from this thing. Because when we start pushing stuff up, when we start pushing our S-bomb up to the OCI layout, there will be multiple tags. And so if we can't query, figure out that we want the latest tag out there, and you see an S-bomb out there, now you can't sign it, that's not a good world. We need to be able to query which tag we want to do this command against. Yeah, and then I'd love to be able to support not only that, but digest multi-platform images. They're big, people have M1s out there, I think. Maybe if you're in the audience right now, if so, your battery life is probably awesome. So yeah, I'd love to see us do that. And then yeah, the working group, if we can get that done. Now we've got a way that we can standardize how we're shipping these S-bombs. So that now whether it's Cyclone DX, whether it's SPDX, pick your format, I'm not gonna pick a winner in that fight. We can ship it in a standard way up to a registry and get this pushed out there so that now someone who's pulling down this image can know that, oh, by the way, I've got this image, but it also has the S-bomb right there. It's also a signed S-bomb. We've standardized all these syntaxes. Hopefully all the tooling can start working together with each other and interoperate. And so we get this pluggable infrastructure. We don't have to say this is our mission controller. Our mission controller only works with this one S-bomb generator. That's not the world we wanna be in. So I'd love to see a lot more portability and plug-and-play in this ecosystem. And so yeah, the end result. Hopefully I'd love to see us get something that's more efficient, more modular, more secure. And then we can get the cat pictures out there, right? So that's the end goal, of course. So you sat through this much. I'm told that if IBM required me to throw this on the slide, they paid me a bunch of money to say this stuff, right? So the image, if you scan the QR code, I don't know. I think a small dinosaur does something funny. I'm not quite sure what that does exactly. So yeah, do that. And of course, lawyers out there saying, you're now legally obligated to visit their booth in the expo hall. I don't argue with lawyers. They tell me it's a requirement. So yeah, don't argue with lawyers. You never win. And if that QR code wasn't enough fun for you to scan, here's another one. The QR code here actually goes to the presentation link up top. So that is where you can download all these slides. The second link in there, the GitHub link, will tell you where that lay, the whole GitHub action that we were talking about earlier that's sitting out there, ignore all the fails in the GitHub action as I was trying to make this demo actually work. There is a successful run at the end. And so you can see this actually does work in a production ecosystem, but this was the minimal viable product because if you really scan closely, you would have seen that I was only building a single platform image. We were working around some of the constraints of the tooling we were working with here. So with that, that is it. Do we have some questions from the audience? I'll open it up. Anybody want to fix a tool? Question from the audience, go for it. So do I see more and more images becoming OCI compliant? And the answer is pretty much yes. If you look at the OCI compliant, I think they actually say that if you're a Docker image, you're still kind of quasi compliant. I think I saw something that a runtime was supposed to support even the Docker images. What I'm seeing though is especially when you start to export stuff to this OCI layout, a lot of the tooling just switches over from generating the Docker schemas, the manifest types and whatnot to generating the OCI manifest types. And so, yes, I absolutely see a lot of that. And from the tooling, we just don't even care. Whether it's OCI or whether it's Docker, it just runs from the runtime. I will say if you're gonna start attaching annotations to your image, you probably want the OCI layout. Not only I'll let you throw an annotation in there, even though you're not supposed to, because the Docker doesn't support that officially on their media types. So once you get into that part, yeah, go OCI. Any others? Yeah, go for it. Yeah, so the question, how far should we take this? Should we be using it just for the Helm charts or should we be putting everything in there? What shouldn't we be putting in there? The question that I've seen a lot from people is, okay, I've generated my S-bomb. And I wanna go ahead and query this. I've got the log4j vulnerability just came out. Tell me all the images out there that have log4j. Well, OCI is not a great interface for querying that. OCI is a great interface for saying, here's a bunch of data, pull it down, and then use it with whatever tooling you need to use it in. And so it's a good packaging format to ship stuff around in. It's not a good format for doing a lot of these query interfaces to say, okay, which one of these things has this one vulnerability or something like that? So I think I look at it much more as the storage and the transport. I don't see it as the query database that you would use for a runtime or something like that. So it's that middle step. So that's the best answer I can give you. If it's something out there, I've seen people put Debian container images. Akahiro is going out there doing all kinds of awesome stuff, throwing things in OCI registries. Go for it. Throw everything in a registry. I love it. Despite the title of the talk. Start from the front and I'll work my way back. Go for it. Yeah, so thinking about all the metadata together and how we understand image with the layouts and the annotations and the attestations, we're not. I think attestations is gonna stick around as external metadata that people are gonna look at just kind of like for auditing. They wanna know that the security team comes through and says make sure you actually did these steps. I think annotations is gonna have a fair bit of that metadata shoved in there. Probably more than some people are comfortable with. But I think a lot of that's gonna go in that path in terms of we're expanding it. And then once you get something that's too big to fit into that little space, we say we don't want you to go more than like 4K in that length of what you're putting in the annotation. I think that's when you're gonna start to see it. It's just gonna go into an external file. That's where the S-bomb goes. The S-bomb says it's gonna be shipped as a blob. It's not gonna be shipped as an annotation. So there's, once you cross a size threshold, you have to switch over. And so I think tooling just needs to understand which side it needs to fall on depending on what kind of stuff it's pushing up to the registry. Yeah, working back. How do I make sure it's not what before the final push? It hadn't been tampered, okay. So everything in those blobs, it's all content addressable. All those digest, if you tamper the contents there, you're changing the digest. So it's the digest of the content of the file so you can't change it without someone being, someone able to see it now. I do intentionally tamper. When I change all those timestamps and stuff, I am changing the digest of those blobs by doing that. So I am myself tampering it. That's me knowing that I did it myself. Scanning, vulnerability scans I would hope. Yeah, okay. Going back still. Yeah, so my philosophy on that one, if you're doing something like an image signing, my philosophy is the person doing the signing should be able to say these are the different names that I expect this to show up with later on because if you're signing it on a staging server, you're gonna have the staging server name in there if you probably don't want that. So I think it would be nice if you were to say, I will sign it and say, I as a signer am saying that this is gonna eventually be known as this official repo out there in the real world and so I'm gonna create that and I'm gonna push that up to my OCI layout and then ship it to the staging and then ship it to somewhere else in production and it's just gonna be the same annotation everywhere. Yeah, so rather than embedding that, just to embed the actual final name. Cool, I think we got time for maybe one more question out here. Last chance, anybody wanna be last? One from the back, go for it. I mean, if you've got a step that's not gonna fit in your CI pipeline just cause it's like a long term thing, you need to do it after the fact, then yeah, you can do it after the fact. I'm not gonna be too picky there, but I'm otherwise you out. It'll be nice if everything that needs to happen within that CI pipeline, especially the stuff that's security critical all happens within that file system that gets shipped as one unit. So I think that's all the time I got time for today. Thanks so much.