 Thank you, everybody, for coming to our talk on a Friday afternoon, last day of KubeCon, for leveraging OCI 1.1 for enhanced S-BOM integration and vulnerability scanning in Haber. I'm here with my co-speaker, Shenmin Yu. My name is Anes Orles. I'm jumping in for Tepe, who can sadly not be here today due to family commitments. Hi, everyone. I have to see you all in this KubeCon session. My name is Shenmin Yu, a software engineer from Boca. I've been in the computing industry for about three years now, and currently I am a maintainer of the Haber project. And my daily work includes release management of the Haber project and Haber OSS and issue-charging PR review of both of them, and Jenkins pipeline maintenance, also bug fixing. And I'm really honored and thrilled to be here, and I can't wait to share with you the topic of leverage OSS-BOM 1.1 for enhanced S-BOM integration and vulnerability scanning. Awesome. So this talk was originally submitted as well by Tepe Fukuda, who's working with me on the open source team at Aqua Security. Now he's the person who created Trivi originally, which will be used as well during this talk. Now I'm an open source developer advocate at Aqua Security and closely working with Tepe and on Trivi. Now for the agenda, for those who are also watching the recording, this is what will be covered in this talk. First, a little bit of an introduction and background on what are S-BOMs, why do we even need S-BOMs? Then we will look at the OCI spec version 1.1, as well as their referral API. We will look at the S-BOM integration in Haber and the Haber pluggable scanner spec version 1.2, as well as how we integrate it in Haber scanner. Trivi integration, we will have a live demo and hopefully lots of time for Q&A. All right, and I'm just curious like how many of you have heard of Haber? Please use your hand. Cool, nice. And how many of you have regularly or occasionally used Haber, I've probably registered for your company or for yourself to serve images? Cool. For those of you who are not familiar with it, Haber is a CNCF graduate project with more than 22,000 starts and around 2.5,000 folks in GitHub. And Haber is also an open source registry that allows users to do with role-based access to poor and poor images. And I'm using it to be the cloud, to be the cloud native repository for Kubernetes. Awesome, who has heard of Trivi before? Who uses Trivi without Haber? Who uses Trivi in Haber? Awesome, so maybe after this talk, more people are gonna use Trivi within Haber. So Trivi is an all in one cloud native open source security scanner. It has lots and lots of different functionality, but in this talk specifically, we are gonna focus on vulnerability scanning and S-bomb generation. Now we've just crossed 20,000 stars about a month ago, so if you use Trivi, we would highly appreciate a star. Same with Haber. As mentioned, Tapper was the one who started. Trivi was originally just a vulnerability scanner for container images. You can find more on the Trivi.dev website. Also, if you want to connect with Tapper, then the Trivi GitHub repository is probably the best place. Now getting started with S-bombs, since that's gonna be a common theme and common component in this talk, what are actually S-bombs? Who is using S-bombs on a day-to-day basis? Okay, a few people. So S-bomb stands for Software Bill of Materials, and it basically allows you to generate an inventory list of a resource. Now that resource could be a container image, it could be a file system, and the S-bomb basically details all of the libraries, all of the components used within that container image. So while the container image has several different layers, the S-bomb exposes all of the components within each layer. Now, usually when we talk about incidents, security issues, we talk about how we can make our services more observable. We don't necessarily talk about what can we do to prepare for security incidents. Now one of the things that organizations can do is generate S-bombs. S-bombs allow you to easily identify what resources you use, so in a case there are new vulnerabilities released, you can easily identify if you are affected by those vulnerabilities. The overall average cost per incident per year increases each year, and organizations have to find new ways to minimize the impact of these incidents. Now, S-bombs are also enforced by, for example, the White House executive order that any organization that works on a national basis has to generate S-bombs for their software. Now, S-bombs can be pushed to OCI registries, similar to how you can push container images, helm charts, signatures for your container image, or attestations. Attestations are basically claims that you make of your resources. All of that can be pushed to container registries, to OCI compliant registries. And basically the OCI framework allows you, OCI specification, allows you to push all of these different artifacts to any registry, so you can interact with all of the registry in a similar way. Now, this is how an S-bomb could look like. This is an SPDX S-bomb. Another popular format is Cyclone DX. You could generate both with Trivi. S-bombs are not supposed to be human readable. You are not supposed to go into the SPDX JSON file and actually read your S-bomb. The S-bomb is supposed to be used in combination with your other resources, such as container images, to understand what's going on within. Now, you could push the S-bomb to other OCI registries. The thing is, if your registry is, uses OCI version 1.0 or lesser, then you cannot correlate resources, meaning you might have lots of S-bombs, lots of container images, signatures, but you can't actually attach them to each other. For that, you will need OCI version 1.1. And OCI version 1.1 has to refer API. The refer API allows you to do this. Basically, you can have, for example, a container image, you can sign the container image, push that also alongside the container image to the registry, attach an attestation, a claim that you make about your container image, you can attach the S-bomb, and so forth. So you basically have a tree of dependencies of your different resources. Now, if you want to learn more about OCI spec version 1.1 and the refer API, here are two really great talks that I highly suggest you to check out. All right, let's move to the section of S-bomb integration in Harbor. Firstly, we will cover one concept called accessory. Then we will talk about two methods to fix the S-bomb files into your artifact in Harbor. One of them is to manually put an S-bomb file using third-party CRR tools, such as 3V and ORAS. And the other way is to automatically generate a S-bomb and associate it to an artifact long image pushed to Harbor. Then we will have one plan feature which is scan a S-bomb for vulnerabilities instead of scanning the artifact itself for vulnerabilities. What is an accessory? An accessory is just like an ordinary OCI artifact but linked to a subject artifact via the subject descriptor. In Harbor, we have an artifact accessory table storing the relationship between the subject artifact and the accessory. The mapping between the accessory and its subject artifact is N2-1, which means multiple accessories can be attached to one single subject artifact and as we can see on the right side of this page. Well, you may have questions like why we need accessories. Accessories extends Harbor's capabilities of storing extra data such as S-bomb and signature information like coser and notation. And it is also the way Harbor incorporates OCI spec 1.1 and the reference API. And how about the life cycle management of accessory? It can be deleted independently in the Harbor portal web UI and it will be garbage collected along with its subject artifact. When a user triggers a replication job, if there is any accessories in the source registry, it will be replicated to the destination registry together with its subject artifact. This page contains the command to generate a N2-bomb file using QV, and it also shows us how to manually associate an S-bomb to an artifact using QV or ORS. This is what it looks like on the Harbor web portal after manually associating an S-bomb file to an artifact. Please note that the type of accessory that are manually associated to the artifact is subject.accessory. And we can also display a tree view of the relationship between the accessories and its subject artifact using ORS CLI. Then like how to automatically generate an S-bomb and associate it to a subject artifact. We can do that go to the project configuration tab and select the automatically generate S-bomb on push checkbox and then click the save button. After that, we can duck or push an image to this project that was configured previously. And we can see there is a S-bomb automatically generated as an accessory to this op-amp image. And please note that the type of this automatically generated S-bomb is Harbor.S-bomb instead of the accessory, subject.accessory. We have one planned feature, which is scanning S-bomb for vulnerabilities. This feature will be designed to be configurable via the system settings. And it scans the S-bomb automatically generated by the Harbor beauty and trivia adapter. So for those S-bombs that are manually attached to artifact in Harbor will not be scanned. If there is no such automatically generated S-bomb present, then it will fall back to the subject to scan the subject artifact itself. By having the feature of scanning S-bomb for vulnerabilities, it will eliminate the procedure of repeatedly analyzing the container images and making the scanning process much more efficient. In addition to that, it also provides us with a comprehensive visibility. Next, let's move to the section of Harbor plugable scanner spec 1.1. Essentially, there are three APIs defined in the Harbor plugable scanner API. The first one is the get method data. And the second one is post scan. And the third one is get the scan report. After finding the request to the metadata API, it will return a JSON string containing the scanner information such as the scanner name, the vendor, and the scanner version. And it also contains the capability info and the properties info we can see in the next slides. On the left-hand side is the probabilities info. We can see that there is a bunch of environment variables returned by the scanners. And on the right-hand side is the capabilities info. Essentially, it tells us what the scanner is capable of. In this case, the scanner is capable of doing vulnerability scanning and S-form generation. These capabilities are feared if to make sure that Harbor and the scanner are on the same page. When posting a request to the scan API, it requires some kind of payload. In this payload, we can specify the registry information, the artifact information, and the values for enabled capabilities. These enabled capabilities are essentially to ask the scanners exactly what to do. In this case, Harbor is asking the scanner to generate an S-form in SPDX format. If everything goes well, then a 202 will be returned as a scan response along with a scan request ID. And this request ID will be used to retrieve the scan report, which we'll see in the next step. In case of anything going wrong, then different error code will be returned based on different scenarios. A scan request ID is required to get a scan report. And we also need to specify the S-form middle type in the query path as a parameter to get an S-form report. The value of accept the header to get a vulnerability report, the value of accept header to get the vulnerability report is different from the one to get the report of the S-form generation. And this is the response to the request of getting an S-form report. And we can see that the S-form data is wrapped into the S-form field. And the middle type field tells us the S-form is in SPDX format. So as you can see, most of the work has actually been done within Harbor as part of the Plugable Scanner spec version 1.2. So Trivy itself can't directly communicate with Harbor. It doesn't know about the Plugable Scanner spec by itself. For that, it needs the Harbor Scanner Trivy integration that we have written in addition. And that basically implements the APIs that you have just seen. So basically, the Harbor Scanner Trivy is implementing the Plugable Scanner spec version 1.2. And that makes it possible for Harbor to basically request from the Harbor Scanner Trivy, hey, what capabilities do you have? What can you actually do for me? And it's basically then able to translate the Harbor Scanning MPI into Trivy commands. So it then is able to serve Trivy functionality to Harbor and say, here's actually what I am able to do. I can generate S-forms for you or I can perform vulnerability scanning. So let's see how that works in action. So I'm here just in a normal Ubuntu VM where I have Harbor installed and running. So we can go to my UI. And here is just my Harbor interface. If you have used Harbor before, this will probably look very familiar to you. Now for those who are new to Harbor, here basically my different projects and projects correspond to, if you're familiar, in Docker Hub with user IDs. So for example, in Docker Hub, my user ID would be a nice release. Versus here, I can set up my project names. So I'm going to set up a new project name, going to call it dev2, set that up, and I can go into my new project. As you can see, there are no resources yet. I haven't pushed anything yet to my dev2 project. Now what I want to do first is, as detailed before on a screenshot, I want to set up automatic S-bomb generation when I push a new artifact. Because that makes my life a lot easier if Harbor just does it for me instead of me having to care whether my container images have S-bomb. Again, if you don't have a registry, if you don't use a registry with the latest OCI spec, then you cannot correlate container images to S-bombs. They will be separate, and you would have to make the connection manually. Now if we have an S-bomb attached to our container image, then a security scanner, such as Trivi, can use the S-bomb instead of the container image for scanning. So it doesn't have to, as mentioned, analyze the separate layers that we have in our container images. Now I'm going to be very boring, and I'm going to copy paste commands because my hands are very shaky when I'm presenting. So here we're going to first pull a new container image. This is just going to pull from Docker Hub. And if I now do Docker image LS, I can see here is my new container image that I just pulled. Now I can go ahead and directly push it to Harbor. I first have to re-tag it to tell it actually now, instead of Docker Hub, where I got you from, you need to use Harbor instead. So we're going to re-tag that. As you can see here, we are going to use the new project that I've just created in Harbor. Once we have re-tagged it, we can go ahead and we can push it to our Harbor instance. Now this should be in Harbor now. So if I go back to my projects, I go to my dev to project, maybe fresh. It's not here. Access repository. I have to look in first. Cool, I didn't do that. OK, Docker log in, right? Is that how you? OK. Oh, yeah, admin. Let's do it again. One second. Live demos, huh? Yeah, now you see my password. Great. One second again, as I said, nervous. Cool, let's push our container image again. Now that we've done that. OK, so here we have the digest 2628. We can refresh. And here we have our container image, our Alpine container image. And within here, it already performed the vulnerability scan, just checking that I'm in the right project. It already performed a vulnerability scan. And here I have the image digest 2628. Why is it done now? No, I did it exactly negative before. One second, let's go back to the configuration. It worked exactly before. We have enabled it, maybe because I locked in again. See, it's not showing this time. We have a little hiccup. So usually, you would see right here your S-bomb attached, which it worked every time we did it before in the live demo. Now, at this time, it doesn't want to work. So let's retry that. Let's we save this. Going back to our project. And then maybe let's just delete it. And we push it again. Good thing we have time. For the new project, maybe? A new project? Let's try to push it first again. Let's go here in it. No, it doesn't want to generate it. OK, let's do it a new project. Three. Yeah, let's go into the new project. Configurations. Automatically generate the S-bomb. Save to our projects. Let's push it to Dev3. Let's go into Dev3. OK, good thing you can also manually push an S-bomb, generate an S-bomb, and push it. So that's what we're going to do next, which I wanted to show you anyways. So we are going to generate a new S-bomb for SPDX JSON for our Alpine image. And we're going to push that to Harvard. So here we have our new Alpine SPDX JSON. Now, as you can see, you wouldn't necessarily read it through the S-bomb yourself. You would push it to your container registry, which we're going to do next. And we do that with the Trivi-Vefura plugin. Now, how Trivi works, we have several different plugins which expand upon the core Trivi functionality. You can generate your own plugins as well. This plugin is specifically written for you to interact with OCI registries and look at all of the, for example, accessories that are in Harvard. You can use the Trivi-Vefura plugin for that. So we're going to put our S-bomb to it. And nothing wants to work this time. Error for our getting image error, getting your S-bomb there. Oh, because I have to retag it. One second, let's put it into our Harvard registry. Okay, successfully pushed the referral. Let's go back to our registry and refresh. Let's look at the app and container image. And now we can see here is the S-bomb that we have manually pushed. Now, obviously, this would be different. This would say Harvard S-bomb if it was automated. I'm not sure while it doesn't work this time. I literally did the live demo three times this morning. So should we try one more time? Or do you want to try? Maybe it's my hand. Maybe I did something. Yeah, we did a preparation on this demo. If today it works perfectly, I'm not sure what happened. And yeah, but in fact, the automatically, the S-bomb generation and those applications should be working fine in the Harvard project. And if it's not in the latest Harvard release yet, it will be available in Hubble 2.11.0, which is our next minor version. So stay tuned. Should we try one more time? We have, oh, well, we are close to time. Yeah, I'm not sure why it doesn't work this time. One more time. Let's delete it. Let's delete our specific container image and go ahead and, not this one, and push it again. Maybe it just takes some time. Maybe it's the conference Wi-Fi voices. Yeah, we can just wait for the, just wait for the, waiting for our release of Harvard 2.11. It should be available there. Awesome, we will publish a working team later on. But ultimately, how it would look like is on the screenshot. You would have, instead of the accessory, you would have harbour.s-bomb written there, so you know it's the automated version. Now, here you can find additional resources, also the discussion to harbour. So if you're interested in the latest development in harbour, get involved there. Give us feedback, give them feedback on the UX and just your use cases as well. And if you have any questions, once you try it out yourself, also reach out in the CNCF Slack channel on harbour. And we also have an aqua security open source Slack where you can reach out to the trivia maintainers. Now, we would highly appreciate your feedback of this talk. We have some time for Q&A. But if you could leave some feedback, that would be amazing as well. Thank you. If you have any questions, there are two mics in the middle of the room on each side. Is there anything in there? I didn't do anything. I did it three runs this morning. So I have a quick question. So I'm, we don't use OCI images yet, but I'm thinking of using them for configuration. So not container images, but just Kubernetes configuration. I'm also not very familiar with Sbombs. Would I use an Sbom for an OCI image where I've pushed just YAML configuration? Would that be useful at all? Or essentially, like, you can first set any files to the harbour artifact using an unclient trivia or this file will be attached to the web saved in the harbour repository as an accessory. Yes. But just to clarify, just for your YAML manifests, you wouldn't generate an Sbom. The Sbom is, for example, for your file system, for your source code to know, as you, for example, ship release, you could tell people here are all the libraries used within. So that would be more the use case for an Sbom. Or, for example, you would attach the Sbom, provide that on your container registry, if your images are used by third parties. So you wouldn't directly, you wouldn't use it for your configuration files. Thank you. So thanks for the presentation. I'm curious whether there's something planned regarding vaccinating or implementing vaccinating for Sbom there on harbour. Because I think yesterday it was a presentation about vaccinating images and getting rid of specific vulnerabilities, for example. Like, you can basically ignore them if they're not relevant for the service that you're building. Something like that planned as well for harbour. Is there plans for VEX documents in harbour? Yeah, VEX documents. For VEX. You know, VEX vulnerability exchange, exploit exchange? Any, actually, like any files or data can be associated, could be associated to artifacting harbour like that and... So for example, if I'm company A and I have, I'm providing the container images with Sbom's on whatever God knows what in my registry and people are using it. It's on the vendors where you're using or on you where you're pulling the container images from. For example, if you pull third party images, you scan them before you push them to your own registry, harbour in that case, you could also pull their VEX documents, put them into harbour in the same way. It's just on you to basically pull and push the documents. So it's for you to then do something with it or for the vendors, your vendors, to provide them to your VEX documents. Yeah, so we are building our own images and then pushing them to harbour to just fetch them. And so we would need to create these documents on your own, but yeah. You could create an issue, for example, for the pluggable scan us back and that it also supports right now. So Trivi supports Sbom and generation and vulnerability scanning, but Trivi itself can use VEX for file systems and container images. So you could put an issue in with a request for that additional support. That would be an option. So you can more automated with Trivi as well. All right, thank you so much. Thank you. Any more questions? Thank you. Thank you all for coming.