 starter. I'd like to thank you, everyone who is joining us today. Welcome to today's CNCF webinar 3D open sources scanner for containers images, just along and run. I'm Paul Cimón, solution engineer, club native leader at Oracle and CNCF ambassador. I will be moderating today's webinar. We would like to welcome our presenter today, Tepe Fukuda, open source engineer at aqua secured and a few housekeeping items before we get started. During the webinar we are not able to talk as an entity and this is in our presentation today is so long so our key way will answer on our blog, right? This is an official webinar of the CNCF and as a subject to the CNCF code of conduct. Please do not add anything to that to the chat or questions that will be in violation of the code of conduct. Basically, please be respectful of all of your fellow participants and presenters. Please also note that the recording and the slides will be posted later today to the CNCF webinar page at www.cncf.io slash webinars. With that they all hand it over to Tepe to kick off today's presentation. Tepe is with you. Okay, thank you for the introduction, Paulo. Hi, everyone. I'm Tepe Fukuda from the open source team in aqua security. Today I'll be talking about TV, which is a vulnerability content, vulnerability scanner for container images. Okay, so let me share my screen. Okay, so today I'll cover these contents. First of all, I'll be speaking a little bit about what's vulnerability, why is a vulnerability scanner necessary, because I think some users are not familiar with vulnerability. So I just have a quick introduction. After that I'll introduce TV and explain basic features, the bounced features, and the new features we recently released. First of all, what is a vulnerability? According to Wikipedia, a vulnerability is a weakness which can be exploited by a threat actor, such as an attacker, to perform an unauthorized action within a computer system. There are a lot of software vulnerabilities such as meltdown, shell shock, heartbreak, and dirty car. Heartbreak affected open SSL, and dirty car affected to the next corner, and so on. You need to know whether these vulnerabilities affect your environment. If these affect you, you have to deal with it immediately. If you don't do this, the attacker can exploit your system. And the most famous vulnerability ID is a CVE ID, which is managed and assigned by MITRE. NVD, National Vulnerability Database, lists all vulnerabilities assigned CVE ID. If you want to know the detail about vulnerability, you can visit NVD and search CVE ID. Now, I'd like to look at an example of heartbreak. This vulnerability affects open SSL and is assigned CVE 2014-160. This affected the latest version at the time. What is important here is that your system will be vulnerable suddenly, even if you have changed nothing. There are two types of vulnerabilities. The first one is known vulnerabilities, which are assigned the vulnerability ID such as CVE ID, I explained. The second one is unknown vulnerabilities, which are not disclosed yet. And there are two types of scanners corresponding to the type of vulnerability. The first one is scanner identifying components with known vulnerabilities, such as 2B, I'm explaining now, and Clare, or Aqua. The second is scanner detecting unknown vulnerabilities. So the vulnerability scanner for web application detects vulnerabilities your application has. Those vulnerabilities are not widely known to the public. And the housing tool also detects unknown vulnerabilities such as buffer, buffer, and DOS, and so on. You can find the vulnerability in open source by using the housing tool and report it. The vulnerability after that becomes known vulnerability at the time. So the tool checks if known vulnerabilities affect your environment. So the target is known vulnerabilities. So we have to prevent a container including vulnerability packages from deploying to our environment. The good solution is to scan a container image stored in an image registry, container registry. So it's called shift left. As shown here, shift left means we should identify security issues at an early stage. So for this purpose, you can use a vulnerability scanner according to Wikipedia. A vulnerability scanner is a computer program designed to assess computers, computer systems, networks, or applications for known weakness. Okay, so how can we identify image vulnerabilities? First, we identify the packages and versions in the image. Next, we cross-reference them with a vulnerability database. It sounds easy, but unfortunately, there are a lot of Linux distributions in the world, such as Retard, Debian, Ubuntu, and also Linux, and so on. So they provide a different share in its system and the package manager. So we need different implementations for each distribution. And in addition, vendor backpots security fixes. For example, OpenSSL in the upstream updated the version to 1.0.1, and it fixes certain vulnerability. But OpenSSL in Retard doesn't use the version as is. They backported the security patch to 1002 EL7. Debian backported it to 1009 U1. So they fixed the same vulnerability with the different versions. So we have to fetch security advisories from each vendor. Okay, so this is today's main topic. I'll talk about Tribi. I developed Tribi as my hobby on a long vacation last year. Eventually, the Acro security came across Tribi and offered me the position within their company. So I decided to take up great opportunity to join this awesome company Acro security. And this is why Tribi and I are here. All right, so let's move on. As I said, Tribi is an open source scanner for container images. It is hosted under the Acro security organization and developed in 2019. So it's been around one year, so it's still young project. It got 4,000 stores on GitHub so far. As you can see in readme, Tribi has six main features. So let me explain then. Okay, so the first one is about type of vulnerabilities Tribi can detect. How does software get into a server? I think there are three types. The first one is the system package manager, such as yarn and apt-get. The second one is the application package manager, such as npm and bundler. The last one is self-installation like installing by make or downloading the binary and put it in the proper path manually. Tribi supports the first and second one. Actually, most scanners support only either one. So this is the advantage of Tribi. So Tribi supports the first one and second one. As for system package manager, apt-yam-apk are supported. Besides, it supports application package manager, such as bundler, composer, ppm, poetry, npm, and so on. So we support many types of package managers. And the supported OSCs listed here. Tribi supports a wide variety of OSCs, including Amazon, Oracle, Susie Linux, and Photon OS. But we don't support Federa and Windows. We have received requests from many users to support Windows. So we are considering it. But for now, we don't support it. And you can install Tribi easily. So when you use Red Hat or CentOS, you can install Tribi with three commands. Let me show the demo. So this is CentOS 7. And at first, you have to configure the Tribi repository. So I already set up. So you have to specify the URL to download to the binary. Okay, after setting up, you update the index. And now you can install Tribi through YAM. Sorry, YAM install. Now the latest version, 0.9, is installing. And then it's finished. So now we are ready to use Tribi. So it's simple. And of course, you install the Tribi through YAM. So you can update Tribi through YAM, like the other packages. Okay, so it's very simple. Going back to the slide. Okay, so if you use Mac OS, just run a new install of a security Tribi through YAM. Of course, it's easy to install on Belian, Ubuntu, and other OSes as well. Also, you can take advantage of the install script. The install script downloads the Tribi binary based on your OS and architecture. So you don't have to care about OS and the architecture you are using. So you can install Tribi with one line regardless of your environment. Okay, so let's try it. This is Alpine 3.11. And so, okay, let me copy one liner. Okay, so this is just a download and install script from GitHub and execute it. Right. Okay, now it's checking GitHub's latest tab. And it found 0.9.0. This is the latest version. And the binary was installed. So now you can use Tribi. Yeah, so it's very simple. So I'll show you this shell script detects Linux and 64 bit. So the script downloads the binary according to the environment. All right. Okay, so the next Tribi is so simple and fast. Take a look. After installation, all you have to do is to specify an image name that you want to scan. Okay, so that's it. Let me show the demo again. So Tribi minus help. So you have the sub command image. Okay, so Tribi image. Okay, this is a usage and the Tribi image and image name. So the option you have to specify at least is only image name. Okay, so for example, our pine 3.10.2 scanning this image. Okay, finished. So Tribi detects some vulnerabilities regarding OpenSSL. And yes, of course, this image is our pine 3.10.2. That's it. So it's so simple to use. All right. Of course, now I scanned our pine 3.10, which is an official image. But of course, you can scan your custom image, like I've got security Tribi or anything you built in your local or you pushed through the registry. Okay, so it usually takes a while to download vulnerability information on the first run. Since it needs to download data from NVV, Red Heart, Debian and so on. So as I explained before, we have to download security advisory from each vendor. Also, some of them don't provide a stable and fast API because it's not supposed to be called frequently. In my experience, it takes about 30 minutes or an hour. But it is not the case for Tribi. When you don't specify any option to be download the full database by default, it finishes within 10 seconds, even on the first run. When you specify minus, minus, right option, it should be download the right database, which doesn't include the details such as description and the references. Because Tribi is supposed to be run on CI-CD. So some user actually needs only CBID and the stability because they can search the CBID on NVV website and knows the detail. So for such a user, I recommend using minus, minus, right option. Actually, it takes a few seconds on the first run. Okay, let's try it. Okay, so this is our API and all the Tribi is installed. All right, so just specify minus, minus, right and our API 2.0. Okay, so now it downloads the database, but as you can see, it's only four or five megabytes. So it takes only a few seconds to download this database. And after that, it takes a few seconds to download the image, Alpine 3.10, but the entire processes has finished within five or, I don't know, but a few seconds. As you can see, so this result doesn't include the description. So just CBID and packaging and stability. But as I said, you can search the detail on the NVV website with this vulnerability ID. So it's pretty fast. All right. And so in a normal way, scanner needs to download data from each source. Unfortunately, they are not stable and slow as I mentioned earlier. And duplicate it. For example, NVV has a description like, this is a denial of service in NGX. Probably, Retard, Runt and other vendors have a similar description for the same vulnerability. But such duplicate information is not necessary for users. So they have duplication and are not sufficient. So why is TV so fast? Because TV doesn't download the data directly. The script on GitHub Action downloads all security advisories from vendor. Then it removes duplication and build database on GitHub Action every 12 hours. Actually, the NVV has all vulnerabilities that CBID is assigned to. It means it includes vulnerabilities are not related to containers such as IoT and iPhone or Android. So this process removes those data as well. After that, GitHub Actions uploads the database to GitHub Release. At last, just to be just download the database from GitHub Release and use it. So this database is optimized and hosted under GitHub, which is stable enough compared to the vendor APIs. Also, it's echo for vendor APIs, since TV calls these APIs only once every 12 hours. In a normal way, the scanner downloads the data through vendor APIs directly. So if there are a lot of users, they call many times. So, vendor APIs. So as you can see, these database files, to be right, db.jzip, uploaded by GitHub Actions. The right database file is 4 or 5 megabytes. And the full database is about 17 megabytes. So it's small enough. And TV uses both db, which is a single file database and embedded key value database. So TV can use this database file after downloading it with any setup like relational database such as MySQL. So you just download the database and load it into the memory. Yeah, that's it. You don't need any setup in advance. Okay, so TV detects vulnerability with a high degree of accuracy. For example, many open source stanners depend on RPind 6BB on GitHub to detect vulnerabilities in RPind Linux. However, the purpose of this database is to make it possible to know what packages have backported fixes. If RPind maintainers don't backport the fix and just update the version, the vulnerability is not listed in this repository. So this database, this repository doesn't have all vulnerabilities regarding RPind Linux. Also, as you can see, this repository is not updated frequently. Four months ago is too old for a vulnerability scanner. In other words, the vulnerabilities disclosed within four months not detected by scanners depending on this database. So as for TV, in addition to RPind 6BB, so we close the RPind Linux package repository as well. This repository is hosted on GitLab and contains APK build files for each and every RPind Linux package. Along with the required purchase and scripts, if any, it includes security fixes, of course. For example, this precast fixes a vulnerability in JSON-C, CVE 2021 to 7.62. This precast was opened and merged five days ago, so it's really fresh. As you can see in GitDev, the difference is the line of package release. Package release was updated from 0 to 1. It means 0.141 fixes this vulnerability. In other words, 0.14.0 or less is vulnerable to this vulnerability. Now, we can detect this vulnerability in RPind Linux because we already have all the information we need to detect vulnerabilities. So if you want to know more about accuracy, you can read a blog comparing Claire, Anker, and Tribit, published by Boxbot. In this blog, Claire and Anker didn't detect any vulnerability for RPind latest, while Tribit did. You may have wondered why there is such a difference before this presentation, but you already know the reason. We use multiple data sources for each programming language for the vulnerability detection of the library, such as NPM and Composer. The important thing is that these databases are not a subset of each other. For example, Friends of PHP. Actually, the security advisor from Friends of PHP has a vulnerability which GitHub advisory database doesn't have. The opposite is also true. In short, using both databases should improve accuracy. So yesterday, we released the version 0.9.0 supporting GitHub advisory database as well. So before, we supported only the one database for each language. This feature was implemented by Masahiro331, who is not a member of Aqua. So I want to thank you for the great contribution here. And Tribit is suitable for WebSecOps. You can use Tribit in CI CD easily. In this example of Travis CI, we have two lines. The second line has exit code 1 and severity critical options. It means it will fail. It will fail as a test if your image has a critical vulnerability. On the other hand, even if your image has a vulnerability with high severity, the first line, the test will pass. But the vulnerabilities will be displayed. So because we specified exit code 0, so even if we find the vulnerability, it will not fail. Of course, you can use it. You can use it with other CI CD tools such as CircleCI and Jenkins and so on. We have GitHub action in GitHub Marketplace, but it's still experimental and doesn't have so many options. On the other hand, Azure also provides GitHub action using Tribit. It's also the free release, but it's easy to use and flexible. To be honest, it is better to use this GitHub action rather than ours for now. But we are working on the official GitHub action, so it should be getting better. So I hope our official GitHub action is better than Azure one. Okay, so we also provide the GitHub integration. We provide a template for GitLab, so all you need to do is include a template and use it. So it's around five or six lines. And you can see the result on the GitLab dashboard. This was added by non-ACA members, so I'm really grateful to them. Thank you for the community. Okay, so Tribit supports multiple formats. At first, Tribit can scan an image in the container registry such as Docker Hub and Harbor. Next, Tribit supports an image in Docker Engine. Usually, the Docker Engine is running as a daemon in your machine. So you can scan your local image. Then you can use a tar file exported by Docker Save. And finally, the Tribit can scan Khanico image and OCI image. Okay, so I'll show the demo about scanning OCI image format. Okay, so in this example, in this demo, I'm using the Scorpio from Red Heart. This is a CLI tool converting the image format. For example, you can download the image from Docker registry and save it to the local Docker Engine. Or you can extract the image from Docker Engine and export it as the OCI image format. Okay, so this time, I copied from, I'm copying from the Docker Hub, Alpine 3.10.2 to the OCI image. Okay, now Scorpio is getting an image from the Docker Hub and export it as the OCI image format. Okay, now we can see the Docker image OCI image format in the local machine. And Tribit can scan this directory directory. So Tribit image and minus, minus input and dot slash Alpine. So if you scan the image in Docker registry or Docker Engine, you don't need the input option. You just specify it like this. But if you want to scan the local file or directory, you have to specify the input option like this. Okay. Okay, that's it. So you can see Alpine 3.10.2 and dot slash Alpine. And also some vulnerabilities are detected. So we can scan OCI image format in the local file system. All right. Okay, going back to the slide. And for more details, visit our readme. You can see the more detail. Okay, I explained the basic features of Tribit. So this section describes the advanced features. The first one is the client server mode. There are client and server. You have to run two processes, but you can use the same binary with client and server subcommands. I'll show you it later. And first of all, a server downloads vulnerability database from GitHub releases. So this database is built on GitHub actions every 12 hours. And this is supposed to be done before scanning an image. And after that, the server downloads the database automatically every 12 hours. Next, a client pulls the layers from our container registry such as Docker Hub. Then the client analyzes the layers and extract OS and package information. And the client sends the information to the server. Actually, this information including the OS and package name and package versions. And the server stores the information in the cache for each layer. At the last, the server responds, the server detects vulnerabilities on the server side based on the information sent from the client. And the server responds vulnerabilities to the client. So I'll show you the demo. At first, you have to run the Tribit server. Now the server is listening at 49.54. Then you go to the client. And actually, Tribit has a subcommand client on the server. So specify to the client and the image name like this. Okay. So it works as a standalone mode. But you can see the log in the server. So the server sets detecting our vulnerabilities. So Tribit client doesn't detect the vulnerabilities. Just sends the information of this image. And on the server side, the server detects vulnerabilities and returns the result to the client. And the client just displays the result got from the server. All right. So, of course, you can specify the port. Listen, 0000 and 10,080. Okay. You can see the server is listening 10,080. And you have to specify the remote option. So HTTP, localhost, 10,080. Of course, you can specify the remote host. So in this case, I'm using the localhost, but you can use a global IP address or private IP address on the different host. Okay. So it's very fast. Okay. So it's also the server sets detecting our vulnerabilities. All right. So, okay. So what is the advantage of the client's server mode? Imagine that there is an image one based on API 3.11. And it has one layer in storing the bash. Let me call the first line layer A and the second line layer B. Let's think about scanning image one by client A. At first, client A through the layer A and B from the container registry. And analyze them. And after that, it sends those information to the server. The server stores them in the cache. Next, there is image two based on image one. It has the same layers as image one. As you can see, the first and the second line are the same as image one from Alpine 3.11 and installed bash. So, but in addition, it has another layer in storing the car, CURL. Let's call this this line layer C. If you want to scan the image two on the client B, what's going to happen? The server already has the information regarding layer A and B. So, the server tells the client that the server already has data about A and B. So, it allows client B to skip downloading layer A and B. It just needs to pull only layer C and analyze it. And at last, client B sends the information to the server. And vulnerabilities will be detected on the server side. Of course, the information of layer A and B are extracted from the cache. So, if you have multiple 3B clients on different hosts, you can reduce network costs and execution time with client server mode because the cache is shared between all clients. It is useful if you run 3B as a ground job or multiple hosts or you have on CI pipeline on your server. In that case, you can run 2B server as a demo and run 2B client in each CI job. All right. Next, the feature is integration with open policy agent. I was supposed to give a presentation and publish this feature in Cubicon Europe 2020 in March. But, you know, it has been postponed to August. So, this feature is not released yet. But let me introduce a little. We are going to provide two integrations, standalone integration and Kubernetes integration. Today, you can feel the vulnerabilities to be detected with severity. For example, you can display only credit card vulnerabilities with the severity option. But you can't apply a complicated rule to the result. For example, in your environment, you can accept DOS in Bash because Bash is not exported to the Internet. Why? You can't accept DOS in NGX because if the NGX is down, your website cannot be accessed. So, this integration allows you to filter vulnerabilities using Regal. It means you can apply a complicated rule. As for Kubernetes integration, let me explain the overview. And for more details, please join my presentation in Cubicon Europe in August. Okay. This is the architecture of the Kubernetes integration. As shown in the figure, you have to deploy Truby Enforcer and Cube Management, developed by Opatin in your Kubernetes cluster. Then, you scan some images by Truby and register the results as custom resources in advance. Also, you need to upload Regal to filter vulnerabilities as ConfigMap. Cube Management, which is ConfigMap, and loads Regal from ConfigMap into OPA automatically. Okay. So, we are ready to deploy a container image. So, you apply Kubernetes YAML, including deployment, demo set, or resources having an image. When applying your YAML, Admission Controller sends Admission Radio to Truby Enforcer. Then, Truby Enforcer gets vulnerabilities from custom resources and asks OPA whether these vulnerabilities can be accepted or not. If OPA doesn't accept those vulnerabilities, according to regular rule, you upload it. For example, the image has critical vulnerability. In that case, Truby rejects deploying the image. So, returns deny to the Admission Controller. So, if you try to deploy a vulnerability image, Truby Enforcer rejects it, and it fails to deploy. Okay. This is our overview of Kubernetes integration. Recently, we released significant new features and talk about them. This is not a feature of Truby, but let me mention it here. Harbor, which is a famous open-source registry, ships Truby as a default scanner. From my details, from version 2.0. From my details, you can read the blog and the CNC webinar from the Harbor team. Also, this is not a feature of Truby, but this week, we released a new project, Starboard. Starboard integrates existing Kubernetes tools, not just from Acura, but also from third-party projects into the Kubernetes experience. Starboard enables results from vulnerability scanners, workload auditors, and configuration benchmark tests to be incorporated into Kubernetes CRDs, custom resource definitions. And from there, access to the Kubernetes API. Of course, Starboard uses Truby internally. Users that are familiar with tube control or with a dashboard to write octant can find security-risk information at their fingertips. And from my details, you can read our blog or read me or Starboard. From version 0.5.3, Truby shows which layers vulnerability comes from. For example, this vulnerability is introduced by this layer. This vulnerability is added in this layer. This vulnerability comes from this layer, like this. So you can know which layer you have to fill. For now, you can't see the visualized result, because Truby is a CLI. And Truby just displays a digest and a diff ID in the result JSON. Not table format. So if you want to know the digest and diff ID, you should export the result as JSON. And a digest is a hash of the compressed layer, while your diff ID is a hash of the uncompressed layer. Docker engine stores uncompressed layer. So if you don't see a digest in the result JSON, if you scan your local image, you see the digest as well when scanning an image in the container registry, because the compressed layer is stored in the container registry. At the beginning, I said Truby is an open source scanner for container images, but it is wrong now. In the latest version, Truby became open source scanner for artifacts. Okay, let me explain. From version 0.9.0, we released yesterday, Truby can scan file systems, such as host machine, virtual machine, or an unpacked container image file system. If you specify the path to your project with file system or FS subcommon, Truby will look for vulnerabilities based on log files, such as Genfile log and package log JSON. Okay, I'll show the demo. Okay, so now we have Truby, and Truby has subcommon file system or FS. And now I have the project in this directory. This project includes PIP file log. So this is written in Python. Okay, scan this project with FS subcommon. Okay, that's it. And one second. I don't know what's happening now. It doesn't work. I don't know why. So I have a video in such a case. Maybe it's a bit small, but I hope it works. Okay, this is a project directory, including PIP file log, and scan the Truby file system with the directory. Okay, so you can see the vulnerabilities. And of course, you can filter the vulnerabilities by stability like this. So in this case, the PIP file log has two high vulnerabilities. Okay, so now Truby can scan the file system. And also, Truby can scan your container from inside the container. All you need to do is log into a container and run Truby with FS slash. In the previous example, we specified the path to the project, but in this example, we specified slash. This is the only difference. If you want to scan an entire container, you specify slash. If you want to scan your project only, you specify the path to your project. Okay, so I'll try the demo. Hopefully it works this time. Okay, so this is inside the container, R-point 3.10. Okay, so to the FS and slash. Okay, it works. So as you can see, this is R-point 3.10. And the host name is 619, blah, blah. Yes, this image. So you can scan the container from inside the container. Okay. And next, as mentioned earlier, you can scan not only a container, but also a host machine and virtual machine. Note that Truby traverses all files on the root directory if you specify slash. When you have a lot of files in your machine, it takes a few minutes or more, so be careful. And in addition, you can scan your image as part of the build process by embedding Truby in the Docker file. Maybe it's easy to understand to see the demo. Okay. So I have Docker file in this directory. Okay, this image is based on R-point 3.7. At first, we install the dependency and install Truby and scan with slash. And also we specified exit code 1. So if this image has vulnerability, this line will fail. At the same time, of course, the Docker build also fails. Okay, let's try. So I didn't aggregate these instructions because I wanted to use the cache for the shortcut. But in the production environment, you should aggregate instructions to reduce the image size. And also you should remove the cache of Truby and remove the Truby binary after scanning for the image size. Okay, it's a build. Okay, it's finished. So as you can see, this line failed. And this took a build failed with exit code 1. So because this image has one vulnerability. So you can scan the image in the Docker file. Okay. So this approach can be used to update Docker files currently using Accra's micro scanner. Okay. And at the last, Truby scans a remote bit repository with repository or repo subcomment specifying repository URL on the repository into your local machine and scan it. Okay, I also showed a demo. Okay, Truby has subcomment repository or repo. So repo help. So the option when you have to specify at least is repository URL you want to scan. Maybe repo, gps, github.com, Accra security, CI test. So this repository includes multiple log files. And also they have some vulnerabilities. So you can scan a remote get repository by repo subcomment. I hope it works. But okay, so maybe something wrong. So I recorded a video just in case. Sorry, I didn't record this video. I don't know why it doesn't work, but sorry. So anyway, this repository subcomment downloads github repository into your local machine and scan it. So now it supports public repository. For example, you are considering using a certain tool, but you want to know this tool has a critical vulnerability or not before using it. This feature is useful in that case. Okay. If you have any questions, please feel free to reach out to us via github issues, Twitter, email, CNCF Slack. We don't have a channel in CNCF Slack, but our members are there. Liz Lies and me and Sima. So you can send us a direct message. Okay, that's it. Thank you for your attention. And hopefully my presentation is useful for Summer. And I hope that everything is going well to you in this situation. Thanks. Thank you. Thank you for a great presentation. I learned a lot with you. And I will see you again because I should learn much more. A very large and great presentation. Thank you so much. Hello, everyone. As I said, we don't have time enough to QA, live QA, but the question will be answered on our blog, right? Thank you, everyone, for this moment, for joining us today. The webinar recording and the slides will be available later today. We are looking forward to see you at future CNCF webinar. Have a great day. Thank you so much.