 Okay. Thank you, everyone, for joining us. Welcome to today's CNCF live webinar, Secure Software Factory, adding SBOM and code signing to your security checks. I'm Libby Schultz and I'll be moderating today's webinar. I'm going to read our code of conduct and then hand over to Ariel Schupper, Engineering Product Manager at Cisco, and Shai Simbom, Senior Developer at Cisco. A few housekeeping items before we get started. During the webinar, you are not able to speak as an attendee, but there is a chat box on the right hand sidebar of your screen. Please feel free to drop all your questions for Ariel and Shai there. We'll get to as many as we can at the end. This is an official webinar of the CNCF and as such is subject to the CNCF code of conduct. Please do not add anything to the chat or questions that would be in violation of that code of conduct. And basically, please be respectful of all of your fellow participants and presenters. Please also note that the recording and slides will be posted later today to the CNCF online programs page at community.cncf.io under online programs. They are also available via your registration link and the recording will be available on our online programs YouTube playlist. With that, I will hand things over to Ariel and Shai to kick off today's presentation. Thank you Libby. Thank you very much. Welcome everyone to our webinar. And we won't talk about, of course, the hot topic of security software factory. So we'll talk about some things on supply chain security, where it started, the big things around it, always lately. We'll talk about the different where to use them. And then we'll put a special focus on some open source tools that are dedicated for the runtime aspect of the supply chain security. But let's first start with talking about like what it is or why we are all concerned from software supply chain recently. So I think, you know, software supply chain security was always an issue. It was always kind of about something that people can temper. It was like a rumor that things can go wrong and things can change. And in Docker Hub, there were many kind of malicious things, you know, container images. But I think the turning point was towards the end of 2020 when the SolarWind attack, you know, took place. It was a massive attack that, you know, started somewhere in 2018. And it took almost 15 months just to discover this attack. And SolarWinds is a software manufacturer that was breached and malicious actors manipulated the software update that it produces. They added malicious code into the update while the audit was built. So when the software was delivered to all the 18,000 SolarWind customer, it was signed and verified. But no one even considered the fact that SolarWind was breaching it such a way that someone, you know, inserted malicious code and consequently all the 18,000 customers, you know, SolarWind customers were infected and got hit by this attack, which was, you know, super sophisticated. And that's why it took like almost 15 months to discover this special command and control protocol and mechanism was really impressive. But nevertheless, it was the most, the first most significant supply chain attack, which had a huge monetary consequences for SolarWinds and all the other, you know, custom-rated venues in their system. Now, I'm calling it a turning point, but if you look at the timelines and you look about between 2015 and 2020, you know, almost, you know, five years or six years of small amount of attacks. And since 2020, and we are now almost towards end of 2022, but still we have a good, you know, good, you know, third of a year to go. You see the amount of attacks is just, you know, really increasing significantly. So yes, software becoming a target. It's easy to target it. Some software really use, I mean, codecov, you can see here in the middle, was another famous attack that had a huge impact on a lot of customers. And the understanding that it's much easier to penetrate users through the software they consume became a noticeable fact that, you know, today use one of the facto popular risk and attack that need to be considered. Now, there's more attack, there's more risk. Obviously, there's also some more standards and regulation that which are emerging. So I think the most notable one was the presidential order that anyone who wants to sell to the US government needs to stand or provide proof to the software supply chain that it produce. But much beyond that, there is Salsa, which is an open SSF project that aiming to provide secure software, secure open source software. Denise generated their code of conduct and their recommendation on how to secure software supply chain, Microsoft, its initiative, and even the CNCF as a reference architecture and a diagram of the best practices to secure software chain. So definitely there's a hot topic, a lot of, you know, both regulations, standards, you know, even just recommendation which with no, you know, superpower, but again, there's a proliferation of recommendation and document that trying to address this aspect. So let's talk about what it is and what do we secure. So Salsa supply chain process, you know, that cover your, let's cover your code development. It starts when developer code, when the code is built, and of course when the code is running in anything, you can temper and you can modify it your malicious code. Now, as you can see from this diagram, all those like small icons of hackers. Ariel, I'm sorry, I'm going to break in real quick. I don't think you're sharing your screen just yet. Oh, thank you, Libby. I'm not seeing that slide. Sorry, I was just thinking, I don't think he's still intro-ing. So there we go. Thank you Libby. You're welcome. All right, so thanks for this note. So, yep, let's talk about, those are the amount of attacks which I mentioned, the standards, different standards, and let's talk about, you know, the supply. So thank you Libby, and I apologize for stop sharing just before the beginning of the show. I didn't mind it. I'm keep sharing my screen. I'm sorry for that. So when you look about, you know, the supply chain, you look about, you know, where you can hack the code to, obviously you can see those small icons of hackers, you know, with the hoodie. It should be also black, but never mind. You can see it usually being, you know, being sent to build a system, in the build system, with the dependencies, when it's sent to, you know, to the, to the different registry before it's being pushed to deploy, even post deployment, those are the places where, you know, the code can be changed and modified. So through the entire lifecycle of the code from the time that the developer wrote the code, and all the way until the time that the code is running, those are the areas, the potential locations where people, where malicious actors can intercept the code, modify, change, and add their malicious packages. Now, the biggest focus area is, you know, on the build process, because this is probably where you generate the most amount of server-specific copies. And the part for which focusing on, you know, how you're making sure that you're getting the build provenance, and how do you do, you know, maintain version control and authentication of the build, and how every step can be auditable and the integrity of the different steps, and probably the most tough recommendation is how you're making sure that the build is hermetic and it's reproducible, so if you can run into a parallel build system, they will reproduce the same exact, you know, artifacts. So all those aspects are targeting the build environment and making sure that the build process by itself is secure and is not tempered, which makes perfect sense. But we need to understand that just securing the build is not enough, right? And while you can argue and say that when developer is, you know, writing his code, it's probably inside the organization, whether it's, you know, inside the ground connected to the network, it's probably when the code is running, it also can be tempered, and there is also a plate that can be changed and need to be taken into consideration. And the purpose of this talk today is to really focus on these aspects, on how do we increase the security level, the supply chain security level of our runtime environment. So when you talk about, you know, building blocks in software supply chain, so one of the first item is the software build of material, right, or what's known as S-Bomb. And the reason why the S-Bomb generates so much interest and a lot of, I probably heard it a lot and in many places is because it offers some transparency level into the executable that you are running. So if I'm getting an executable and I'm running it and I have no idea what's coming in, if I get the software build of material and I see all different, you know, libraries and dependencies and everything which the code is built from, I get better visibility, I get better transparency into what I'm using, which of course is super important when there are some, you know, critical vulnerabilities, but even without it, it will have some, it's not just like food. I want to know what exactly the food that you are eating is made out of. The same thing it relates to the software which you are using. I want to know what is the nutrition fact table of this software, this is what S-Bomb can provide and why is it good because it gives you some indication on the maturity level of the security program of the software, the software state, if it's maintained, if it's usable, if it's supported, if it's outdated or not, it also gives you very important information about vulnerabilities level. So vulnerabilities are based on packages and if you get the full list of the packages you can see which one of them is vulnerable and vulnerable to what and also from compliance perspective, there's information about the licenses, the open source which you can get yourself or at least get the compliance fulfilled with understanding if all the packages you are using are compliant with the organization policies. Now as well as different formats, the S-B-D-X and the Cyclone-D-X, okay, the S-B-D-X is pushed by the Linux foundation, the Cyclone-D-X by the OSP. While the S-B-D-X is more focused on licenses view and considering what licenses are being used, the Cyclone-D-X in my opinion is much more impressive. It's security oriented so there's a lot of good security metadata in the S-Bomb that produces with the Cyclone-D-X format. So if you're looking about the CVEs and exploitability and you know, remediation, it's really very useful information. It also supports all the advanced programming languages, you know it's extensible, you can extend it, it's tertiary proof, it's really at least in my opinion it's a very impressive format or provide that good useful information. Now generating S-Bomb is typically something you do in your CI, okay, because this is where you produce your software, but generating S-Bomb of your runtime environment has some significant advantages. I think probably all the audience recall the look for J-Fire drill and if you really want to get an accurate snapshot of all the effective CVEs, the wear in your environment, you have, you know, critical items like look for J, then it's really good to know what's running in your environment. Also exploitation insights, right, so look for J-Fire, everybody was you know rushing to replace it, but there were many places where the look for J-Fire is not even exploitable. And then with replacing and bringing down those services, so it's good to understand and to see the context of where does it run, where you have it, and also if you want to get again a good compliance to the policy of the open source policy that in your organization it's good to know what is running in your environment. In the cube clarity, this like beautiful icon you know on the top is an open source tool that Cisco donated and contributed to the to the community and generate dynamic S-Bomb in Kubernetes clusters. Let's take a look just a minute. Let's take a look on the other side. Okay, so this is the cube clarity dashboard. I run it in my Kubernetes cluster, so I have a simple cluster running on GKE. I deployed a few containers as a deployment. I used the Sock Shop demo application in order to get and I'm just running this you know this dashboard which is connected to my cluster and doing port forwarding for my cluster into this local host. In order to start seeing everything in the dashboard you need to start scan, so you can schedule the scan, you can select the namespace that you want to what names in the cluster. You can see all the list of the namespace that you have in the cluster and you can select which of them. I select the default namespace in the Sock Shop. A name says you can decide if you want docker status benchmark or not, alter it off because now part of the S-Bomb and you can decide if you want to do it to what or no. You click save, you can and then you can start scanning and then immediately it will start scanning a scanner environment and you can filter everything based on those scans. Now once we have this scanning I already ran this scan before, you can see it by the way it's a very fast scanner just you know already done before, you get this the dashboard is start populating, you're getting to see you know all the the cumulative number of vulnerabilities, how much of them has a fix. You can see different packages per license, so you can I can show you what how many GPI licenses I have, MIT licenses or if I want something else. I can see the package breakdown based on programming languages, I can see how many applications I have, resources, packages and here I can start playing with applications, I can start playing with the top one of all resources or even the top one of all packages that I have and it's sorted out by the severity, not by the code number but by how many you know based on the severity of the vulnerabilities then I can start going to the nice thing I can check start searching for vulnerability you know based on the package it exists or vice versa, I can look at the different packages that are there and try to find you know specific, you can see who's using those packages and then I can again check all the different resources and I'm there being used so it's very useful for me to get a good visibility on whether it's licenses or vulnerabilities per package, where exactly this package is, how many are you using it so it's very useful because if I'm using like a base image which is standardized I want to do it once and I want to deploy it anywhere and this is where you get this great great visibility so this is cube clarity, it's part of it's available on github in the open clarity repo where she's called is you know convening all the open source to the cloud 80 open source tool and we'll be happy for people to start using it more and more so this is the cube clarity and this allows you to address you know sbom aspects in runtime but let's talk about something else so sbom is important but it's not enough we also want to make sure there is a code integrity, what does it mean code integrity so code integrity is of course we want to make sure that the code isn't tempered isn't modified isn't change one it was built right and it's not only in the build phase but also beyond because code can temper it I can act if I have access to your registry or if I have access to your where you keep where you store your your container even if I have access to your cluster I can temper or change doesn't have to be Kubernetes can be also into your cloud account it can be into virtual machines it can be to your service functions and the way the code signing works is you know you you sign the code when you produce it with you know a cryptographic signature then part of it is you know store the private key or the private signature is stored with the code and it's pushed to the registry together with the code so kind of a metadata that's pushed with the code and then I have the public key which is then used to validate so in the validation phase I'm matching the part in the public and I can see if it matches or if the code is was tempered and changed now one of the more interesting things that I think today is existing is the six store six store is an open source security framework that creates a new standard for signing verifying it really it's an open source but it can use it and truly aim to increase the security level of I know software production so it's part of the open ssf the open source security by the Linux foundation and it's really a great contribution security of software in general now one of the the the interesting thing they introduce is a unique structure for keyless signing so one of the challenges in keyless in keys or the classical code signing is that you have long lasting keys they can get discovered and you can then you know temper them but when using keyless this is much safer approach and then it's primarily focused on container images and they can sign them you know with skill assigning and then you can use any admission controller so six or it's only mission controller but you can use any you know OPA or key verinobu also mission controllers then they can apply that it only signed and verified images are being deployed in the cluster which again is is a great tool that provide a higher level for the you know the codes which are running in the cluster but here we want to discuss not just containers we also want to discuss serverless functions and while six stories really focusing on containers serverless functions is a larger security challenge because unlike containers in serverless function there is no sha there's no hash that can be used in order to verify and even if you try to do it you know different deployment frameworks has different way to calculate it and different things they include in the in the zip file so it's really hard to get a unified standard just like we have a sha for an image and even if there was like a unified hash there's no validation so we're still missing the validation option now i'm saying still talk about most of the clouds AWS has their code sign which you know is a great service code sign allow you to sign functions so once this function is uploaded you can use the function you can sign it you can add a signing profile to tell you what to do if something doesn't match and then you can select which lambda will get which profile and then before the lambda is executed code sign to verify the function hash that was not changed so this is great but it's only post deployment remember we want to make sure that the entire chain is covered and it's slightly cumbersome or slightly hard to configure and ideally we would like to have the same concept as six store signing also enabled for serverless functions so for that we created function clarity and again it's an open source tool that sees the community for serverless functions it's extending the six store concept also to serve as functions so it allows users to sign the functions which is great i mean you can do it without it but it you know at the missing point of the validation functions you can really validate those function in any cloud environment and you can get it making sure that only function which were not tempered are being used in the cloud moment now how does it work so when you write your service function and you're doing after doing your testing before you make the deployment we are inserting a step into your ci pipeline and then this i is using cosine in order to sign the image you can use it with key pair but you can use it with keyless so you can create identity get an entity from full ceo use this identity you know upload the entity to the to the record or if you're using a key pair then you can you know upload the public key to the cloud account and then in every cloud account when you want to install cube clarity there is like the infrastructure that first we need to listen to the events we get the notification about an update or change or a new function was created which trigger a dedicated validating lambda so this lambda this lambda either check versus record or if you're using public key get the public key and verify it with the private key of the function and then allow you to perform actions so if you need to decide if you want to get alert if you want to get locked if you want to get notified uh or on the consequent action that post the validation so the function is what it can sign and it's correct thumbs up you can move ahead uh if the if the function um is not or it's tempered uh then you know you can decide what you want to do with it so this is function clarity it's going to be released soon and available in our repo and i'll be i'm happy to invite chai to the demo so i'll stop sharing uh in chai please remember to share your screen hey yes i will share my screen in a minute and so let me know when you see my screen not just yet right again that's strange so it states that the that we can't access your screen check your system settings and then restart your browser ah let's see try to refresh the right everybody hang tight while he refreshes that browser and are there any quick questions we could maybe answer right now while we're waiting aerial or anyone in the audience have anything we could chat about while we're waiting all right you are still muted i'll try to refresh my browser part part of the part of the solutions here so it didn't work all right while you're working on that aerial we do have a question what should be the starting point of implementing sbom is a part of our release maybe we can chat about that while we're waiting on chai's browser we're getting some audience answers we're gonna bear with us just a moment all right here we go okay now let me know if there it is okay that's great so hi everyone so my name is chai and i'm from the etni group in sisco where i really is our product manager so what i'm going to show here in this demo is i'm going to first of all the the project is not yet released so we are going to be published in the in several weeks and what i'm going to show is how you can sign the piece of code in your in your own environment and then you after you sign it you what we actually do is we generate an identity for this code and then we sign the identity and we upload the certificate or the signature depending off it's a keyless solution or if you decided to use keeper so yes we upload the signed code into your cloud account at the moment we support the aws so this will be also the context of the demo and i will show how i deploy two functions one on top of the signed code and another one which is not signed and then i will show how the the the function clarity is triggered and then it realizes that the sign code is okay but the function with the unsigned code is was not verified by function clarity and in this case we have several options i will show how we block the function and we also send the notification in aws context we send it to an sns queue and i did some cool integration where it will send me an email that will notify me that an unrecognized function tried to be created and it was blocked so let's start the demo so first thing i'm going to do is i'm going to sign a piece of code in my in my computer so what happens in keyless is that you are authenticating yourself using one of these three options this this is the way you claim to be who you are and once you do that we use the certificate that was produced from the six store keyless project and we sign the code and this is the certificate who signed the code and it is registered somewhere at record and full show so now that i have signed the code i'm going to create two functions one with the signed code and the the other one with the code which is not signed and after i do that now the functions are created at aws so this is a process that can take maybe a minute or two where the we have in aws the solution is to use cloud trail that sends all the events to cloud watch and we send those cloud those log events into our own verifier lambda and once our lambda is triggered then it will process the the function that was created and it will check if the function is is verified it will take the function it will download its code in this case and it will sort of reverse engineer create the identity and then will seek for a appropriate certificate or a signature that the code was signed with and if it will find it it will try to open of course the signature using the the signature or the certificate and if it's successful then we are okay and if not then several options here so in a minute we'll see if the if the if the events were already picked up by our lambda and then we can see the results so in case that the the the code is verified we will for example add a tag to the function that states that if the function was verified by function clarity let's wait another minute and in case it was not signed by us then we will see that in this case i chose to block the function we'll see how the function is blocked from running meaning in the in case of AWS it's throttling the function here as you can see the function is throttled meaning we reduce its concurrency to zero and you can see that now i received an event that stating that the function clarity alert we failed to verify this lambda and this is the action we took in this case so this is for in this case and let's see that the one that is verified yeah we see that just we just tagged it and said okay this function was verified by function clarity and it's okay it's more informative and now i can also do some cool stuff for example if in this in the function that is signed if you try for example to just change in the function at runtime then an update function event should be triggered and then again it would maybe take a minute or so and then you can see how this function now that we got the update function event we do this process again we try to understand whether this code is signed by function clarity and if not we will want to in this case since the user chose to block the function we will block this one as well so let's wait a minute or so and if someone has questions in the meanwhile then feel free to ask so i see a question here that states uh how can i get notified if an unsigned code is deployed so how can you be notified so we have several options of a post verification actions so one of the we have at the moment detect which means let's tag the function we have a block that states let's block the function and besides that we have the options to send a notification to a predefined queue and from that point on the user can decide whatever he wants to do we can integrate with this queue and do whatever he wants in the case of for example un verified function i hope this answers the question so here you can see that now the function that was signed the event was picked up and now where the function was stopped and we had a relevant tag as well that function is not signed and you can see here that i got of course another event now that the function is is now blocked as well since someone changed the code without signing it first and i see that another question if i'm showing is does it work for images as well so yes the the new the new thing we brought in this project is the code signature but we also support image signature which hernes a six store cosine and we use it to sign images as well in case of image signing the the signature let's say or the the code the the signature of the signed image is realized in the repository itself where the image exists so this is how six to work when signing images so if there aren't any more questions then i'm done from my side are you do you want to continue okay are we what's next are we waiting on another presentation or should we go into questions shy what do you think i'm done with my demo of the function clarity and code signing uh i guess that may be a real knows if he has some more topics to talk about i'm sure can you hear me i'm trying to talk can you see can you hear me or you cannot yes no we can yes we can hear you now perfect now i was trying to talk so i'm saying that's the end of our webinar so i have no more slides we'll be happy to go for questions so if i just want to summarize i'll just try to summarize what you know what we saw so we talk about you know generating runtime as one shy also did us a demo of how you create signatures are you making sure your code integrity is not tempered and we show it on serverless which is a very special use cases so if anyone in the audience has questions we'll be happy to answer it we'll waste three minutes Libby anything from your side nothing for mine have we worked through it looks like we have worked through all the questions in the chat does anyone have anything else to add any other questions see if anybody pops anything else in do y'all have any links or anything you want me to add into the chat or do you want to add into the chat for folks to follow up or keep it keep up with what you're working on yes absolutely so definitely i'd love to share here the open clarity so follow our ripple function clarity cube clarity is there function clarity is going to be there shortly and we of course would love to get users to address it you know hear about it hear this you also shared our open source project would be great okay let's see here is a question to address to the audience does anyone have a requirement to expose their S-bomb let me guess anyone in the audience feel free to respond to that in the chat and speakers if you have anything to add into that as well you're welcome to all right if we don't have anything else and no more questions we can wrap a little bit early but Ariel and shy do y'all have anything else any parting words you want to leave everyone with um no i think you know we covered it thank you Libby thank you shy we do want to make sure that people are keeping their environment secure and just you know using open source free simple tools that can keep them much more secure than before that's all from my side anything from your side shy and no i have nothing to add all right well everyone thank you so much for joining us for another live webinar thank you to Ariel and shy for their time and presentation really appreciate it well like i said this will all be up on the website shortly and be sure to look for another webinar or online program from cncf this week and thank you everyone for joining us so much