 The 2019 SolarWinds hack represents a new threat milestone in the technology industry. The hackers, they patiently waited and evolved their intrusion over several years, literally. They lived in stealth. They tested and they retested their techniques and used very sophisticated methods to get into email systems, networks, authentication systems, and numerous points in the software supply chain to replicate the malicious code at massive scale. They would use techniques like inserting malware and then they would steal data and then they would remove the code before it was discovered. And they used many other advanced approaches to cover their tracks. The really scary thing about this breach is, you know, people often think, oh, well, I'm good. Thankfully, I don't use SolarWinds. But it's not true. You're not safe. The domino effect of this hack, it's created a massive, massive concerns throughout the industry. We actually, to this day, we don't know the true scope of this attack and we don't even know who was impacted. We may never know. So connecting all the dots on this breach is extremely difficult. Moreover, new threats like those exposed in the recent log 4J vulnerability, they seem to hit the news like weekly. And they further underscore the risks that organizations face, not just large companies by the way, small businesses, mid-sized organizations, and individuals. Hello, my name is Dave Vellante and welcome to theCUBE's special look at managing risk in the digital supply chain made possible by Red Hat. Today, we're going to hear from some of the top experts that will help you better understand how to think about the exposures in the software supply chain, some of the steps that we can all take to reduce our risks, and how an endless game of escalation is likely going to play out over the next decade. Up next is our first segment hosted by Dave Nicholson of theCUBE. He's with Luke Hines and Vincent Dana of Red Hat. They're going to talk about where the greatest threats exist and how to think about open source versus other commercial software and discuss ways that organizations can reduce their risk going forward. Let's get started. Welcome to theCUBE. I'm Dave Nicholson and this is part of a continuing conversation about managing risk in the digital supply chain. I have with me today Vincent Dana, Vice President of Product Security from Red Hat and Luke Hines, Security Engineering Lead from the Office of the CTO at Red Hat. Gentlemen, welcome to theCUBE. Thank you. Great to be here. So let's just start out and dive right into this. Vincent, what is the software or digital supply chain? What are we talking about? Yeah, it's a good question. Software supply chain is basically the software that an end user would get from a vendor or in our case we're talking about open source, so upstream. It is the software that comes in that is part of your package operating system, applications. It could be something that you get from one vendor, multiple vendors. So we look at, in the example of Red Hat, we are one part of a customer's software supply chain. So it's interesting that it's coming in from different areas. Do we have a sense for the ratio of kind of commercial software versus open source software that makes up an enterprise today? I think that's a really hard thing to answer and I think every enterprise or every company would have it a little bit different. Depends if you have an open source vendor that you choose, you may get a significant amount of software from them. Certainly you're not going to get it all, right? For example, Red Hat provides thousands of open source packages. We certainly can't provide all of them. There are millions that are out there. So when you're looking at a specific application that you're building, chances are you could be running that on a managed platform or an enterprise supply platform. But there are going to be packages that you're going to be obtaining from other sources in other communities as well in order to power your applications. So, Luke, that sounds like kind of a vague situation we're looking at in terms of where all of our software is coming from. So what do we need to know about our software supply chain in that context? What do we need to understand before we even get anywhere near the idea of securing it? What are some of the issues that arise from that? Yes, so as Vincent touched upon, it's a very wide range in ecosystem. Multiple sources, when we're talking about open source. So essentially, awareness is key, really. I think a lot of people are really not aware of the sources that they're drawing from to create their own supply chain. So there's multiple supply chains. You know, you can be somebody like Red Hat that provides software, OK? And then people will leverage Red Hat for their own supply chain. You see, and then you have the cloud provider and they have their own source of software. So it's really I think that the key thing is the awareness of how much you rely upon that ecosystem before we look at the securing of the supply chain. It's really it's really understanding your supply chain. And just to follow up on that, what so can you I'm sort of checking my own level of understanding on this subject when you talk about open source code, you're talking about a code base that is often maintained essentially by volunteers, isn't that correct? A mix of volunteers and paid professionals where a company has an interest in the open source project. And but predominantly, I would say it's. Well, I'm not entirely sure, but volunteers make up a substantial part of the ecosystem, that is for sure. So it's a mix, really. Some people do it because they they enjoy writing software. They want to share software. OK, other people also enjoy working software, but they're in the position that a company pays for them to to work on that software. So it's a mix of both. Vincent, give us a reminder of reminder of why this is important from kind of a, you know, a little bit of a higher level. Step back from from the from the data center view of things, from the IT view of things, just from a societal perspective, Vincent, what what happens when we don't secure our digital supply chain? What are the things that are put at risk? Well, there's a significant number of things that are placed at risk. The security of the enterprise itself, right? So your own customer data, your own internal corporate data is placed at risk if there were supply chain breach. But further to that for a software provider, and I think that in a lot of cases, most companies today are software providers or software developers. You actually put your own customers at risk as well, not just their data, but their actual, you know, the things that they're working on, any workloads that they may have, you know, in order that they might place as an example, right? So there's a number of areas where you want to have the security of that supply chain, the software components that you have figured out, right? You want to be on top of that because there is that risk that trickles down when it comes to any event. I mean, we've seen that with breaches earlier this year, you know, one company is breached, multiple companies end up being breached as a result of that. So it's really important. I think we all have a part to play in that. I always view it as it's not just about the company itself. So I mean, speaking from a Red Hat perspective, I don't look at it as we're just securing Red Hat. We're securing our customers and we're also doing that further for their customers as well, right? Because they're writing software that's running on the software that we're providing to them. So there is this trickle down effect that comes. And so I think that every link in that chain, I mean, it's wonderful. That's called the supply chain, right? It's only as strong as its weakest link. So our view is, how do we, how do we strengthen every link in that chain? We're one part of it, but we're kind of looking a little broader. What can we do upstream and how can we help our customers? To ensure the security of their part in that supply chain? Yeah, I want to talk about that in a broad sense, but let's see if we can get a little bit more specific in terms of what some of these, what some of the chains look like, because it's not just really one chain when you think about it. There's the idea of inherent flaws that can be caught. And then there are the things that bad actors might be doing to leverage those flaws. So you've got all of these different things that are converging. So first, and Vincent, if you want to toss this to Luke back and forth, it's up to you guys. What about this issue of inherent flaws in code? We referenced this idea of the maintainer community. How do you, what are best practices for locking that down to make sure that there aren't inherent flaws or security risks? I'll take a stab at it and then I'll let Luke follow up with maybe some of the technologies that Red Hat provides. But and again, speaking to Red Hat as a part of that chain, when we talk about inherent risks, there's a vulnerability that's present upstream. You know, we pull that software into Red Hat. We package it as a component of one of this, you know, some pieces of software that we provide to our customers. It's our responsibility to pay attention to those upstream potential vulnerabilities, potential risks and correct them in our code, right? So that might be taking a patch from upstream, applying it to our software might be grabbing the latest version from upstream, whatever the case might be. But it's our responsibility to provide that protection for that software to actually remediate that risk. And then our customers can then install the update and apply the mitigations themselves. If we take a look at it from, you know, when we're looking at multiple suppliers, right, you'd asked earlier about what part of it is Red Hat and what part of it is, you know, self-service open source. When you look at that, the work that Red Hat's doing there as a commercial provider of open source and end user for that little bit that they're going to grab themselves that Red Hat doesn't provide, it's going to have to do all of those things as well. They're going to have to pay attention to that risk from upstream, they're going to have to pay attention to any potential vulnerabilities and pull that in to figure out, you know, do I need to patch? Where do I need to patch it? That's something we didn't really touch on, was an inventory of the software that you have in place, right? I mean, you don't know that you need to fix something. You don't even know that it's running. So, I mean, there's a lot of considerations there where you have to pay attention to a lot of sources. Certainly there's metadata, automation, all of these things that make it easier, but it doesn't absolve us of the responsibility across the board to pay attention to these things, whether you're grabbing it from upstream directly or from the vendor, and I said, vendor's responsibility to then be paying attention to things upstream. Yeah, so Luke, I want you to kind of riff on that from the perspective of that. Let's just assume that Vincent was just primarily talking about the idea that, okay, we've established that this code is solid, and we've got a, you know, gold copy of it, and we know it's okay. There aren't inherent problems in the code as far as we can tell. Well, that's fine. I'm a developer, I go out to pull code in to use. How do I know if it's not been tampered with? How do I know if it's, in fact, the code that was validated during this process before? What do you do about that? So there's, yeah, there's several methods there, but I'd just like to loop back to that point because I think this is really interesting around. So if you look at a software supply chain, okay, this is a mix of humans and machines, okay, and both have flaws, probably humans a bit more, okay, and a supply chain, you have developers, okay, you have code reviewers, you have your systems, administrators that set up the systems, okay, and then you have your machine actors. So you've got your build systems, the various machines that are part of that supply chain, okay. Now the humans, there's an attack vector there, okay, because typically they will have some sort of identity which they leverage for access to the supply chain, okay. So quite often a developer's identity can be compromised, okay. So a lot of the time people will have a corporate account that gives them some sort of single sign on access to multiple systems, okay. So the developer's account, and this could be somebody in the community as well, their account is compromised, then they're able to easily backdoor systems, okay. So that's one aspect. And then there is machines as well. There's the whole premise of machine software not being up to date. So when the latest nasty vulnerability is released, machines are updated, then the machines have their flaws, they can be exploited. So there's, I would say there's a, you know, it's not just a technical problem, there is a humanistic element to this as well around protecting your supply chain. And I would say that a really good perspective to carry when you're looking to how do I secure my supply chain is treat it like you would a production system, okay. So what do I mean by that? When we put something into production and we've got this very long legacy of treating it with a very strict security context around who can access that people, okay. How much it's upgraded and it's patched. And we seem to not have this same perception around our supply chain and our build systems, okay. The integrity of those, the access of those, the policy around the access and so forth. So that's one giveaway that I would say is a real key focus that you should have is treat it like a production system, okay. Be very mindful about what you're bringing in, who can access it. Because it is the keys to the kingdom. Because if somebody compromises your supply chain, your build systems and so forth, that they can compromise the whole chain, okay. Because a chain is only strong as the weakest link. So that's what I draw upon it. And around the verifications, that there is multiple technologies that you can leverage, okay. So Red Hat, got a very robust signing system that we use so that you can be sure that the packages that we get, you have non-repudiation that they've been produced by Red Hat, okay. When you update your system, that's automatically looked after, okay. And there are other systems as well. There's other new technologies that are starting to get a foothold, okay. Around the provenance of aspects of your build system. So when you're pulling in from these multiple sources of open source communities, you can have some provenance around what you're pulling in as well. And yeah, there's, you know, I don't want to bite you too much on the technologies, but there's some exciting stuff that's starting to happen there as well. So let's look at an example of something because I think it's important to understand all of these different aspects. Recently, I think actually still in the news, we found that some logging software distributed by Apache that's widely used in people's websites to gather information about to help, you know, to help from a security perspective and to help developers improve the things that are going on in websites. Vulnerability was discovered. I guess first, Alibaba, some folks reported it directly to some folks at Apache in the Apache organization. And then of all people, some folks from Minecraft mentioned it in a blog. That seems like a crazy way to find out about something that's a critical flaw. Now, we're looking at this right now with hindsight. So with hindsight, what could we have done to not be in the circumstances that we're in right now? Vincent, I'll toss that to you first, but again, if Luke is more appropriate, let us know. No, it's a great question and it's a hard question. How did you let this happen, Vincent? How did you let this happen? It wasn't me, I promise. But I mean, it is, it's a challenging, and there's a number of areas where we focused on a lot of what we perceived as critical software. So it comes to web server applications, DNS, a number of the critical infrastructure that powers the internet. Right or wrong, do we look at logging software as a critical piece of that? Well, maybe, maybe we should. Logging is definitely important as part of an incident response or just an awareness of what's going on. So I mean, yeah, it probably should have been considered a critical software. There's also, I mean, it's open source, right? So there's a number of different logging applications. I imagine now we're scrutinizing those a little bit more, but looking beforehand, how do you determine what's critical until an event like this happens? And it's unfortunate that it happens. And I like to think of these as learning opportunities, certainly not just for Red Hat, but for, I mean, this touches on, of course. Yeah, this certainly, this is not, yeah, this is not an indictment of our entire industry. We are all in this together and learning every day. It just highlights how complex the situation is that we're dealing with, right? Oh, it really is. And I mean, a lot of what we're looking at now is how do we get tools into the hands of developers who can catch some of these things earlier, right? And there's a lot of commercial offerings. There's a lot of open source tools that are available and being produced that are going to help with these sorts of situations moving forward. But I mean, all the tools on the planet aren't gonna help if they're not being used, right? So I mean, there has to be an education and incentive for these developers, particularly maybe in some upstream communities where they are labors of love and they're passion projects. They're not sponsored or backed by a corporation who's paying for these tools, right? To be able to use some of them and move that forward. I think that looking at things now, there is work to be done. Obviously, there's always going to be work to be done. Not all of these tools, and we have to recognize this. They're not all perfect. They're not going to catch everything. These tools could have been, I mean, I don't know if they were running these tools or not, they could have been. And the tools simply could not have picked them up. So part of it is the proactive part. We talk a lot about shift left and moving these things early into the development process. And that's great and we should do it. It certainly should never be seen as a silver bullet or a replacement for a good response. And I think what the really important thing to highlight with respect to this, and I mean, this touches on the supply chain issue as well. Companies, especially those who never maybe saw themselves as a software development company, really have to figure out and understand how to do appropriate response. Part of that is awareness. What do you have installed? Part of it is sources of information, like how do I find out about a new vulnerability or a potential vulnerability? And then it's just the speed to respond. We know that a number of companies, they have maybe it's a patch Tuesday, maybe it's a patch 26 of the month, maybe it's patch day of the quarter. We have to learn how to respond to these things quickly so that we can apply these mitigations and these fixes as quickly as possible to then protect ourselves and protect our, the end users or customers that we have or to keep the kids from using some back doors of Minecraft as a word. Yeah, this, look, this is an immensely important subject to wrap us up on this. Luke, I'd like you to pretend that you just got in, got into an elevator in a moderately tall building and you have 60 seconds to share with me someone who already trusts you. You don't have to convince me of your credentials or anything. I trust you. What tools specifically do you need me to be running? Tools and processes, you've got 60 seconds to say, Dave, if you're not doing these things right now, you're unnecessarily vulnerable. So ready, ready and go. Luke. Okay, so automatically update all packages. Okay, always stay up to date. So that when an issue does hit, you're not having to go back 10 versions and work your way forward. That's a key thing. Ensure that everything that you pull in, you're not gonna hit 100% but have a very strict requirement that there is non-repudiation, it's signed content. So you can verify that it's not been tampered with. Okay, for your developers that are producing code, run static, dynamic analysis, API fuzzers, all of these sorts of tools, they will find some vulnerabilities for you. Be part of communities, okay? Be part of communities. Help chop the wood and carry the water because the log for J, the thing is that was found because it was in the open. Okay, if it wasn't in the open, it wouldn't have been found. And I've been in this business for a long time. Software developers will always write bugs. Okay, I do. Some of them will be security bugs. That's never gonna change, okay? So it's not about stopping something that's inevitable. It's about being prepared to react accordingly in a right and correct manner when it does happen so that you can mitigate against those risks. Well, we're here on the 35th floor. That was amazing. Thank you, Luke. Vincent, you were in the elevator also listening in on this conversation. Did you miss anything? No, I mean, the only thing I'll say is it's really helpful to partner with an enterprise open source provider, be a redhead or anybody else. I don't wanna toot our own heart, right? They do a lot of that work on your behalf that you don't have to do. A lot of the things that Luke was talking about, those providers do. So you don't have to, right? And that's where you, I like that you talked about, hey, you don't have to convince me that I'm trusted, right? Or that I trust you, trust those vendors. They're literally here to do a lot of that heavy lifting for you and trust the process. Yeah, it's a very, it's a very, very good point. And I know that sometimes it's hard to get to that point where you are the trusted advisor. Both of you certainly are. And with that, I would like to thank you very much for an interesting conversation. Gentlemen, let's keep in touch. You're always welcome on theCUBE. Luke, second time, getting a chance to talk to you on theCUBE personally, fantastic. With that, I would like to thank everyone for joining this very special series on theCUBE. Managing risk in the digital supply chain is a critical topic to keep on top of. Thanks for tuning into theCUBE. We'll be back soon. I'm Dave Nicholson saying, thanks again. In a moment, Kirsten Newcomer, who is the director of Cloud and DevSecOps strategy at Red Hat, joins Dave Vallante. You're watching theCUBE, the global leader in tech coverage. What's the most powerful technology ever invented? Electricity, the internet, nuclear power, the printing press, laser swords. Maybe our most transformative tool isn't technology at all, but a simple conversation between two humans. So join me for technically speaking, an all new series of informal, casual conversations exploring all things technology, from edge computing to Kubernetes to hybrid cloud and what's on the horizon. Hi everyone, my name is Dave Vallante and we're digging into the many facets of the software supply chain and how to better manage digital risk. I'd like to introduce Kirsten Newcomer, who is the director of Cloud and DevSecOps strategy at Red Hat. Hello Kirsten, welcome. Hello Dave, great to be here with you today. Let's dive right in. What technologies and practices should we be thinking about that can help improve the security posture within the software supply chain? So I think the most important thing for folks to think about really is adopting DevSecOps. And while organizations talk about DevSecOps or and many folks have adopted DevOps, they tend to forget the security part of DevSecOps. And so for me, DevSecOps is both DevSec, how do I shift security left into my supply chain and SecOps, which is a better understood and more common piece of the puzzle, but then closing that loop between what issues are discovered in production and feeding that back to the development team to ensure that we're really addressing that supply chain. Yeah, I heard a stat. I don't know what the source is. I don't know if it's true, but it probably is that around 50% of the organizations in North America don't even have a SecOps team. Now, of course that probably includes a lot of smaller organizations, but they're not the SecOps team, they're not doing DevSecOps. But so what are organizations doing for supply chain security today? Yeah, I think the most common practice that people have adopted is vulnerability scanning. And so they will do that as part of their development process. They might do it at one particular point. They might do it at more than one point, but one of the challenges that we see first of all is that that's the only security gate that they've integrated into their supply chain, into their pipeline. So they may be scanning code that they get externally, they may be scanning their own code. But the second challenge is that the results take so much work to triage, right? This is static vulnerability scanning. You get information that is not in full context because you don't know whether a vulnerability is truly exploitable unless you know how exposed that particular part of the code is to the internet, for example, or to other aspects. And so it's just a real challenge for organizations who are only looking at static vulnerability data to figure out what the right steps to take are to manage those. And there's no way we're gonna wind up with zero vulnerabilities in the code that we're all working with today. Things just move too quickly. Is that idea of vulnerability scanning almost like sampling where you may or may not find the weakest link? I wouldn't, I would say that it's more comprehensive than that, the vulnerability scanners that are available are generally pretty strong, but they're, again, if it's a static environment, a lot of them rely on NVD database, which typically is gonna give you the worst case scenario. And by nature can't account for things like was the software that you're scanning built with controls, you know, mitigations built in, right? It's just gonna tell you this is the package and this is the known vulnerabilities associated with that package. It's not gonna tell you whether there were compiler time flags that maybe mitigated that vulnerability. And so it's almost overwhelming for organizations to prioritize that information and really understand it in context. And so when I think about the closed loop feedback, you really want not just that static scan, but also analysis that takes into account the configuration of the application and the runtime environment and any mitigations that might be present there. I see, thank you for that. So, you know, given that this digital risk and software supply chains are now front and center, we read about them all the time now. How do you think organizations are responding? What's the future of software supply chain gonna look like? That's a great one. So I think organizations are scrambling. We've certainly at Red Hat, we've seen an increase in questions about Red Hat's own supply chain security and we've got lots of information that we can share and make available. But I think also we're starting to see this strong increased interest in security bill of materials. So I actually started working with automation and standards around security bill of materials a number of years ago. I participated in the Linux Foundation, as PDX project. There are other projects like Cyclone DX, but I think all organizations are going to need to, those of us who deliver software, we're gonna need to provide S-bombs and consumers of our software should be looking for S-bombs to help them understand to build transparency across the projects and to facilitate that automation. You can leverage the data in a software package list to get a quick view of vulnerabilities. Again, you don't have that runtime context yet, but it saves you that step, perhaps of having to do the initial scanning. And then there are additional things that folks are looking at, attested pipelines is gonna be key for building your custom software. As you pull the code in and your developers build their solutions, their applications, being able to vet the steps in your pipeline and attest that nothing has happened in that pipeline is really gonna be key. So the software bill of material is gonna give you a granular picture of your software and then what, the chain of provenance, if you will? Well, an S-bomb, depending on the format, an S-bomb absolutely can provide a chain of provenance, but another thing when we think about it from the security angle, so there's the provenance, right? Where did this come from? Who provided it to me? But also with that bill of materials, that list of packages, you can leverage tooling that will give you information about vulnerability information about those packages. At Red Hat, we don't think that vulnerability info should be included in the S-bomb because vulnerability data changes every day. But it saves you a step potentially, right? Then you don't necessarily have to be so concerned about doing the scan, you can pull data about known vulnerabilities for those packages without a scan. Similarly, the attestation in the pipeline, that's about things like ensuring that the code that you pull into your pipeline is signed, right? Signatures are in many ways a more important piece for defining provenance and getting trust. Got it. So I was talking to a CISO the other day and I was asking her, okay, what are your main challenges, kind of the standard analyst questions, if you will? And she said, look, I got great people, but I just don't have enough depth of talent to handle the challenges. I'm always sort of playing catch up. And so that leads one to the conclusion, okay, automation is potentially an answer to address that problem. But at the same time, people have said to me, sometimes we put too much faith in automation. Some say, okay, Kirsten, help me square the circle. I want to automate because I lack the talent, but it's not sufficient. How do I, what are your thoughts on automation? So I think in the world we're in today, especially with cloud native applications, you can't manage without automation because things are moving too quickly. So I think the way that you assess whether automation is meeting your goals becomes critical. And so looking for external guidance, such as the NIST secure software development framework, that can help. But again, when we come back, I think look for an opinionated position from the vendors, from the folks you're working with, from your advisors on what are the appropriate set of gates. And we've talked about vulnerability scanning, but analyzing the config data for your apps is just as important, right? And so I think we have to work together as an industry to figure out what are the key security gates? How do we audit the automation so that I can validate that automation and be comfortable that it is actually meeting the needs? But I don't see how we move forward without automation. Excellent, thank you. So given, I mean, we were forced into digital without a lot of thought. Some folks, it's a spectrum. Some organizations in better shape than others, but many had to just dive right in without a lot of strategy. And now people have sat back and said, okay, let's be more planful, more thoughtful. So as you, and then of course, you got the supply chain hacks, et cetera. How do you think the whole narrative and the strategy is going to change? How should it change the way in which we create, maintain, consume software as both organizations and individuals? So again, I think there's going to be, and there's already a need, request for more transparency from software vendors. This is a place where S-bombs play a role. But there's also a lot of conversation out there about zero trust. So what does that mean in, you have to have a relationship with your vendor that provides transparency. So that you can assess the level of trust. You also have to, in your organization, determine to your point earlier about people with skills and automation, how do you trust but verify? This is not just with your vendor, but also with your internal supply chain. So trust and verify remains key. That's been a concept that's been around for a while. CloudNative doesn't change that, but it may change the tools that we use. And we may also decide what are our trust boundaries? Are they, where are we comfortable trusting? Where do we think that zero trust is a more applicable place, a more applicable frame to apply? But I do think back to the automation piece. And again, it is hard for everybody to keep up. I think we have to break down silos. We have to ensure that teams are talking across those silos so that we can leverage each other's skills. And we need to think about managing everything as code. What I like about that everything as code, including security, is it does create auditability in new ways, right? If you're managing your infrastructure in a GitOps-like approach, your security policies with a GitOps-like approach, it provides visibility and auditability and it enables your dev team to participate in new ways. So when you talk about zero trust, I think, okay, I can't trust users. I gotta trust but very much users, machines, employees, my software, my partners. Every possible connection point. Absolutely, and this is where both attestation and identity become key, right? So being able to, I mean, the SolarWinds team has done a really interesting set of things with their supply chain after they were in response to the hack they were dealing with, right? They're now using Tecton CD chains to ensure that they have attested every step in their supply chain process and that they can replicate that with automation. So they're doing a combination of, yep, we've got humans who need to interact with the chain and then we can validate every step in that chain. And then workload identity, right, is a key thing for us to think about too. So how do we assert identity for the workloads that are being deployed to the cloud and verify whether that's with SpiffySpire or related projects, verify that the workload is the one that we meant to deploy and also runtime behavioral analysis. I know we've been talking about supply chain, but again, I think we have to do this closed loop, right? You can't just think about shifting security left. And I know you mentioned earlier, right? A lot of teams don't have SecOps, but there are solutions available that help assess the behavior in runtime and that information can be fed back to the app dev team to help them adjust and verify and validate where do I need to tighten my security? I'm glad you brought up the SolarWinds to Kirsten what they're doing. And I remember after 9-11, I was afraid to fly, but it was probably the safest time in history to fly. And so same analogy here, SolarWinds probably has learned more about this and his reputation took a huge hit. But if you had to compare what SolarWinds has learned and applied at the speed at which they've done it with maybe some other software suppliers, you might find that they've actually done a better job. It's just unfortunate that something hit that we never saw before. To me, it was Stuxnet-like. We've never seen anything like this before. And then boom, we ventured a whole new era. I'll give you the last word, Kirsten. No, just to agree with you. And I think again, as an industry, it's pushed us all to think harder and more carefully about where do we need to improve? What tools do we need to build to help ourselves? Again, S-bombs have been around for a good 10 years or so, but they are enjoying a resurgence of importance. Signing, image signing, manifest signing, that's been around for ages, but we haven't made it easy to integrate that into the supply chain. And that's work that's happening today. Similarly, the attestation of a supply chain, of a pipeline, that's happening. So I think as an industry, we've all recognized that we need to step up and there's a lot of creative energy going into improving in this space. Excellent, Kirsten Newcomer, thanks so much for your perspectives, excellent conversation. My pleasure, thanks so much. When we return, Andrea Hall, along with Andrew Block, both from Red Hat will join me. You're watching theCUBE, the global leader in enterprise tech coverage. And we can have this in market by end of quarter. End of quarter? Yep. Well, there might be a few global requirements that you forgot to consider. All of our data has to be stored on private clouds. We can do that. We only need to go inside, not in the back. That's fine. Our developers need to be able to use the tools that they already have. You got it. We have critical services on VMs. Not a problem. Not a problem. We're building on a really flexible platform. Life really flexible. Okay then, let's get started. Okay, we're here talking about how you can better understand and manage the risks associated with the digital supply chain. How in this day and age where software comes from so many different places and sources throughout the ecosystem, how can organizations manage the risks associated with our dependence on software? And with me now are two great guests, Andrea Hall, who is a specialist solution architect and project manager for security and compliance at Red Hat. She's got a focus on public sector. And Andrew Block, who's a distinguished architect at Red Hat Consulting. Folks, welcome. Welcome. Thanks for having us. You're very welcome. So Andrea, let's start with you. Let's talk about regulations. What exists today that we should be aware of that organizations should be paying attention to? Oh, sure. So the thing that comes to mind first being in the US is the presidential executive order on cybersecurity that came out a few months ago. Organizations are really paying attention to that. And in the US, it's having a ripple effect with policy. But we're also seeing policy considerations pop up in other countries, Australia and England. So the supply chain is a big focus right now, of course. But we see these changes coming down the road as more and more government organizations are trying to secure their critical infrastructure. Is there kind of a leadership approach? In other words, as somebody saying what the UK does and saying, okay, we're going to follow that template or is it sort of just a variety and a mishmash with no sort of consolidation? How was that sort of playing out? I see a lot of organizations kind of basing their requirements on NIST 800171, I'm sorry, NIST 853. However, each organization has its own nuances. Each agency has its own nuances to how it wants and implemented. Great. Andrew, maybe you could chime in here. What are you seeing when you talk to customers that are tuned into this issue? As Andrew just mentioned, having that North Star in terms of regulations is so fundamentally great for them because many of them especially in regulatory industries look to these regulations on how they apply their own policies. So at least we have some guidance on how to move forward because as we all know, the secure software supply chain is getting news every day and how they act to it is something that I know all their leaders are asking themselves, especially those IT leaders. So Andrea, when I talk to practitioners, sometimes they're frustrated, they understand they have to comply, they know new regulations are coming out, but sometimes it's hard for them to keep up. So it would be helpful if you're sitting across the table from somebody who's frustrated and they ask you, what are your expectations? How do you see, what are the trends in regulations? How do you see the current regulations evolving to specifically accommodate the digital supply chain and the security exposures and corollary requirements there? I see a lot of organizations struggling in the sense of trying to understand what the policy actually wants. Definitions are still a little bit vague, but implementation is also difficult because sometimes organizations will add more tools to their toolkit, adding a layer of complexity there. It's really automation has to be pulled in that's key to implementing this instead of adding more workload and more burden to your folks. It's really important for these organizations to hold the stakeholders in the organization together. So the IT leaders bring together the developers, the security operations sit at the same table, talk about whether or not what needs to be implemented or what's proposed to be implemented will affect the mission in any way or disrupt operations. It's important for everybody to be on the same page so it doesn't slow anything down as you're trying to roll it out. And one of the things here is that we're seeing a lot of change with these new regulations and with a lot of organizations, any type of change is scary. And that is one area that they're looking for guidance not only in the tooling, but also how they apply it in the organization. Well, I'll add on. Please. I'll add on to that and say, organizations really need to take into account the people side of things too. People need to understand what the impact is to the organization so that they don't try to find the loopholes. They're buying into what needs to be done. They understand the why behind it. For example, if you walk into your house, you normally close the door behind you. Security needs to be seen as that as well. That's the culture and it's the habit and it's ingrained in the fabric of the organization to live this way, not just implement the tools to do it. Right, and the number of doors you have in your infrastructure a lot more than just a couple. Andrew mentioned sort of guidance. Governments are obviously taking a more active role. I mean, sometimes I'm a cynic. I mean, great that President Biden signs an executive order, but the swipe of a pen doesn't really give us enough to go on. Do you think, Andrea, that we're going to see new guidance from governments in the very near future? What are you expecting? I expect to see more conversations happening. So I know that organizations, the who develop the policies are pulling together stakeholders and getting input, but I do see in the not too distant future that mandates will be rolling out. Yes. Well, so Andrew, of course, Andrea, if you have a thought on this as well, but how do you see organizations dealing with adopting these new policies? Slowly, don't bore the oceans. One thing I tell everyone of them because a lot of these tooling, a lot of these concepts are foreign to them, brand new. How they adopt those and how they implement them needs to be done in a very agile fashion, very slow and prescriptive. Go ahead and try to find one area of improvement and go ahead and work upon it and build upon it because not only does that not only make your organization more successful and secure, but also helps your organization just from a morale standpoint. One thing that you need to emphasize is that don't blame anyone because a lot of times when you're going through this, you're reassessing your own supply chain and might find where you could see improvements that need to be done. Don't blame things that may have occurred in the past. See how you can benefit from these lessons learned in the future. You know, it's interesting you say that about the blame game. I mean, it used to be that failure meant you get fired and that's how it obviously has changed. It's not about, as many have said, you know you're gonna have incidents. It's how you respond to those incidents, what you learn from them. Do you have, Andrew, any insights from specifically working with customers on securing their software supply chain? What can you tell us about what leading practitioners are doing today? They're going in and not only assessing what their software components consist of. Using tools like an S-bomb, a software build materials to understand where all the components of their ecosystem and their lineage comes from. So we're hearing almost every single day new vulnerabilities that are being introduced in various software packages. By having that understanding of what is in your ecosystem, you can then better understand how to mitigate those concerns will be forward. So Andrea, Andrew was just saying, one of the things is don't just dive in. You got to be careful. There's got to be ripple effects is what I'm inferring. But at the same time, you know, there's a mandate to move quickly. So are there things that could accelerate the adoption of regulation or even the creation of regulations and that guidance in your view, what could accelerate this? As far as accelerating it goes, I think it's having those conversations proactively with the stakeholders in your organization and understanding the environment. Like Andrew said, go ahead and get that baseline and just know that whatever changes you make may be going to be audited down the road because we were moving towards this kind of third party verification that you're actually implementing things in order to do business with another organization. So the importance of that, if organizations see that gravity to this, I think they will try to speed things up. I think that if organizations and the people in those organizations understand that why that I talked about earlier and they understand how things like solar winds or things like the oil disruption that happened earlier this year, the personal effects of cyber events will help your organization move forward. Again, everybody's bought into the concept, everybody's working towards the same goals and they understand that why behind it. In addition to that having tooling available that makes it easy for them. You have a lot of individuals who this is all foreign providing that base level tooling that aligns to a lot of the regulations that might be applicable within their realm and their domain makes it easier for them to start to confine and taking less burden off of them to be able to be successful. So it's a hard problem because how do you, Andrew, how do you deal with sort of the comment more tools? Okay, but I look at that the Optif map, if you've seen that, it makes your eyes cross. So you've got so many tools, so much fragmentation. You're introducing new tools, can automation help that? Is there hope for consolidation of that tools portfolio? Right now, this space is very emerging, very emerging, very fluid to be honest because these actually mandates only a year or two old as they come over the course of time. However, I do see these types of tooling starting to consolidate where right now it seems like every vendor has a tool that tries to address this. It's being able to have the people work together or have more regulations that will come out that will allow us to start to redefine and solidify on certain tools like ISO standards. There's certain ones I mentioned on S-Bombs previously there's now a ISO standard on S-Bombs there wasn't previously. So as more and more of these regulations come out it makes it easier to provide that recommended set of tooling that organizations are leveraging instead of vendor A, vendor B. So Andrea, I said before I was a cynic, but we'll give you the last word, give us some hope. I mean, obviously public policy is very important, a partnership between governments and industry, both the practitioners, the organizations that are buying these tools as well as the technology industry got to work together in an ecosystem. Give us some hope. The hope I think will come from realizing that as you're doing this, as you are implementing these changes, you're in a sense prevent trying to prevent those future incidents from happening. So there's some assurance that you're doing everything that you can do here. You know, it's a situation, it can be daunting. You know, I'll put it that way. It can be really daunting for organizations, but just know that organizations like Red Hat are doing what we can to help you down the road. And really it's just continuing this whole shift in left mentality, this offer supply chain is just one component but the introducing DevSec off security at the beginning that really will make the organizations become successful because this is not just a technology problem, it's a people issue as well and being able to kind of package them all up together will help organizations as a whole. Yeah, so that's a really important point. You know, you hear that term shift left. For years, you're, you know, people say, hey, you can't just bolt security on as an afterthought, that's problematic. And that's really, that's the answer to that problem. Right, is shifting left, meaning designing it in at the point of code, infrastructure as code, DevSecOps, that's where it starts, right? Exactly, being able to have security at the forefront and then have everything afterwards, you know, propagate from your security mindset. Excellent. Okay, Andrea, Andrew, thanks so much for coming to the program today. Thank you for having us. Thank you. So look, I wish I could say there's an end to these threats, there isn't. They will continue indefinitely. The adversaries, they're well-funded, they're motivated and they're sophisticated. Your job as practitioners is to try and make it less profitable for the hackers. At the end of the day, this is a business for them. And the hackers, what do they want? They want value, it's all about ROI for them. That means benefit over cost. If you can increase the denominator, it lowers their value. And they're going to go elsewhere and they'll fish in more productive places. The hard reality is that bad user practices will trump good security every time. And that's where the vulnerability starts. So shoring up the basics, that's table stakes. Now beyond that, working with strong technology partners can bring expertise to compliment your team's skills and reduce the threat against these sophisticated attacks. We hope this program was informative and will inspire you to take action. All these videos, they're available on demand at theCUBE.net and both theCUBEs and Red Hat's social channels and a variety of other places that we'll share with the community. Thanks to all our guests today for Dave Nicholson and the entire CUBE team. This is Dave Vellante. Appreciate you watching, we'll see you next time.