 Hi, I'm Vipin Vallatham. I've been involved in Hyperledger since it started, probably in 2015-2016. And I'm the chair of two important or foundational groups in Hyperledger. One is the Hyperledger capital markets and the other is the identity working group. And this was prompted, this particular talk was prompted by all the news about software supply chain and the intersection of the software supply chain with the open source community. And we'll go through this presentation in the beginning, just exploring the software supply chain and the software supply chain security. Then we talk about open source and what we can do in Hyperledger repositories. And I can also talk about what we do in Hyperledger repositories. I've demonstrated some issues to people. Anyway, the reason this is called the poisoned well is because in ancient times, the well was a common place where people would come to get water. Everybody, it was a shared resource. It was the commons. And I have to say that once the well is poisoned, then everybody gets poisoned, which is a metaphor for open source. So we have to talk about the software supply chain and the attack surface of the software supply chain and the trends in the software supply chain compromises, the evaluation of data on actual attacks, the solar winds hack, a deep dive, and a special case of open source software and finally mitigation. Equal time will not be spent on each one of these. You know, some of the items get more priority than others. I don't think we need to talk about this particular problem here. This particular issue. I mean, set of things that we need to talk about, which is software is ubiquitous software is evolving fast, and software is complex. That we that much we can agree on. The consequences of this phenomenon are that most enterprises use commercial of the self software or commercial open source software like, for example, what is in red hat. There is a lot of reuse being done and outsourcing. There is a phenomenon of outsourcing complexity. Right. There was a, you know, this is not new. There was a thought experiment from Ken Thompson in 1984. The real source is a air for Air Force critique. Now obscure Ken Thompson is of course the creator of the original unix and the C compiler. If you had been writing C, you would have read Thompson and Richie's C programming. The C programming language, which is, which was the seminal text of the 80s and 90s. Ken Thompson showed in code how the C compiler can be hacked to insert a Trojan horse, leaving no traces in the source. Basically, he did that by creating the source. Well, the C compiler itself is written in C. This is the beauty of the whole thing. So the C compiler source is changed to insert the Trojan horse. And then he removes the source after compiling the second, the new C compiler, which contains the Trojan horse, and all traces of this hack disappear from the source code. So the compiler and the source code do not correspond to each other. Of course, as Ken said, the level of, as the level of programming gets lower. And especially in microcode in those days, microcode, but in these days, these days with IoT devices and so on, that kind of well installed Trojan horse inside a microcode will be very, very almost impossible to detect. So this should send shivers down everybody's spine because it means that if the microcode or or the IoT devices hacked. There is no way to find out what Trojan horse lurks inside each one of them. There are ways in which we can detect this mostly by the behavior of the component. And these days when you have AI monitoring for security, you can probably figure out that one of the components are misbehaving. So you go to the solar winds hack, which was very prominently in the news just recently solar winds is a company that manufactures a product called Orion. Orion is used for many administrative tasks all over the place. The point is, it is used across more than 18,000 enterprises, and many sensitive federal and local agencies. So talking about the US Treasury Department, you're talking about, you know, even probably parts of the Defense Department. So the reach is tremendous of once you get into the solar winds product Orion, then you can get into all these other products. So initial hack is very innocuous, meaning they probably use some kind of an account compromise using a using phishing or something else to get into the into the IT management product build tooling. The details are there in the references that I provide in the end. Basically, it monitors the running processes for MS build.exe. I have done this myself, meaning, I'm not saying I did it for the purpose of hacking the build, but in order to, you know, to detect a running program is very easy. And then to replace one of the source files. So these are not complex hacks. Thing is, the accreted value of the of the hack is tremendous because the scale of the compromise is magnified by the use of Orion in multiple places. So overall, safeguards were added to the sunspot sunspot is the actual name code name given by the analysts to avoid the builds from failing due to this inserted code. And the beauty of the thing is similar to what Ken Thompson did which is, you make the change, you do the compilation. The file is compiled. You back out the change, and you restore the old file. So no traces of the hack remain in the source files that are on SolarWinds. SolarWinds servers. And I said it was used extensively in enterprises. Now. So any supply chain hack can happen at various points in the design. I mean in the in the process in the build process in the implementation process in the iterative testing process in the deployment process in the updates and maintenance process. SolarWinds of course is sunburst is the code name was in account. In the account access process, but it can happen at any point in this, in this build chain, build and deploy chain, which is very interesting out of the 18,000 only a small portion of the victims were chosen. And then they compromise the using this service. The identity and access management software from Azure were compromised to gain access to many email accounts, including many sensitive email accounts. There was a cloud based compromise as well as a sign of a state actor with time, money, patients and discipline, which means, after the initial penetration, they stayed quiet for a long, long time. They activated some of these changes in the builds, then the exfiltrated exfiltrated meaning took out the data from these from these compromised accounts, they used only us base servers. Through VPNs or whatever to exfiltrate the data to take the data out and store it. And they did it in a way that it would not raise any red flags, including making sure that the traffic from the compromise servers. Followed the usual patterns. That means not too much data was pushed out in burst, they slowly filtered it out. And the major point that should be noted is the software supply chain attack lasted months, and it was a major major strategic failure for the US. Obviously, nobody knows exactly who did it. But all fingers seem to point at Russia. But you don't need this kind of state actor to compromise the software supply chain. You just need somebody who has the money and the willingness to attack the software supply chain. Now, that brings us to the open source world. The open source world has more than 1.5 trillion downloads in 2020. High performance high performers who use open source detect and remediate open source software well vulnerabilities 26 times faster. High performers are 51% more likely to create a software bill of materials which we'll get into in a minute. And there is a 40 430% year on year growth in cyber attacks targeting open source software projects. Nearly 40% of all npm packages rely on code with known vulnerabilities. On average, there are 38 known open source software vulnerabilities per application. You know, all these numbers, they're tremendously sort of. They create a situation in which we are, we should be very vigilant when we release open source software, especially blockchain based blockchain. Based open source software, you know, blockchain frameworks on open source software, which is the business of hyper ledger. Whenever they talk about exemplary projects. They say they are, you know, 530 times faster or at updating dependencies. The aim that we have should be to become a high performer to become an exemplary project. When we have a project inside hyper ledger. Our software needs to be among the high performers. And it's not just small companies that use open source software. We can see that 373,000 average enterprise downloads of open source software components happen per year. Many of the very sensitive financial applications, many others use open source software extensively, even though the people inside the enterprise are not even aware of it. So the aim for us should be to get to be high performers. And the defense of this software starts, you know, so we are talking about now about defense starts with the software bill of materials. The bill of materials is it's like a bill of materials for any product. That means you break down the product into its components and the components are further break broken down into their components and so on, until you reach the raw material stage, or at least the material that is very simple. So this is a known way of analyzing any product and the logistics of supply chain for a normal product. So the software bill of materials is very similar to the software bill of materials for a industrial product. It's a physical object or physical product like a laptop laptop has probably thousands, hundreds of thousands of components, which are sourced from all over the world. So first of all, this cannot be done by humans. I mean, maybe you can try. But the software bill of materials should be created using a software component analysis tools, also monitoring and response have to be as close as possible. And of course, after the hack. There was an executive order by President Biden that asked NIST to develop a way to tackle the supply chain security. NIST had already had many meetings, many ways of dealing with supply chain attacks, but obviously the breach showed that they are nowhere near securing the software supply chain. So this has to be taken up with the most urgency. I mean, hypoallergic repositories could manage the software bill of materials through a database. I tried to automate the SCA software component analysis using a tool directly linked to the GitHub repository. We have to rebuild and redeploy the project, if any of the dependencies have reported vulnerabilities. And we also have to communicate to users saying that our original build had vulnerabilities and there is a pathway to upgrade once we've once we've created the new build. In the universe of software supply chain attacks. This is only one one vector, but it's an important vector. So I'm going to share what is called the crop circle diagram from the software bill of materials practice in supply chain supply supply chain. It all points to what should be stored for the component identity related to the component identity. There should be authors, there should be a license. There should be a formulation, which is basically a build system, which is how the component is built. What is the usage description. And what are the known vulnerabilities and what vulnerabilities do they expose the protocol and the format and the components that the component this component depends on. So as you can see, I've shown this with the dashed line that that this can be one too many. That means the component identity can point to so many other components, and each of those components can point to others. So there is an explosion in the dependency tree from this component to the other components. Let, let's let's talk about what I've done in terms of what I've run. I have a creative branches in many of the projects in hyper ledger my own private branch, but of course since it's public software I can do that. And then I can, I plugged in two things. One is the white source bolt. So this only handles the known vulnerabilities because basically what it's doing is it's comparing what the product uses first first it does a component analysis software component analysis and finds out all the components and for each component and version. If there is a particular vulnerability reported in the press or not the press sorry in the CBE which is the, which is the place where they collect all the vulnerabilities. Of the software, and if that saw piece of software the component is available on that P on that website, then it reports it as an issue. So I ran this against some of the components in hyper ledger some of the projects. Some of them picked up quite a few vulnerabilities serious medium, and so on. So I presented that fact to the projects. And we have to set up a methodology by which we can mitigate and upgrade to the latest version. When we are doing this, there are a couple of problems. One problem is the problem of false positives that is the software component analysis is reporting a problem where none exists. That can cause the developers to ignore, ignore the, you know, the fact that a compromise was detected. We have to somehow configure the white source bolt which is what I used, or any other product to only report things that above a certain threshold above, let's say, at least medium vulnerability. So if you think about this possibility that you're using a vulnerable component, what is the problem. The problem is that you are no longer your software that you created inside hyper ledger is no longer just dependent on the security of your own software, but it is dependent on the security of the components that you used. And any one of those vectors can be used to attack a user of your software compromise them, get into their, their servers, similar to what happened to Orion. That piece of software by solar winds. And we also see that that can then spread all over the place, depending on the reach of the software. So obviously, all you need to do is to compromise. Solar winds, and then using that to compromise the identity and access management system inside Microsoft Azure. Then you have the whole web based platform of, you know, multiple organizations that can be breached. So for us, what we can do is a first level defense of figuring out which components we are using that are problematic and fix them. I've already started a blog. I mean, I'm going to set out a blog to hyper ledger and also communicate with the maintainers and the security practitioners inside hyper ledger. And maybe we will come up with a process and a workflow to do this component software component analysis on a rigorous way and find out how we can fix the software that we are building and make it as safe as possible. Another fact is that the earlier you have this practice of software hygiene. The better it is going to be for the final product because finding and fixing problems. Before you're about to release production version is not going to be easy compared to inculcating that practice right from the get go. Anyway, thank you all for listening. I think I have come to the end of my 30 minute talk. And in case you want to contact me. My email address is VIP at DLT dot NYC, which is on every one of these slides actually it's not but the only my website. But I want to also share with you the references that I have here. And I am thankful to many of these. Many of much of this literature. In order to do my research and to come up with practical solutions. Thank you again.