 So you know those open source libraries? Well, they kind of disappeared a little bit. So we're going to be talking about the unexpected demise of open source libraries. Or it's going to disappear, and we're not going to be talking about anything I don't know. So next step, we have Larentel. So I'm probably going to be the only person who's stopping you and the after party up happy hour thing. So no pressure there. So yeah, the unexpected demise of open source libraries or basically where do libraries go to die or libraries die hard, all of those kind of titles of us using open source projects but maybe not knowing exactly what's happening there. So open source has fundamentally changed my life. As a teenager, for me, I was going on to things like the Linux kernel mailing list, hanging out with people, learning about open source, using tools, basically finding collaboration and communication paths with like-minded people. And it enabled me to learn and do many, many good things, I hope, and just put them out there. One of those was the projects that I actually built about 15 years ago. I uploaded it to an open source page called SourceForge for some of you who might remember it. And to date, I've actually went back and seen how much that has actually been impactful. And I've actually seen this project called Deloradius as part of the resume of some people that have their skills and experience laid out on LinkedIn, which is pretty cool. So it actually changed my life, but it also allowed me and enabled me to basically improve and change other people's life to the better. So I'm glad for open source. My name is Irantal. I'm a developer advocate at SNCC, where we build developer-first security tools, basically help empower developers to not just find but fix security issues along the way, ID is, Saturday and all of those things. Also doing a bunch of stuff on the ecosystems of Node.js and PM, open source all around. So if you have any questions on this in any different way, I'm happy to chat about it afterwards just to find me here. So talking about open source, this is a pretty interesting topic and the success of it. I think it's been successful so far. So I went back and found some information that was really interesting for me, one of which was almost half of the projects at around 2015 on GitHub are MIT licensed, which is interesting by itself. But actually, this is all the projects, everything else. If we look down specifically on things like ecosystem, language-based ecosystems, like NPM, like Ruby, Python, PyPI, things that people store there, we're actually getting to about 90% of those projects being used using licenses that are very, very permissive. And I think that is something that has indeed contributed to the fact that open source has really been growing and advancing at a really increased state. So one question when we're talking about here and the unexpected demise of open source libraries is have we used so much open source that at this point we're actually taking it for granted? And then what happens when we do? What happens when we use open source and just consume it? It's just another S-bomb entry. So with that said, I want to share with you a few stories that really happened in the ecosystems. This one specifically called Left Pad is a story of how legal actions were driven and taken by a commercial body towards an open source maintainer that at the end of it, back in 2016, at the end of this action, it had actually caused widespread disruption of build and systems impacting projects like the Node.js project, like Babel and other in the JavaScript ecosystem. So what exactly happened there? Well, if we take, for example, a regular software development team like this one, well, there are some fallacies there. For example, we have Murray, Jonathan, and Mina doing open source development, which is great. And they're doing also internal development for the company. And it's been scientifically proved that to be a developer and what it actually means is essentially you take some coffee. That's right, you know it. And some code goes out. And so you know about this one already. There's another thing that we've been witnessing recently, and that is that they use a lot of open source packages. Up to 90% of those statistics have been shared out once and about every time. And they use a lot of those open source packages. One of them, by the way, is this person called Azur. He's an open source developer and maintainer of many, many modules. One of them, this team actually uses. So in essence, one of the fallacies is this team is not really a three-people team. They're like a four-people team. Actually, many, many more people because we're using a lot of open source packages. Behind those projects, behind those packages leave other maintainers and other developers that impact us. So let's talk about Azur for a second. Azur is an NPM developer, you know, building stuff on the NPM registry for JavaScript. He has this module called KIK, which accidentally is also a name of an instant messaging application that's been there and a commercial entity that exists on this kind of like a platform. And so he was asked by this legal entity, you know, several emails, exchange, and so on to basically give up the name and use some other name, just, you know, give it up and allow this other company, use this KIK to own that actual name. That didn't really work out as, you know, could have guessed. So as a protest of, you know, not really agreeing with that, then NPM, the registry itself, the team behind it, not able to actually bridge between the two sides. And eventually what happened is he did not want to get a lawsuit or any other kind of like legal actions. So went on and just as a protest action removed all of his modules from NPM. He had many of them, about 250 back then. Just removed all of them and said, you know, bye-bye. One little thing that happened there is there's like this little little model, 17 lines of code called left pad, left padding strings. And it's downloaded just a little bit, about 2.5 million times. And so when that got removed, essentially a lot of projects that used it as a nested dependency somewhere down their transitive dependency tree have actually broken up. And this is how this story became to be how one developer actually broke whole ecosystem. So some learnings from this story. First of all, reinstalling those packages all the time when you do NPM install on CI or developer machines and you go off and fetch the actual already fact, like this key module or left pad module or something else, you want this to be a very deterministic and proven way of doing that. So obviously one of the ways of doing this is to put something like a caching proxy, like something that actually takes both the fact that it also saves up, not just the fact that something could be a network error or just begun from the registry for some reasons, but also the fact that you wanna speed things up and do those kind of installs really fast internally. So Verdecio is an open source private proxy server that does all of those kind of things. It's pretty fast, basically the way that it works. You kind of configure package managers to use that as the upstream or as the actual registry. And then that one is Verdecio itself is used as that source to which upstream are you allowed to actually contact and bring all of these things in. So Verdecio is really cool and it's a great open source software for us to do this. And I think a lot of companies have learned this in the hard way of using those internal proxies, not just for speeding up reasons, but actually to prevent those cases from happening. But also the fact that when this happened, when all modules got published, this was an unprecedented event in the NPM ecosystem because no one thought about the potential threat attack that someone would actually go and unpublish their modules, especially if they're so popular and everyone used them. But when they do, it just impacts everyone else. So it actually brought in NPM to introduce new limitations as if you cannot unpublish your modules if it is popular or used by other dependencies and so on and so on. So we've learned from it as an ecosystem as well, but we've had to learn the hard way. Another story is this story about someone who actually stopped their project from being alive, from being maintained anymore, just because they wanted to be a bit more greener for the earth. So I don't know how you would feel about it, but I'm pretty sure like for a lot of maintainers like gaining the popularity and reputation of using an open source project is something they probably don't wanna cut in just midway as you're developing it. But for this person, not so much. So Meet François is the developer and maintainer of this Faker PHP library. It's essentially a library that allows you to fake data, to random generation of data for tests, things like that. It's pretty cool. It's been there for since 2011 and it has been used to be an SDK for like different languages like Python and also provide different locales like over 70K. So it's like a very, very popular project when you download it as a DevTool, it's three megabytes of size, pretty, you know, not that much if you compare it to like end-to-end tools and like bigger things that you probably pull in. So this is a project that became so popular that by 2019 it had actually been downloaded over 120 million times, many, many contributors, many, many products being dependent on it. But the thing is, at this point of popularity, if something, if you wanted to be conscious of climate change, would you actually consider the size of your library an issue? Sure, maybe it's an issue for front-end and optimization and stuff like that, but for climate change, that is something that François really had difficulties agreeing with and working with. He did not want to be a person who contributes in a bad way to climate change and he actually figured out that potentially what's going on there is about 11 metric tons of CO2 emissions just by all of those downloads. That's a lot and he said, I do not want to do this anymore. So what happens next? Well, first of all, do we get new maintainers? How does open source work? So François had, he's a passive sin and understood like he has been on the open source ecosystem for a bit and he had like really bad experience with transition in his projects to other maintainers. One of his stories is he actually owned another, maintained another project called Propel, a PHP ORM library and when he handed that off to a friend to continue maintaining it, that friend handed it off to another friend, which handed it off to another friend, which they ended off to another friend that said, this codebase is not really good. I'm gonna rewrite this whole thing and do a 2.0 version. That was six years ago, nothing came out of it. So he had a really bad experience and he did not want to just try to continue the project by handing it off. So he said, I'm archiving the project, archiving the GitHub repository, no more releases, no more issues, nothing else if you want, you can forget. You know, he just added a comment. You can forget, it might be better than Faker, the PHP library, just remember it might not be so much fun. So here's another open source dependency just unexpectedly dying off. First of all, appreciating maintainers I think is a really good thing that we should adopt and that relates to the whole sustainability aspects of open source and discussions around this. And of course, you know, health support, the climate change is an important thing and it's great that maintainers and developers are conscious to this. Our last story is about how Mr. M, this person, did not want to do any more free work as part of their open source packages. And I'm pretty sure there's some people here that are contributing to open source, maybe even maintaining it. So we're all using open source projects, but how much are we actually giving back to people? How much are we actually giving back to the projects maintainers themselves and developers? I promise that, unlike the other two stories, I'll end this fairytale one in a positive note. So meet Mr. M. They are an open source maintainer and they have a bunch of projects. One of them is this called Faker.js. This is by the way not the same project. The idea is the same, but coincidentally it is of the same name, just not the same one. Different maintainers, different projects, by the way, different outcomes. It's pretty popular as well. This one as well is used to generate random data used for tests and things like that, but it is pretty popular as well, 12 million monthly downloads and just like many other projects on the ecosystem, most of this work is unfunded. It's maybe sustainable, maybe not, but no one contributes to the project. So this person, the maintainer behind a project called Faker.js, went on in 2020 and said in a GitHub issue, I am not going to support this project or you anymore, no more free work from me. And I think as you can see maybe a bit by the reactions on GitHub, most of the people actually agreed with this and support this because it's a project that's used by Fortune 500 companies and we're not doing that well of a job contributing back to all the peers and the colleagues that actually work on open source. So this was his statement and the question is, was this heard? This was back 2020. It wasn't that heard. One of those things that happened is that by 2022, the beginning of January, this person, the maintainer, actually pushed codes into the repository that essentially nullified everything else. They forced push a commit. So all the source code had existed for this Faker.js PHP library was completely removed only if you archived it or forked it before or clone it or whatever, nothing exists. Also they pushed a new version called Faker 6.6.6, understand where this is going, which indeed broke the ecosystem as well. It had nothing in it. Essentially it was an empty project. If you try to use it, it would fake. It would not work. And so he'd actually lived up to his statement, right? Said no more free work, so no more free work. You're not getting any of those packages anymore. Question is, did anyone really care? To understand that, we actually need to dive a bit deeper into what happened there. And so Faker, this project, if the maintainer actually pushes a project with 6.6.6 major version, that is broken, but is this actually affecting all of us? Not exactly. The main stable branch or version, same range of Faker, the library on NPM, actually major version five. The way that NPM works behind the scenes is where usually it auto fetches, minor and patch versions, depending on how you configure things, but you have to be very explicit to just take everything in with a wildcard. And people usually do not do that and good thing that they don't. So the fact that a new version 6.6.6.6 is published doesn't really impact many, many people because the amount of people actually use Faker as a new project or automatically get it by builds isn't that big. So something like this wasn't really hurting anybody, even though we wanted to make a statement as this open source maintainer did. But here's a note. This person, a maintainer on NPM, actually owns many other libraries, not just Faker. For example, colors is another important and very important to the ecosystems, but also very prolific one. And it's so popular used by Nessa dependencies that it has over 100 million downloads a month. So Mr. M actually owns both of them. Here's the maintainer of both of those. So what would we do? Perhaps inject something into colors if 1.4.0 is the latest stable version that gets those 100 million downloads a month. If we release 1.4.1, for example, a patch version, probably a lot of automated download tools will actually start getting this. And this is exactly what happened and also developers at least in JavaScript from what I've seen are very keen on just getting the latest and bleeding edge which is, we'll leave it to the end to talk about it. But what's in 1.4.1? We usually don't even look at a source code to realize what we're getting. We just want to be at a bleeding edge. Want to get the new features, the new bugs, new bug fixes. Well, here's one of the change logs. I'll go ahead and zoom in for some of you. This is what was added in 1.4.1. I realize that maybe not everyone here are programmer, but essentially by the for loop and a case of minus of less than infinity. You can understand that this is a denial of service of a loop that once you include and use colors 1.4.1 in your project, the moment that you do and this is getting used, essentially, this is a denial of service attack for anyone because nothing will actually continue working with your application, especially Node.js being single-threaded and so on. So this is what happened. Now remember we talked about Mary and Jonathan and Mina at the beginning and we said they are part of open source ecosystems. This is the fallacy. We do not understand, but they are. So let's see how far this went out. Well, Mary actually works at Amazon and she is working with AWS CDK, which got impacted because it uses 1.4.1. So all of the users who used Amazon CDK have actually had this problem coming in. Remember Mina? He is actually working for Heroku's Ocliffe. Essentially, that also uses colors behind the scene and if you were using that, you would be pulling in automatically potentially this denial of service attack to your application. How about Jonathan? He's working on Jest. That's like a test runner for JavaScript. Very famous one built from Facebook or Meta these days. So that's also been pulling in colors and all of these projects and many, many more. So we are very much part of open source and we cannot deny it anymore and all of those things are actually impacting us. So if something goes bad there, regardless of security vulnerabilities, like projects dying or in some way getting added some malicious code into them that just is bad, we are impacted by this. Let's see further out how impactful this is. So as they later published version 1.4.1 and version 1.4.2, you could see that really, really fast in a matter of hours and days the amount of downloads per each version. We're talking about more than 500,000, half a million downloads within a day or two. These and 1.4.1 and 2 are the impacted one. So it shows you how fast and how impactful this has been on the ecosystem if you have been impacted by colors. Well, let's talk about colors incident learning. What could we have done to have prevented this? Well, first of all, if you use a log file and don't just install everything all the time, then you have a lot of assurances, like a checksum, like a Shah, and all of those other things. But essentially, it allows you to not just get every install, new versions, Semver, whatever, that just went out yesterday, but you're always getting deterministic installs. There are specific commands in NPM that you have to follow both in CI and your development environment to make sure that indeed this is what happens, because you can still have a log file, but use different commands that override it. This is like a bad practice. The other thing is, if that would have happened to you and you wanted to mitigate it, if this is a nested dependency, not a direct dependency, which you can just update, if this is a nested dependency impacting you, then you cannot really do anything to change it. If another maintainer pulls in, for example, Jest, that pulls in colors, 1.4.2, you cannot do anything to tell Jest to do 1.41, until this feature from NPM, now supported by other package managers like Yarn and PNPM, and that allows you to specify this thing called overrides, which we used it also for, as you can see in the reference here, for the node IPC, which is, you didn't cover it here, but that's like another interesting protest work. Let's call it like that. So, this is not a way of doing that. The other thing that I actually want to share with you is, there are many automated dependency upgrade tools, like the one you use and see in GitHub, and there are probably other vendors that do it. The thing is, you have to be very conscious on what you're updating. We're just talking about now, about packages that have been harmed or just stopped in one way or the other, but they might by the maintainer. That's not even like an external attack vector, someone taking over a package. That's like someone that you're supposed to trust within the ecosystem, and all of those automated dependency upgrade tools that you actually get all the time, updates and updates, think about for a second, what if you could have been able to just delay the upgrade, right? And so, this is something that we actually looked into about three years ago, back at Snake when I joined, actually looked at many other malicious incident that happened, and we looked at the data frames for how much time it took for a malicious package to actually be known. So now, unless a dependency is needs to get a security fix, like unless this is a security fix that needs to go in, all of those updates that you will get from the sneak bot, for example, will be delayed by about three weeks, which is what we found to be a very reasonable time for you to update, just for new features or new bug fixes if needed, automatically, but also be a bit on the safe side, give you some buffer time, so the community can vet it and things can get worked out by a bunch of other people and more eyeballs looking at it, if to quote, para quote someone else. So, I said this is gonna be a fairytale ending, so what happened here? I'm gonna go back not to collars, but to Faker.js of the same maintainer. Luckily for us, Faker.js on the JavaScript ecosystem is a very big project. Like we said, many Fortune 500 companies have been using it. So the beauty of open source is that in the case of Faker.js here, this maintainer of collars and Faker.js, another group of people, amazing humans who came up and said, hey, you know what? We're gonna fork the project and it is very influential. There's like a lot to do. People use it all the time. It's a very good project and they have continued it and provided the roadmap and everything that they updated the documentation. It actually brought a lot of different forks and different people who maintained like documentation and stuff like that aside, brought them together so they can actually cooperate on a new place for Faker.js. So big shout out to this team. Big shout out to open source. To everyone here who's doing that and to that team specifically. I wanted to take in just an example of the metric. This is from a screenshot from yesterday. You could see that from just the last month. The activity that has been on the project has been to close down, merge 83 pull requests and more and more. So it's amazing to see also the continuation of projects in the open source ecosystem beyond them being sometimes discontinued in ways that are unexpected. So to end the whole stories here, I'm gonna give you a bit more takeaways unrelated to the specific stories. Well, first of all, I think when developers essentially look for packages, we use the currency of stars, the currency of downloads. If I want to look out for a package, for example, an HTTP library, a client-side, a client library for HTTP calls, I might go into a place like NPM.js and probably the one thing that I'll use to judge is how many downloads are there for it? Actually, it looks like this is a trending up graph event. So it looks like requests here, this library gets in about 15 million downloads by other users, by our developers. So I don't know, maybe it's safe, maybe I'll use it. Maybe, but there are more perspectives that we could look at around the spectrum of what makes a project, what makes a community for a project that are good indications that we shouldn't deny them. So on this one specifically request, actually got deprecated a few months after this screenshot thing. So what led to that is if we could have looked at something that gave us more information. For example, I open it up retroactively now, for example, in the sneak advisor tool, which is a free tool that you can use to look at packages and get more package health information. And you could see, for example, we look at maintenance and we see that it didn't have any commits or any releases in the last two years. So yes, it's getting downloaded 15 million times, but it's not really active. That's a signal that something could actually turn bad. It doesn't mean that it's bad internally or in any other way, but you probably do not want to continue new greenfield projects and might want to consider moving on to other HTTP client libraries on NPN that are not this. Let's take another example, that's the other side of the flip side of the coin. This is marked, a library, a JavaScript library that translates markdown and converts that to HTML. It's a pretty cool library. I've been on the access them also for a long time. And it had its share of vulnerabilities too, but you could see that many of the latest versions, first of all, there are many of those throughout 2021. And even though many vulnerabilities are being triaged to the project, they're getting fixed and new releases are going out. Also, you could see the pulse on the maintenance side of things, showing you that commits are happening all the time, showing you an idea that, again, is not bulletproof and doesn't mean that this is not gonna be discontinued tomorrow, but it's a good indication that says, this is a pretty positive, healthy project. Whoever heard about this mom, right? But I want to give you a different perspective. I want to lend a new opinion here for you to think about here. Not just the software bill of materials. That's fine, we talked about it a couple of times. That's okay. How about the fact that projects are not materials. They are built by humans, by people. They are very dynamic in nature. They change, they evolve. And so to think about humans as materials is a pretty bad concept, I think. So how about we talk about SBOH? How about we talk about the software bill of health? Not just what we use. Not just the vulnerabilities of it, but all those other signals that we may have in the project and around the project and how it's actually being used and consumed and does it have tests and so many other kind of signals that we can get into how healthy the project is as an open source project for us to use. So hopefully some of this has resonated with you. I'm here for a bit. Thank you for having me and use open source safely. It is great. It's amazing. That's it. Wanted to end with a bit of an ending here. This is the one. So the only things that last forever are not really dependencies, as we've seen, and not really air miles. So I hope you used all of those when flying here on COVID and all of those things. I think it's cat videos on YouTube. That's the only thing I could think of that probably lasts forever. And so with that, really thank you so much and thank you for having me. Any questions? Eager for the happy hour. Not sure yet. Okay, cool. We'll have fun people and enjoy the conference.