 Thank you for coming today. I know it's the last talk of OpenJS World, so give everybody a round of applause for making it this far. I quickly just wanted to thank everybody that made this happen, so everybody that's behind the scenes. It's awesome that we can come all the way here and I still see familiar faces, which is great and some new faces. If you want to follow along or you want the slides, there's a QR code there. If the link doesn't work, let me know. There's also a Bitly link there as well. If you still use Bitly, you like it or you don't. Today, I'm going to be talking about the future of JavaScript package management and hopefully most people here care a little bit about JavaScript and package management in some sense. Hopefully that's why you're here. Give you a little bit of insight about who I am. My name is Darcy Clark. I've worked with a lot of brands, big names in my tenure as an engineer. I've also, most recently, worked at GitHub. We were acquired via NPM acquisition in 2020, so worked very closely with and managed the NPM CLI team as well as the GitHub CLI teams at GitHub. I also co-founded a company called Themify. Anybody ever do any WordPress work in their lives? Yes, some of you, hopefully. You use the WordPress blog, I'm sure. Yeah, that was a company I started in my early 20s. I'm very proud of that as well. I also do a ton of work in the open source. I work with a bunch of folks here. Contribute back to OpenJS Foundation, know it as much as I can, and I'm based at a trial. In December, I left GitHub to start a new company called Volt. There's some stickers up here if you want them. I'll talk a little bit about what we're doing at Volt later on in the presentation, but you can also check out and sign up to get early access if you want at Volt.sh. So the agenda of this talk is basically broken down into two parts. We're going to talk about the current state, which you may know of about package management within JavaScript, and you might know some of the issues that maybe you or your teams are facing. And then we're going to talk about the future state and even maybe some tools that you haven't heard of that exist today. And we're going to learn some scary, interesting things along the way. So hopefully you're excited. I want to start this talk with a quote from Steve Jobs says, anybody ever seen this quote or heard of this sentiment? He back in 1997 was quoted as saying the best or sorry, the line of code that's the fastest to write that never breaks that doesn't need maintenance is the line you never had to write. And I think that our ecosystem if you write JavaScript has like taken this to heart a little bit too far in terms of the dependency stack. But I do think it's true right like leveraging a strong web of trusted dependencies can really alleviate a lot of headaches that you have in writing code yourself. So the results of this obviously are a huge ecosystem. MPM is the largest package registry in the world. There's over 2.5 million packages there. It sees over 210 billion downloads for those packages every month, which is just insane. So we've been very successful at consuming other people's software and not having to, you know, repeat ourselves and reproduce a lot of the same code. And primarily how we got here, especially in terms of those really large numbers is actually through transit of dependencies. So primarily when you think about, you know, your huge node modules folder or the packages and dependencies you're relying upon, there's actually way more. It's an exponential factor, transitive dependencies than direct dependencies. And then this is just a quick diagram of what that means when we say that if you haven't heard that term before. And actually, we got some data back in 2020 when I was still at GitHub about the actual state of transitive dependencies within GitHub repos. And we calculated that there was on average 683 transitive dependencies in JavaScript projects on GitHub. And when you compare that to pretty much every other language, it's just a crazy number of dependencies that you didn't rely on directly that are coming along for the ride, which is kind of scary. Of course, you probably have seen memes as well. If you've been in the industry long enough, you start to make these jokes yourselves. Yeah, so you start a small project and then you layer on a whole bunch of olive oil, aka node modules. And of course, the one that kills me all the time is your node modules are like the heaviest thing in the universe, which is pretty sad. So one solution that some people have talked to me before and I've actually seen teams try to implement this is only to concern themselves with the actual direct dependencies. The only things that they can control, they know that they can update them, they actually know them by name, right? You know that you rely on React, you know that you rely on Next.js. And so you can easily manage and update those things. Unfortunately, it's just it's not it's not reasonable to take that strategy. It's estimated that there's same 5% of all the actual vulnerabilities that your code is going to come up against or residing in those transitive dependencies. And unfortunately, year over year, the attacks on the supply chain, the tax on on your package, the package ecosystem is growing exponentially. So year over year, it's grown by 742%. What that looks like in a nice graph is this. So if you like skateboarding, this is like cool. I got a half pipe and it's getting worse. We're seeing, you know, hockey stick growth in this area. And when primarily these issues are arising in transitive dependencies, we really need to think about solutions at that scale. So whose problem is this? You're not going to like the next slide, I'm sure it's yours, right? Everybody here has felt this, I'm sure. But licensing within open source, if you hopefully you've read many licenses of open source projects that you consume. They're providing you this software as is. And so when we talk about supply chains, we've talked about package management. We really need to think about what's the dynamic that we have with the maintainers, quote unquote maintainers, but essentially the creators of the software and what are the expectations that we're putting back out there when we are talking about package management tools and solutions. And of course, the entire ecosystem, the industry has moved and pivoted to consuming more open source. 90% of the companies out there, and I think it's larger than the stat, use open source. And so really becomes like a you problem and it becomes like an us problem. Right. So all your teams have to worry about, you know, supply chain attacks, they have to worry about the state of their dependencies. And I think just to sort of encapsulate this whole trade of thought, I think it's because we have a really tough time, both as management, but as teams as individuals, actually. Calculating the total total cost of ownership. If you've ever had to do this for anything, there's like a quick equation that can help you kind of like visualize this, which is like the purchase price is only like 20% of the whole right like the 80 20 rule. The rest of the cost is actually hidden behind the scenes right so you buy a lemon as a car, you're actually going to spend 80% of the money that goes totally into that car into maintenance and fixes. And when you're actually consuming from open source, that this equation like completely blows out the water, because you haven't paid anything for the software and so 100% of the costs of basically consuming from open source are in the maintenance and supporting of that software. And it gets it's sort of getting to I think a precipice in terms of how developers are looking at these problems. There's recently been surveyed, tidelifted a survey and found that up to 50% of teams were spending sorry up to. Yeah, up to 50% of the time developers were spending on managing tool chains so they were having to deal with alerts security alerts vulnerabilities having dive into updating essentially deep tech stacks, and it's like really eating away at actual time of building business logic. So what's the role of the package manager in this sort of crazy Wild West that we still are in. It's kind of a Swiss Army knife, and it does a lot of things if you've used npm for just installation you're only using one piece of this Swiss Army knife. But yeah, a package manager helps you with discovering packages, authenticating yourself against configured registries permissioning building tooling testing. All these capabilities usually come into a package manager. And we think about sort of the larger ecosystem. There's actually a number of tools that have started to get into sort of package management in terms of what they provide as capabilities. So you can see dependent bot and men's renovate bot which you might be using in your CI environments. Those are helping you sort of manage your dependencies right so when we talk about package managers actually we're sort of expanding the scope of what we mean when we talk about that. Of course I'm going to speak primarily to my experience with npm, since I managed it for three and a half years. It's also the biggest package manager out there as far as I know we made up roughly 2% of all the registry traffic in our downloads alone for npm. And so yeah, I kind of look at like this like npm is definitely the biggest of the Swiss Army knives. So let's look at the good in terms of the current state of package management. So almost all package managers now have the concept of workspaces you might be using mono repos as the terminology but basically letting you organize and orchestrate multiple projects in the single repo. And so all of the package managers there up there all support some form of workspaces which is awesome and it's pretty easy to find these and sort of manage package installations across and updating your dependencies across all these projects. So some of the benefits obviously you can run scripts and sort of coordinate changes across these projects all in one which is great for why you might want to use workspaces. It's also easier to do troubleshooting as a maintainer of open source software can be a lot easier to do issue trials as well if everything's in one place. Current state of overrides as well. If you've ever used yarns resolutions npm has overrides. This is also great, you know, benefit today you can essentially use specific field in your package JSON to define essentially a way to resolve your dependency graph different than we would normally. This is a, you know, better. There you go. Getting a lot of tap. So the benefits of this again are you can do specific versioning patching and forking of dependencies, and you can block bad or no or bad known versions of packages so you can basically take control of how your dependencies are resolving. And again, pretty much every modern package manager has these capabilities now. And what is gaining a lot more traction is a recently ppm sort of rolled this out is the idea of a globally linked global store of packages. And so pretty much every package manager, including npm this past year, shipped a subset of the capabilities for a linked store support this and it's great in terms of sort of performance installations if you want to speed up you have like, you don't want to have multiple copies of the same packages. It saves on file system size, it's pretty awesome and you actually get safer module resolution. There are some downsides in terms of the legacy ecosystem sometimes is expecting your known modules folders to be in certain places and and that is both. I think it's safer technically to use a global store that's being linked, just because you really shouldn't be relying on something that you haven't defined as a pen to yourself. So that was the good. Now let's look at the bad of the current state interoperability. How many people have tried to migrate from one package manager to another ever. Okay, how many were successful. Somewhat. That's really sad. Like I think that's a huge issue. Most of the package managers that you've heard about ever since npm in the JS ecosystem have coined themselves as npm compatible and it's just not true. And most recently bond has done this as well. Their marketing was that they were dropping replacement for node and a dropping replacement for npm install and it's just it's not true the capabilities compatibility isn't there. And so it actually takes a lot of work if you want to migrate from one package manager to another, which is super unfortunate. What's the current state of search and discovery. How many people have tried to like find a package on npm through search. Yeah, most some. Have you ever been like confused about the entire UI experience like what those little like bars are and the PQM scores like. Yeah, yeah, it's okay. You can say yes, I used to work here. This was my problem at one point. Yeah, right now when you go to search like you have very limited ability to drill into metrics that you might care about for the quality of or just search and discovery is generally really limited in terms of finding good packages. So you end up going to Google usually and then you're like okay well I found finally found a great package and now I have to like just pray that some of the information on this page. Makes sense that it's going to be a good good package that downloads a lot of other people are using it so I should use it right like that's that's I'm I've done that many times that it's you know it's 20 million downloads it must be a good package. I can install it. At least we're all in it together if we're all doing that. So it's also the state of authentication. Well if you run MPM login the last decade. What you might have seen is that it actually generates a auth token in the registry itself. This is good. This is essentially acts as a almost like a session token similar to if you logged into a website. And when you actually run and log out you'll remove that just like you would expect a session token to be blown away. But you might be surprised by the fact that the token that is created is actually like a published token which is the most permissive of the token types. So you get read and write access when you do MPM login and also it doesn't have an expiry which is a bit scary. Especially when you run login twice and you don't ever run log out it actually generate will continue to generate tokens without cleaning up the old session token. So this is something that I tell folks to please go look at. You might actually have a whole bunch of tokens that are still valid and have not been expired that we're all based around login and this flow. This what I consider to be inaccurate usage of tokens as session tokens. And of course an easy way to figure out if these are web or sorry MPM login tokens is their name will start with like an MPM underscore they won't be named. And so you'll want to want to delete these current state of installation which is probably the biggest area of concern and where we spend most of our time. You probably think about when you talk think about package management and there's been a recent trend of sort of this no install behavior. Originally Tink was an experiment that former MPM engineer worked on yarn has a variant of this and deno and bun most recently have added basically a no install behavior into the runtimes themselves. And so I like to say this and I hope this catches on at some point that no install is basically the new serverless. So there's a server somewhere right serverless was just it's branding and just like no install well you still have to install the packages at some point. They're just making it harder for you to actually find where they're putting them which I actually think is a bad thing. And so when you hear about this behavior when they say that's a feature and it's great capability that they're doing all this work for you automatically. Magically just be mindful it's actually going to make it harder for you to do introspection build great good security best practices and have good hygiene around your pensies if they aren't really making this visible to you. Of course when everybody thinks of MPM install they probably think of I run the command magic happens my files end up on system right. And now with this new sort of no install behavior you know I run this program files up and up on the system and then program is run. So I want to quickly look to show you at least one example that I put together actually for the last open source summit and things haven't changed. This is a create react that projects where you can see it actually has seven direct dependencies so these are all things that you might know by name. And I ran essentially a test against all the different package managers just to give you a sense of sort of what happens when you run install against a standard you know project. And this is what happens the number of dependencies varies widely across all the package managers and this should be really scary to you if you work with teams that might be trying to transition from one package manager to another. Or you find that you know people are using different versions of the package manager. This is a huge problem I think for the security industry is for us we should be mindful that when you say you just ran install with yarn it's not going to give you the same packages as if you ran with MPM etc. And so there's a difference of over 850 between the largest and the smallest number of packages downloaded. So you might be like what like you're this is crazy I've never heard of this. This is insane that there's such a difference in the way that these tools interpret my projects and the dependency graph. And it's because of these things and a few more but package managers take liberties and make changes to the way that they behave around different types of dependencies. Optional dependencies peer dependencies how they implement the overrides features and capabilities we were talking about before. They also may or may not support life cycle scripts which opens a whole whole can of worms in terms of the arbitrary code execution that can happen to install even more packages from one one package manager to another. There's also a problem here in terms of the package specs itself. There's non determinism and sort of mutability in the references we use for packages. So if you've ever seen a package reference that says you know pkg at latest that's referencing essentially a distribution tag. It's not pinned to a specific version and that is a mutable reference that can change over time. And so there's other ones like that that are even more risky in terms of remote tarball your balls that actually can be found in packages and package references that essentially alias you know third party remote packages that might be installed as react or something like that. And this is all supported by every package manager that you have out there today. So it's a bit scary when you think that you could potentially be hitting an unknown host or untrusted host for your software. Again some people say well just use a lock file it should protect you unfortunately lock files aren't completely immutable. And so they do not always reproduce the same installation each time and time again lock files do not actually store configuration in them. So if you want the same install to happen in your CI as well as it happened in your development environment you actually have to pass the same configuration options to that environment. And this is something that has bit quite a few people especially as MPM specifically introduced support for peer dependencies. People found that they could not install using MPM CI with just a lock file loan because we were not storing information about the configuration they used and they would have to continue to pass the flag legacy peer debt pansies if they didn't want to install peer depths. And this kind of exacerbates this whole problem of this inconsistent mutable behavior actually gets exacerbated by the fact that we don't check in our node modules. I'm not sure some teams here might be doing that which amazing if you're doing that but what you actually end up having mostly doing is pushing code minus the node modules into your repository in the cloud. You expect the CI environment then to build and install and then you have no clue probably what transitive to pansies were installed in that environment and then shipped production. And so we're really removing ourselves from the problem which I think is kind of scary. And then there's this great meme life is like an MPM install you never know what you're going to get it's very true. And it's it's sad. Cranstive security and auditing tooling within package management is also really poor. Unfortunately I don't mean to be a super big downer. But back in twenty twenty one Danny Vermont did a really good write up about MPM audit and the CV ecosystem that has been built up and that's respected by the audit tooling which has just created a ton of noise for maintainers and for our ecosystem. And this is actually really good read if you get a chance to if you haven't already you should read through it. But again going back to our example I couldn't even just show you right here that MPM audit the former tool I used to manage does its best to show you that you know you are potentially vulnerable to some vulnerabilities. And it says oh there's six you know high vulnerabilities here's an action to take here's some automated tool to essentially resolve this. And for the people that know don't don't say anything but does anybody know what happens when you run MPM audit fix or can you guess what happens. No no takers fix the issues OK right that's what should happen for sure that's what should happen. And so you're like OK cool I'm going to run the command and what happened we actually ended up with exponentially more vulnerabilities than before. Right. And and so this is a current behavior that you might have experienced. I'm sure you have run into this before I did pull yesterday in a room. Every pretty much everybody I knew that said they had run MPM install said they got warnings during install but almost none of them said they were able to resolve it with like MPM fix MPM audit fix. And so this is a huge problem this is the state of the largest package manager in our ecosystem and it's not alone in terms of the problem that this problem trying to navigate updating them to panties and patching to panties. And of course like to add insult to injury MPM shows in the log output oh would you like to run MPM audit fix again. It's like no I don't want more vulnerabilities right like it's it's you know it's like stop hitting yourself this is this is terrible. So I feel really bad for like new engineers into the ecosystem because they probably run in with these tools and they just are like I have no clue what's going on here. I have no clue why this is going on and I'm and I even have a hard time reasoning about this. And unfortunately a lot of people just stay silent and it's it's becoming a problem. And it's becoming a problem because you're likely to run into security alert in the next year. It's almost I can almost assure you with more than 60 percent probably 100 percent accuracy that you're going to run into vulnerability or alert in your projects within the next year. And it's because we continue to add to the malware advisories to the the CV advisories that we have in the database GitHub's database and there's few different databases out there. And on a really interesting stat if you're just thinking that you're in the wrong industry or you're in the wrong room you're not. JavaScript makes up 80 percent of the Panda Bot alerts on the GitHub platform. So at least you could say we've got a lot of work to do and you're going to be able to be employed for a long period of time because we got a lot of work to do. But it also just shows the size and scale of the problem and specifically our ecosystem obviously being the largest that this is truly a problem and an area that we should be the ones to pave new paths in. And good example of this as well. This doesn't need the numbers I showed before don't even include it most recently a spam attack that happened to the MPM ecosystem where over 500,000 packages were added and then removed by the MPM team like month or two later. And no there was no advisories put out for those. So if you happen to accidentally install one of those there just isn't an information out there telling you that you actually have a spam package which is pretty bad. So we're not even documenting all of the issues in the ecosystem. Chris they have CI solutions obviously not much better. We have two of the biggest names the Panda Bot and men's renovate bot. This is just an example from my own experience. Oftentimes I was seeing you know 300 open pull requests on average you know hundreds of open pull requests for updating my dependencies. And this is after being really diligent. My team was a really small team and we had closed 3000 of them and we just continue to have this backlog of updating dependencies and having to manually install PRs. And it's just crazy to think that we're going to have to stay on top of all these pull requests. And on average we're seeing you know somewhere in the range of like four day or it takes four days to to actually review pull requests within teams. So these these potentially vulnerable unpatched dependencies are sitting there in your backlog and they're just they're just waiting and you're hoping and praying that there's no sort of exploit that happens. So the turnaround time is just way too long as well. And these are a bunch of the exploits that we've seen talked about quite a lot and not just vulnerabilities but there's there's kind of compromises. And most recently I actually put out a blog post about manifest confusion which is sort of well dive into next. So I've been talking about a lot of bad things. Take a breath. Right. It's all going to be OK. But MPM is broken. Fundamentally like absolutely fundamentally there's a there's a huge problem. I won the jazz party podcast to talk about this blog post they wrote. I tried to advise GitHub about this and I've been talking with all our security companies. But essentially when you publish a package to MPM when anybody does that there's a behavior in which you pass both a manifest file which tells MPM about the contents. Or dependencies and all the other information. And then there's the tar ball the actual packed package. And what has been documented and what you've been told for the last decade is that actually the tar balls package JSON gets included in the manifest information. And it does it does actually do that if you use one of the package managers they'll pull out that information and then publish it. But because you can essentially publish these two things independently it actually causes a problem with the source of true source of truth for this information. And so what you get is actually inconsistency between the manifest information and what's actually in your package packages. And so it's kind of be like going to the grocery store and going to the checkout like the automated checkout and saying like here's my receipt for this. Apple but really inside the package is a car and you walk away and you just stolen like a car. And so really what MPM should be doing this whole time is validating these two things against each other and it hasn't been. And so it's a huge problem. Almost all your caches or should be invalidated likely. It's a huge problem within the ecosystem that is just it crushes everybody. The other option instead of just invalidating or validating these two things against each other and throwing at the time of publish MPM could have also essentially pulled out extracted the contents like you probably thought it was on the server side and then distributing good or bad packages. So this affects everybody. This affects every single security product you probably use because most of them end up using the manifest API and not actually pulling down the package and extracting the contents. But unfortunately all your package managers actually do just that they pull down the actual package. They use the contents and so you have this to the two sources of truth that are not that oftentimes actually are conflicting. There is hope. If you've heard about socket socket dev they actually were one of the first security companies to come up with a detection for this. I highly recommend checking out their product. They are one of the few companies that have actually solved this and I've talked with quite a few very large companies about this. Let's quickly talk to you a future state of tooling because I think I'm running out of time. What do I got time check. Anybody know. Five minutes. Okay. I will try to go fast just like this says. Future of tooling right. Everybody's been talking about bun. Well I can give you some peace of mind. Like I said it's not a drop in replacement for MPM or node in the last month. There's been over 600 issues filed against bun. Unfortunately they're taking the approach of really make the tool fast and not necessarily make the tool right. And I really hope that people are interested in making tools that are right. So there's this concept of reproducible installations within MPM. I've proposed a new package distributions. Conceptually this would be codifying the need for post install scripts. So oftentimes the reason why you actually do see post install scripts is to install some sort of native specific variant of a package. But you can also get even more detailed with this in terms of specific variants that might be for production or development etc. And so this is a huge opportunity for us to add a feature which takes away a security risk. And of course what we're talking about or what I care about most about is standards and introspection. And I look at sort of the ecosystem today as kind of like the wild wild west of or the browser wars back in the day before we had HTML5 and sort of a standard way of interpreting your HTML documents. We didn't really know how the browsers were handling all these edge cases and errors. And so we need something like that within the package management ecosystem so that you can have safety and confidence that your projects are being installed the same way all over the place. And so that requires essentially codifying and creating specs for different pieces of a package specifically the package JSON. There's some efforts in this today that are going on. I'm a part of some efforts in the OpenJS Foundation and NodeJS working groups. Also we need to essentially codify and standardize semantic versioning because this package specifier needs to be interpreted by all these tools in the same way consistently. So again we go from hopefully a place in the future where you run NPM install and the magic happens to a place where you run NPM install and you actually know there's a codified playbook for how to handle the resolution of your dependency graph. And then we can have great tools because that and specifically one that I created is called a new language that I specced out is called DSS. It's a dependency selector syntax. It steals from CSS. So if anybody writes CSS here or care or like it, I used to love CSS. But essentially this gives you a new expressive way of actually querying your dependency graph. You can do some amazing things with this. You can create policies, you can do auditing, you can do validation, you can do all the type of sort of linking. And so here's a whole bunch of examples of how you could essentially write a query that does introspection into your dependency graph. And if we had a standard way of creating that graph then you know this language could permeate it into a whole bunch of tools. So here's some more examples of just how you could find different licenses with the query syntax. It uses attribute selectors. So again, if you've seen CSS as a JavaScript developer, this is going to feel really familiar. And you'll even note that we have like not selectors and has. Has isn't even in Firefox yet as far as I know. So this actually launched in MPM. You can use it today. You can run MPM query against your projects today and start actually doing this validation. Cool. And actually just like in the DOM in the browser, we created actually a programmatic way to run those queries via the MPM CLI Arborist Rule Library. You can run query selector all and pass a selector, which is just amazing. So again, it's MPM query. We're hoping that catches on. There's validation that eventually this will unlock for things like CVs and vulnerabilities. So net new syntax can be invented that looks like pseudo classes, but they become pseudo classes and pseudo states for free packages. And again, if you think of an amazing policy engine, maybe you have JFrog or you have something else today that you're writing really verbose, weird syntax for your policies or gating installation or warning for your teams. This again is really expressive to essentially write those policies and choose what you exactly want to happen. And quickly going further, I got one, two slides left. I'm building a new package manager and registry. It's going to be all of these things and more if you care about that kind of stuff. If you care about any of these issues and key takeaways here. I hope that the future is fast, open source, standard, accurate, secure and hopefully sustainable within package management. And yeah, thank you. I don't know what we're at at time. On time. Okay. Two minutes more. Okay. Well, I could go back to any slide or somebody has any questions. Let me know. Go ahead. So I come from JVM world and like, you know, Scala Java, Haskell, Rust, Python. So it seems to be pretty reasonable, you know, ecosystems. Why is JavaScript so kind of prone to the package manager management problem? Is this the fact that it's moving very fast that, you know, CI CD is not as solid? I wonder, like, is there a difference in culture? And like your technical solutions, do they reflect this difference? Right. I just want because like, just to give you an example, right? Like in Java is used in a bunch of banks and people are generally very paranoid people who use Java like strong types. Right. I like strong types. And so the attitude, right? Like if you want to sneak a jar in the bank, like you can use Maven because it's far world. Like it will be put in a holding pen and inspected by security. So people around kind of JVM ecosystem, they have this kind of paranoia built in. Maybe that's preventing them from this kind of security exploits. I just wonder what like, what do you think is kind of cultural traits of JavaScript ecosystem should enable this kind of abuse? And which technical solution will attack it in the best way? Sure. I'll answer this one. And then I think that's probably all we have time for. I think the biggest reason why culturally and within this ecosystem. We have this problem is because we pivoted from a package management ecosystem prior to MPM to that was fully gated. If you ever tried to get your software into a package manager, it was essentially a library that was gated by a few privileged few. And in fact, a lot of ecosystems are still like that. And when MPM was introduced, they essentially unlock the ability for you to distribute through the package manager, almost any software, like the barrier to entry into the JavaScript ecosystem to distribute software is very low. Like there's almost very little criteria that you have to, you know, and that's why it's easy to automate attacks like you saw. And so because the barrier to entry is so low, you can essentially automate this. And MPM never built into it any kind of like staging, like to staged review process for adding packages into the registry. And so what you get is actually a lot of low quality packages. So that number is nice and big and we can all feel good about it in terms of how many packages are in the ecosystem. But the actual quality of a lot of them is probably pretty low, unfortunately. But that, you know, it's a two sided, there's trade offs there because you really do want to add mechanisms to allow for people to contribute and start to get involved as quickly as possible and not to have sort of what we had prior to MPM, which was a lot of gated communities. And but there's there's definitely more that could be done. So awesome. Well, thank you folks and I'll talk to you soon.