 Hey everyone, my name's Danny Makovay. I am a co-founder at Jaff Labs or just another Framework Labs. And today I'll be talking about package management for WASM components. So where this work is primarily being done is actually in the SIG registries group under the Byte Code Alliance. The project that we're working on is called WARG, which is named such for the WebAssembly Registry. So WARG is a protocol spec that is currently in development, and it has several interesting characteristics. It's got some cryptographic verifiability, and it's also federated. And some of these things are things that people have already presented on publicly at other conferences in the past. My mayor Lee Kyle Stone has definitely talked a lot about the cryptographic verifiability stuff in the past, but essentially one of the things that we have now is a reference implementation of the WARG protocol. We do anticipate there being many implementations of the WARG protocol, but right now there is a CLI that you can interact with, and there's a reference server. And what this is doing primarily is providing a package management experience for you using WebAssembly. And the folks here are the people who have been attending the WARG meetings most recently, but we've had different people who contribute quite a bit that come and go. So I guess for all the developers in the room, most of us are probably pretty intimately familiar with at least one package manager. If you are a JavaScript developer, you're probably using NPM. If you're a Rust developer, you're probably using crates. Basically every programming language has a package manager that is a pretty integral part of the development process. When you have these categories of problems that you know there is somebody who's already implemented it, you go and you reach for your library and you use that instead of rewriting a REGX parser or base 64 encoding from scratch. But now with WebAssembly and some of the language interoperability characteristics that it has, it kind of makes it interesting to think about package managers a little bit differently than we have before. I decided to share a tweet that I have from a recent friend of mine, or a recent tweet from a friend of mine. He was basically working on a Go project and had a need to, yeah, format Rust code inside of his Go and nobody had written anything that does that in a Go package. So he decided to go ahead and compile Rust to Wasm so that way he could use the Rust formatter inside of his Go code. And that's pretty cool and kind of completely showcases part of what people are really excited about with WebAssembly as this language interoperability. But also nobody should have to do this ever again now that Shiv has already done this. He should basically be able to publish this to a registry so anybody could go and reference his Go code and that will bring in the Rust formatter as a dependency and nobody has to compile it again because it's already been compiled. And kind of I guess what we're seeing is what we're not accustomed to is when you go and you look at your package manager because of the language neutrality characteristics of WebAssembly, you actually have utilities available to you that aren't authored in the language that you're currently writing in, which is pretty cool and pretty exciting. And what you'd end up with is a dependency tree, very much like the one that we have here, although probably much, much bigger in practice. But you'd have somebody who's like authoring the package all the way at the top and in this case, they've got like five packages that they depend on directly and then those depend on more and then those depend on more and this could go on and go on and go on. And again, these would all be implemented in different languages. I would like to take a moment just to like appreciate some of the things that hold for package managers that we don't necessarily comment on all the time because I guess we take these things for granted because of how integral this stuff is to our development workflow. But I guess just the person who's authoring the package at the top familiarizes themselves with the code that is exposed from their direct dependencies. And their direct dependencies also familiarize themselves with what is exported from their direct dependencies. But they're kind of shielding the author from having to go through the mental energy of understanding their dependencies themselves. So even though you're using the code that is below the people that you depend on directly, you are totally not needing to consider any of the things that were actually done. And I guess this is largely what is responsible for our sense of productivity as developers and what prevents us from having to boil the ocean every time that we sit down and write code. And it's also largely responsible for how we measure the maturity in a programming language ecosystem. When there are a bunch of packages that do things for you, that's how you know that this technology has matured. And when people say that something has not matured, they're essentially saying, I expect there to be an HTTP server. I expect there to be a way of interacting with SQL. And that doesn't exist yet. And so maybe I don't wanna be an early adopter here because it's too much of a pain because there aren't a bunch of packages for that language yet. So some of the functionality that we wanna see here in the Warg registry is a place where you can go and you can discover libraries that have been published at Warg, package versioning, so that way an author, when they modify a package, can do a version bump and then you can opt in when you're ready, but not before you're ready so that way your code doesn't break. We'll also talk about some of the reproducibility that we have via locking, which is gonna be essential for production deployments, module linking and bundling so that we have one large executable that is created from all of the dependencies that are referenced in the registry and then also language neutrality because again all of these packages might have been created from distinct languages. And so the benefits that we have is getting to leverage existing ecosystems, which I was talking about before, only again cross language this time, things being much more easy to distribute and making it easy to generate deployment artifacts. I do think that as we've been seeing these conferences, we do see people showcasing a lot of really cool WebAssembly stuff and then people, one thing that holds people back is talking about being able to actually recreate things and not have to go through these complex readme's and familiarize myself with all these new concepts and instead I should be able to just type something into a terminal and it should grab all of the things that I need and configure everything that I need and that way it'll be easier to solve and easier to reproduce. And then one last thing that I'll mention is that this has the potential to make it so that there's less burden for new languages. Again, going back to this maturity thing, a lot of the time when there's a new language, yes there's a grammar and there's a syntax that's well-defined but if there aren't these packages that I expect to exist that exist in my programming language, I'm going to be hesitant to go and adopt this language but if I'm able to actually take advantage of utilities that were written in other languages, a potential pretty cool side effect would be that we might actually see more emergent languages happening more easily because there's a reduced barrier to entry for them. So, pretty exciting stuff. I do also, before going into exactly how the package manager works and demonstrating some of its capabilities, want to talk just a little bit about why the component model specifically enables us to actually have these transitive dependencies for the first time. We have seen various people hosting WASM in various ways online but you're typically just like grabbing an executable and executing it and it's like flat and there's none of this transitive building of all the things that you depend on. So first, just a little bit of vocabulary. Generally people are writing code in a language and they're compiling that to WASM and they'll be referring that to that as the guest code so that would either be the language used to generate the WASM or the WASM itself and then the host code is the code that's going to be using that WASM and taking advantage of the functionality that it offers. So, a really common example that might help to like pin things down as I'm talking about this more concretely is like maybe you're a Rust developer and you want to have something that runs in the browser. Generally, Rust can't run in the browser so you're gonna compile its web assembly and then import it in JavaScript so that way it can run in the browser. In that case, JavaScript would be your host and Rust would be your guest language. And here we have just an example of some Rust code so we see these like WASM bind gen macros that are and there's this extern keyword which is essentially saying hey, this is a function that I expect to import from my host. So this would be like a JavaScript function in this case. And I don't have a function body that I'm defining, instead I just say this is what the inputs are that it expects and what the output's gonna be and then I'm able to execute that JavaScript code in the middle of my Rust code. And then I can also, and the other WASM bind gen macro above our function, we are able to export a function and then our JavaScript host is able to actually execute this function that was implemented in Rust inside of the middle of our JavaScript. And so here we have logic passing across the WASM boundary. But something that you, well, I guess before I go into that, when you are actually doing this and you don't have access to bind gen tooling, you do have to think about things in terms of bytes in memory, which can be like quite tedious and challenging and intimidating for some people. And so we do have this bind gen tooling that makes it so that you don't have to do this. But essentially without the bind gen tooling, what you are passing across the WASM boundary is pointers and lengths that specify, hey, here's a string, this is where it is in memory, this is how long it is, or here's a list, or here's a struct, and it can be a little bit challenging to use. So we luckily do have quite a bit of bind gen tooling that does a lot of this heavy lifting for us. The reason that I bring that up is because when you are referring to these things in memory, the way that you do it is actually going to be dependent upon the memory layout in the guest language that you're doing. So the representation of a list might be different in Rust than it is in Go, than it is in JavaScript, which are all these languages. And so you kind of immediately encounter this thing, which is that you don't have guest-to-guest interactions. You only have the ability to pass this logic across the WASM boundary when you're using core WASM from the guest to the host and vice versa. But if you start thinking about what it would take to be able to use a dependency tree, like the one that we had initially when talking about package management, you kind of immediately need guest-to-guest interaction. Each of these arrows going from one package to another package is a guest-to-guest interaction. And so if you didn't have the ability to have guest-to-guest interactions, what you'd end up with is actually each of these packages going up to our host and going, hey, JavaScript, can you please enable me as one of your node modules, sort of crates or packages, can you go talk to this other dependency for me? So for every one of these arrows, there would actually be multiple arrows going back up to the host. And the host would be basically translating between all of these different memory layouts for us, which would become really unwieldy, really quickly, and to the point where it would be impractical. And so luckily we do have solutions that are made available to us by the component model that enable us to have these guest-to-guest interactions, which is like what's enabling us to have these transitive dependencies for the first time. And the way that it's essentially achieving that is via this canonical ADI, which is essentially the memory layout that you're using when you're in the component model. And the way that you refer to it in a human-readable way is kind of with these wit files where you're referring to data structures that map very well to the data structures that you'll tend to encounter in your guest programming languages. So you'll have a record here in our example with one field that's a U32 and another field that's a string. There's structs, there's errors, there's results. There's all these things that you would expect. Perhaps there might be some academic language that has some interesting data structures that might not map well, but for the most part, I haven't yet to encounter a guest language that doesn't map well to these wit interfaces. And then essentially, within the component model, you do have these additional instructions, canonical lifting and canonical lowering. And what they're doing essentially is enabling you to pass from the memory layout of the guest core wasm to the canonical ADI. And so you do that via these options, which is the string encoding, memory and an index, and a reallocation function. So you do have as a guest this responsibility of providing a reallocation function, a lot of the time the component model tooling is going to handle providing that for you. It kind of just depends on what the tooling is for the guest language that you happen to be trying to interact with. So now that we know how the component model enables guest to guest interaction, which is what enables us to have transitive dependencies, which is what enables us to build things via the way that, to meet the expectations that we have of a package manager generally, we can talk a little bit about how we go about doing some of the things that we do here. So in the component model, basically you'll have one component that like imports another component that might import another component. And when you do this, you'll actually like go, want to go and instantiate each component in order to make it executable. And what's interesting is that you can actually instantiate each component like multiple times and each time that you do this, it will bring its own memory with it. So it'll actually kind of be like a mini process, which is interesting. And it's something that we will likely use down the line to do more interesting locking of these components and creating these like build and executable artifacts that we like are actually doing. But so like here, you have one package that like has no imports. So you're able to just instantiate it. And then you have like another one that has one import. And so you can see that when you instantiate it, you actually provide the first instance as an argument to the instantiation. And that's how it knows what its import is for that instance. And we also have a couple of different import statements that are like there are syntaxes that are available in the component model. One that is like stable right now is this interface import, which like if you've done some compilation of guest code to the component model, you've probably seen this. And these are generally going to be referring to WASI interfaces, which is the thing that you can talk about a lot. There are several talks about it here at the conference, but essentially host functionality that is like, you know, providing things similar to like a virtual file system for you that you're able to depend on. So not quite the same thing as like an actual dependency, but something that your runtime is going to provide to you. These other two syntaxes that are used for creating the artifacts and the executables from transit registry dependencies are actually not merged into the main branch yet, but you know, we are discussing them. And there is an implementation that I will be showing off in the demonstration today that you know, does work. And there's just a couple of things that we're deciding before we merge. But once we do, then other people should be able to interact with the stuff and then, you know, do the things that I'll be doing later on. But we have basically an unlock dependency which is going to enable you to use a semver version range. So that way, you know, if you wanted to, as people are publishing new versions, you could like automatically opt in. And then we also have a locks dependency, which, you know, that's going to be pinned to a specific semver version. And because of that, you're actually able to have in the import statement like an integrity shaw, so that way you can know that you actually got the content that you were supposed to get from the registry. Yeah. So what you'll end up with then are kind of these unlocked components and these locked components. So the unlocked components are basically going to be what you would see if you're browsing like through a UI for the registry. These are the libraries that are available to you and they're basically inside of them going to have imports that are unlocked imports that have the semver ranges. You can also specify like an exact semver version. And then you'll also be able to create from these unlocked components, locked components, which are similar to like a lock file in a lot of different programming languages. But this itself is semantic and in certain cases will even be executable. So it's going to, you know, have locked imports inside of it that specify pinned versions of its dependencies. And because of that, we're able to have our integrity digests just as you would have like in a package lock file. And this is going to be like your reproducible artifact that you would want to use in your production deployments that you're able to, so that you're able to, you know, know safely that you have the exact same thing that is being like reconstructed for all of the various instances of your application that you have running. And then right now, there are not really registry aware runtimes but we do foresee that there might be. But so for the time being with what you'll have to do is actually bundle your locked component, which is going to go and look through your locked component. And anytime that it encounters an import statement, actually just inline the definition of that component. So that way you have like one large bundled executable. But again, as we are, you know, potentially having registry aware runtimes in the future, the locked component itself could be executable. So we're getting close to me demonstrating things. I figured I'd just introduce you to some of the commands that I'll be using. So that way they're not foreign as I'm typing them into the terminal. So we have warg init, which is going to be initializing a new package. We'll have warg release. So after you've initialized a package, you're able to actually do a version bump whenever you change the binary that is the component. Update is gonna enable you to get the latest versions of all of the dependencies that you have in a project. And then these last three are ones that are also are not merged because they're dependent upon this, you know, syntax that is not part of the binary format yet, but they are dependencies, which enables you to see all of the dependencies recursively that you end up with from your package. Lock is how you're gonna create a locked component from your unlocked component. And then bundle is how you create that bundled component, which is going to be executable since the runtime that we're gonna be using is not registry aware. And what I'll be using in order to execute these components is JCo, which I believe Guy Bedford spoke about at this conference. He's the primary maintainer of it. And so it's going to create bindings in JavaScript from a component and we'll actually see that the way that I'm executing the logic that we're bundling together out of several distinct wasm components is going to be imported into a JavaScript project and we're just gonna execute it as a JavaScript function that's exposed. So yeah, let's go ahead and look at the demo. I decided for the first set of dependencies that I'm gonna create for us, I'm actually gonna use handwritten wasms because I think that that'll be easier in our locked component to follow what's going on from our dependencies because we won't have any WASD interfaces to look at. And then after, yes, I can. Is that good? Cool. But yeah, so we won't have any WASD interfaces, which I think will make it easier initially. And then after we work with those dependency trees, then we should be able to move on to guest languages and we'll see JavaScript and Rust and Zig all working together and creating one large wasm out of all of those. So this very first one is called add and it's basically just a component that adds two numbers together. And I'm not gonna spend a ton of time going over how to actually read Watt, but here, if you are familiar with it, you'll be able to see that it is just adding two numbers together. So we'll use the commands that we had, which was Ward, init, foo, add. Oh, hang on. Oh, sorry. Ward, publish, init, foo, add. And so this is making the registry aware that this package exists. We can then do a release on this. So this will be foo, add, and this will make it so that we have version 1.0 available of this. And I think because it's big, it's being spread across multiple lines, but there we go. So then we also have this other one that I have called ink, which is basically doing an unlocked import. This is that syntax that isn't in the binary format yet. And it's importing this add function and using it in order to define another function which just adds the number one to an input. So we can do an initialize on that. And now after this is done running, it is aware that the ink package exists and we can do a version bump on that. So that is foo, ink. And then while that's publishing, we can also look at five, which does almost the exact same thing as our increment package. It's just importing the add function that we originally published and using that to define a function that adds five to an input. So again, we'll do an initialize of foo five. And then we'll do a release of foo five. And then finally, I have this meet one, which is going to import both the increment package that we've just published to the registry and the five package that we've just published to the registry. And then it's gonna use those to define a new function which basically just uses each of those on a different input. So two inputs, increment one of them, add five to the other one, and then add those two together. And if you're familiar with the fact that addition is commutative, this is basically just going to add six to the two inputs that we have as arguments. So we can do an initialize of foo meet and we can do a release of foo meet. And now we can do an update. And we should at this point be able to use the dependencies command on foo meet. And so what we see here is that we actually have a dependency tree. So it knows about imports that weren't specified like in our top level import by like analyzing this recursively. So we have foo meet, which was released at version 1.0. It depends on foo ink, which is released at 1.0. And foo five directly, which was released at 1.0. And each of those depend on foo add. Also at 1.0. So then we can do a lock of this component. And then we can print that guy out. Ooh, before I do that really quick, I do want to show that this is actually the place where when we do an install that all of these dependencies are stored locally. And so I can actually delete the binaries that we have. So this would be like akin to deleting your node modules or like your target folder and Rust. And if when I run that dependencies command again, it not only is like creating the dependency tree for us, it's actually going and fetching all of those binaries from the registry for us. But anyways, yeah, so we do a lock now on this. And then now that we've done our lock, we can like print that and we can look at it a little bit. And so what we see here is that this lock file is, let me get rid of the terminal really quick, is going to import each of the packages for us with the integrity show for like that specific version of the package. So we're importing foo add all the way at the top, then we're importing foo ink, then we're importing foo five and then we're importing foo meat. And then that first one was foo add and it didn't have any imports. So it just gets instantiated. And then that is being passed as an instantiation argument to both the increment package and the increment by five package because those both depended on the add component. And then finally, meat depended on both ink and five. And so they are both going to be passed as instantiation arguments to our meat function. And then we can do a bundle on this. And if we print the bundle and open it, it's actually going to be like exactly the same as our locked component. The only difference being that instead of having import statements, it's actually inlining the contents of each of the components that we imported. So this is going to be like our large, generally executable component. Since again, JCo is in registry at this point in time. And then we can use JCo on our bundled component. And we can go take a look at the JavaScript that's generated and here it is. And it's depending on a bunch of wasm that it creates out of that like large bundled component. And then finally, over here in index.js, we can import the function. So foo was the name of the function that takes two numerical arguments and then basically add six to the whole thing. And let's get the one that we want. And when we do node index.js, we get 11, which is six more than two and three. And so we have all of those dependencies like working together transitively, which is pretty cool. That is using handwritten wasm and the more interesting thing that we can now look at now that we've like seen all of these dependencies working together is when each of these are compiled from distinct guest languages that are all targeting the component model. So I'll actually show like a little bit of the experience. I know that we've had some talks about like various guest languages, so I don't want to like spend forever on it, but we can start with JavaScript. So JavaScript is basically all these languages have different like maturity levels of these tool chains for creating components and they do it via different ways. So JavaScript is the only dynamically typed language that we're going to be looking at. And it's actually going to use componentize.js, which is also primarily maintained by Guy Bedford. And it's utilizing spider monkey and it's actually embedding JavaScript rather than targeting it directly. And because of the way that it's implemented, it's actually going to look basically identical to regular JavaScript that you would end up writing. We'll see with some of the other languages. There's a little bit of other, it's not as idiomatic necessarily as it would be for other languages, but here we just have a function that I'm calling because of leak code. It turns Es into threes and it turns Os into zeros and we're going to end up composing that with other components later. But you basically can just do node componentize. And when you do that, boom, it creates a JavaScript component for you. I've actually already printed it out ahead of time over here. And the only thing that we did instead of writing our regular JavaScript was writing that width that I was describing at the beginning of the presentation that takes one argument that's a string and exposes another argument that's a string. And so this function is leaked. We also have cargo component over here. And so here I'm actually, and this is a project that's primarily maintained by Peter Hewn. And this also is very idiomatic code. The one thing is that when you do cargo component new, it's going to do some binding generation for you, which we can't see right here, but if I do cargo expand, we can actually take a look at those bindings, which is gonna do that like heavy mechanics of like using pointers and lengths to represent the data structures for passing things across wasm boundaries. And this is like what the actual bindings would look like that do that like heavy lifting mechanics. And so what is exposed from cargo component is basically just a trait that you implement, which should be like familiar to anybody who's like doing a decent amount of rust code. That's like basically the only difference is that you do have to actually implement this trait when you're doing this. And so then you can just do a cargo component build. And it's also really easy to make these components from rust. And so here we have an up casing function in this component that is just going to take a string as input and then spit out another string. And then finally, ZIG has like a, not a like de facto well-known tool for the component model, but still the by code alliance has some tooling available that makes it like pretty easy to generate components from ZIG, even though there's like not like support from the official ZIG laying for making components. And so the way that we do that actually is we have to, let's actually start with the ZIG code, which is over here, my component ZIG. So this both exposes a function add that adds two numbers together and can cat, which can catnates two strings together. And this is gonna be probably the least idiomatic code out of all the things because of the fact that it's like our responsibility ourselves to create a core wasm module that is compliant with the canonical ABI. And so that's like why I have this at symbol here. It enables me to use illegal characters in my function names because we expect for there to be like identifiers in such a way with the canonical ABI. So we have foo and can cat here and this pound sign is enabled here by the at symbol. And this is basically going to enable me to, oh, and then I also am exposing these like reallocation functions which are used for those like lifting and lowering operations that I described. So this is actually going to enable me to compile this ZIG code to core wasm that is compliant with the canonical ABI. And then I also again have this wit where we're describing, hey, there's one function that adds two things together. There's another one that concatenates two strings together. And we can actually do an embed command that is made available to us by bytecode alliance tooling and wasm tools that will just embed that and make some wasm binary that represents those data types. And then we can actually do a new component from that core wasm that has the wit type descriptions embedded in it. And when we do this, we're creating a component from that core wasm initially created with ZIG. And if we wanted to look at that a little bit, this is what it looks like. And we can see that it has actually these like lowering and lifting operations that are enabling you to go from its memory representation to the canonical ABI. So given all of those components that we've created different ways, we had one that was JavaScript that does the leading of turning Es into threes and os into zeros. We upcase things using Rust and we concatenate strings using ZIG. We can publish all those to the registry and then combine them all together and have like one thing that does things with all those dependencies that were implemented in different languages. So the very first one I actually ahead of time brought all of those files over here. So pardon me as I publish each of these to the registry, we'll do a concat. So that's the concat nation function that was implemented in ZIG. Then we can do a release of foo concat. Then let's do our init of foo upper, which was our Rust function that upcases things. And let's get that version 1.0 out there. And then we had the JavaScript foo leet and version 1.0. And then the last thing is again, because the syntax hasn't been merged into the binary format yet for doing these component imports, we actually don't have an option right now other than handwriting our wasms for importing each of those component imports. So what I have here is something that composes all three of those functions together. And it's basically yeah, importing foo concat, importing foo upper, importing foo leet, and then actually doing some manual memory allocation in order to be able to compose all of these functions with each other. Once we get these things merged, we've been talking about adding it to cargo component or other tool chains so that way you could actually have your consuming component of these be in a guest language so that you don't have to bother wasm by hand because that's kind of painful. But that soon to come once we get the syntax merged also. So let's finally do our initial foo stringy which is what is going to compose all those functions together and then a release of that. All right, so then we can do an update. We can do our dependencies and we see here stringy released at 1.0. It depends on our zig component, our rust component and our JavaScript component did an install of all of those. We can do a lock of foo stringy and we can print that. And this time when we look at our locked component, we'll actually see those WASI interfaces being included because each of our guest components required one of those. So here we have things for like WASI IO and WASI file system which is like a whole thing that we could spend more time talking about if we add more time. Then we're going to actually do our bundle of each of these. So that's foo bundle foo stringy and print it. And again, this is going to do the exact same thing as our locked component. Only it is, my computer's freezing. There we go. Only it is going to inline all of the component definitions for us. And then finally, we can do jco on it and then we go over here. The function that we expose is called upcat because it uppercases and concatenates things. One second. My cursor is like not responding. I think my memory is bad, not my computer. Funny. Well, there we go. When I do this, we see that we have a Rust component composed with a Zig component composed with a JavaScript component that is upcasing everything, concatenating it together and replacing our vowels with numeric values. And all of that was built out of several dependencies using packages that were published to work. So that is it for the demonstration. I did provide some links and QR codes for anybody who wants to get involved. There's lots of stuff for us to do. We've got some more stuff to do with various guest language support and some Sembrer-like dependency solving stuff. We didn't really go into that today at all. But anyways, the one on the left is the bytecode aligns Zulip chat where you can just ask for meeting invites or just questions if you want to try to use these things. And then also, if you go to our GitHub repo, you can see the open issues and the pull requests that we actually have opened that are active work being done. And with that, I think I'm pretty much out of time. Thanks, everybody, for coming. And I'm happy to talk to any of you afterwards.