 Hello, everyone. I am James Snell. I hope you are enjoying your OpenJS world so far. I am here to talk to you about aligning NodeJS with the web platform. A little about me first. I am the head of research at NearForm, a consultancy services provider. The focus is primarily on Node React Native, that entire related stack of technologies. I am a core contributor to the Node project. I'm a technical steering committee member there, and I've contributed quite a few things to that platform over the years. The URL implementation, text encoder, text decoder, performance hooks, web crypto API, HB2, and most recently I've been working on an implementation of the new quick protocol. If you look back at the contributions that I have made, you'll see that there is a common team. A lot of it has to do with aligning Node to the web platform. When I started with Node back in 2015, there was a common reframe that we would hear from a lot of the Node core contributors, that this assertion that Node is not a web browser. It shouldn't act like a web browser. It shouldn't have the same things that a web browser has. There is this what I call this gulf of unsame-ness that existed between Node.js as a JavaScript run time and the JavaScript that you would run in browsers. To a large extent, that is true. Node doesn't do rendering of web pages, for instance. There is an area of overlap that does exist. This gulf of unsame-ness, this idea that these are completely different environments, that they should operate in different ways, that they should do things differently. It's actually fundamentally wrong. There is an area of overlap. There are things that both browsers and Node do. Crypto is a good example of this. You may want to encrypt or decrypt some bit of information in both environments. You may want to encode from some bit of text from UTF-8 to UTF-16 or whatever in both environments. You might want to generate a UUID in both environments. There are common tasks that you're going to do in both of these places. The idea of aligning Node to the web platform is really about finding those areas of commonality, those areas of overlap, where you might have to do one thing, encrypt some data, but you don't want to have to do it in two completely different ways, or worse, three or four or five completely different ways, depending on where your JavaScript happens to be running. What we want to do is find the commonalities and look at the APIs and figure out where it makes the most sense to bring those together. Whenever this discussion comes up about aligning Node with the web platform, there is often criticism as to where does that stop. Are we going to implement everything that you find in browsers in Node? I just want to pop over and look at this real quick. If you look at the MDN page for web APIs, these are all of the APIs that are available in web browsers today. What you'll find is there is a massive number of object types, of interfaces, of different APIs that do different things, SVG manipulation, HTML, DOM access. You have in here time ranges for timing events. You have local storage, you have video playback, even new APIs for virtual reality display events. You have all of these APIs that exist within the browsers to do these extremely rich user interface, user interaction types of applications. The question naturally comes up, do all of these APIs make sense to implement in Node? The answer clearly emphatically and absolutely is no. There is never going to be a time where we will actively go through and make Node do everything that a browser does. There will never be a time where it implements all of these APIs built into the core of Node. It's just never going to happen. But what is happening, what we are doing, are finding the areas of overlap, finding the areas where these things are similar enough or where you're doing the same task, and you want to be able to write some bit of JavaScript that works in all these different environments. This is really a follow-on to a talk I gave previously at another event where I introduced some of the APIs that we were working on within Node that really speak to this alignment. Here I just want to touch on a few brief examples, but then dive deeper into a specific set of APIs that are leading us closer and closest towards a Node core implementation of the Fetch API, that exists in browsers today. Let me take a step back just a little bit and let's look at where we have been, look at where we are now, and then look at where we are going with these APIs. It was a number of years ago now that we introduced the WhatWG URL parser, and really this was the first of the web platform APIs. When I say web platform, I mean additional APIs that are not part of JavaScript, the language itself. These are APIs that are defined externally to TC39, whether it's in browsers, whether it's part of W3C, whether it's part of the WhatWG, all these other areas, other venues where these APIs are created that they typically would go into the browser but aren't part of the language itself. URL was our first example of this, and really the motivation behind adding the URL API was that the existing URL parse, that existing API in Node for parsing URLs actually has a number of fundamental bugs in terms of how it is handling things like usernames and passwords, how it handles certain edge cases and how the URL is structured. And the implementation has been so performance optimized that it is extremely difficult to make changes to fix bugs without negatively impacting the performance optimization that's there. For example, when I originally tried to fix the parsing of the username and password, ended up being something like a 75% reduction in performance of just the typical parsing operation. It's the kind of code that if you look at it wrong, it slows down. So I really wanted to just kind of leave that existing URL parser alone and not touch it anymore. But we still needed a way of parsing correctly without the bugs. And about this time, I started looking at, well, what if we just used the same API that's in the browser? What if we just introduced a new parser and had it be a standard interface, so you could do new URL in the browser, and you could do new URL in Node, and it would just work the same way. This actually ended up being fairly controversial. I actually had some of the other Node core contributors up in arms about this idea that we would have this second different way of parsing a URL. What would we do with the existing one? What would we do with URL pars? Would that go away? Would that be removed? Would code suddenly stop breaking? And the answer was no. This is just an additional way of parsing URLs in Node. Today, it is fully spec compliant. New URL is the recommended kind of default way of handling URLs in Node. We've verified the implementation on the same web platform tests that the browsers use. We know we are spec compliant. In fact, I'm proud to say that Node is even more spec compliant on URLs than the Chrome browser is. It's been a while. It's been a while coming, but new URL is now the default of standard way of handling URLs in Node. But that was the first step. As we just saw in the MBA, there's so many more APIs. What other APIs make sense to implement? Well, text encoder and text encoder. Being able to handle text and convert from multiple character encodings is something that you're going to be doing in both environments of Node or in the browser, server side, client side. It doesn't matter. You have to be able to handle text. Text encoder and text encoder were the second kind of standard browser APIs that were added to Node. Those went in much quieter than new URL. It was much less controversial. I think we finally got that first one done. The next one after that came much more naturally. After that, the worker threads API was added thanks to a tremendous amount of effort from Anna Headingson who basically just said it's happening. Worker thread is going to be there. It's going to implement standard APIs as much as possible. We're going to have message channel. We're going to have message ports for passing messages back and forth. It was just all going to work. Now, the worker threads API does not match exactly workers in the browser. There are a number of important differences, but in the primitive APIs that are part of it, like message channel and message port, the APIs and the way that they operate is exactly the same as in the browser. You can pass messages back and forth across threads. You have the structured cloning where objects are either cloned or transferred over between. All of that just works now in Node. Using the same APIs, the same mechanism that exists in the browser. More recently, I also added the broadcast channel, which is fairly new to browsers as well, that gives you a one-to-many messaging model, whereas message channel and message port is always one-to-one. Every instance of a broadcast channel that is listening to the same ID can receive a copy of the message. It gives just a little more flexible messaging model when you're dealing with multiple threads, but it's the same API that exists in browsers today. A couple of other critical ones include event target and abort controller. Event target has existed in browsers pretty much forever. Node has always had its own event emitter, which was similar, did essentially the same thing, but was almost entirely different than event target. We had this event model in both environments, but the way it worked, the various nuances between them, very, very dissimilar APIs. For the longest time, the fact that event emitter existed in Node was a reason not to have event target. Why would we want to have a second way of handling events? Eventually, we just got to the point where why not have it? It's not difficult to do. The API is there, it can be available, it can be used. Why not just have that same API available? As long as event emitter isn't going anywhere, which is not, it's still going to be a node, it's still supported, it's still one of the core APIs, and it is still the most commonly used, one of the most broadly used APIs in the entire Node ecosystem. Nothing is happening to event emitter, but now we have event target there, and we can use that same event model that exists in browsers in Node. That is important because of abort controller. Abort controller is a way of allowing us to inject a signal to cancel asynchronous operations. Abort controller became popular as part of the Fetch API in browsers, where you could start a request and then use the abort controller to cancel that prematurely if necessary. It's paired with another interface called abort signal. You have a signal. The signal itself is an implementation of event target, so it has an abort event. When you call the abort method on the abort controller, it will trigger an event to be dispatched on the abort signal object, causing that event to be fired. That has become the, as now, and this is recent within the past year, abort controller and abort signal have become the de facto way of canceling asynchronous operations that are happening in Node. And we now have support for using abort controller and child process event target file system, HTTP net read line and Ripple, streams API, awaitable timers, UDP and Dgram, and more in terms of where you'll see support API being implemented. So this browser API, this thing that originated as part of Fetch, has now become a critical component of the Node API, because it serves a very important purpose and it's something that we need in both places, browsers and on Node. So it's something that just made sense to implement. I do want to point out, there are some tricks to using abort controller. There are some things you need to know. And very soon, I have a blog post coming out. Here's just a draft. I'm going to have a blog post coming out on the Nearform blog going into detail on how to use the abort signal in abort controller correctly. So keep an eye out for that. I think it's going to be useful. So switching back here, going on to some of the other things that we have done. We've implemented the performance API from browsers in Node. So you have what's called the user timing API. Performance.now for getting the current time. This performance observer object that allows you to listen to, for performance related events. You have this performance mark and performance measure. These are all APIs that originated in the browsers for measuring performance related events that occur. A couple of years ago now, it decided just to go ahead and implement this for Node and make it possible to use these common primitive APIs to do very simple performance measurements within Node itself. Previously to this, folks had to go up and use dependency libraries for injecting instrumentation for measuring how long it would take to run a particular function. There's dependency libraries out there for benchmarks, for APMs out there for inject this functionality. With the introduction of the performance API as a module in Node, what's called Perfix, the basic mechanisms for doing very primitive, very rudimentary performance measurements are there as part of the platform now and are there ready to be used without any dependencies needed. So it's all there and it's the same API that you would find in the browser. So this has been a slow march. This has taken several years to get to the point where we've only implemented a limited handful of the APIs that exist in the browsers, but it's a very powerful set of APIs. It enables quite a bit. We have the message passing for working with multiple threads. We have an implementation of the web crypto API. So if you want to digitally sign something or decrypt something, you can do it the same way now in Node as you do in the browsers. You can write that code once and it works in both places without any modification. So with this limited set of APIs that exists so far, it's already very powerful in what you can accomplish in terms of writing clear consistent code that just works across these several different environments. But there's more to do. There's more to come. And one of the things that we get asked about the most, the one API that I think that we've consistently seen folks ask for time and time again is Fetch. Now Fetch is simple. It's just a method that triggers an 8-seats-p request. It returns a promise that resolves, that promise fulfills when that HP request has been completed. So you want to go fetch a document, you want to go get an image, you want to go to a server and get a page, return that content. So you call Fetch, pass in the request information that you have, and you get back a response. The API is simple, but Fetch itself is really quite complicated. There is a lot to it. Fetch isn't just one API. It's several APIs working together. And I kind of want to break down some parts of that so it's clear kind of what we need to do in order to implement Fetch properly in a spec-compliant way in Node. So one of the things we have to implement, one of several, is Blob. Blob is an API that exists in browsers. It's part of Fetch and it's part of the file API that's there. And essentially what a Blob is is an immutable opaque data source. So if you think like a typed array, it acts like an array. It stores data, but you can kind of randomly access any of the bytes, change them, modify it. It's not immutable in any way. A Blob is entirely opaque. It stores the data, but it doesn't provide any accessors into individual bytes. It doesn't allow individual bytes to be modified. It just encapsulates that data. The other characteristic about Blob is that it's not just a single group of data. It's not just a single chunk of data in an array buffer in a typed array. Underneath, that's still just a single chunk of allocated memory. A Blob can actually be made up of multiple pieces. Each piece, what I'm calling a data source here, is itself a chunk of data. Now it could be a string. It could be an array buffer. It could be a typed array. It could be a readable stream. It could be another Blob, whatever makes up that data source. The Blob itself is a sequence of these data sources so that when you read a Blob and you get a string, what you're going to get is a concatenation of all the data from each of the member data sources that's there. Now the data for those data sources might not actually exist when the Blob object is created. It might be a stream. Think of an HP request. You submit the request, you get back, but there still might be a stream of data being returned by the server. What the APIs are going to do is it's going to create a Blob for you before it has received all that data. All the data is going to stream into that Blob as part of that data storage. That Blob can grow over time. The data that contains can evolve, but several things about it are interesting. One, like I said, it's immutable. The data can't be changed once it's there. Reading it is idempotent. What that means is when I read it once, I'm going to get some data. The next time I read it, I'm going to get the exact same data. Once the data has been loaded into the Blob, like I said, it may not be there all right away, but it's going to stream in. Once it's loaded, I can read it again and again and again, and I'm going to get the same data back. It speaks to that immutability of what that Blob is. But again, there's no API for getting the bite at index 50 in a Blob. What you do is you get a promise to get a string that has the entire content, or a promise to get an array buffer that has the entire content, or you can get a stream readable that will give you the content as it is available. There's ways of consuming the Blob, but it just doesn't have that same level of data access. Now, why am I talking about Blob so much? Well, fetch includes support for the Blob API. When you submit a request, that request, that input data, could come from a Blob. When you get the response back, Blob is one of the options for getting that data back. Now, there's other options, or string and array buffers, other things, but if you look at the fetch API, the standard, the spec, it says one of the options is to get a Blob. Blob doesn't exist in Node. If we want to do fetch correctly in a spec-compliant way, then we have to implement Blob. It really is a simple of that. Now, we do have an initial, simple implementation of Blob in core, experimental right now. There is additional work that needs to be done. Right now, it only works with static data sources, where the data is all available when the Blob is created. We have to further enable that to support the streaming in of data for the individual data sources, and that's something that's going to take some time to complete. This is just one small part of the overall fetch implementation. Something else that we need for fetch, if you look at the spec and it says, hey, it's there and needs to be supported, is form data and headers. These are two objects that the fetch API is designed to work with. If you want to send HTML form data, the content of an HTML form serialized on the wire, you would use the form data object to create that. It's a simple object used to parse and produce this encoded form data, and you could use it as the body of a fetch request, or get it back as a response. We have a question. We have a decision to make with a node. Do we want to implement this? Do we want to support this? Is this part of the fetch API that we want to support in a standard way? I have it marked as optional, it's still up in the air whether within node it makes sense for us to actually provide this API. The header's object, similarly, is for working with HTTP headers. It's a standard part of the fetch API that if we want to implement fetch correctly, we have to do this. The challenge is where do we use this? If we implement the header's object, should we integrate it with the existing HTTP API that we have in node? Or is it just a standalone thing that only works with fetch? It's one thing just to implement something. It's another thing to figure out where all it fits, how it fits with all the existing legacy API that we have. We have to address that concern before we can really move forward. Otherwise node just becomes this grab bag mess of things that aren't really well integrated with each other. We have to make those decisions on how to implement these and how to integrate them as opposed to just simply implementing them. The other big thing we need to figure out before we can do fetch properly really is figure out the implementation of the WhatWgStreams API. Fetch is largely built around WhatWgStreams. It's become critical to many browser APIs. I don't think anyone in node is excited to add yet another Streams API to node. We already have one. It's extremely complex. It's gotten through many different versions. It has all kinds of warps and everything else to it. Just this prospect of adding another API, particularly one that's going to be slower. It's not something that's very attractive to folks. So it's something we're working through. It's something that needs to happen. Yes, and I do anticipate it happening for sure. It just might take some time. There are other considerations for fetch as well. Fetch requires support for cores, which is cross-origin resources and this model about limiting requests across origins. Well, node doesn't have any concept of origin. Cores doesn't really make sense in the node environment. Is this something that we would even implement? Is this something we even need to worry about? Do we need to introduce this to node, this concept of origins? These are things that are just open questions. Fetch has support for HTTP caching. Node has no implementation of HTTP caching. Do we need one? Is that something we need to go through and implement? We also need to figure out what we're going to do about HTTP state. Fetch in the browser assumes that states, any state that's maintained, cookies, all that kind of stuff, is bound to the origin model. The cookies that it stores are going to be scoped only to the origin in which the current web page is running it. But again, node doesn't have a concept of origin. Unlike the browser, which has a single user, node could have millions of users simultaneously. Global states is a big problem and Fetch relies a great deal. The API, if you look at the actual API spec, it relies a great deal on global state being maintained. We need to figure out kind of how that fits within the node model. It's not just about implementing the API. We have to figure out how it integrates with everything else. And then another challenge we have is figuring out seamless version discovery between HP 1, HP 2, and HP 3 because of the very organic way and incremental way that these APIs evolved over time. Within node, they are not very well integrated. In order to do Fetch correctly with a good performance, we really do need to get those integrated well. In the meantime, while you're waiting for Fetch, you really should look at Undici. Undici is an alternative HP client that's starting to be developed as part of node with the intent of it becoming the next node client API. And it is very similar. If you're familiar with Fetch, you should be able to look at Undici, work with Undici. It's very, very familiar way of doing it. That is the quick version of this. Unfortunately, I've run out of time to go into more detail. The whole point of this conversation really has been to go through and outline where we're at, where we're going, why certain decisions around browser APIs take a long time, what the considerations are when we're going through this. It never ever is just about the implementation of the API and that's it. All of the difficulty comes in where it fits in the larger platform and the larger strategy so that we are delivering benefit for the entire node ecosystem rather than just implementing an API for the sake of having that API. With that, thank you all for joining me. I hope you enjoyed the rest of the sessions and the discussions around OpenJS world and I'll catch you all later. Bye.