 Hi everyone! I am Matteo Collina and today we are going to talk about the Node.js HTTP client. What can be so interesting about it? Well, what if I could tell you that, you know, you could just double the throughput of your HTTP client or even better maybe, you know, triple the throughput. So, you know, we can do that and we'll see how. A couple of things about me before we start. I am Matteo Collina. You can follow me on Twitter at Matteo Collina. Please, you know, follow me back. I'm also part of NIRFORM, technical directors there. We are a professional services company based in Ireland. So, if you are interested, you know, working for us, check us out. We hire globally and remotely. So, hey, we are also, I'm also a member of the Node.js technical steering committee. So, I'm part of Node and I'm the co-creator of A Few Bits and Bobs on NPM. I also write every week and I take and we then use letters called NodeLand. So, you know, check us out. Anyway, I maintain a lot of open source including Node. So, as part of my activity of both as a consultant and as a maintainer, I tend to strike and maintain a balance and feedbacks my learning from when I help companies to when I maintain things in the ecosystem. As part of this activity, I tend to write and build new things when the opportunity arises. So, most applications start as a monolith. You know, there's nothing bad about talking about monoliths. Most apps start as a monolith. They're great. And, fastify the framework that I've built is great for building monoliths. So, you know, it has almost 900,000 downloads per month and 15 collaborators. We are working on version 4 now and we have a lot of ecosystem plugins. So, it's pretty great. And it works really well. It's also part of the OpenJS Foundation. So, I have another talk about it. Sorry, I'm not going to spend so much time about fastify. However, once we build our monolith, one of the hardest questions is how do we start scaling our teams? How do we start scaling our teams? How do we start improving our system? Well, we'll see that in a second. So, what we want to do in those cases is that we want to move to what's called a microservice architecture. And where we have some sort of gateway that talks to several microservices, as you can see in the slide. So, however, this is not just that because in the most complex enterprises and projects, the reality is way, way, way more gets very complicated very quickly because you have several layers of microservices that talk to each other. Now, the throughput and latency that each one of those introduce, it's critical for the system. So, and it's one of the most bits that tend to not tend to be overlooked when opting for a microservice architecture. So, and by the way, Fastify is one of the architectures. Fastify is great for building microservices, also lambdas, by the way, but it's great for building microservices. So, please use Fastify when we're building microservices. As part of this talk, we are going to go through and look at these links between the microservices. And now we can use HTTP to provide a link between the microservices, easily debuggable, but also very, very performant. So, let's dig deep into this topic. So, it's, you know, look at this, because it's if you want to make it fast, it's, you know, that link can add up pretty quickly, because if you add more and more levels of latency, you know, if that link becomes low, then all your system is low. So, let's take about a little very simple server. This server is an HTTP server directory node that, you know, just respond with a given time out. Easy. And the client, the HTTP client for this is, again, it's very simple. You call HTTP.get, but you can also use axios, nodefetch, request, got, whatever you have. And you can call, and then you can just, you know, pipe it, pipe the answer back somewhere, somewhere. Note that this call, this system by default allocates a new TCP socket for every TCP, for every HTTP request. So, one new socket for every request and we get, we dispose it. This is not efficient. Creating a socket will see it's, this is not efficient, because creating a socket involves several round trips. So, on the, on our, on the, on the server. So, essentially, when you establish a new socket, you need to do a SYN, you need to send a SYNT IP packet, which then needs to be replied by the server. Now, this is a back and forth and you are actually losing milliseconds here. So, if you're on a very fast network, like a server, this is actually not a big problem. But, you know, you still have to allocate, you know, file descriptors in your open system and so on and so forth. So, there's a limited amount of these things that you can use. Note that you get a little bit of, of latency as well. And part of the problem is also due to the so-called TCP congestion window as shown in this diagram. By the way, these diagrams are from a book called I Performance Browser Networking, which I would highly recommend you to, to go and read. So, this book, it's, so this, this congestion windows, what you have is, you can see that if I'm sending a big packet on a freshly created socket, say, sorry, if I'm sending a big file on a freshly created socket, you know, I need to do a lot of round trips, because every once in a while I need to send a knack to the server. This is very important for HTTP, for TCP, because it ensures that all the packets arrives in order. However, you know, the congestion windows grows over time. And at some point, it's, it's with using an algorithm and so on, it stabilizes. I don't want to enter all the stuff. It's very important. But, you know, outside of the stock, what is important is that if I have, if my socket has been open for a while, the congestion windows is higher. And because it's higher, we can only, we can send a lot of data before sending a knack. Now, in the example on the right, you can see that if a big, if we have a biggest congestion window, we can only, we can send all our data in without receiving a knack from the, from the sender. This, in this way, we are actually reducing a lot, reducing the latency quite a lot. So in order to ensure the maximum bandwidth and the minimum latency, we must reuse existing connections, right? Is that, you know, it's, it's, this is a fair assessment. So what it means, what we were doing before creating a new connection every time is not efficient. So in, in not core, we offer a construct to do this, which is creating an HTTP agent. HTTP agents can keep the connection alive. So it avoids the end shake and maximizes the connection window. It uses the keep a HTTP 1.1 keep alive. It's a key feature of HTTP 1.1. This is actually critical for TLS connections, because on top of the TCP end shake, you will also have the TLS end shake. So if you don't have an agent and you're calling HTTPS services, you, you are setting yourself up for trouble. Note that this is not the default. So you need to either to configure, you need to configure it manually for your HTTP client or set it out to configure as a default in node. So it's very important that you do because the difference can be staggering. Now let's turn the idea into reality. One of the, the server scenario that I'm going to test, it involves doing a 500 parallel request to one server. And it's, which in turn does, you know, another five, five requests to other microservices. So essentially, it's, you know, it's a lot of, it's a lot of requests. Okay. The server takes 10 milliseconds to process the request and the client has a limit of 50 seconds. So all of these is fictional. So don't worry too much about it. Do your measurements. But the difference is staggering. So if you don't do keep alive, you will be in very, very huge trouble. Use HTTP agent with keep alive. Full stop. The difference can be enormous. So the key, the secondary question then after we have seen how we can improve the, our client, you know, it's, we can use an agent then, right? But can we still, can we improve things further? Well, we can actually, we can. How? Well, you know, we need to go back into the spec and look what's there. So is there something in HTTP that can allow us to actually, you know, work at a higher speed? Well, that is. So one of the important bits is this thing called HTTP 1.1 pipelining. HTTP 1.1 pipelining is allows to send more than one request at the same time using the, more than one request at the same time. It's great. So minus one thing, you know, all the responses with HTTP 1.1 needs to be received in order. So it's software for the so-called head of line blocking. So if the first request takes a lot of time, everything will have to wait. However, these, it's, it's a good technique. It's important to know this is possible because we can actually use it on our server to server. Not that head of line. We need to talk a little bit about head of line blocking though, because if you're doing this and you start losing packets, or, you know, having a single slow request, you can actually block all the incoming requests for a while. So be careful on how much you are pipelining. The other important part that we need to to talk before, you know, making a recommendation and, you know, discussing how and why we can actually improve the speed is the event loop. In the event loop, if no JS event loop, we have an event that, you know, when we actually, we have, we have called events, so I called event loop, you know, events are are not JavaScript though. So we have, you have IO events. IO events are can be produced by the kernel or can be produced by thread pool. And those events get put into a queue. Once, once an event is there, no JS can fetch that event and process it with JavaScript. That's what it does. That's all, all of it does. And with JavaScript, you can schedule more events to happen in the future. And then those would be cute. Or maybe we can just say, Oh, I finished processing this event. Please send me the next one. That's what it does. How does this rating to pipelining? We'll get there in a second. But one of the important bit is to tell, to note, how can we make things fast in, in, in node. So in order to make things fast, you need to understand when the event loop is blocked. So if you're doing massive IO, like in this case, you need to, you want to maximize the, you want to maximize the time that node.js is doing IO. This means minimizing the time the event loop is blocked, right? Well, the event loop is blocked when we are executing JavaScript. So in order to make things fast, we need to reduce the amount of time we spend in JavaScript down to zero, if possible. And it's a, this is the key technique that we have used in, in, we can use to improve things. Okay. So, because, because we can, because we can schedule things, you know, with HTTP pipelining, one of the, that technique combined with the event loop logic can make sure that when we are processing things, we are, you know, processing a lot of events, a lot of things from the server or for the client and so on and so forth. So it's, it's important. Note that it's, it's also important to note that with HTTP pipelining, you are going to have a lot of econ resets. You're risking econ resets. So if the other side is truncating your connection before you send any data. So recently, we have changed the KIPA live, the KIPA live agent to a logic of LIFO. So last in first out, this means that this reduces the amount of, these tend to use the most fresh sockets. So it reduces the risk of them timing out. The still, the risk is still there though. So you need to configure your KIPA live timeout well. So considering all of the knowledge that we have went through so far, I am going to show you the one of the best thing that I wrote in a while, which is this new library called Undici. Why Undici? Well, Undici, it's an Italian word that means 11. Why 11? Well, one, if you consider the number 11, then we have HTTP 1.1. So you see Undici, which is pretty great. Note that this is totally a Stranger Things reference. So if you don't, you know, if you, if you're wondering if it's why it's called Undici, it's also, it's totally because Stranger Things, because when I was starting working on this library, I ended up writing Stranger Things was just came out. So in effect, I've been developing this library for quite a long time. So how does Undici work? Well, Undici is a new library for Node and it KIPA lives by default. So you don't need to configure a KIPA live agent. It also adds a LIFO, it also had a LIFO, a LIFO scheduler. So by default again, it does not do any pipelining by default, but it can be configured to do so and it can create a limited connection. Or by the way, it can also follow redirects. Note that this is really fresh. It's a fresh syntax, it uses promises, can also use callbacks if you want, but maybe not. And basically you can just, you know, just use the things that you like the most. It's very simple to use. Note that you can, Undici is similar to Node as a concept of agent. And you can configure a global agent if you want to do so. You can configure, for example, the pipelining factors. So Undici is fully capable of doing HTTP pipelining. Don't pipeline too much because you risk a lot. But you can also configure the number of connection for to each destination if you want to do so. So it's actually pretty great that you can configure all those things. Note that if you're using Undici for testing, you might want to disable or reduce, deeply reduce the keep alive. So you can actually change the setting and essentially configure the global dispatcher for everything. You can also use the lower level API. So you can create a pool for a target destination. So it creates a pool with a target pipelining and connection numbers. And then you can just call request as it was before. You can create a client, a client map, one single socket. Again, you can do all those things, you know, it's one at a time. We also have these interesting methods called stream, request, dispatch, we have pipeline, we have a lot of things, we have a lot of options in terms of integration that you can do. Oh, by the way, I was almost forgetting, we also have support for mocks. This is one of the greatest things that I really wanted to get in Undici v4. This is not in v3. Undici v4 is coming out these days. When you watch this, it might be already get out, but it's still not at this time. So it's still in the release candidate phase. So you can install it with MPMI v4.0.0-rc-4, for example. With the mock, you can actually, you know, configure a global agent, a global dispatcher, for that will mock the response. And you can also enable a pass-through mode, so you can only mock certain things. This is really important because in, in order to support mocks, for example, in a knock or in the node core HTTP, they had to rely a lot on monkey patching internals, node core internals. However, we don't monkey patch. With this, it is no monkey patching happening at all. You can just create one and it will just work. So it's pretty, pretty great because you can also use it for testing your libraries. So I really love it. How does this compare? Is this fast enough? Well, let me show you. So if we are considering just a very, a very simple system with the system, simple system as before with keep alive, if we don't enable pipelining, there's not much effort here. But if we enable pipelining, we can dramatically and drastically increase the number of requests per second that node core can send. Why? Well, because we are minimizing the number of round trips to the underlining to the kernel and the operating system. Anyway, and we are using our shock at the best, the most essential. So for me, this is pretty, pretty great. I also done some benchmarks on HTTP 2. Using this library called Fastify HTTP proxy. It's a simple HTTP proxy system that is built on top of Node.js and Fastify that can do HTTP 1 to HTTP 1, but also HTTP 2 to HTTP 2, but also HTTP 2 to HTTP 1.1 and vice versa. So it also uses by default for HTTP 1.1, for HTTP 1.1, which is great. So you see it's fast actually. And, you know, I'm pretty, pretty happy about this. So hey, it's actually pretty good. Now, I know that this can be improved quite a lot potentially because this is using one single connection and we might be eating some sort of HTTP 2 limit. So yeah, yeah, this can be improved even, but I'm pretty happy about it. So I am almost wrapping up. I just want to say that I want to recommend to you always use, if you haven't watched this talk, you want to just get the most out of it. Always set an HTTP agent. Check out Undici v4. And if you, if you have the problem of doing a lot of microservice system and HTTP calls, Undici can actually drastically reduce the overhead for your distributed system. So hey, pretty great. We have a new docs website. So HTTPS and undici.nojs.org. Yes, it's part of Node. Okay, Undici is part of Node.js project now. So it's, it's pretty great from, from my point of view. We need help. There is, we need people to use Undici and file bugs so that we can fix them. Please do that. We can also send PRs. There's a lot of activity is one of the most active projects in the Node.js organization. So hey, pretty neat, right? I also want to, to show, oh, nice. Here we go. Like, I also would like to show you a little bit of, of a demo of, of Undici. So here we go. So let's see that we have a server. So this is a server that does a few things. And, you know, you can see it, it's, it's pretty, it's pretty new syntax. So we're using the new ESM. We can actually iterate over the incoming events, incoming requests. We await for the number of, to the server for be listening. And then we start posting our request. Pretty nice, right? I, I, I pretty, I, I like this syntax. So then I can actually do the Node server, right? So if I do that, I then I can say, for example, I can call it and here you go. Yeah, yeah, it actually, you know, works. Cool. Now, how can you use you, Undici to call, to query this server? Well, we can actually open up these other code, which use the global request method from Undici. We extract a bunch of things and then we console log it essentially. So let's see if it works. Cool. Whoa, it seems working because we are actually replying that it's, it's saying that we do, we have a date. It's a 200. And it, it tells us that the server want to keep this connection alive for five seconds. And it has a content length of 11 characters. Those 11 characters are a low word. Pray. And if you're looking on the server, you see we have access slash through. So it's pretty great from, from what I can see. Going back to, to our slide, we can, I just wanted to, to finish up by pointing out to the fantastic high performance browser networking book by Ilya Grigoryk. You can also read it online for free at hpbn.co. Check it out. I talked a lot about Fastify. So Fastify.io. Here you go. And then we have Undici and there is a guide on the event loop and finally note clinic if you want to optimize your servers. We are about to wrap up. So I will just thank you very much for having me. You can reach me on Twitter at matthewcolina. Also send me an email asking for anything essentially like the universe and everything, everything at matthew.colina at nearform.com.