 All right, so it's 9 a.m. I'll go ahead and get started Good morning everyone. Thank you for coming to my talk I just want to let you know ahead of time that the projector is having some issues yesterday and today So if it cuts out, it's not my fault So the title of my talk is LibuV. What's a unicorn velociraptor? So if you were to enter that into Google you would see some results similar to this So the the picture in the top left is the actual LibuV Logo and then things just kind of go off the rails from there a little bit We have a dinosaur riding a unicorn For some reason the Chicago Bears football player is on a dinosaur But yeah, so what I'm talking about is LibuV the platform abstraction library that no JS sits on top of LibuV is written in C It's not C++. It's actually I believe C 89 so it runs, you know in most places The the point of the library is to do asynchronous IO So it's gonna be what gives LibuV or I'm sorry gives node It's you know asynchronous IO operations event loop things of that nature Here we go My fault That's right I'm glad I warned you guys ahead of time about that. All right, so LibuV is used by node obviously But it actually has a number of other really big consumers. So a language called Julia Neo Vim C make a bunch of others At the bottom of the screen here, you will see a cartoon head. That is Saul He is another one of the LibuV collaborators. That's his his github icon And he he said this really nice quote to me one time and I wanted to you know Attribute it correctly not not take credit for the quote myself, but it basically is we write the if-def so you don't have to So if you look inside the node JS C++ code base There there is some branching some if-def based on what platform you're executing on But it's really not that bad. If you then dive down into the LibuV source code It's a whole another story. There are if-def's all over the place There are actually two different source trees one for windows and then one for everything else So there's there's a lot of branching logic there that that kind of gives a nice Consistent API to people who want to build on top of LibuV And as I said, it is a cross-platform C library So we we support a large number of operating systems Some more than others. So we have a three tiered support system So tier one is kind of you know, we we test this in the ci Anything that you know might cause a regression on any of those platforms is going to block something from landing And then tier two is it's it's considered officially supported It might not necessarily be tested in the ci And we'll we'll try to the best of our abilities to to make sure that it never breaks I'm not really sure why we have the distinction between tier one and tier two because I do believe that all the things in tier two are currently tested in the ci Uh, and then tier three is going to be community maintained platforms. So that's going to be android IBM i although I think they're currently in the process of trying to add support, uh, you know a higher tier support right now Uh, and then just some other, you know different random platforms A lot of times people will show up with platforms that I've never even heard of and you know They'll start trying to add if defs into the code and see how it works So, you know as long as it's not too intrusive into the code base We're usually okay with that But we can't really make any promises that it won't break because we're not testing it anywhere So if you you know if there's a platform you care about and you want to see it in there You are kind of on the hook to make sure things don't break So some of the features that come from libu v uh the event loop obviously is a really big one in node TCP sockets so in node that basically translates to the net module We have dns resolution. So In the dns module a lot of the or some of the system calls are from a library called c aries that i'll talk about On the next in a couple slides But libu v also implements some stuff there UDP sockets is going to be basically nodes d gram module File watching and file system operation. So just about everything in the fs module is going to go through libu v Child processes and threads, you know, obviously that translates to the child process module and worker threads And then we have other things like synchronization primitives Mu texas and things like that And then we offer a high resolution clock So if you know date dot now is just not accurate enough for you You can use inside of node process dot hr time to get a little more high resolution timing And i want to talk a little bit about The high level architecture of libu v so i like i like to think of this as broken up into two rows So basically the top row which has the network i o all the way across to like file i o dns and user code That's more of The public facing api What users of libu v are going to consume and then the bottom row with like the iocp and the thread pool and things like that That's more of the guts of libu v You can actually get pretty far using libu v without having to interact with any of that stuff One of the things that's interesting on the bottom row is You know iocp is is basically how We do iopolling on windows But we also have things like epoll kq and event ports. So libu v will basically pick the best I guess primitives for what operating system you're running on So kq will be used on bsds and the mac Uh epoll is used on linux and then event ports are uh solaris So in order to really use libu v effectively, there's a couple things you have to understand a few like concepts The first one is handles So these are an abstraction for what it's typically a long-lived resource So this might be something like a socket or a child process Um tty's if you're dealing with your terminal timers and things like that We also have a couple of different types of handles that are used for interacting with the event loop So something called an idle handle, which is actually poorly named because it runs on every iteration of the event loop So there's not really anything that's idle about it And then prepare and check handles which run before and after you do before libu v does its iopolling And then async handles which can you know do things like be used to wake up the event loop if it's sleeping and things like that And then handles have a concept of being active. So if a handle is active, it'll actually keep the event loop alive So for example in node whenever you start a server if you just you call server dot start and nothing else You'll notice that node doesn't exit That's because the event loop sees that there's at least one active handle alive and or remaining and keep the event loop open And then there's also an operation called unrefing And then the inverse of that is reffing So when a handle is created, it's in a state of being referenced And that is what will help you keep the event loop alive But if you unref that Then the the event loop no longer considers that as something that will keep your application open So these are actually exposed inside of node. So if you've done a lot of node programming you there's a chance you've seen dot unref and dot ref And that's basically what it translates to down in libu v So in addition to handles, we also have something called requests I like to think of handles as more of an object Whereas a request is more of a function or a method I say function or method because sometimes they are involved with a handle and sometimes they're kind of their own standalone thing These are typically shorter lived operations So things that happen when you're doing file io If you're doing dns lookups Or if the user passes in their own custom work that they want to execute in the thread pool Then that's going to be more of a request type operation instead of a handle But like handles they can also keep the event loop alive So for example, if you you know start node and you do fs dot read file Um It's nodes not going to terminate until that read operation is complete if that's the only thing that's happening Um, if if the if the request wasn't something that could keep the event loop open You would call fs read file and then the program would just exit immediately So if you're ever curious why it is that node doesn't exit in some cases It's usually because there's a request or a handle somewhere Uh, and then one of the more I guess famous things that we get out of libu v is the thread pool. So The whole point there is to move computations off of the main thread. Um JavaScript as a language with the exception of workers, which are relatively new is single threaded. So You know in a in a server application you probably are going to have a lot of things going on at the same time You could have you know hundreds or thousands of requests being processed simultaneously Um, and if everything ran on just the one main thread things would slow down pretty quickly So we use the thread pool to offset some work onto, you know worker threads Um, one common misconception is that everything runs in the thread pool. That's wrong Uh, only file io, uh dns lookups. So get address info and get name info And then custom work that a user might actually off put into the thread pool Those are the only things that actually run on the thread pool And uh by default there are four worker threads in the thread pool But you can actually control that with an environment variable called uv underscore thread pool size So if you pass that to uh, that's propagated up through node two. So if you start node with uv thread pool size equals, you know 124 that's how many threads you'll you'll spawn Unless you actually need to do this you should probably be careful because More threads are not always better if you don't have enough hardware to keep up with all the threads They can actually compete with one another and all the context switching in between them can actually slow your application down Uh, so this this picture is taken directly from the libv documentation. It's basically explains how the event loop works so Every tick through the event loop we calculate the loop time. So we have some reference for what time it is Um, and this is kind of an expensive operation So we cash it at the beginning and then we are going to check is the loop alive or not And by is the loop alive? I mean are there active handles and requests that Are outstanding if there are none then the event loop can exit and then you know in node that'll propagate to the process exiting But if there is still work to be done There's a number of steps that that libv goes through so first It'll look and see if there are any timers that are due So, you know, if you've called set time out or any of those like ready to to be processed From there it goes on to pending callbacks. So these are going to be your, you know in node js callbacks It pretty much we pass the functions down to libv as well. So are there any callbacks that are ready to run? Next it'll look for it'll process idle handles. These are the things that I said before had kind of a bad name Because they get processed every time through the event loop And then we do something called prepare handles. So this is Basically, we're about to go into a polling for i o Prepare handles give you kind of a hook into the event loop if you want to do anything before pulling for i o So then we move into the actual i o polling and then when we come out of that we have check handles So that's kind of the inverse of the prepare handle So these that gives you a good way to hook into your event loop Then finally any close callbacks that are outstanding we execute and then we loop all the way back up to the top Compute the time again and and basically start from scratch and that is That's basically one tick of the event loop This is all that i'm really going to cover on the event loop But if you're interested in more about it, I would recommend Burt Belder's talk from 2016 node interactive It is you know a few years old now, but the the structure of the event loop hasn't changed So the the information there is still relevant So next I want to talk I talked a little bit about how libv works now I want to talk about how it fits into node j s So at the very top of this diagram in yellow, I have uh, that's going to be your applications java script code And then it's going to call down into nodes standard libraries. So, you know the fs module dns Child process all those And that's going to be the the second layer of yellow there from there It's going to call down into the purple layer c++. That's the binding layer It's a really ugly code to kind of interface between, you know java script and v8 and libv And then the binding layer I I kind of listed some of the major libraries that are part of node here I put libv on the left by itself because it is You know what we're talking about here, but in reality, I would say v8 is the biggest dependency And then probably libv But you can see some of the other dependencies here. So, uh, c aries is a Dns resolver So you can actually there node has two different ways to do dns lookups There is the way that goes through c aries, which is always going to make a network request And then there is the the the one that libv implements, which is going to be dns lookup in node That actually uses the system resolver So it's going to use the same lookup mechanism as like ping and any other, you know applications you might be running on your computer One thing to note about that is Until I don't know about a year ago It was actually possible that if you issued a bunch of dns lookup requests That it would you know go down into the thread pool And so if you're doing like five six seven eight dns lookups It could actually block other things in your application that we're using the thread pool from running So people would issue a bunch of dns requests and then start doing file io And they wouldn't understand why their file io wasn't working It's because the threads were tied up with dns lookups So that's where it would be a good use case to use c aries because c aries doesn't go through the thread pool But either way that's no longer the case. So we've now Inside of libv started to distinguish between different types of thread pool operations to make sure that they don't step on each other's feet too much And then finally at the bottom is just the operating system Libby v calls out to that but you know for the purposes of this talk, it's not really important to think about So on this slide saul is back to talk about the onion architecture Where basically the more layers that you peel away the more you cry and The reason for this is Inside of node you might have used something like net dot socket which has a nice javascript api It's built with streams and it's just fairly easy to use But if you look in if you look closer at the source code, it references something called a tcp wrap which is In purple. So it's one of those nasty binding layer objects So this is written in c plus plus. It's a lot less nice to work with And then that actually wraps something called a uv tcp t Which is a libv type written in c for interacting with sockets And then inside of libv. We actually wrap that inside of an os specific handler or os specific handle So, you know a windows socket or a unix socket things of that nature I want to talk a little bit about how we test this thing. So We had issues in the past Where, you know, we would run we would create a release of libv And you know all the tests we would pass and then we would go to update node and then all of a sudden, you know Things would break inside of node libv has close to 400 tests But they're written in c c is kind of a pain to to deal with I mean, I think we're mostly javascript developers in this room So it's it's tedious And like I said, we had issues where for example libv 119 came out Everything was fine went to upgrade node and the ci, you know came back red So we didn't actually land that but unfortunately there are people in the community who as soon as a new libv release comes out They build with that version, you know, they compile node with that version of libv themselves So even though we hadn't strictly broken node, we did break some users Who were, you know, a little more brave? And so We very quickly reverted and got a new a new version released and you know life was good again But it was it was an issue that happened more than once and it was kind of frustrating to deal with So We kind of thought about it for a little bit and we're trying to figure out a good way to To make sure, you know, that node was going to be fine before we actually Made a libv release And so the the build node jas build working groups stepped up big time for us there They actually created a new ci job where before we create a libv release We actually take whatever is the latest in node and whatever is the latest in libv Compile them together and then run nodes test suite and see what happens So node has, you know, over 2,800 tests. Uh, they're it's easy to write tests in node because they're javascript And ever since we kind of took this approach we haven't had any issues where, you know There there have been bugs, but we haven't had any of those known regressions that cause us to do cycles of multiple releases So that's been a nice process improvement So now I want to uh, actually trace through uh, thread pool operation So all the way from user land javascript code all the way down to the thread pool So in this case, we're just going to do a copy file operation So we're going to use fs.copy file. Uh, there's three different variations of that. So there's the synchronous version The promises based version and then a callback based version And so the the first one shown here is the synchronous version Second is promises and then the one at the bottom is the old school callback Based approach So from the code that we just saw the first thing that would happen is we would call into nodes fs module In this case, I'm showing the code for the promises based version because it's just it fits nicely on a slide a little better Um, but you know, if you look inside node, there's similar code for synchronous and callback versions Um, but so all we're doing here is we're passing in source the destination So, you know where we're going to copy the file to and then certain flags that that uh, the operation takes So for an example flag would be, you know, if the file exists already, do we want to overwrite it or not things like that? Uh, we're going to validate both the source and destination paths inside of node Uh, the flags we're going to make sure that the flag is an integer. So that's what the or zero is that's kind of a uh, javascript trick, um, javascript doesn't really have Different types of numbers, but under the hood the js engine does So the or zero will actually convert it to a signed 32 bit integer under the hood And then we're going to call binding dot copy file and pass in the the you know normalized paths the flags And then a special symbol here called k use promises This is just a symbol that um under the hood will tell the binding layer that we're doing a promises operation as opposed to synchronous or callbacks And so if you look through the node code base, these are the three different ways that we would call binding dot copy file You'll notice that the first three parameters in every case Are the source destination and flags because that's going to be the data that we're operating on But then the you know the remaining arguments differ between the implementations So I already mentioned k use promises is a symbol that tells you to use the promises implementation The the synchronous version passes undefined followed by a context The context is what will be populated with any results and errors that might come back from the binding layer And then the callback based version just passes something called rec Which is a a c++ object that's used for in the callback code So from here, we're going to be leaving javascript. We're going to be entering the the really ugly c++ code that I talked about a little bit Can everybody in the back read that Not that it's nice code, but So this is actually the c++ code that gets executed anytime you run copy file The first you know collection of statements You'll see a bunch of check g e check not in all things like that. These are Basically a last ditch effort to validate user input If any of these checks fail node will hard crash And we're okay with that for a couple reasons First off, you're really not supposed to be using the binding layer directly So if if in your code you're calling this directly, then you know, you're kind of on your own The other reason is we have already done we've already validated all these same things in javascript So we're just trying to make sure that you know, people aren't going to be passing in garbage data to us From there, we're actually going to then call something called get rec wrap And that is how what we're going to be able to use to determine At what type of operation it's going to be so synchronous asynchronous And then if it is asynchronous is it callbacks or promises? Um, and I'm not going to go too much into what all these different arguments are because you can see there's there's You know a lot of them, but you'll see that copy file is there You'll see the source and the destination the flags And then most importantly you'll see uv underscore fs underscore copy file That's the function inside of lib uv that this operation is going to execute So from here, we're actually going to be leaving node completely and going into lib uv Um, and this is that same function that I just mentioned. This is What the signature looks like So the first parameter is going to be the event loop For synchronous operations that can actually be null, but It'll be there in all the calls The rec the second parameter is It's going to be basically a file system operation request. So, you know earlier I talked about handles and requests This is one of those requests So node is not really responsible for attaching all of the information to the request There are some macros down later in this code that I'll go over that kind of populate that some more But then the source and destinations are passed as path and new path The same flags that we gave in javascript are passed as the flags and then the uv fs Callback that's going to be either a callback function or null if it's synchronous From there, we're going to call in it. So in it is a macro It's going to populate that request that I talked about with it's going to basically Tell the request that this is a copy file operation So everybody who calls into libv doesn't have to specify what operation they're running It'll just you know automatically know by the the function that you call Then we do some some flag validation So we want to make sure that you know people are passing in garbage values in the c layer too So if anyone passes in a flag that we don't recognize we'll return e and val Next we do cap fs capture path The point of this function is to basically take the path that was passed in and make a copy of it because If we're doing an asynchronous operation There's a chance that that that memory could be freed by the time the the operation completes If that happens, you're probably going to run into a hard crash And then finally we we add the flags to the request and then we call post So post is another macro that is going to send your work off to the thread pool It's worth pointing out. This is the actual full implementation on windows There are similar ones on on mac and linux But because the windows code is a little smaller and cleaner i'm going with that So this is what the post macro looks like Basically it checks the callback if it's null or not so it knows if it's synchronous or asynchronous And then if it's asynchronous it's going to register the request with the event loop So once it does that it it'll keep your operate your process alive If the while the work is still happening and then it'll call work submit which basically takes You know your request and sends it off to the thread pool You'll see uv fs work is one of the parameters That's basically a giant switch case where it looks at what type of operation you're trying to run and then You know calls whatever the code that you need to call for that And then uv fs done is what's going to be called once the work is complete in the thread pool That'll basically propagate you back to the binding layer and then back up to javascript If you're using a synchronous operation, we just call uv fs work directly So it'll do the copy and then it'll just return the results directly because there's we don't really need to involve the thread pool if we're blocking So this is the windows internal copy file implementation We're again going to do some flag validation. This is going to be operating Operating system specific flag validation Because certain things are supported on unix and mac that aren't supported on windows And then you'll see the copy file w call. That's actually a windows api call that will handle the copy for you And then the rest of the code at the bottom is Because of a little bug on windows where it'll return e busy if you're trying to copy the same source and destination So we try to do the the copy operation if it fails with e busy Then we stat both of the both of the files and if it's the same then we know the operation actually succeeded And it's not a genuine error But from there, uh, we're basically, you know, going to go back up the stack. So we'll go back to the We'll exit the event loop go back up to the binding layer back up into javascript Um, next I want to talk a little bit about the possibility of a lib uv 2.0 So This has kind of been an ongoing thing for a while. Uh, lib uv 1 was released in 2014. So it's now what five years old The project has been api and abi stable since then with a few, you know Basically a few oopses on our side But In order to do some cleanup that we'd like to do and add some new features that are breaking changes We would have to bump up to 2.0 The problem with that is at this point that would be a rather large like delta But also we have a small team. So there are you know hundreds of people collaborating on node I'd say there's less than 10 collaborating on lib uv So it's it's a pretty big support job for the people who are working on lib uv to support a 1x and a 2o Even though some projects out there are already actually using the the the fake 2o So we have v1x, which is what node and most people are using and then the master branch in github is what would be the 2o I think julia lang at least is already using that But you know, we've we've been going back and forth on this for over a year now and We're starting. I think we're starting to come around to the idea of just staying on v1 forever There's you know, something to be said for stability and things like that It would avoid extra work on node side of having to create you know apis and napis so that So that lib uv changes wouldn't break node add-ons and things like that So I think now we're leaning more towards, you know 1.x forever But we are still adding a bunch of new features. So this is just some of them that have landed in the past year One of the one of the ones that I guess depending on your use case could be a big feature is The maximum thread pool size has been increased. So, you know by default, it's still going to be four threads But if you really need it, you can bump it up to 1,024 threads Prior to that change the maximum that you could do is 128 People have made the case that you know as computers continue to get better. We can run more threads We've added something called uv random. So this is going to be lib uv's answer to generating cross-platform random numbers UDP connected sockets are a new thing. So You know UDP sockets you generally think of you just kind of broadcast your messages out and maybe it'll be received Maybe it won't but it's actually possible to connect a UDP socket So that anytime you do to send it'll always send to the same destination We've added uv osu name So if you're interested in getting more information about what platform you're running on that's going to that's going to be useful there And then we've actually taken that in nodes os module is now built on top of that Uv get constrained memory is interesting. So One of the problems that People have is you know, especially v8 would set its Its memory limit to like, you know a certain amount of memory like whatever is available on the computer But some operating systems like linux have things called c groups And then there can be other you know right now we only look at c groups But in the future it can be expanded to other things where you can actually impose artificial memory constraints on your Application and so if something like v8 would try to use all of the system memory. It's not going to be able to do that anyway So uv get constrained memory lets you actually query the operating system to see how much memory you're allowed to use So that's a useful one Um threads can now actually set what they want their stack size to be So that's just a little tweak for you know, if you know how much memory you're going to need you can configure that Um a really big one was streaming reader. So this request goes back like five years Node has fs dot reader But under the hood that actually calls scander and that buffers all of the requests at once So you could see if you're trying to read a very large directory how you could run into memory issues Um, so people have wanted streaming reader. We had a poor request that changed hands like three or four times Um, and then it was also targeted at the master branch. So within the past year We actually you know got that under control got it targeting the v1x branch and we're able to get it landed and you know As of a couple months ago. It's now shipping in node Um, and then just uv get time of day. This is the uh, you know, basically the the c equivalent of date dot now Um, and uv fs mks temp. So it's a call to make um a temp file for your application And then I wanted to finish up with just One thing that's I guess tangentially related to lib uv. Um, but it's my talk my rules. So I want to talk about it um It's called uv wazzy. So wazzy is the web assembly system interface Uh, it's relatively new. It came out within the past year Um, and it basically gives web assembly applications a way to access the underlying operating system Uh, because by default, uh, web assembly code is sandboxed So I you know as a lib uv maintainer when I first heard about wazzy. I was like, oh, that sounds like lib uv for For web assembly. I was like, let me try to build that So I did I built it on top of lib uv for, you know, maximum portability Um, and then as of node 13 dot three, which I don't know came out in the past month or so It's shipping in nodes. So There's documentation you can do require wazzy and and play around with that if you want Um, hoping to do more work on that in the future The the github repository is shown on the slide Wazzy.dev is the kind of homepage for wazzy So if you're interested in web assembly or anything like that, I encourage you to go play around with that And that is all that I have. Thank you