 We have Girish Punathil. Hopefully, hope I said that right. Girish, are you around? Let's see if he's able to Join us here Can you hear me? Yes, I can. Excellent. Thank you for joining us today Hello, everybody. Hi John. All right. Enjoy. Can you create crowd here? Great job John. It's really enjoyed it. Thanks Excellent. So you're going to be covering Node.js core APIs, right? Getting right to the basics That's right. Yeah. So basically the essence of the topic is to cover the most important APIs which not necessarily based on how the project deals it, but based on how the ecosystem has been has been using it. For example, I do a lot of support for end users, helping customers and end users and even students to understand the asynchronous event-driven architecture, etc. And then these are the There are regular set of questions that come in all the time. For example, how does this specific API work? And then over the years, I have seen certain pattern that is settling in. So I thought it would be a good idea in these kinds of conferences to showcase the key APIs which are always hotspot and talk about the definitions the fundamental form of the examples and share some of the best practices Awesome. We're excited about it. Well, feel free to, do you have a screen share or slides for this talk? I have a screen share. All right. But it's popping up the system preferences window. Okay. I can see the entire screen or maybe the window. Okay. Can you see my PowerPoint? Let's see. I don't see it yet. Or in the Mac security and privacy, do I need to do any settings? You might. If you haven't shared your browser before, there might be an extra step there. And depending that step might ask you to reboot your browser. So we'll see. I'm not seeing it yet. If it's a simple deck, I could flip through the slides for you if you have a URL. I can do that. Yeah. Teamwork. Let's see if I can do this. All right. Now I'm, let's see if I can figure this out. I just slacked you. Can you see it? I don't see it yet. Let me make sure I'm in the right slack. Probably got the wrong slack open. Is this in the definition slack? Yeah, that's right. No, it's a definition slack. Yeah. It's still uploading. Maybe it takes a few more seconds. Oh, PowerPoint. All right. Let's see if I can, let's see how this works out. Google. Let's see if I can open this. When I shared through the green house, green room, this problem did not occur. Really? I'm sorry about that. Too bad we can't just present out of the green room. You're not still presenting on that one, are you? I have not. Yeah. I was trying to think of what might be taking over video. Let's see. So I got the... But it's okay if you can project it. Yeah, yeah. Let me see if I can figure out how to open it. I don't have PowerPoint installed, so I'm trying to go to drive.google and then upload this file. And I love the macOS Finder, but I can never find my downloads. It's supposed to go to the downloads folder. Here it goes, desktop. See if I can get it open. I did upload, and here we go, core APIs. Let's see if I can present this. Opening it now. Thanks everyone for hanging with us for this. It looks like I have it open. It doesn't look pretty, but I do have something open. Let me see if I can share this. This will be a little tricky. Let's see if this works. Let's get the share. Let's go for entire screen and... Swap over. Okay. How does that look? Can you see? Yeah, I can see that. The text is a little bit off. Sorry, I think that's a translation issue, but I can flip through, hopefully, on this end. Okay. Can you go to the next slide to see how much is the damage? Okay. I think it has affected all the slides, but yeah, let's get started. Good morning. Good afternoon. Good evening, everybody. I don't know if it is a morning for anybody. It's still morning for me, just barely. Okay. For me, it's nearing midnight. Anyway, thanks and apologies for the small glitch. Thanks for making it. This is as I provided an introduction. I've been helping a number of folks in the community. I'm a member of the Node.js Technical Steering Committee, and as part of that responsibility, I've been looking at various issues that are reported by end users, and I thought sharing the key essence of all those help and support would be somewhat useful for the community. And this fast crash course on the core APIs is essentially aimed to help the audience to understand some of the subtleties of Node.js as the asynchronous event-driven architecture and how the API subsystem is developed and matured over time to support that specific cool aspects of the language runtime, and the platform. So that's the essence. The idea is basically centered around top 10 APIs by virtue of how the users feel difficulty in understanding the abstraction at the language level to how to use those APIs in an application program. So from that perspective, we took these APIs and in the subsequent slides, I'm going through each of these APIs and define what are those in layman terms and provide an example and then share some of the best practices. So let's start with buffer. What is a buffer? Buffer is an abstraction around row binary data. So as we know, the most common use case for Node.js is for web applications and the data that comes from the client and undergo some business logic processing and eventually flow back into the client. In all these transactional routes, remember that data doesn't have a type in most of this part and data can be of any length that can be of any type. Data can be representative of any real-world business data such as images, multimedia text, or social media text, all sorts of things. Now, how do you cater to such a heterogeneous type of data in a language which is dynamically typed, which does not impose strict data typing? So that's when the abstraction around the data in its purest form or in its basic form makes a lot of sense and buffer is such an abstraction. Now, also added by the fact that the data flows through the network and the network needs to adhere to certain byte order, etc. makes it really a necessity for a language to treat the data in a raw form at least for majority, at least for a good part of its data processing. So buffer is a convenient API abstraction for dealing with data of arbitrary size data of arbitrary type and with certain specific operations that is possible on that. Now, if you look at the buffer APIs, you can see 50 plus functions basically about how to transform the data underneath, how to swap the bytes, how to concatenate bytes, how to translate buffer into different encoded values, how to create buffer from various known types such as string arrays and numbers and things like that. So if you're in a program where you are actually wanting to transform these things into concrete data types and finally put it into a data structure with specific types, those APIs will come handy to be applied onto the buffer. The next slide please. So some of the best practices around the buffers as we have seen in the field is buffers can hold large amount of memory and it's essentially important to know where you're creating the buffer and what is the life cycle of the objects that you're creating because it has implication to the JavaScript heap. There is one specific variant of the buffer called Alok Unsafe. There is a trade-off between performance and security. If you're creating the buffer through the normal process using Alok APIs it allocates the required amount of memory and it also initializes to zero. So the additional effort is required to initialize, but then the data is clean and you are not exposing any previous data that is lying permanent in your system. Whereas if you're using Alok Unsafe API, you might be using the raw memory available in the system and potentially exposing some of the private data that was stale as you can see from the screenshot. Next slide please. So that's pretty much about the buffer. The next API is the stream. This is very important API just like the buffer. While buffer is talking about how to abstract the data, stream is caring about how to worry about the data transport or how to deal with the data transport. So as we again revisiting the common use case of Node.js, the data comes through the network, the data undergoes processing and eventually flows back into the network for the destination clients. Now, as anybody who is moderately skilled in the art of network programming would know that the data doesn't come in one shot, it comes in chunks. Based on the network settings, based on the TCP implementation specifics and based on the bandwidth of your network, data can come in 64 kb chunks or 1 kb chunks and whatever irrespective of what is the size of the data that you sent. So that means there is an inherent nature of the data that comes through the network as chunked. And then think about a streaming scenario, think about a transformation scenario where you're changing a video format into another. In all these cases what we see is a huge amount of data is coming and it is getting translated into another set of huge data and you will need to hold all these data into the memory which can be really an overhead to your virtual machine itself. And think about so many concurrent requests coming for similar operations it can really break the system. So instead, as in when the data comes from the network if you have a mechanism to operate on the data that is available right now and then push it to the destination or the sync and then wait for the next set of data to come in in its natural course of action from the network and that becomes the most reasonable thing to do for such data and the stream is exactly that. In the example we can see that we are reading from a file as if it's a stream of bytes and then we are wanting to write it into another file and we are just piping both the streams so the data actually flows from one file into another with respect to the size of both the files through this mechanism exposed by these APIs. The stream is not native to NodeJs. The concept exists for years and the cool thing with NodeJs is that it's one of the first-class citizens one of the fundamental API abstraction and it's a perfect fit for network programming. Next slide please. Some of the best practice one of the noteworthy thing is around stream is a complex class if you are inheriting or if you are consuming the stream object it is better to know what are the life cycle events of a stream from the creation up to the destruction specifically around how the data flows and how do you intercept the data flow what is back pressure, how the NodeJs API itself is taking care of the back pressure what is water marks and what are various events such as drain, pose, resume, etc. So if you understand these life cycle aspects you can actually work wonders with the stream if you are dealing with large network data and you can super tune the efficiency of your program as well and remember that the stream do not impose any data types so what the type of the data that you are pulling through the stream the stream assumes that specific data type Next one please. So the third one is cluster API Why do we need the cluster API? So one of the noted drawback of NodeJs API is that it is, sorry, NodeJs platform is that it is single threader for the application that means it is doing good the single thread is consuming most of the CPUs allotted to it but then most of the modern computers are multicore and high end server machines have tens hundreds of CPUs if you are installing a NodeJs application as your mission critical service 99% of the CPUs are idle is it optimal? That was a question that is addressed by the cluster API not necessarily so how do you exploit the multicore that is available in the modern hardware by making use of the cluster API and fork out as many child processors as many worker processors as you have the CPUs the example shows you are forking 32 children processors now how does it help? the children processors which are spawned by the cluster API are not just for road processors they work in synergy they work with high synchronization with the parent process and the whole parent and the children process work as if we are running a distributed computing for example if you pump 100 requests to the server the single port listens to all the requests and once the request is arrived in the master or the parent the request is dispatched to any of the child process or the worker process and the request response cycle is managed very well think about Kubernetes or the open shift where the load is balanced in an intelligent manner and that whole complexity is completely hidden from both end user as well as the application developer so the cluster API provides a similar mechanism though not with that level of sophistication as the Kubernetes provides but at the API level it provides the complete lifecycle operations that is required for vertical scalability in terms of best practices cluster is not the solution for all sort of high level of concurrency that is required because at some point in time you will need to look at horizontal scaling for higher availability and other peak load scalability etc. and don't mix the cluster with the child process because cluster itself is a little complex and adding child process into the cluster can really make your code a little messy and the last one is sessions are not managed automatically within the cluster you will need to take care of the sessions moving on event event is a program abstraction around the event-driven architecture and it follows an observability pattern observable pattern is a first class semantics that is well understood by the JavaScript programmers and this is the piece of code that hide the complexity of the event-driven architecture if anybody has lack of understanding about how the event-driven architecture event-driven programming works look at these four lines of code and see where the event is getting intercepted how the event is propagated and how the event is handled so everything comes in four cool lines of code and then read through the event-driven architecture again you should be able to understand it completely in this code we are emitting an event and we are intercepting that as simple as that best practices event listeners use closures one of the known problem is memory leaks because each closure remembers the context in which the closure is created and that remembrance can cost a lot of memory if not used properly that is one of the main best practice I would say next slide please timers are very popular APIs futuristic execution you have two types of timers one is one time execution into the future second one is repeated execution in the future set time out and set interval very intuitive powerful APIs used across the board in many many applications nothing to talk about here but best practices very key information is there for example many people ask me the set time out is not working because it's not returning at the time that is expected they have made program logic constructs which statically which assumes how the code order would happen based on the actual timer interval that is wrong because when you say set time out 60 milliseconds 60 seconds here it is an indication to the underlying platform you cannot hard assume that it is going to be exactly after 60 seconds starting from the point of execution of this line of code that depends on a number of factors in the underlying architecture software stack middleware stack and so many other things and the rest of the code in your application as well it doesn't mean that this is just a random and arbitrary there is a specific order that can be determined but definitely these are not hard real time values so that's one of the very strong key take away from the timers next slide please next slide please one more npm needless to say everybody knows about npm nodej's package manager is the default manager package manager for nodej's and one of the most most popular software library most the library with maximum number of reusable modules the last time I checked it is 1.5 million why do we need this because javascript ecosystem is proliferating and everybody made their contributions what is a good thing about npm is that when you start thinking about writing serious applications any sort of applications in any domain you realize that you don't need to write anything from scratch every business logic that is posable to think about is already implemented in the form of a module and kept it in the public registry you just need to kind of understand which API implements what and just make use of it so that's the one of the most powerful aspect of the platform itself there are several best practice around nodej npm main thing is make sure that the modules are maintained modules are maintained from the security perspective and there is an active maintainers group around the look at the GitHub of the module and see it is actually maintained that would be one of the main advice I can provide on that there are well known modules with zero maintainers there are you know very low very less popular modules with a lot of active maintainers so the key would be to strike a right balance is it something useful for you and is it maintained if it is not maintained it's going to be a problem for you in the future moving on NetEverdowns just like nodej this is a platform that abstracts various capabilities including the operating system underneath it doesn't mean that all the system calls and all the operating system capabilities are available through the limited set of javascript APIs always there are one of the other use cases where you still want to make a operating system call that is not available or you want to do some custom task in your own manner in probably C++ so that's the time you call a foreign function interfacing for a native implementation which most of the managed on time such as python and java also do and the javascript the nodej's way of doing it is through a native add on so here is the C++ counterpart that you implement bind it with a javascript API and expose that API to the external world any javascript program can directly call the javascript API internally it can translate into the C++ code and get your things done so one of the best practices don't use directly deal with v8 because v8 version changes are not in line with the nodej's version changes because these are developed by two separate communities so there is a beautiful wrapper called nan and node API etc make use of that they basically abstract the v8 version changes and help you use a high level abstraction over there and the resources that you make use of in the native add on is something which you need to control yourself as opposed to the nodej's or the underlying platform next one is json json stands for javascript object notation I don't need to provide an introduction to that it is one of the coolest thing in javascript the good thing with json is it is native to javascript that means if you have a json object it is directly serialized and is serialized by the language by the application no need of transformation that is required the two powerful APIs in json is stringify that serializes the object into stream of bytes and parse that takes the stream of bytes number of bytes and converts that into an object some of the best practices json serialization operation can be costly based on the volume so try to minimize the volume of the object if it is not possible then think about some of the asynchronous library otherwise the single threader program can be kind of throttled just because of the json operations the next one is module module is just like a reusable API abstraction this is the semantics through which you basically expose your capability so that other part other application can make use of it you use the export keyword and embed all your functions and other values that you want to export to external world provide a name to that so that the external world programs can require your module in various ways that I have shown and then start using your APIs some of the best practice around modules are the most important thing I want to say is the avoid global state in modules avoid side effects for example process never exit from a module because you don't know who is going to consume your module that means you need to well define the scope of your module who will be calling at what point your module will be calling and at what point your module will be exiting so exactly do what is within the scope of those APIs next one please the last one being the promise as being a promise is a mechanism to sequence plurality of asynchronous functions asynchronous operations one of the known problem when people migrated to JavaScript and asynchronous style of coding was readability was very poor they don't call back inside a call back inside a call back the promise provides a convenient way to sequence the code that should appear one up to the other with a little trade off on the performance for example you need to register the promise you need to do the plumbing work internally to sequence these things and then recreate the asynchrony under the cover so that cost a little bit but otherwise looking at the code we can say the next one comes first and the next one comes next and things like that that was a great relief for a huge set of programmers who came from non-javascript background next one the best practice I already said about the trade off between performance and readability so look at evaluate which is important for you if performance is super critical for you still look around call back style of programming I love call back style of programming but look at around and figure out which one fits best mixing call back with promises is not a good idea it can lead to you know various problems such as code doesn't behave according to your expectation not necessarily there is a bug underneath but because you don't understand what you wrote after some time that's pretty much it is I just want to conclude my talk saying that why should somebody look at Node.js seriously these are the reasons I would quote one of the top cloud runtime today is Node.js because of its cool capabilities such as high scalability and high high concurrency and it's one of the top GitHub project high developer productivity because of the NPM reason I quoted you don't have to write all the code for the business logic that you have it's already out there you just need to figure it out full stack development for web applications front end and back end people think top and work in the same manner so that there is another productivity aspect to that highly performant I already told about that and there are hot cool jobs out there so if you are not looking at Node.js seriously these are the reasons the community is very very open very inclusive very accommodative we have special programs for newcomers we have special code contribution opportunities kept open for newcomers so that they have a very wonderful good exciting experience working in the community so I strongly encourage and advise everybody to look at Node.js as a serious thing for you I just want to conclude by saying that thank you so much