 Hello everyone, welcome to Smart India Hackathon 2019 Designing Digital Solution Program. My name is Archana Ghatigar. I work with persistent systems as a technical trainer. I have around seven years of experience in trainings. I mainly train on front-end technologies and the mean stack related technologies. In today's session, I am going to be talking about the very popular server-side technology that is Node.js. This session is divided into two parts. The first part, I will be discussing mainly on the overview of Node.js and touching upon the concepts of what is single-threaded architecture, why Node is so popular, what is server-side JavaScript and so on. In the second part, we will be mainly looking at some sample demos which is going to see the Node API in action. So let's begin. So the agenda for the first session will be understanding what is the concept of server-side JavaScript. Then we will look at the Node architecture. We will also try to understand why Node is so fast and scalable and we will see how to use the asynchronous Node API. And lastly, we will conclude with looking at some use cases for Node.js. Now JavaScript, JavaScript is today's hero of web application development. It is a buzzword everywhere and JavaScript seems to be used at a lot of places. Mainly the features of this language, that is the usage of JavaScript so far has been mainly for client-side scripting, which takes care of event handling, form validations. You would have used it for Ajax requests mainly to create the XML HTTP requests and to update the DOM for updating the pages and so on. So JavaScript is a language on its own and Node.js is a program for running JavaScript at server-side. So Node.js is not a language, it is a program which mainly runs the JS. Now not on the browser but it runs it at the server-side. So what is the server-side JavaScript? The first incarnation of JavaScript lived in the browsers. So browser was the context for executing the JS code. But JavaScript is a complete language, it has its own features and advantages just like any other language. So the authors thought why not use JavaScript on the server-side. So now the context is not the browser but the context is Node.js. So Node.js mainly executes your JavaScript but not in the browser but in the back end that is in the server-side. So Node was created by Randall starting in 2009. It is a command line tool that lets you run JavaScript programs. So remember Node runs JavaScript but is not JavaScript alone. Now what is Node.js? Node.js is just not a runtime environment, it comes with also a library, library of some inbuilt modules. So just like how any other technology like you can take Java which comes with a set of basic classes, basic packages, similarly Node.js also is packed with a library of inbuilt modules. So we can start our program by using these libraries, using these packages and build upon your applications. An important point when it comes to Node.js is the Google's V8 engine. Now so far if you have used JavaScript you would know that there are certain steps that are required while executing that is it needs to be interpreted, it needs to be executed and of course rendered into the browser. So if you have a browser like Chrome it would have its own JavaScript engine and Chrome has the V8 engine that is running inside. It will be interesting for you to know that when you use Node.js what runs inside is the same Google's V8 engine. So the performance benefits that you get inside the browser are same achieved inside Node as well. Let's move to understanding the Node architecture and dive into what really happens inside this particular technology. So I have referred to a simple analogy here using the doctor's office reception line as an example. Let's consider a simple queue that exists at any typical clinic or a hospital. There are people waiting in the queue either for making inquiries or either trying to take an appointment with the doctor. And the general process is to fill up certain forms before visiting the doctor. So the form filling procedure is quite time consuming due to which other people in the queue are blocked. Maybe a person wants to simply make an inquiry but he is simply blocked because of another person trying to fill the form. If you notice even the receptionist's time is not optimized because he is simply sitting idle until the person finishes. Now how can we optimize this process? How can we make this entire process better? What can be done is we can distribute the forms for people who are waiting to visit and ask them to fill them up in the waiting area. So as in how they finish, they can come back and join the same queue. With this, the receptionist's time is optimized because he is always serving the people either for inquiries or maybe for appointments or any other miscellaneous tasks. And the people are not blocking each other because they go and sit in the waiting area, fill up at their own pace and come back and join the queue. So what all can we infer from this analogy that operations or tasks that are time consuming can be moved out of the main queue? When the long running task is completed, so that task comes back to the queue for further execution. Remember the queue is single. We do not have multiple queues for serving different types of requests. So the same queue is used for newly arrived people or requests as well as the old ones to complete. And a single person is handling or serving these requests of the people. Now Node.js follows a very similar architecture that is it follows a single threaded architecture. That means a single process or a single thread is executing all the operations and the requests. Let us understand this a bit more by moving into a bit of in the program. Let us consider a typical program which consists of a set of statements here. So there is a console.log, there is a read file, there is a process file contains, compute some and lastly we have another console statement. Now JavaScript has got a single call stack. So to represent that we have a call stack and all the statements are added onto this call stack one by one. So first the statement console statement then we have read file which is executed popped out, process file contains, compute some and last statement as well. So remember in JavaScript we have a single call stack which executes one method at a time. This is also called as synchronous way of executing. That is one by one one after another statements get executed. Now when we are using Node.js we are going to execute the same JavaScript on the server side code. Now when we are talking about server side technology then it would involve a lot of IO operations. IO operations can be file IO, it can be read write append, it can be database operations wherein you are reading from the database, writing to the database, updating, deleting and so on. Remember it can even be network IO wherein you are reading and writing over the network using HTTP or sockets. And remember any IO operation that you take are always blocking calls. So to visualize a blocking operation we have statement one, statement two and after that we have certain IO operations and you can see that it is blocking the next two statements that is three and four. The overall time taken would be 60 milliseconds. The same operations which are non-blocking that means it would be occurring in asynchronous fashion you can see the gap is reduced. That is because the IO operations are going to be executed asynchronously. All the APIs in Node.js which are pertaining to IO are asynchronous by default. So we can see now the total time has come from 60 to 40 milliseconds. So let us see the same demo in execution. I have the similar statements, there is a console.log, there is a read file method, there is a process file contents method which is simply printing this file data. I also have a method compute sum and lastly another console statement. So the functionality is nothing butch as of now just to keep the demo simple. I have some console statements here as well in read file which is simply printing a hello world data. Processing file contents is also doing something similar and computing sum is just printing that it is going to compute some numbers, sum of some numbers. So if we execute this as it is, so this is the output. We have the console statements, then we have all the methods getting called one by one and lastly the program call stack ends. So here you can see there is a single call stack used by JavaScript and one by one functions are called in the synchronous manner. So the same sequence is what you can see here in the output. So Node.js is a single process, a single thread that is executing these functions, these methods one by one in the synchronous fashion. Now let us see a demo which is going to stimulate the file reading process and we will see how it causes a blocking operation and blocks the other APIs as well. So something similar that you saw in the last demo, some statements and some function calls like read file, process file and compute sum. The logic that I have added is read file here and I have added a very huge loop. An empty loop which is not really doing anything, this is just to add the delay to simulate the reading of a file. Let us run this and see what really happens. So we can see the first statement getting printed that is program call stack starts, then it starts reading a large file and now it is waiting till the complete file reading is complete. That means it is waiting for the loop to finish and you can see now it prints the remaining part of the call stack that is processing file contents, computing sum and the program call stack ends. So this is called as a blocking operation. Whenever there is an IO involved, file IO, DB IO or network IO, then this is how it causes a blocking operation and all the other methods are simply delayed until this finishes. So if we try to see it logically here, there is this read file and process file contents of course has to wait till it gets data from the file. But what about compute sum? Why should compute sum be delayed until reading of file and processing file contents finishes? So we can see here that there is a delay caused for this function unnecessarily. So if we can move this reading of file into another separate process or a separate thread, then all the other remaining methods which are independent of reading of file can occur seamlessly. So that is exactly what Node.js does. It moves all IO operations into a separate asynchronous execution. So I have tried to simulate this as well, the asynchronous execution into another demo here by using an asynchronous method which is setTimeout. So we have the same read file which has the similar big loop here. The only thing is it will be executed asynchronously because we are using setTimeout. Please note that in JavaScript, timers are executed in asynchronous fashion. So to simulate it, I have used this function here. The rest all remains the same, the program is the same as it is. Now let us execute this and check the behavior. Check we have the console statement reading a large file. Now it is asynchronously and it is not blocking computing sum of numbers. Even the call stack is ending here but the program has not yet finished. You can see that the cursor is still blinking here. So once the loop finishes, reading of a large file is completed and we have the program execution which is finished. So that is the difference between synchronous execution and asynchronous execution. In synchronous, if there is a large execution, time consuming execution it will be blocking all the others whereas in asynchronous it is not going to block. Let us have a look at it again. So that is how we can see what we really mean by asynchronous execution. But we have a problem. What is the problem? We have processing file contents which is simply getting executed and having an undefined value. If you see the code, there is read file and after that there is process file contents. Because this is going asynchronously, this particular method call also gets executed and because file reading is not yet complete, it will get the file data as undefined and that is what we can see here. How do we solve this problem? We want processing file contents to be called only after read file is complete and not before. So to solve this problem, there is a concept called as callbacks. If you have heard of callbacks which works in this way, after the file reading is complete, we invoke this callback. That means, so what this means is only after file reading is complete, this particular method will be called. So that will take care of the undefined problem and not. So let us see that. So we have first statements, reading a large file, computing some. Now you can see processing file contents is not getting printed, that is because it is not getting invoked. Let us wait until it finishes. So here it is. Reading a large file is complete and only then we are calling processing file contents which is simply printing out hello world. So have a look at this callbacks. So callbacks is used heavily in Node.js while using the methods asynchronously. So inside a read file, we pass the callback and once file reading is complete, we simply invoke this callback here by passing the data that which has been read from the file. Remember this is just a simulation of file reading that we have done and in the subsequent module that we are going to be covering, I will show you how the actual file APIs work inside Node.js. This demo was mainly to only show the difference between synchronous and asynchronous executions. Now how does Node.js really handle all these IO operations asynchronously? So although at the top level we are saying Node.js is a single threaded environment, internally Node.js does make use of a thread pool. So you can see here there is an event queue which is having all the operations, all the APIs listed or the requests that are coming in and we have an event loop that is executing these requests one by one. As soon as it finds there is an IO operation, it will delegate it to an internal thread pool. So a particular thread is used for executing all the database calls, file Ios, network and so on. But this thread pool is abstracted from us. So internally it makes use of it but for us as the developers, we are going to be using a single thread, single process at our level. When the operation completes that is any of the IO operations are finished by the threads, they come back and they return to the original event queue. That is the single queue that we have been using so far. And that is how the event loop continues to process all the operations. But not everything goes to worker threads. There are some asynchronous operations that also occur at the OS level. So node internally either makes use of a thread pool or it may make use of the OS calls for executing operations in asynchronous fashion. So it is up to node on deciding what really goes to the thread pool and what really goes to the OS level. So this is known as the asynchronous processing model internally used by node. Now let us look at some use cases of this technology. The most common usage is building a web application server with node on the server side and we can have any of the client side frameworks on the browser side. A very common usage is also a RESTful server, wherein node only acts as the RESTful server and serving using the APIs. And we can have any of the frameworks like maybe Angular react on the client side for building the client side applications. Single page applications are very popular nowadays and node forms an integral part of such kind of single page apps. A chat server where there is a lot of data being exchanged. So there is a lot of network IO involved. So that is where again node plays a very important role in building such chat applications. And yes I am sure everyone would have heard the mean stack keyword where M stands for MongoDB, E stands for Express, A for Angular and N for Node. So we have a server being written in the Node.js and over on top of that we have a framework called as Express which is used to build the RESTful API. For database we use MongoDB and Angular on the client side. So these are some popular use cases of Node.js and these are some popular names who all make use of this popular technology. Now the question is when to use Node.js? Do we replace all our existing server side technologies with this? Not really. It means that it is not a single silver bullet that is going to solve all the problems. Instead it is a platform which is actually going to fill a particular need. So applications which require very fast response which requires scalability which have a lot of network, a lot of IO involved. Such kind of applications are suitable to be built with Node.js. And not for applications which have lot of CPU intensive operations because if it has then it will nullify all the advantages of Node.js architecture. So something which is very heavy computation which is going to be long time consuming tasks are not really suitable for building with Node.js. So the edge that this technology has over others is one is efficiency that it takes because of the raw speed that you get with V8 engine. It uses JavaScript. So if you are a developer who is already working on the front end and already knows and has the knowledge of JavaScript then it is very easy for you to move to the server side technology as well. JSON data which is very popular nowadays for exchanging data between client and server also helps us to ease the data conversion between both the sides. Since we are going to use JavaScript on client as well as server then the JSON is a very common format that can be easily used seamlessly to exchange data. And yes the raw speed that we already talked about that is Google's V8 engine is being used here which gives us the advantage over other technologies. So having looked at the concepts behind Node that is single threaded architecture, why is it so fast, what are the different concepts like server side, JavaScript and so on. Now let us look at how do we actually work with this technology. What is the software, what are the prerequisites for this? So one of the basic things is we need to have at least basic knowledge of JavaScript in order to work with Node.js. So it can be very simple things like how you write a function, how you create variables, how are the method calls and how do you create objects, JSONs and simple things like that. You need to have the Node installation in place. The ID that we are going to use is Visual Studio Code. There are a lot of other IDs which are available with support Node and give you the context help. So here I am preferring Visual Studio. Let us get started with a very simple Hello World program in Node.js. Let us see how we can write code and how do we execute from the command line. We will see this example first and then we will switch on to the IDE that is Visual Studio Code to execute the rest of the demos. So here I have a file which is created with the name Hello World.js and a very simple statement I will write here which is a console.log. Now when we execute JavaScript on the browser then we would see our console statements in the console tab of the browser. And now let us check how do we run this and where can we see the output of this console statement. I have my command line with Node.js already installed. So I can check my Node version by using Node minus V that will give you what version of Node is installed and I am already in this directory where this file is being placed that is HelloWorld.js. To execute the Node program we simply run the command Node followed by the file name that is HelloWorld.js the extension can be skipped here and we can see the statement of console statement is being displayed here. So this JS file can contain typically any of the JS features, language features like we can write our variables, we can define if else statements, we can have switch cases, we can write our loops of course functions, we can define objects and have them converted into different formats we can make and so on. So now we talked about Node.js being two things we said it is a runtime environment and also it comes with a built-in API in the form of modules. So let us explore that the API is in the form of modules which is divided into three types. We have built-in modules there is third-party modules which is available via NPM. I am sure everyone is familiar with NPM but we will anyways be looking at an overview of this and how we can write our custom modules in the Node.js. So let us start with the first one how do we work with built-in modules. So built-in modules they come prepackaged with the Node installation and in order to use any of the prepackaged or available modules we simply need to add a require statement. Some common inbuilt modules over here are HTTP files for accessing the file system that is the file API that is FS module to access OS related information we have the OS module and so on. So just like an import statement in any other language here we refer to it with the require statement. So if I have to make use of the HTTP module in order to access the API related to HTTP requests I will add the line require followed by the module name. So we can see all this documentation provided on the Node.js website. So here are a couple of modules listed on the left hand side. So this is HTTP module here which has a list of methods, a list of properties, a very simple one that is OS if you want OS information like what is the host name what is the platform. So these are the different methods available. So module is nothing but a simple package containing a set of methods and a set of properties. So to make use of this let me add a require statement for the OS module. So this means I am importing the OS module and now I can print certain information like what is the OS type by saying OS.type. So all this list of methods are listed here we can always check out what each one is doing and accordingly make use of it. Let me also print the host name here and let us rerun this. So we can check here the OS type is getting displayed along with the host name. So this is how we use the require statement and use the inbuilt modules. So these are the inbuilt modules in Node.js. Similarly, if I had to use the FS module that is the file system then I would do something very similar here. This gives me access to the various APIs for working with files. So let us have a look at that. We have file system. So it has a stats property with all these statistical information like is it a directory, is it a file. Then we have various other methods like copy file, append file, exist. Then the very common make dir, read file, read directory and so on. So every module is used for certain purpose. It has certain common functionalities. So here FS mainly deals with working with the file system. So this is what we already looked at. Next is the second type. Sometimes we may require some functionality which is not really available in the default Node modules. So what comes prepackaged with Node installation might not be sufficient. So there is a community called as NPM. So it is basically like a repository you can say where people have contributed their modules or packages. So in the form of short JS functions or libraries people have contributed and hosted it on the single repository called as NPM. So our job is to simply search for the required package that we are looking for, take the version that we are interested in and simply get it or download it and use it in our projects. So this is the second type of module that is the third party modules from NPM. So it is a growing repository every day people have been contributing to this and you will find a huge list, huge collection of packages where people have written a lot of utility codes, lot of code or logic which is frequently used. Now the question is how do we get this NPM? So we do not need to do anything specific. NPM comes automatically with the Node installation. So if you have Node then you already have NPM along with it. So NPM is a command line tool. It comes with a set of commands to basically install modules, to uninstall modules, to update modules, to delete prune modules, to check the version, to update the versions and so on. So the first step is how do we install a module using NPM? So the command is very simple. It says NPM install followed by the module name and after we install it then we simply need to add the required statement and make use of the API. So earlier we directly made used of the required statement and provided the default or pre-packaged modules. Now first we need to install it, get it from NPM that is it downloads it and places it in the local directory and then we still need to add the required statement. So let us have a look. So here I am going to try and install a module called as MathJS. So I am just taking a very simple module here that is for doing mathematical calculations. Let us say I want to find square roots, cubes and I want to find logs, you know and other arithmetical calculations and I quickly want to use some utility methods. So what do we do? We can simply go to NPM and look for the packages that we want to. So here I can search since I already know this package MathJS. So there are lot of others which are listed. We need to use the one which is the most stable, which is most popular and look at the API. So the install command is given here NPM install MathJS and here are some of the methods that it provides which can be used for doing arithmetical calculations. So here in order to get I will need to run this command NPM install MathJS. So where does this get downloaded? This package gets downloaded from the NPMJS into the local repository. So if I look at my folder structure there is a folder which gets created called as node modules. All NPM modules are listed in this folder called as node modules. So if I try to look for MathJS I will find it here. So I have lot of other packages which are required for my demos coming in line and that is why you can see a whole list. So this is the place where it places it keeps all the NPM modules together. Now since we have executed NPM install MathJS and it has downloaded it into the node modules. Now let us check how do we use this. So I am going to close and make use of another file. So the required statement now will be require MathJS and let me try something very simple like finding the square root. So I will say math dot square root and we can print it. Similarly I can print the value of pi for example even very simple things like sum of two numbers can be printed. Let us run this program. So square root of 64 it is getting printed pi value and sum of the two numbers also is getting printed. So notice that now this is a module from NPM and not what is provided by default node installation. So this is the second type now that we are dealing with which is an NPM module. You may require to set certain proxies if you tend to get some error sometimes when we are trying to do an NPM install. So you can check out what is the command for setting proxies and accordingly do the configurations before running NPM install. So these are some useful and most popularly used modules like you know if you have to parse the HTTP request then you will be using body parser. To continuously monitor a node script there is nodemon, there is morgan, there is a very popular framework called as express which is also an NPM module and we will be looking at a demo in the next session. There is mongoose, there is socket IO, nodemailer, math, validator and so on. So there is a huge collection and it is up to you to go and explore and accordingly use them. Now comes the third type of module that is writing custom modules. How do we write our own logic and put it into a module and use it in our node projects? So what is a module? First of all a module is a very simple division of tasks where one file or you can say one script does a certain set of functionalities. It can be a very simple single function that it does or related functionalities can be all packaged into a single module. So it can be an authentication module or it can be a very you know even simple like a calculator which does set of calculations for you which has a list of formulas pertaining to the project. It can also be an object if we are going to send certain objects which are you know defined in the form of an array then we can even have the entire data segregated into a module and that can always return that particular data. So the modules can be any JavaScript object which has Boolean values, number, date, JSON, it can have functions, it can have arrays and so on or it can even be a set of functions utility functions that we are making use of in our programs. Now here I have a very simple function here which is just adding to numbers. So initial number is 5 and it takes another value and adds 5 to it and returns it back. Remember the required statement remains the same. So in spite of writing your own module in a different file called as MIS.js, you will still need to require it. So even if it is an inbuilt module, if it is an NPM module or your own custom module, you will still need the required statement which is going to add a path to the file that you are requiring or you are importing. So we add a required statement and call the function add num MIS.add num which is going to pass the required value. This should be able to print the value that we have added here that is 10 and 5 should be printing 15. But we have a problem. So when we write our own modules and we simply add the required statement and try to access, this would not work. Why? Because an important step needs to be added before this and that is exports. We need to export this function out and only then this add num function will be accessible or visible to the other scripts. Otherwise the scope of this add num is only for this particular MIS.js. So this is an important step. Any function, any object that you want accessible outside needs to be added to the exports object. So here we are exporting two things, exports.num that is this number 5 and also the function add num. Now when we try to execute this it should give you the required output that is 15. Let us look at a demo to write our own custom modules. Now let us switch on to the ID of Visual Studio Code which will have the terminal as well in the same integrated manner. So I will switch on to the ID now. So I already have this file for custom module and I am going to write the custom module into another file called as cal.js. So let us write a function here for sum and take two numbers which is going to return the total of 2. So this is the function and this is the file cal.js. So in order to make use of this module, custom module into the main module what we will do is we will add a required statement just like how we did in the other cases. So here, so we can omit out the extensions by default it is always going to look for .js files. If we simply say cal then it will try to locate it into the default node installation directory which it will not have there is no module called as cal. So we need to give the exact path here that is .slash because it is in the current directory I am providing it as .slash followed by the module name that is the cal. So we have required it here and now let me make use of it by saying cal.sum and passing some values to it also trying to print it. Let us run this code and see. The advantage of this ID is we can have the integrated terminal right here. So we can write the commands and execute. So this leads to an error saying that cal.sum is not a function. Why does this say so? That is because we have not added the exports function and that means the sum the function of sum is not accessible outside this particular file. So let us add that let us say exports or you can say module.exports.sum equal to this function. So here what we are doing is exports is an object and I am adding another property called as sum to it and assigning it to this particular function that we have defined here called as sum. All right. So once we do this now it will be accessible and we can invoke the function. Let us rerun we should be able to get the total of numbers as 13. Now there is another way that we can also define a module we can write a constructor and this is a very common usage wherein if we want to define a constructor for user then we can write it in a separate file called as user.js and export the entire constructor function and the usage of this will be you require the user file that is this particular script and keep on using it just like any other constructor. So let us see a demo of this. So we have this user.js here already have module.exports equal to a function that takes name and age as the parameters and here we are setting it to the current function the properties name age and another function called as get details. All right. So it is a function that has all these list of properties and because it is added to module.exports I can call this constructor function in any other script. So let us make use of it here in the custom module.js first let us add a required statement say dot slash user I will create a new user equal to new user remember I am referring to this variable here and it requires name and age and we can print out the details by calling new user dot get details. Let us run this so we will be able to see the details getting printed because it is invoking this method get details. So that is how we can make use of the constructor functions defined in a separate module. All right. So we have come to an end of this particular module now and to summarize the points we have mainly touched upon what is server-side JavaScript what do we mean by single threaded architecture and how this architecture is working inside Node.js. We have looked at what are blocking and non-blocking operations and with that we have also covered synchronous and asynchronous execution of APIs and lastly we have seen how and what kind of applications are suitable for Node.js. I hope you found this module interesting and carrying forward now in the next module we are going to see the Node API in action with some sample demos. Thank you everyone.