 Hello everyone, welcome to the 18th lecture in the course design and engineering of computer systems. In the previous two lectures we have studied how file systems work. In this lecture and the next we are going to study how network IO works in an operating system. So, let us get started. So, in all modern operating systems provide this abstraction of communication between two machines via a concept known as sockets. So, any computer system you need machines to exchange messages with each other and a socket is an abstraction using which you can actually exchange these messages and a socket is a way to communicate between two processes. These processes can be in the same machine and different machines does not matter the idea of the sockets remains the same it is a way for two processes to communicate with each other. So, one process P1 opens a socket, another process P2 opens a socket then these two sockets connect to each other and then P1 can send messages in the socket, P2 will receive it, P2 can send messages into the socket, P1 will be able to read it from its socket. So, you can think of sockets as two end points of a pipe. So, normally there is the notion of a client and a server in a socket that is the process that opens the socket first is called the server and another process which connects to the first guy's process is called a client. So, anybody can be a client anybody can be a server it is just that who starts first and who starts second. Then so that is why sockets are said to follow the client server communication paradigm. Then the other thing to note about the sockets is it is bi-directional just because one person started first the other person started second there is really no difference between them once they establish a connection. Anybody can read into a socket can write into a socket the other side will read this side can write this side will read it is a bi-directional communication and these processes can be on the same machine or on different machines. So, in this lecture what we are going to do is we are going to understand these system calls that correspond to communicating using sockets and in the next lecture we are going to study how these system calls are actually implemented inside the operating system. So, now let us see what are the different types of sockets. So, roughly depending on where these two communicating processes are there are two types of sockets one is what are called Unix domain sockets or local sockets that are used to communicate between processes on the same machine and internet sockets which are used to communicate between processes on different machines. So, in the case of Unix domain sockets or local sockets one process opens a socket and gives it a certain name you know some path name slash slash something something it will give a name and another process will also open a socket and it will ask it will make a system call to connect this socket to the first socket by giving this name. So, this is your server socket and this is the client and once these two processes are connected they can exchange information with each other. So, typically processes on the same machine also have to communicate with each other if you have a large computer system it is not like all the logic will be there just in one process the logic will be split across many different processes on a machine and all of these will have to exchange information and that is done one of the ways to do that is Unix domain sockets we are going to study more about this in a later lecture. Then the next thing is if these two processes are on different machines a process on one machine has to communicate with another process on another machine then these processes will open internet sockets a different type of sockets will be open to communicate across machines. Now you can no longer use some name or some path name to address a socket on another machine right how do you know on that machine what path name is being used in all of that it is very difficult. So, you have a different way of addressing sockets across machines. So, on the internet every machine has a unique address that is called the IP address. So, next week when we study networking we are going to study this in a lot more detail but for now understand that an IP address is nothing but a unique address that is given to every machine on the internet. So, now on the server machine this machine has a certain IP address and at that IP address on this machine there could be multiple sockets open multiple processes could be communicating and each of these sockets will get a port number a 16 bit unique number that is unique within a machine that is called the port number. So, when a server opens a socket this IP address plus this port number is a unique way to identify the server process and therefore when a client opens a socket it will so another client on another machine there is a process if it knows the IP address and port number of the server socket then it can connect to that server socket and communicate with it. So, this is how sockets are addressed differently based on whether they are local sockets or internet sockets. So, note that the client and server they are primarily differentiated by who starts first. The server starts first and it opens its socket at a well known address and the client process must know that server's address in order to connect to it. So, how does the client know the server's address? There are many ways to learn the server's addresses that in when we study about the internet in the next week we are going to know more about this. But for now understand that when you want to talk to somebody else you need to know where they are if a client socket wants to connect to a server socket the server socket must be at a well known address that the client can talk to. Next type of sockets that you have the next classification is based on whether these sockets are connection based or connectionless. What does this mean? In a connection based sockets two sockets the client and server are connected with each other are tied to each other are sort of like married to each other. So, that when this socket you write a message into this socket you can read it only from here when you write a message here you can only read it from here. So, after connection the two sockets can only communicate with each other. The other type of sockets are connectionless sockets that is you have a socket here this can send a message to this socket or this socket or this socket it can send messages to different sockets at different points of time. On every message as long as you mention the address of the other endpoint you can the message will reach the other endpoint that is this socket is not connected to any one socket but it can communicate with any of the other sockets. So, these are called connectionless sockets. The difference here is that when you send and receive a message here once the connection is done you do not have to explicitly say the address on every message whereas here for every message you have to give an address of which endpoint it is going to. So, now depending on your type of socket whether it is local or internet or connection oriented or connectionless all of these type of sockets are specified as an argument when you create a socket to the system call that actually creates the sockets. The one thing I would like to point out is connection based sockets are also called TCP sockets and connectionless sockets are also called UDP sockets. So, what are TCP and UDP they are nothing but protocols that are used on the internet. For example, TCP will ensure that on a connection data is being transferred in a reliable manner whereas with UDP since there is no notion of a connection there is no reliability there all of these different properties out there for these sockets that next week when we study how the internet works we are going to understand what is a TCP and what is a UDP socket in more detail. But for now you can remember that TCP sockets are connection based sockets and UDP sockets are connectionless sockets. So, now that we have seen the type of sockets let us understand how you create sockets. So, there is a system call the socket system call which is used to create a socket it takes various arguments like the type of the socket and so on and it returns a socket file descriptor. So, this is very similar to a file descriptor that is returned when you open a file. So, the socket file descriptor is also the index in the file table file descriptor array of a process which points to some socket based data structures which are useful to for future operations on the socket. Therefore, whenever you open a socket you get a file descriptor and then any system call you want to do on the socket you will provide this file descriptor as an argument. So, that is about opening a socket and once you open a socket the next thing you can do is you can actually bind it to an address that is you can give a path name or some IP address port number as arguments and you can bind the socket to an address. So, server sockets must always bind to an address why because the client must know how to connect to the server socket. The client must know that the server socket is there at a specific address. Therefore, server sockets must bind to well known addresses and the client should know about these addresses. But client sockets need not bind if you just want to be a client and connect to somebody then you do not have to bind you do not have to do the bind system call the OS will just assign some temporary address to you as needed. So, this is about opening a socket similarly there is also a closed system call just like with files to close a socket. So, now that we have created sockets how do we exchange data across sockets. So, let us begin with understanding how data is exchanged on connection less sockets. So, you have a client and a server that have both opened a socket using the socket system call and the server has done this bind system call to bind its socket to a specific address a well known address and the client knows about it somehow. Now the client can use or the server also can use the send to function to send a message to the other side. Now the client knows the server's address. So, into this socket it will send this message to this address. You have the client socket, you have the server socket and when the client writes a message into the socket gives the server's address the message will reach this server and the server can do receive from and read this message whatever message was sent here that message will be received here at the server process. So, the send to system call is used to send a message from one socket to the other note that these are both connection less sockets why because on every message you are providing the address after sending one message to this server the next message can go to another server from this client ok from the same socket you can send to another server also. Therefore, there is no notion of a connection here between these two sockets. Therefore, for with every message every time you send a message you will specify which socket what is the message to send you know some user space buffer will be there which has the message and which address to send it to. Similarly, when you receive from a socket the receive system call will use as arguments a socket file descriptor the message the buffer into which you want to read this message that is received you will also have some you know character area or something into which this message has to be copied by this system call and you also provide an address structure into which the address of the remote end point is filled. Now when the server receives a message the kernel the system call will also tell the server or this message came from so and so client at so and so address ok. So, why does the server need this address because if the server wants to send a reply back to the client then it will use this address of the client and then do a send to to this address so that the message once again will reach the client ok. In this way you can exchange two processes on two different machines can exchange data using connectionless sockets using this send to and receive from system calls. The next thing is what if you want to connect sockets ok. So, if I want two sockets to only explicitly communicate with each other and to nobody else and on every message I do not want to put an address then you will have to connect these sockets to each other ok. So, these connection oriented or TCP sockets must be explicitly connected before they can start exchanging messages. So, how do you do it? Once a server opens a socket and binds it to an address it will use a system call called a listen. This listen system call says that this server is open for new connections from clients ok. Now the client it cannot directly start sending messages to the server if you are using connection based sockets if this socket is a connection based socket you cannot just directly send message you have to first connect it to somebody. So, the client will connect this socket to the server you will know the server address and you will say connect this socket to this server socket ok. And this connect system call what will it do it will cause this machine to send some messages will be exchanged back and forth at the end of which this connection is established as socket at the client and a socket at the server are bound to each other. And this connect system call blocks until all of this message exchange happens and after the connect system call returns now whenever you write anything into the socket it will only reach this server at the other side it will not go to anybody else. And similarly similar to connect at the client you have the accept system call at the server which is this system call when the servers has accept this system call will block until a new connection arrives and when a new connection arrives then the client and server can talk to each other. And one very important thing to note here is that this accept system call returns a new socket file descriptor that is explicitly and exclusively connected to this client that is your client has one socket and your server has one socket that is listening for new connections ok. And when the client sends a message to connect to the server then what happens the server will create a separate socket after it accepts this connection from the client after it tells the client ok fine let us talk it will create a separate connected socket that is bound to this client socket ok. Why because you want to keep this listen socket free for other clients now you cannot say I will only talk to one client at a time now this server may be having to talk to multiple clients and it multiple clients may be sending request to connect to the server. So, it will keep this listen socket free to listen to request from all clients and for every connected client it will assign a separate socket that is exclusively dedicated to talk to this client ok. So, at the server there will be one listen socket which will continue to accept now this listen socket will continue to accept new connections on this socket you will only keep on doing accepting and every time there is a new connection this accept will return. And in addition to this there is also a separate connected socket now this new socket FD you write into this it will reach the client you write here it will reach here. Now these two sockets are connected to each other ok. So, there is a listen socket and there is a connected socket there is one connected socket for every client ok. So, if you have multiple clients who are connected to you there will be multiple connected sockets and one listen socket at the server ok. So, how do you do data exchange using connection using connected sockets the client will open a socket connected to the server the server will open a socket bind it to a well known address do listen and accept a connection. Once the connection is accepted this new socket FD and this socket here these two are directly bound to each other. Now the client can send a message on this socket the server can receive it note that it is the connected socket here not the listen socket the connected socket will receive this message then when the server sends a message on this connected socket the client will receive it ok. The pairs of sockets are used to exchange data and note that it this is at the server you are reading and writing into the per client socket on the listen socket you can continue to do accept you know you can accept for new connections and get more connected sockets. But this connected socket will be used to send and receive messages and the system calls here are send and receive also called read and write there is very just slight difference between these two ok. And note that for these system calls you do not have to provide the address of the other side you are just sending a message because it is connected you know where it has to go to there is no need to explicitly specify a address on these sockets whenever you send data. And the arguments are of course the socket file descriptor whatever message you want to send the size of the message and you will read it on the other side whatever buffer and length you have you write into the socket you will read it here ok. And the return value will basically say how many bytes have been read or written or if there is some error and these flags control behavior like for example should I block what if there is no data here should I block all of that information is put in over here. And there is no need to specify a socket address every on every message why because these sockets are already connected ok. So, that is about data exchange using connected sockets you will first connect accept a connection you have separate sockets that are bound to each other which are used to exchange information. So, now the next question comes up what if a server has to simultaneously talk to multiple clients. So, this is a valid scenario you have a web server any commerce website it is not like you are talking to one client at a time. So, now let us look at the issues that happen when a server has to talk to multiple clients a server is connected to client C1, client C2, multiple clients at the same time ok. So, in this case what is happening at the server you have a listen socket and you have one connected socket for every client that is currently connected and the server is listening for new connections on the socket and on all of these sockets you might be getting messages you have to read and reply and all of that ok. So, the server has to manage multiple sockets at the same time and note that each of these system calls can block the accept system call will block until a new connection comes the receive system call can block until some data comes. So, now how does a server know which socket should I look at suppose the server there is a single process at the server should I be waiting for new connections should I you know receive data on the socket should I receive data on that socket and you know if I keep waiting here what if I miss messages coming from some other client you know if the server pays attention to one guy the other guys can get neglected. So, then in such cases what should a server do you cannot constantly keep switching between all of these sockets right the server becomes very confused as to what to do. So, therefore you need more techniques at the server you cannot just write a simple program like this to handle multiple concurrent clients you need to do more in order to correctly handle multiple clients if you just have a single threaded single process server like this then if you are waiting on one socket other sockets can get neglected. So, typically what real life computer systems do is one way to solve this problem is you create multiple child processes or threads. So, there is the main server process or thread and there are multiple child processes or threads created this main server process will wait will keep monitoring this listen socket keep listening for new connections and these other child processes will keep monitoring their individual connected sockets you know every child process or thread is given its own socket. You keep taking care of this socket read from the socket process this request send a reply to the client you take care of this socket and if a new connection comes in you create a new process or thread and assign it its connected socket. In this way multiple threads or processes is each monitoring its own socket and it can monitor it effectively without any one socket getting neglected ok. So, this is called the one process or thread per connection model this is one way of designing real life web servers or application servers and the advantage of doing this is now you have multiple processes and threads even if one of them blocks the other can run they can run in parallel on multiple cores. So, you can effectively use your CPU and new connections as well as existing client connections are all equally handled with the equal amount of attention. But the problem with this method is that you cannot support a large number of clients if you have millions of clients connected you know you there is a limit on how many processes and threads your system can support. Therefore for very large number of connections this model can get somewhat inefficient ok. So, then what are the other options we have? The other option is what is called even driven or asynchronous IO. This is a different way of managing multiple sockets where a single process can correctly handle multiple sockets ok. And this uses a different set of system calls which are called select or EPOL ok. There are many APIs to do this. Let us look at the EPOL API here. So, what this EPOL API does is it gives the server the single process a way to give the OS the multiple file descriptors and say that I will not constantly keep looking at each of them. You tell me which of these I have to look at if a new connection has occurred or new data has arrived whatever has happened you tell me what to do I will look at that. That is a way in which that is the high level idea of how the EPOL API works. For example, so a server that has multiple sockets to monitor what it will do is it will create what is called an EPOL instance and it will add all the file descriptors to this EPOL instance. You know there is this listen socket some other socket all of that information is provided to the operating system using this EPOL API and then the server itself it will not block on either accept or receive or read it does not know where will it block it does not know. So, therefore it will call this function EPOL weight and it will give it all of these file descriptors and this EPOL weight will block until there is an event on any of these file descriptors. So, now the process does not have to constantly check each file descriptor the OS is taking care of this and the OS will make this process block and when any event happens on any of these file descriptors when any of these file descriptors become ready if a new connection has occurred or if data has arrived on a connected socket whatever whenever any event occurs then this EPOL weight will return the OS will tell you which are the file descriptors that are ready that have events and then the process will go handle that event it will accept a connection or read or whatever it has to do and then it will go back into this EPOL weight loop again. When EPOL weight returns you will handle the event now that you know there is an event you want block accept do not block because there is a connection receive need not block because the data is already arrived when things are ready then EPOL weight returns the process the single process at the server can handle all of these events and it can go back into EPOL weight again. In this way a process is efficiently handling multiple sockets efficiently handling multiple clients connected clients at the same time concurrently connected clients. But one thing to remember is that inside this EPOL processing you cannot block you cannot do any operation that blocks like for example you cannot do a disc read or something because why now this is your only process only thread in your process and you know the OS is coming back to you only when there events to be handled so you have to quickly handle the event go back to check on EPOL weight again. So this is how event driven APIs work for network IO this also similar things exist for disk IO but they are not very popular with disk you still use the blocking based you know read write APIs but with network IO this event driven APIs are fairly popular. So most components in any computer system need to do some network IO as clients as servers for example you know there is a web server a client connects to the web server then the web server becomes the client again and connects to some database to get some information then returns a reply there is a lot of network IO happening in real systems and you there are many different APIs many different libraries and different programming languages also available for network IO you need not use simple this socket based APIs that I have described in this lecture there are also more complicated simpler APIs available which are basically wrappers around this the OS provide socket you can add more functionalities over it for example there are RPC libraries where you know you can invoke the server code like it is a regular function call without actually worrying about this messages and reading and writing and all of that. So we will study these when we study application design later on but there are many ways for machines to communicate with each other but with any API if you look deep enough you have this design choice either you have one thread per connection and you use a blocking synchronous API or you use fewer threads and you use an event driven asynchronous API whichever programming language you are using whichever framework you are using you will always have the choice between you know a client and a server the server is connected to multiple clients should the server have a separate thread or something for each of these clients and block on them or should it use a fewer number of threads and use an event driven API this is always a design choice that has to be made and what is the trade off here event driven APIs usually have lesser overheads because there are fewer threads to manage and so on but they are also somewhat harder to program because every time there will be an event then you know your application code is actually split across these event callbacks and it can get kind of difficult to program they are usually non-intuitive and they are also difficult to scale to multiple cores if you have just one process thread that is doing or handling all of these events then you know how do you scale it to multiple cores becomes a little bit harder than when you have multiple threads one per connection. So the summary is that these are two different ways of doing network IO and any API you use any language any programming language any library you use for network IO always think about is it giving you a blocking synchronous API or a asynchronous event driven API and accordingly you have to think about how to structure your application. So in this lecture I have given you a summary of the socket API that is used to communicate between two processes we have seen local remote sockets connectionless connection oriented sockets we have seen how to do synchronous and asynchronous communication also across these sockets. As a programming exercise I will ask you to try to write code for a simple client and server that communicate with each other then try to extend that code to make your server communicate with multiple clients say using ePoll or by creating a separate process or thread for every connection. So try to play around with these APIs so that you understand these concepts better. So thank you all that is all I have for this lecture in the next lecture we are going to understand how the OS actually implements these system calls. So see you all then thank you.