 Hello everyone again welcome to the Linux networking class this is the the the topic is actually the Linux programming languages as you know today we are still going to continue the Linux networking today is lecture 5 we will be talking about the DNS where we left off last time so let me just do a small recap we talked about the DNS of the remaining system we talked about the point-to-point protocol and then essentially like I mean the remaining the major recap we will do it a little later because we have one more topic to cover in this section so I just want to leave it leave you with just that so we know that the domain name system is used to map the domain names into their IP addresses and we know the IP address is form all of this four acts three of them denotes the network which to which the host belongs to and in the last one itself is the host number in the network or how to identify the host inside the network and then we know that actually there is the subnet mask which can use it to separate the network address from the host and host is typically not the last octet but it is also like the name either it may be a subset of that octet or even a it can approach into the third octet also so and then we also saw basically how the point-to-point protocol works through the gateway basically like whenever there is a message that needs to be transmitted essentially like I mean if it is within the network it communicates directly to the host and basically let us exchange the message if it wants to go outside the network then it goes through a gateway and then the default gateways are all more than the directories are actually stored in computers that we saw and then how they interact we will come to that one a little bit later and we will formally take you through that section any activity for today actually I am not passing any activity for this lecture again we will complete the in the email aspect as to the lemon to applications of the domain name system itself is systems and then the other one is the email we also briefly saw like you know I think the third lecture how a web server works or how a client server architecture works where we open port and this is the communicate to go server and basically exchange that it is back and forth using the same IP protocol or the OSI protocol that we talked about and we learned earlier so let me now talk about the remaining section for this topic oh before I go there I also wanted to give you a brief look into the domain name registration itself the domain name registration for the worldwide domains is controlled by this corporation for the internet cooperation for assigned means from numbers we refer it to as I can I can publishes the complete list of the top level domains domain registries and the domain name registrar there is another one called D NIC D-NIC which is the German unique that holds all the Germany base data and then similarly there are other play other network centers essentially like which also holds the top level domain data for various countries and these so big corporations and then there is also this company called verisign which used to actually are even now with controls the dot com and the dot net domain names so anytime if you want to buy a dot-com domain name you can approach verisign and buy it actually they do charge money there are several companies which give you domain names today like Google and several other companies for a charge from like $10 to $50 $60 and you can also like apply directly to I can register for a fee also charge fee for that this these kind of the top level domains essentially like and then you can actually find multiple domains and the Googles of the world actually can distribute domain names because they have a sublet or subcontractor they are the subcontractors for some domain names so so that you can buy from them so and then they also probably charge some maintenance and they do some value added services setting up a webpage for you and populating with some additional data things like that so I think it is a profitable business proposition for everyone I just want to keep you keep it in keep this in mind when we go through all these things that it is all like this commercially motivated so let us look at email so the email actually is transferred on the internet using this SMTP very widely known which is a simple mail transport protocol email itself is divided into three categories male transfer agent which is actually moving the mail from this is from one server to another mail delivery which places the mail into the mailbox and then the male user agent displays and manages email messages for a user this gives kind of an overview of how things do work as you know like I mean the contribution of mail is fairly easy nowadays so you need to know who is the mail server essentially so you need to register that server into your machine and then you need to identify yourself as an address so once the address is known and the mail server is more then you will be able to send the emails out which basically goes to server and server distributes to the whole internet and so we will go through these things in detail so the user accounts have associated mail accounts there so every user has a related mail and the email is placed in this this particular directory that is flash bar flash pool a school slash mail again go back to your lecture one and lecture two or from the Linux system where we mentioned that actually everything is represented as files and folders and Linux and then this mail delivery mail also kept in a file under this directory and then the mail is retrieved using these one of these days one is post office protocol or pop three which is it used to be like very popular until recently this pop three essentially like I mean its server is download messages to the computer and then nowadays more prevalent one is the IM AP or IMAP which is the internet mail actual protocol the IMAP servers are again they just form into like two different classical servers nowadays the more prevalent one is the IMAP service and then the finally like your web browser so can access a mail client that's very popular when you will talk about like Yahoo mail or Gmail in fact behind the Gmail actually the IMAP server is so they are painting for Yahoo Yahoo and I Gmail code provide like IMAP service and then those servers deliver the mail to the web browser so mail client so now let's talk about the pop mail this is like a email filter so the previous one like I mean we talked about this all these various IM I'm apps and pop trees they are like they come under the the mail user agent essentially like I studied here I will they will talk about the mail delivery agent so the pop mail so the pop mail is a special delivery agent that acts as a filter and process the email now based on some user defined criteria it's it's difficult to configure but sometimes if you are receiving a lot of messages and also you want to save them like in a very ordered fashion then pop mail the configuration may be worth it pop mail is installed by default you can think of pop mail in the form of automatic filing so you have many folders that contain names and then you can automatically file them into various things which one that you want to look at first prioritizing and also like some males like and directly go into jump mail things like that so you can set up the rules and recipes and formulas for that the the proc mail itself checks for a system-wide configuration file which is stored in slash ETC slash prop mail RC and then it goes into your home directory to look for a dot proc mail RC file so there are some the configuration is done in two ways so there is a global configuration that resides in the the slash ETC slash proc mail RC that file you can override that with the dot proc mail RC again this file will have the recipes and formulas and all those form what you want to state action on for a given message so now like let us look at the reader itself this is the game the user is a text mode email reader I think like you are all like familiar with the familiar with the various other email clients outlaws or yahoo mail or email this looks like slightly primitive but here it is basically like I mean this is the way that even it is programs can communicate to you so here the program is actually sending you messages relating to the jobs and also like some we will learn about how to send the mail to yourself from programmatic way things like that in when we talk about the programs themselves like Perl and things like that there it is easy to actually put some mail programs here you can see that basically the comes to the the first mail and it wait for the command so here the command is pretty much like a Unix VI based command J is moving down K is moving up and then you can also like delete mail and un-delete mail delete is D and you and then if you want to mail a message you just type on M and then if you want to reply to this mail then type R and or if you want to forward the mail to another user then you press Q will quit and again you can think of this as another common interpreter there it basically like presents all the data and then it waits for your command so based on the command that you type in it performs the next action so that's that's a very easy to understand email reader and some writer so slightly more complicated like I mean so here is one setup using the next somewhat old thing close here again it's a pop server basically and this is the incoming is pop and outgoing is an instant TV server so here is set up all those things basically and then the where you want to keep the mail that's the root and this is Netscape mail so you keep it there and then you can actually like I mean this is the mail server setup so and then you can add multiple incoming pop servers so if you have like we want to consolidate account from multiple sources then you can just add all those types and then you can also the outgoing server is typically only one and then here the local host is the server and then there are other options here like I mean like secure socket layer which is all like much more secured communication so essentially like I mean you can set it up as whether you want to use it all the time or never use it or wherever it's available then use it and so pretty much this is this is what you will use to set up the mail kind of it's a little bit old this particular setup because Netscape communication is something that we seldom use nowadays either like Google Chrome or in Linux the most common one actually yeah so that that's pretty much concludes the actually yeah what I was going to say was the Firefox browser is much more common in Linux Firefox also allows all these so now let's we come to the end of this lecture so we started our lecture with the domain name service itself looked at the resolver basically which is essentially the client portion of domain name service and the resolver is nothing but it has the domain name that's the IP address so the fully qualified domain name the fdn consists of a host name plus the domain name of the host that is basically like I mean so it's all there essentially so typical example will be IDM.mydivision.com which is fully qualified domain name and then the point-to-point protocol that we saw was a popular method for making retro connections via modems and point-to-point protocol is provided by the password authentication and the challenge handshake authentication protocols password authentication is a very simple thing just ask for a password and then basically provide the password every time the challenge handshake is more more involved and that's why it's more secure that's basically it's like an RSA token basically there are random number generators which are synced random number generators and then essentially generating that random number and on the server side also it is generating something and then basically you are asked to provide that so you challenge the server with that or essentially like I mean you put your magic pin into this to generate a random number and that pin when it is sent to the server server also generates a random number and then you compare those two responses and if the responses match that's when you can authenticate the machine and then that's where like I mean the communication will start you also saw the W need dial utility that to on the configure the manage the PCP connection from the command line the dial D program this automates the use of dial-up connection where PPP automatically connecting and disconnecting based on traffic so again these are all something that we learned about and essentially like to make sure that make how to make those connections and see how the connections really impact on how the connections can be then we also looked at the dynamic host configuration protocol D HCP where the clients can configure IP networking automatically by receiving the network Atlas information from the DHCP server so this I will say is one of the reasons why internet became so criminal because you look at the IP space itself you know that actually the IP addresses are the octets right the core octets so we have inherently some limitation between 2424 servers all over the world that are possible and because those are the unique addresses that can be generated with 24 bits or the first three octets of the IP addressing but today we can actually put together multitudes of addresses and also we can communicate between these multitudes of addresses much more than to power 24 bit like very very easily and without any loss of communication packets or collision of addresses or address collision the reason why we can do this is because of this dynamic host configuration for protocol which means that when you go into a subnet that we know about you know the term subnet now we learned about this so when you go into subnet the different subnets can have the same model and they can still communicate between each other without any collision because of the DHCP server which can resolve the collision and resolve the addresses and route them properly so a simple application will be like if you are say you are leaving internet connection from either your cable company any of one of the providers ISP basically again ISP has a subnet and the ISP subnet is sending you an office but tomorrow there is one more person who got added to the network or the ISP itself actually has some new customers to serve so there is inherently like I mean ISP can run out of the address space whatever they are so instead what they can do is they can start another subnet so the subnets will be different they have different addresses but they internally will have the DHCP servers and the server will now assign addresses to the various hosts in a dynamic fashion so that means that you guys can have the same address but there is a DHCP server in the gateway which resolves these addresses and make sure that you are communicating and receive the communication appropriate the benefit of this is now the ISP initially can sell only like so many offices or they can serve only so many clients but now with having subnets as the subnet they can serve maybe 10 times more clients than what they originally had with the number of IP addresses and for you also it is a seamless process because the DHCP server automatically sends what the address and what it wants the client to be configured as so you only know like about what address that you are sending the things to and it automatically figures out who to send so most versions of Linux include a DHCP server and at least one of the three common DHCP clients the lightweight directory access protocol of the LDAP that provides the directory services that lets users query a worldwide database or information on the sources so again the we know about the directories essentially the LDAP is essentially like I mean it keeps only like so much information about the neighboring gateways or neighboring posts and then essentially like I mean when I need the additional information it gets it from that those sources and the message is also passed to those main servers and communities so this again made the whole communication process so one of the LDAP servers that's provided with the Linux distribution is the open LDAP and then the the X term itself can execute graph graphical programs remotely by referring to display variable usually like I mean we give like the dash dash display command in the command line option we can also open up a port with the post name which corresponds to a display and then we can display various graphical items into the port so we can do like a remote terminal and steal the remote terminal and display it in our terminal things like that are possible today and this text B MCP this lets users from remote X servers obtain the graphical login screen and begin using the X clients on Linux without first logging into the next we are telling it and then there is this remote utility R starts with R that that are that provide an convenient way to execute commands so to copy files we use RCP and to open a remote shell I think you all know the command it's RSH which is like again once you open the remote shell then you cannot work on that on that machine RCP is or R copy remote copy to copy in between two servers that are in the network or essentially connected in the network and want to copy between each other then we also saw about the UCP protocol or the Unix to Unix copy protocol this is designed to facilitate inexpensive transfers it's mostly like the email messages between the servers and this was again it was used before internet connectivity was widespread there are slightly different slightly different from the STP protocol that we saw earlier STP also like achieve this you know kind of thing one is this one is UCP is mostly machines that are connected in the same network whereas STP can go between networks as well so and then the web browsers many of them are available for Linux we all already saw that the popular being the Mozilla Firefox and the Netscape I will still say that actually the use of escape is kind of waning at this point there are some text mode browsers to call links so the in the graphical area Firefox is probably the most prevalent browser at this point and this is very similar to all the all the features just like either the internet explorer or the Chrome browser so I think like I mean you will be using Mozilla Firefox more often on a Linux box then the internet mail itself that relies on the the mail to move the messages between host we didn't see an example of this because this is pretty much under the hood transmission that happens we go we learn about the mail delivery agent and then the mail delivery agent process the mail as it gets delivered to the user mailbox we saw one example of that and then the mail user agent that is used to read the mail and send messages so the mail user agent reads local mail files or it can also use this pop tree or the IMAP protocols to retrieve messages from the local messages essentially previously people used to send between the two computers and basically that is stored in the mail file that you also already saw the var school dot slash the proc mail is the used as a delivery agent basically as the mail gets delivered it or it provides like the message management automatic filing automatic replies things like that and these things you can construct it as a recipe and then we also saw that there are two basically like two files that are used for configuration this is a simple file and then also a local file and finally like I mean the mail client because there are many popular ones so L is one element fine basically we saw the you can say like they have a new way for the mail user agents we saw an example of the text based mail support ELM this is essentially like I mean the the interface and commands are just like using a visual editor or AI there are other mails also that are kind of popular fetch mail K mail Walter and one thing I will ask you is actually to use the man mail command to just see what happens when you invoke a mail and then see how the mail work or what are the different commands what are the different options available and then are there any arguments so you can also think of that and see any arguments for that so this will pretty much conclude the overall lecture on the mail and DNS hope this one was useful now we go into more distributed file system where we talk about the whole file system structure in the Linux we actually like look at it briefly we know that there are different files like flash ETC flash war flash dev slash root all those different files system now we will learn about how to do it between the networks and what happens when you do that and how do we maintain consistent because there are some metrics that are fairly important to maintain that kind of scenario so let's start looking at the file systems so we will talk about the distributed file system in the context of Linux so in this section we will be talking for an introduction then we will go into the file services architecture and then we will take up two very popular case studies one is the Sun NSS system the network file system this has been like analyzed and over-analyzed and under-analyzed whatever you think of the objectives and we will always quote this one and then the second one that we will look at is the Android file system which is also distributed network file system which was initially started at CMU or the planning in null and unicity and then the part of the Android project essentially so we will learn about that these two I systems as a part of this distributed file system and just compare and contrast what is in each of them so let's look at the file system the file system is essentially originally developed for a central centralized system or a desktop system it was basically it was an operating system facility providing a convenient program interface to this storage so you know that basically the memory itself we will go into the various hierarchies of memory but there is a storage that you can think of in the software complex where the programs are stored I think like when we started looking at the shells we learned about the programs and the processes so at a glance actually like you can think of there is a system there programs are stored and that needs to be bought out whenever they are queried and then basically like that one such a process instance of running program is a process that we saw so now to get that program out the file system is an operating systems way to provide a user method to access these programs or things that are stored so instead of like saying that okay go to sector 12 and on this for to get me my program you can say simply then go to slash home slash x slash y and that's where my program is so it's basically like I mean so there is a lot of translation that goes on behind the scenes but now the interesting challenge is like this is the disk is the flash home this is not on your computer and it's residing somewhere else how do you make sure that you can see that on the other hand you can also think of this as save is that this gives us is not available what happens do you see it do you you don't see it how how the computers will react so we will we will learn about that so a distributed file system support sharing of information in the form of files and hardware resources so there are distributed object systems like the Corba and Java and the web the picture has become much more complex so what I mean is like again exactly the same thing basically now the your disk you no longer resides in your computer it resides far away maybe you have connected through a network into that particular computer and you know how to access it what should be the the name of that can you name it as slash home slash same as another slash x in your computer so think of it this way now the you saw like I mean the addresses IP addresses and translators between IP addresses and there are DNS resolvers similarly you can think of now the files can be distributed and every file needs to have an app or a unique name because if you don't have the unique name then you won't be able to access that or there will be like name collisions so how do you provide that unique name and how do you manage the whole system infrastructure those are the things that you will be worrying about so before we start that let's look at the types of storage systems this is one of the key part and one of the key things that we will go through first of all I want to introduce certain key concepts here so look at in this table look at the all the column headers so we have like sharing is one of the attributes that we will be describing on what the memory storage system should have then the persistence what do you mean the persistence so sharing we know that actually whether the same thing can be shared between multiple resources or just single resource for example a main memory in this case is just shared by just that one city today if you know about the any of the computing systems there are gradations of this memory all the way from the L0 L1 cache to L2 cache to specific L3 cache also and then going to the main one this is especially true when we talk about like the multi-core systems where the machine is not just formed by single core but there are multiple cores that are working and each core can operate one or more threads of computing so we can say like I mean if you have a dual code machine internally can execute four programs simultaneously in the system and so like I mean some program can share the memory some programs may not even main memory can be shared between the two threads whereas the caches cannot be shared between the threads and each core can have its own cache and which it divides it between like a lot of threads also so again sharing is one of the attributes of the second one is persistence so what do we mean by persistence persistence is something that that is the data that we write into a particular storage is always there whether it is always there or not depends on the persistence of the particular system how much it is persistent so for example in main memory the data may not be persistent because wherever we write actually that the data gets overwritten very quickly with something else so usually like I mean this is smaller and but very very fast or having high bandwidth we will also talk about that then there is a distributed cache the replicas basically so when we the cache is nothing but a buffer essentially so if you want to serve a particular entity or serve a node like a CPU then you need to have provided with the data whenever it needs it not whenever the data is available so there is a big difference between those two if you only want to provide when the data is available then probably like 99.1 or 99.9% the CPU will be just starting because it won't be able to get to the data so you want to provide it then it needs it as opposed to when the data is available so a cache is a good means to provide now because you can preface the data and keep it in the cache and the CPU will only access the cache whereas when the cache is getting empty it gets more data so that cache distributed cache and replicas that's again determined that's another determinant of the storage system so in the in a storage system like main memory or even a simple file system you may not have any distributed cache or replicas then the the fourth point is the consistency maintenance here actually like I mean it so depends on how you maintain the consistency of data so the high bandwidth or the memories that are very close to the computing system itself they will have a consistency maintenance property as for one copy consistent here it's represented as a one and so the example is RAM that is what we quoted from me you can think of basically it's it only has only one copy doesn't store more than one copy because that you can think of this as like because it's so close to that computing device the the data is constantly getting updated so you don't you cannot keep more than one copy because suddenly like other copy becomes stable so it becomes very easy so you don't want to have more than one copy by system again the same thing we will talk about the distributed by system and all those things I also want to have add one more property to this one which is bandwidth which is not covered here again so for the past memories essentially like all the memories that are very close to the computing module computing platform we want them to have extremely high bandwidth so that the computer the CPU never starts for data and there are ways of doing that but one thing is to just use the seminal physics which is use the speed up of RAM to make sure that those cities are so well so let's look at the and compare the various main memory if you think of main memory as a storage it has very strict one copy consistency but other than that it doesn't have sharing it doesn't have persistence it doesn't have distributed cache of because now let's look at a standard file system basically confined to that particular plan again that one does not allow sharing it has persistence essentially and it does not have any distributed caches or replicas and then the consistency maintenance is strict one copy consistency now let's look at the distributed file system allows you to share allows persistence of data allows distributed cache or replicas and it is also it manages consistency which is again you don't need to have like multiple I mean a single copy for this so it is kind of consistent basically like slow changing but at the same time any point of time you can read out that and read out the particular distributed file system of storage and then get the same data back so an example could be the Sun nfs which we will now the other one is the web there again it has the sharing persistence it also provides the distributed cache and replicas but it does not provide the consistency so when you go when you do something and then when you come back it may not be consistent anymore so typical example will be a web server and then there are other things basically distributed shared memory which is again the concept is sharing so it allows you to share but it does not provide persistence it allows the distributed cache and replicas and then but it also allows the consistency then there is something called remote the objects are RMI or ORB the example could be like the explore bar that we talked about earlier this provides only sharing no persistence no distributed cash and also like this one copy and then there is something called persistent objects store this provides the persistence essentially and it also allows the sharing it does not allow distributed cash or replicas and then the consistency maintenance is also like it is only one copy consistency then we have peer-to-peer storage systems they provide all of them and then example could be the ocean store so now let's look at the module structure of the non-distributed file system so this one it's basically like since it's a non-network system it has a directory module this basically like relates the file names to the file ID so it's kind of the same as a DNS resolver that we can think of the file module itself relates the file ID to particular files the directory module actually can also have the directory names associated with it here basically like I mean the file module relates the file ID to the particular files and then the access control module this checks for the permission of the operation that is requested we also saw this in the beginning of lecture 2 where we could assign the files with various properties or permissions like read the write execute so the file access module checks for that the block module accesses and allocates the disk more so as I mentioned basically like you know like which sector which block where the program is assigned so that's all taken care by the block module and then the device module does the disk IO and the buffering so it basically communicates to that this and then stores the so this is a very simple single user file system essentially and then the file systems essentially is that's where we organize store retrieve and some name sharing naming and then all of the sharing and protection of various lines so the files can contain data and that so let's look at some of the attributes of the record structure here so the things that you want to store are the filing then it was created a creation timestamp then it was last read read timestamp and then when was it updated with some data which is the write timestamp and then any any other attributes can provide that and then the attributes timestamp also is home and then finally a recent found given and then there is also like like encoding in form of the owner and the file type and then who or who cannot access the file and this is called the access control and that's also provided so that it goes with the software and essentially like I mean goes with the file and if it if the user is not authorized then it won't like let you see you won't be able to read that so in the next figure that will be basically with all the operations on the file that are available to applications in the unit system so when we say open and the name and the mode essentially like I mean so that opens the an existing file with a or open the file with a given name essentially and then when we say like file does equal to create name mode this is actually creating a new file essentially so the file descriptor is in this file DS essentially the answer which is kind of a just a number that it returns it's typically the pointer to the head of keep or section of memory where the particular file is reading store I'll call the name then you have a status that you can set and usually like we do a close and then you can give the file handle so I'm going to refer this as file handle from now on the file descriptor essentially it's like a pointer to the head of the file so that we know that that is the start of the file and then we can even read that the fields that we mentioned in the previous slide which is these things file length creation time stamp etc etc so the file that gives a pointer to that particular head of that particular table we can give the count so here like I mean we just told the count in a variable call count so the command is again read file does buffer and so essentially the buffer stores the n bytes of the read n bytes of the file file yes and it stores it essentially like the buffer and then that's what is in the town so the town displays the number of bytes essentially you can also do write file does buffer on the end they both deliver the number of bytes actually transferred between the advance they both deliver the number of bits which are actually transferred and it also addresses the real pointer to the next location and then the other command is the LC command I'll see the file does offset when essentially like I mean it takes the rewrite pointer to the top set related to an absolute number and that is given by this when variable and then the other one is unlink name this removes the file name from the data constructor if the file has no other names then file is deleted actually yeah then we saw a soft link earlier this is the hard link of the link name one in two this adds a new name for name two to name one thing and it just provides another pointer to the same header information there the file name two or name one asset then the other command is the status command this is STAT or just that and then essentially that gives like the it gets all the file attributes of the file name and then stores it into buffer so we studied about a few things essentially like I mean so we will talk about this briefly and then we will stop at this point and then the next time we will expand this first of all give you all the definitions of all these different terms and then we will move forward so just wanted to give you this basically the distributed file system requirements essentially it requires transparency concurrency replication heterogeneity all tolerance consistency which is different from the transparency and other things security and efficiency so we will talk about what each term means in the context of distributed system and then also how to compare different file systems in the next chapter thanks