 on mechanisms for building distributed file system and its design issues. And in this particular session, we will be mainly focusing on availability and scalability features of DFS. This is Dr. Neeta Pooja, professor in computer science and engineering department at Walchin Institute of Technology, SolarPoor. Now at the end of this session, students will get familiar to how to achieve availability in distributed file systems. And also how to handle the issue of scalability in distributed file systems. Now let's start with availability. Now you know that replication is the primary mechanism for enhancing the availability of files in distributed file system. Every file should have a replica maintained on multiple nodes so that even if one node goes down, the file is available to the clients from the other nodes. It is expensive, that is, replication is expensive as it requires extra storage space for replicas. Overhead incurred in maintaining all the replicas up to date is the most important problem that is to be handled here, that is called as consistency. Potentially, inconsistency problems may preclude file updates, that is, they may not contain the latest file updates that have been done, thus decreasing the availability as the level of replication is increased. Now let's first start with the unit of replication. The most basic unit is a file itself. It is difficult to maintain the replication of a file as the protection rights of the directory are to be stored with each replica of the file. Now all the replicas of the file belonging to the same directory, they may not have a common file server, that is, they may be residing on the different file servers. And hence it requires extra efforts for name resolution to locate all the replicas. Now replication unit can be a group of files of a single user or all the files that belong to a single server. This group of files is known as volume. Then let's see what are the advantages of handling the volume replication. Now in volume replication, replica management is easier because we are replicating the whole volume on multiple nodes instead of replicating the separate files on multiple nodes. So protection rights can be associated with volume instead of with each individual file replica. But volume replication may be a waste as a user typically needs higher availability for only few files in the volume. User may not need the replica of all the files in the volume. Note volume replication is used in CODA file system. Now let's find out the compromise between two. Now what are two things here? One is volume replication or handling the single file replication. So we are finding out the compromise between two because single file replication involves a lot of complexity and overhead whereas volume replication it may sometimes go waste because all the files need not be replicated on multiple nodes. So we find out the compromise between these two. All the files of a user constitute a file group called as primary pack. A replica of the primary pack which is called as a pack contains only a subset of files in the primary pack. It will not contain all the files of the primary pack. So with this method a different degree of replication for each file in the primary pack would be obtained. Note this method is used in locus file system. Now let's see what is replica management. Now as you know that replica management involves creating and managing multiple replicas of the files as well. It also depends on the consistency problem among the replicas that is whether consistency among replicas is mutual consistency is guaranteed by the distributed file system or not. This is very important part of the replica management. Now to ensure this mutual consistency a weighted voting scheme is used. Now in this scheme some number of votes and the timestamp which is also known as quorum are associated with each replica. That is every replica will have the read quorum and the write quorum. Now a certain number of votes or also called as quorum that is R or W must be obtained before read or write operation is performed at that replica. Early votes from the current copies are considered to be valid. Reads can be performed from any current copy and writes update all the current copies. Timestamps of all the participating replicas replicas that is only the current copies are updated when a particular copy is updated. Now by keeping W greater than R that is write quorum greater than read quorum and R plus W greater than total number of votes of all the replicas. It is possible to maintain at least one current copy up to date. It is observed that it is not necessary to keep all the replicas up to date as long as sufficient votes can be obtained to perform read and write operations. So this feature provides increased availability and fault tolerance during system failures. Note voting is used in row file system to maintain mutual consistency. Another scheme to maintain consistency among replicas is to designate one or more processes as the agents for controlling the access to the replicas of the files. That is each file group will have a designated site which is called as current synchronization site that enforces the global synchronization policy. The file open and close requests are routed through this site to a site which has the replica of the requested file. Now the disadvantage of this approach is that it is but obvious that the synchronization site becomes bottleneck because it has to handle the file open file close request of so many clients if the size of distributed system is large. Hence it has poor scalability. So let's see how to handle the issue of scalability in distributed file systems. Now you know that ability of the system to cater to the demands of the growing system is nothing but the scalability. Currently client server approach is the commonly used approach in the distributed file systems. Caching is used to reduce the network latency, the load on the server and improve client response time. But caching introduces the problem of consistency as we have already discussed earlier. Now there are two approaches that is client initiated approach and server initiated approach to handle the problem of consistency but in server initiated approach a server keeps track of all the clients which store the replicas of its files. This is known as server state and the size of the server state and the load due to any validation of the server it will increase when the number of nodes in the system increases that is when the size of the system increases because it has to keep because the number of clients caching these replicas will be more so all these states need to be stored in the server state hence the size of the server state goes on increasing. Another observation is that the data required by the client is often found in another client. So as clients have more free cycles as compared to that of the server, client can obtain the required data from another client without contacting server at all. So the load on the server will decrease here. But only thing is there should be a mechanism which needs to handle where the on which machines the data required by the client are present. Now one more feature that plays a major role in the scalability is the structure of the server process. Server designed with a single process can handle only a single client request at a time. So it will make the other clients wait for a longer time. If multiple processes are used each process handling a particular client request then you know that process are heavy weight then context switch between the process also increases the latency as well as the overhead. So instead of using multiple processes why not use a lightweight threads? Threads can be used to reduce the wait time and overhead due to frequent context switches between clients. Now we will pause the video for a moment and try to answer this question. The pack is the subset of files in the primary pack. Is it true or false? Now this concept we have already discussed where we had found out a strategy which was a compromise between the two that is the replication of a single file and the volume replication. So their single file replication takes too much of overhead or complexity whereas volume replication is advantageous but sometimes all the files in the volume may not be required by the clients frequently that is they need not be replicated on multiple nodes. So we had found out the compromise between two that the pack is the use a primary pack for the set of files a group of files and create a pack of it which contains only a subset of files in the primary pack. So the answer for this question is true. These are some of the references that have been used for preparing this content and decisions. Thank you.