 The foremost basic system calls for dealing with files are open, close, read, and write. Read and write are quite self-evident. Read will copy bytes from the file to memory in your process, whereas write does the opposite. It copies bytes in your process to the file. When reading and writing a file, however, that file has to be open. So first, to work with a file, you invoke the open system call and you pass in the file path to that file and open will return what's called a file descriptor. A file descriptor is basically just a number which in your process uniquely identifies an open file. So in fact, when you invoke the read and write system calls, you don't pass in a file path, you pass in the file descriptor. When you're done with the file descriptor, you should then pass it to the closed system call. Releasing file descriptors this way is not strictly critical, but it is good practice because in particularly longer running programs, if you just keep hoarding file descriptors and never giving them back, you're going to run out eventually. Each file descriptor does take up a bit of operating system resources or some memory associated with each one. So it's just good practice to always close files when you're done with them. Now note here that I've indicated the read and write system calls may block. Now exactly under which circumstances these system calls will block depends upon the type of the file being read or written and also some options that can be specified when that file was opened. We won't get into too many of those details. In the most common scenario though, when a file is opened with the default options, the read system call will usually block whereas the write system call will not. So looking at the write system call, here's what happens in the default case. If we are writing to a file on a storage device, like say a hard drive, the data is not actually directly copied from the process to the hard drive. Instead, the call to write will copy that data from the process to a buffer in memory outside the process and controlled by the operating system. And then from there, the operating system will write the data from the buffer to the actual storage device. Even though the scheme involves copying the data twice, it still makes sense for performance reasons. As we've discussed a bit before, IO devices like hard drives are relatively very very slow compared to the CPU. So we wouldn't want our process to have to sit around and wait while data is being copied to the actual storage device. So if the process only has to write to a buffer in the operating system and then it's left up to the operating system to actually copy the data from that buffer to the storage device, then the write system call can return after the data has been copied to the buffer. It doesn't have to sit around and wait and the process can then continue and get some work done while that data is being copied out to the actual device. So here's what the use of write might look like in code. Say we want to write to the file slash Alice slash Tim. Well, first we need to open it and that call to open returns a file descriptor here assigned to F. Then when we call write, we specify the file descriptor so it knows which file we're actually writing to. And then we also pass in the data to write. In this case, let's just say it's an ASCII string where each character represents one byte that's going to be written to the file. So here we call write twice and the first one writes blah blah to the file and then the second one writes blah blah blah. We'll talk in a moment how exactly you specify where in a file you wish to read or write. In this case what's happening is the first write just writes at the start of the file from byte zero and then the next write gets tacked on after that. So we write five blahs to the file and we're done with the file so we close it because that's good practice. One thing to be clear about with the usual behavior of write is that your process can't really know when the data is actually written to disk or in fact if it ever actually is written to disk you never actually get verification that it actually happens. In fact because it's left up to the operating system to do the actual writing to disk, the process which wrote the data doesn't even have to stick around. It's quite possible that the process can terminate before the data is ever actually written to disk. This is problematic for programs which need with a high degree of certainty to know that their data has been written properly to disk. A database program for example makes data integrity a very high priority so a database just can't accept letting the operating system handle the actual writing of its data without getting any verification that the data was actually properly written. Even if databases can't get 100% guaranteed that nothing will ever go wrong with their data they at the very least need to know when something has gone wrong. So although we won't get into the details of how exactly this is done there are ways in Unix of writing to a file such that you do get verification that the data was actually written. Just understand that by default you don't get such an assurance. Now as for the read system call like with write there's an intermediary buffer in the operating system. When a process invokes read the data is not copied directly from the store's device to the process. First it is copied into a buffer in the operating system and then from there copied to the process. Again this is done mainly for performance reasons. IO devices like hard drives are typically very very slow relative to the CPU so it would take a long time if a process had to sit around and wait for the data to actually be read off the hard drive. What makes this different from calls to write however is that when you're trying to read data presumably you really can't do anything until you actually get the data. You can't sit around and wait for the operating system to then get the data later we need the data now. So the purpose of the read buffer is really quite different. The most common usage pattern when reading from a file is that we wish not to read just a small portion of it but also the portions in the file which follow. So given the typical performance characteristics of stores devices like hard drives it makes sense when reading from a file to actually read more than is requested and then to store it in a read buffer so that subsequent calls to read can read directly from the buffer without having to wait again for data to be read from the hard drive. So typically the first time we read from a file the read buffer is empty so our process actually has to block while data is read from the hard drive to the buffer and then once the data is ready in the buffer our process is unblocked at which point the read system call finishes its business by copying the data from the buffer to the actual process. If our process then invokes read again on the same file it's quite likely that the data is already in the buffer in which case the read system call doesn't need to block. So to put it succinctly the read system call works by default by first checking the buffer and seeing if the data it wants is already there. If not the process will block while that data plus some amount of extra data most likely is read into the buffer and then the process will unblock once the data is there in the buffer at which point the data can then actually be copied into the process. What the use of read might look like in code is this First we open the file say slash Alice slash Tim that returns a file descriptor which we assigned to F and then we invoke the read system call specifying which file we are reading from and the read call whether it blocks or not will eventually return the data read from the file and if we're done with that file descriptor good practice is to release it by passing it to the closed system call. Now the interesting question here is how much data does the call to read return? Well in fact how much data gets returned is not up to you. It's left up to read itself to decide how much data to return. When invoking read you can actually specify the max amount of data you want to return the maximum number of bytes but it's left up to read to decide whether or not to return that much or actually less. And again read works this way basically for performance reasons. Consider for example if you invoke read and request 4,000 bytes well if only 2,000 bytes are available in the read buffer that means your process would have to block to wait for the remaining 2,000 bytes. Read will probably in this case decide rather than have your process block it'll just return those 2,000 bytes. It's then left up to your program to check how much data was actually returned by read and then call read again to get the rest. While it is left up to read to decide how much data to return read is always guaranteed to return some amount of data when there is data in the file left to be read. So this means that read will only return no data in the special case when you are attempting to read at the very end of the file. So in the cases where you invoke read and there's data left in the file but there's nothing in the buffer read will not just return nothing it'll block your process and wait for at least something to be read into the buffer. If read were to return nothing when there's still data left to be read that would falsely indicate to the program that there's no more data left, that the end of file has been reached. Consider now this code which reads in a whole file and prints all of it. First of course we open the file slash Alice slash Tim and we assign the file descriptor to F and then we read from the file with read F this call most likely blocks but eventually it's going to return some amount of data unless of course the file is actually totally empty in which case it will return an empty string and so for our loop we test whether they return data the string is not empty that is whether its length is not equal to zero. Assuming it's not we then print that string and then we want to read more from the file so we invoke read again and assign it again to data and unless we've already reached the end of file that read will return some amount of data so the condition will once again be true and we'll go through the loop again and so we'll keep doing this you know every chunk of data we read we then print and then eventually we'll reach the end of the file at which point read will return an empty string so we'll then leave the loop and then close the file because we're done with it. We haven't yet actually explained how exactly read and write know where in a file you wish to read from or write to. Well quite simply when you open a file with each file descriptor is associated a marker basically just a numeric index keeping track of where your next read or next write is going to take place. By default when you open a file the marker starts out at the first byte and each time we do a reader write with that file descriptor the marker will advance that much into the file so say if you read five bytes then the marker will advance by five bytes. Now generally if you're both reading and writing a file it probably makes most sense because you have separate file descriptors with separate markers so you can use one file descriptor to read and the other to write and they don't interfere with each other. Now you may be wondering what if the marker is at the very end of the file and you wish to write or what if it's near the end of the file and the amount of data you're going to write is going to go past the end. What happens? Well in that case the file simply expands to accommodate however much data you're writing. So in fact if you wish to create a file in Unix all you do is simply open a file which doesn't yet exist and then start writing to it. The file starts out empty but each write just adds more data. Now if you actually wish to shrink a file that's a different story. We have a system call for that called Truncate to which you pass the new size of the file. So if your file is 5,000 bytes in size if you call Truncate with the argument 3,000 that will effectively lop off the last 2,000 bytes leaving you with a file of those first 3,000 bytes. And actually even though the name Truncate implies it's merely for shrinking files you can also expand files with Truncate and that effectively adds bytes to the end of the file and all those bytes will be null bytes, they will be zeroed out until of course we actually write something else in their place. Now of course in many cases you don't necessarily want to start out reading or writing at the very start of the file or for whatever reason you just want to move the marker. So we can do so with the system call called lseq. The seek part of the name makes sense, you're effectively seeking to some part of the file. What the l stands for is sort of lost to history, it probably stands for long as in a long sized integer but we can't say for sure and in any case the name is stuck and it's called lseq. So when you call lseq you simply pass in the byte to which you wish to move the marker. If you wish to move the marker to the very end of the file say you're going to write to the end of the file to append more stuff. There's a special value you pass in to make it go to the very last byte. Also it's actually possible to move the marker past the end of the last byte in which case if you then write you will be effectively expanding the size of the file and all the bytes in between, past the former end of the file and where you're beginning that new write, those bytes get filled in, they become null bytes, zeroed out bytes. Now when dealing with file descriptors it's actually quite important to understand that a file descriptor itself is really just a number in the process used to uniquely identify an underlying data structure what's called a description and the description is the thing which actually represents the open file and which contains the marker. This distinction between descriptors and descriptions is important because there are actually circumstances in which the descriptor can get copied and you end up with two separate descriptors which both point to the same description so you'd have two separate file descriptors but they'd share the same underlying file marker. When we call open however, open always returns a new file descriptor with a new underlying file description. So here when we open the file slash al slash tim twice the two file descriptors f and f2 both have separate file markers. So when we write here with f and then read with f2 the read and write are both done at the start of the file because f and f2 have their own markers. Looking at this code may raise a question and that is when the read call is invoked does the data returned reflect the change in the file as written by the previous write? That is, is the data returned going to start with a blah space blah? Well to answer this question first we have to look again at how descriptors are connected to the actual files stored on disk and that is you have a descriptor which points to a description and when we read or write via a description we're actually writing to a buffer in the operating system not directly to or from a file on disk. The key thing to understand is that in most Unix's including Linux there's only one buffer no matter how many descriptions are open on the same file. So consider say a scenario in which a single file is opened and being used by four different processes. These processes very well may end up reading and writing from overlapping portions of the file but what happens in each case is that each write ends up just overriding what already was in the buffer and each read simply reads whatever happens to be in the buffer at that time. And very important to understand is that reads and writes are by default not atomic meaning when you invoke read or write it may get interrupted in the middle of copying the data to or from the buffer and so say here when we have two writes to the same portion of the file the data being written there may end up getting interweaved. That is portions of the data from one call to the write may get overwritten by data from the other call to write and vice versa. So what we end up with here is not necessarily Herp Derp overwritten by blah blah or blah blah overwritten by Herp Derp but possibly something resulting from them taking turns overriding each other. In the case of simultaneous reads there's generally not a problem because as long as the data is not changing then the reads aren't affected by each other however if a read is intermingled with writes then the changes made to the buffer by the writes may end up changing what data gets read. So the important takeaway here is that by default in Unix the file buffers are read and written with no coordination. Now some programs like say databases do require exclusive control over a file and for that purpose there are special mechanisms which we won't get into in this unit. As previously discussed every file in Unix has an owner and it has a set of permission bits that is it has a read write and execute permission for the user, for the group and for what's called the other class. So when we create a file how do we set its permissions? Well we can do so with a strangely named Umask system call which sets the permissions for any file we henceforth create in that process. The reason it's called Umask is because there is a so called mask, a set of bits each one of which corresponds to the permission bits so we configure this bit mask by invoking Umask and passing in the new mask we want what Umask returns is the old mask, the one with the old permissions. In case we're curious what the permissions formally were and henceforth any new file we open and write will be created with the permissions as set in that mask we passed to Umask. And these permissions also apply to directories which we'll talk about creating in a moment. Now if you wish to modify the permissions of an existing file the system call for that is called chmod as in change mode. We invoke it by simply passing in the path of either the file or the directory and then also the mask of the new permissions we want. Now of course it would totally defeat the whole purpose of permissions if any process could change the permissions of any file So for a call to chmod to be successful the invoking process effective user ID must be 0 the super user or it must match the owner of that file or directory. To change the owner of a file or directory we can invoke chown as in change owner. And of course we pass in the path to the file or directory and we specify the new user and the new group which will own this file. And again of course for security purposes we can't allow just anyone to modify the ownership of a file or directory. So for a call to chown to succeed the effective user ID of the invoking process must be 0 that is the super user or it must be the same as the user which owns that file or directory. Though in that case chown is only allowed to change the group so the user passed to chown must match the user which already owns that file or directory. Before discussing how to work with directories recall that each stores device is divided into possibly one or more partitions. It's within these partitions that files and directories are stored and within each partition each file and directory is known simply by what's called an iNode number. A number which uniquely identifies a file or directory within that partition. Always within a partition you have at least one directory called the root directory which is always designated iNode2. Why iNode2? Well iNode1 is used especially for a list that keeps track of all the bad sections in the partition the parts that are unusable. And iNode0 is used especially like a null pointer it indicates the absence of any file or directory. Now to be clear this diagram here is a bit misleading. It seems to imply that directories and files are always stored contiguously. That is that their bytes are always written in order in adjacent parts of the partition. But in truth a file can actually be scattered all throughout a partition in pieces. In any case to create and remove directories we have the system calls mkdir as in make directory and rmdir as in remove directory. And so here for example mkdir with slash alice slash tim creates a directory called tim in the directory alice. And then if we invoke rmdir with the same file path it then removes that directory. Now to add files to a directory you simply create the file. That is you open the file that doesn't yet exist and start writing to it and that creates it. What that does exactly is it creates a new file in the partition with its own iNode. And then in the directory it creates an association between an iNode and the name of the file. And be clear that files only have names insofar as they are listed in directories. Stored with the file itself is no name whatsoever. Files don't have a concept of names really. The iNode has no name, just a number. Only in directories is there a name associated with an iNode. So in fact what we can do is we can actually place a single file in multiple directories. Once a file has been created we can then add it to other directories with the system called link. So here for example we're taking the existing file slash alice slash in. And we're adding that file as another entry in the directory slash ben giving it the name Jill. So now we have this single file with this one iNode that is found with the name Ian in the directory alice but also found with the name Jill in the directory ben. As far as the file is concerned neither one of these is its true name. They are both equally valid. They're just different listings of the same thing. Now to remove files from directories we have the system called unlink. So here say slash alice slash Ian. This will remove the Ian entry in the directory alice. Now stored with each iNode is a count of how many times that iNode is listed in directories. And so every time an iNode gets unlinked from a directory its link account is decremented. And when that count reaches zero the file system knows that file can then be discarded. Effectively that means that the storage area which that file takes up on disk is now free to be overridden for the use of other files in directories. So it would be clear that unlinking the last link to a file doesn't really delete the file. If for privacy or security reasons you want to make sure that a file is really gone you should overwrite its entire content with random garbage. To read the contents of a directory we have the system called getDNTS as in get directory entries. First off we open the directory just like with a file that returns a file descriptor which actually points to a directory. And then when we invoke getDNTS with that file descriptor it returns some number of entries from that directory. Now for basically the same reasons as the read system call we don't know how many entries exactly getDNTS will return but it's always guaranteed to return at least one. So when getDNTS returns zero entries we know there are no more entries to read. Now each returned entry is a particular data structure containing most obviously an inode and the associated file name but also the length of that file or directory and also something indicating the type that is whether this is a file or directory or possibly one of a few different file types which we haven't yet discussed. In Microsoft Windows which you're probably familiar with each partition of every storage device on the system is given a drive letter like say the main hard drive is usually one big partition which is known as C. So the path to the root directory on that partition is C colon slash and in Windows the convention is to use backslashes rather than slashes though Windows doesn't actually care it doesn't matter which one you use. In Unix things work quite differently rather than assigning each partition its own drive letter each partition is mounted to a directory. So first off in any Unix system when it boots one partition must get mounted to the root partition designated with just a single slash. Once mounted the root directory in that partition is now synonymous with slash so the file path consisting of just a single forward slash that is synonymous with the root directory of the partition mounted at slash. Once a partition has been mounted to slash we can mount additional partitions to directories found on other partitions which are already mounted. So here for example we have partition 2 mounted at root if we have a directory slash Jessica a directory which itself is in the root directory of partition 2 then we can mount partition 4 to slash Jessica. Henceforth slash Jessica now refers to the root directory on partition 4 not the directory in partition 2. The directory Jessica in partition 2 is still there it's just now effectively hidden. We would have to unmount partition 4 if we wanted to get at the underlying directory again. In practice we generally just don't mount partitions to directories which have stuff in them usually we just create empty directories for the purpose of a so called mount point. In any case continuing here if we wish to mount partition 3 to slash Jessica slash Vincent well first there must be a directory there so inside partition 4 inside its root directory which is now mounted at slash Jessica we create a directory named Vincent and then we can mount partition 3 there. Finally to mount partition 1 at slash Alice slash Tim well then in partition 2 we're going to have to create a directory Alice and then inside that a directory Tim and then we can mount partition 1 there. To mount and unmount partitions we use the mount and U mount system calls. Now when specifying the directory to which we wish to mount a partition and from which we wish to unmount partition it's fairly obvious you provide that as a file path but as for specifying the partition itself that's actually specified with a special kind of file which we won't get into in detail in this unit. Anytime a system call expects a file path we can express that file path as either what's called an absolute path or as what's called a relative path. In an absolute path we specify the whole path starting from the root of the system that is the root directory of the partition mounted at slash. A relative path in contrast does not begin with a slash. A relative path is automatically translated into a full absolute path by tacking on in front what's called the current working directory. The current working directory is a directory associated with a process and is set in that process with the chdir as in change directory system call. So here for example I am setting the current working directory for this process to slash ben slash in. When I then call open with an absolute path slash alice slash tim the current working directory is irrelevant. This simply opens slash alice slash tim. In the second call to open here though the path specified is relative so it is actually opening alice slash tim inside the current working directory which at the moment is slash ben slash in. So this opens slash ben slash in slash alice slash tim. What's the point of this? Well simply in some context it's more convenient if we don't have to write out full absolute paths. So far we've only discussed what you might call regular files that is files with data on a storage device and we've discussed directories which are basically just listings of files in other directories but Unix also has a number of other things which it also considers to be so called files. In fact in the broadest use of the term directories are considered to be files. These other file things include what are called symbolic links and also what are called device files or sometimes special files and those come in two types character device files and block device files and then also we have what are called pipes and sockets. Very briefly a symbolic link is a file which is written to disk like any other file but it doesn't have any real content. It just has a link that is a file path pointing somewhere else and it's specially marked as a special kind of file it's not a regular file it's a symbolic link file such that any system call which opens a symbolic link will not open the symbolic link itself but actually the file pointed to by the symbolic link. In practice symbolic links are very much like shortcut files and windows. Device files as we'll discuss in a moment represent hardware devices and they are effectively a clever way for our processes to send and receive data from devices using the same system calls we use to read and write files. These device files don't really represent stored data like a regular file rather they're more like convenient fictions which appear to act like files. Pipes and sockets are both means for inner process communication that is sending data between processes the difference being that pipes can only communicate between two processes on the same system whereas sockets can connect processes on different machines across a network. Again pipes and sockets like device files don't really represent any kind of stored data on a storage device they really are just sort of a fiction that allows us to do this communication using the same set of system calls we use to read and write files. That's the sense in which these things are considered files. So to create a symbolic link we have the system call sim link to which we supply the path to the file or directory to which we wish to create a symbolic link and then we also provide the path of the symbolic link we wish to create. So here this creates a symbolic link slash Jill slash Ken which points to the file or directory slash Alice slash Tim. If we then open that symbolic link what actually gets opened is the file to which it points not the symbolic link itself. That's pretty much all there is to symbolic links they're really not that complicated. Now as for device files first recall the basic relationship between the CPU and an IO device. Devices typically have some number of registers which the CPU can read and write and that is how they communicate. There actually isn't any way for the device to force the CPU to read its registers though some devices may send an interrupt signal to the CPU which then triggers it to go and execute a predesignated chunk of code which then will tell the CPU to hey go read these registers. Other than that the CPU is basically in control. So ultimately talking to a device means reading and writing its registers but at the level of processes we don't want to have to deal with such details we want to work at a higher level of abstraction. So this level of abstraction is what device files provide for us. A device file is not really a file but when we open a device file and read from it we are getting data from the device and when we write to it we are sending data to the device. For this to work though we need a distinction between what are called block devices and what are called character devices. Very broadly block devices are generally devices with large storage areas like say a hard drive is naturally a block device. Character devices in contrast are the sorts of devices that don't really store much data. Data flows in and data flows out but the data is not really retained by the device it's more like acted upon as it flows in and out. Or to think of it another way with block devices it makes sense somehow to read and write to specific locations whereas with character devices the data simply goes in and out in sequence. You are not specifying where in the device you are reading from or writing to. So looking first at block devices storage partitions are divided into units called blocks. Like the bytes of RAM these blocks are numbered from 0 and they are all of a uniform size. Usually the block size on a partition is something which is a multiple of 512. So like say 4096 is a typical block size. Also like the bytes of RAM these blocks have to be each read in their entirety. So say if you want to read something in a block you have to read the whole block you can't just read part of it and even if you want to write to just a single byte in a block you actually have to rewrite the whole block. So that usually means you'll have to read out the block to a buffer modify the portion you want to change and then copy back the buffer to the block. So when an iNote that is a file or directory is stored on a partition it's stored on some number of blocks and those blocks actually don't have to be contiguous as I previously mentioned. So say here if we have an iNote 86 which is presumably some file or some directory it might be stored on blocks here 1, 4 and 6. So one consideration when deciding how big the blocks should be on a partition is if you make them too big then very small files like files taking up only a few bytes are going to end up wasting a lot of space and files always get stored in whole blocks you can't have a block that's shared between multiple files. So every file no matter how small always takes up at least one block. Now when it comes to reading and writing files on a block device as previously mentioned there's always a buffer involved and the way it works in most unixes including Linux is that there's a buffer for each block and as we previously mentioned no matter how many overlapping reads and writes you have from however many different processes they're always just reading and writing to the same single buffer so there's always just one buffer per block. Anytime the buffer gets written to that block is marked as so-called dirty meaning that the content in the buffer no longer matches what's actually on the disk so it needs to be copied to the actual storage device. So that explains what happens when we write files to block devices but what though is a block device file? Well a block device file is a file which effectively represents the whole storage area of that block device such that the first byte of the first block is byte 0 and then the last byte of the last block is the last byte in the file. When we write to a partition this way we are actually circumventing the whole system of files and directories on that partition. This way we're actually reading and writing the raw bytes themselves and so in fact when we write to a block device like this we can very easily screw up the files and directories on that partition because of course when files and directories are stored there has to be some extra information written that keeps track of how the files and directories are actually written on disk namely which blocks they occupy and in what order. So reading and writing a block device through a block device file is not something we normally do but the capability to do so is usually provided because well some programs simply have very special needs. Like for example databases as we mentioned have special needs when it comes to storing and retrieving data so database may in fact have a partition like this set inside which it reads and writes in this manner. Again though this is not the usual thing to do and it probably occurs to you that this is an obvious security hole. Block device files pretty obviously should be given restrictive permissions. You don't want just anyone reading and writing arbitrarily to any partition. Character devices differ from block devices in that first of all there's no concept of blocks so instead of having a buffer for each block there are actually just two buffers one for input and one for output. When a process writes to a character device file it is writing to that device files output buffer and when a process reads from a character device file it is reading from its input buffer. These input and output buffers are usually so called because the character device file usually represents some kind of actual hardware device and so the input which the device receives it writes to the input buffer of the device file and the data written to the output buffer is sent as output to the actual device. So the other big difference here is that not only do character devices have no concept of blocks they also use a separate buffer for input and output. The explanation for this difference is that the input and output buffers of a character device file are both what are called FIFOs. They are first in first out buffers. A FIFO is effectively like a line a queue of people waiting for something. When people join a line they join at the end of the line and it's the people at the front of the line that get served next. Likewise in a FIFO buffer the bytes that are written to the buffer are appended at the end and the bytes which are read from the buffer are always read from the front. So to make this absolutely clear in the case of the input buffer of a character device file the device will send data to the input buffer which then gets appended to the end and then when a process invokes the read system call on that character device file it sends the bytes at the front of the buffer. Once read those bytes are then removed from the buffer. Conversely with the output buffer data is added when a process writes to the character device file and the data at the front of the buffer is read by the actual device and again once read the data is removed from the buffer. Now this is just a logical picture of how a buffer is actually supposed to work. I like to picture it as if when data is read from a buffer any data slides forward to the front but of course that's not what actually happens because in practice what that would involve is actually copying all the data byte by byte each time something's read and that would be very inefficient. So really what goes on in the character device file there's something which keeps track of where the next read in the buffer should occur and where the next append of new data should occur but those are details handled by the operating system we don't really have to concern ourselves with them. On the other hand these buffers are generally capped in size they of course can't hold an infinite amount of data so what these caps can mean is that when you write to a buffer the write may fail because there's not enough space left in the buffer. Aside from that you really shouldn't have to concern yourself with exactly what's going on in these buffers. Let me reiterate the core difference between block devices and character devices. Block devices are for devices that have some large kind of storage space and so each buffer for a block device is actually backed by storage space but character devices generally aren't backed by any storage space. With character devices typically what's happening is that there's these two buffers in the operating system which are serving as a channel of communication between processes and some actual device. A device which typically has a small number of registers but maybe no other storage. So again character device files are usually for the sort of device which immediately acts upon the data which it receives as output. As we'll see in the next unit primary examples of character devices include terminals. How exactly character devices get created differs greatly from one unix to the next. Generally it's something handled specially by the operating system during boot time. By convention though device files to be found in a directory placed at slash dev, dev of course short forward device. So here for example I'm opening two device files one is slash dev slash sda1 and sda1 assuming the system follows the usual convention should refer to the first partition of the first storage device on the system. sd here actually originally stood for a scuzzy device but some people have retconned and now say that it stands for storage device. In any case a device file representing a partition will of course be a block device file. In the second line we're opening a character device file LP here standing for the archaic term line printer. So this should be a character device file which I can send data to to actually print something out on my printer assuming I have one. Don't ask me why line printers are numbered starting from zero but partitions are numbered starting from one. Now that I've opened these two files I can now work with them such as say reading from them and notice that I can invoke lseq on the block device file but I couldn't do the same with a character device file. Character devices have no concept of file markers. So calls to lseq on a character device file won't do anything. It should be fairly obvious what we get when we read from the block device file here. We'll get back whatever data is stored starting at byte 100 on that device but in the case of a character device it's less obvious. In the case of a printer it's obvious why you'd want to send data to the device because you want to print stuff but understand that you may also need to read from the device to say check its status. If something's gone wrong with a printer and is not ready to print that's something your program would probably want to know. In the dev directory you'll also find a number of what are called pseudo device files. These are mostly character device files but they don't represent actual hardware devices. They're simply abstractions provided by the operating system. For example, the pseudo device file slash dev slash zero is a character device which when read will simply return an infant string of nulled bytes of zeroed out bytes. Similarly the pseudo device slash dev slash random will return an endless stream of random bytes. And lastly another commonly used pseudo device file slash dev slash null doesn't return any bytes at all. Instead it simply exists for when we need a file to which we can write data and have the data just go nowhere and disappear. It sounds a bit odd but strangely enough that does actually come in useful sometimes. Honestly I find the name a bit confusing. Zero and null sound like they're kind of the same thing though they're not. Null should have probably been called discard or something like that. Now I think I've been clear that device files whether they're block files or character device files or whether they're pseudo device files they don't necessarily represent stored data but on the other hand there has to be some kind of representation for these so-called files. That is in the dev directory for every device file there has to be an entry of this name corresponds to such and such iNode and there has to actually be an iNode representing the device file. In the iNode stored on the partition is a special indicator that this is not a regular file. This is a device file. So in that sense device files are stored like other files. It's just as no stored data associated with their iNodes. What we call in Unix a pipe is basically just a single FIFO buffer used as a channel of communication between processes. When a process writes to the pipe data is appended at the end and when a process reads from the pipe, data is taken off the front. So when say process B wishes to send data to process A it writes to a pipe and process A then reads from that pipe. Generally it makes most sense to treat a pipe as unidirectional so one process writes and the other one reads. If you want to send data in the other direction you should use a separate pipe. To create pipes or in fact to create device files and even regular files we have a system call called kNode as in makeNode. Here node is used sort of like a generic term for file. When we invoke makeNode we specify the name of the file we wish to create and we specify its type and in the case of block and character devices we have to specify the device number. As we'll explain in a later unit each device in Unix is given a unique device number. Now again be clear that these files though stored as iNodes on disk themselves do not have any data with them. So when we write data to a pipe that data only exists in memory. It is not written to the partition on which we have this pipe file. Now when you create a pipe in this manner as we do here with the file ryan slash tina this is what Unix calls a named pipe because it actually has a name. It has a name in some directory. You can also though in Unix create an anonymous pipe or what's just simply called a pipe and you do so with the pipe system call. What pipe does is create a new anonymous pipe and returns to file descriptors each pointing to separate file descriptions though both of those descriptions are opened on the same pipe. The first one open for reading, the second one open for writing. The idea is that we create a pipe and then fork off another process. Then one of the two processes either the child or its parent can write to the pipe using one descriptor while the other process uses the other descriptor to read from the pipe and as mentioned a pipe is usually for one way communication so if you want communication in both directions you should open two pipes. To be clear when you create pipes in this way it's only useful for communication between related processes that is two processes where one is a descendant of the other. Because open file descriptors are passed on to child processes they can both see the same pipe created in this manner. For unrelated processes you would have to use a named pipe. Another feature of modern Unix is the ability to map a file in whole or in part to pages of memory. What this effectively means is that pages in a processes address space get specially marked such that reads and writes to those pages actually trigger reads and writes to some underlying file. So here for example we're opening a file slash brad slash mic assigning it to a file descriptor F and then when we invoke mmap to map pages of our address space we pass in first the number of bytes we wish to replicate but now we're also passing in a file descriptor and then specifying with a file offset which part of that file we wish to map to memory. So what this called to mmap will do is map the 500 bytes starting at byte 200 in the file to pages in the address space such that when we now read and write from those addresses we're actually reading and writing from that part of the file. When done with this work we should then release the pages of memory and also close the file. Actually it would have been fine if we'd closed the file after we did the mmap. We don't have to keep the file descriptor around for the sake of the mapped memory. You're probably wondering what is the point of memory mapped files. Well it tends to make sense when your reads and writes are not contiguous and clumped together they're just spread everywhere. When this is the case it could be considerably easier to write code with a memory mapped file than with regular reads and writes and also it may end up being more efficient. In fact depending upon your unix system and depending upon some options when you invoke mmap the memory mapped this way into your process are generally the very same buffers that are usually used when we read and write from files. So this avoids the usual extra work of copying data from your process to the buffer or from the buffer to your process. I would say that memory mapping files is used much less commonly to read and write files than simply using the read and write system calls. But it is something that comes in handy for the programs of good use of it. What unix calls signals are sometimes called software interrupts because they're somewhat analogous to the hardware interrupts sent by hardware devices to the CPU. A so-called software interrupt however is sent by the operating system to a process which then must somehow deal with a signal either by performing a default action associated with that type of signal or by invoking a handler function registered for that signal or possibly by blocking the signal or ignoring it. Blocking here meaning that the signal is queued up until the process then unblocks that type of signal and ignoring here meaning that the signal is just discarded and totally forgotten. Now in most modern unixes there are between about 30 or 40 different types of signals and the reasons for why these signals are sent varies from type to type. Many of the signals are sent in the event of some hardware event especially some kind of error. So for example four signals found on all unixes include SIG SEG V where SIG of course stands for signal and SEG means segment as in segmentation fault don't ask me what the V stands for actually and then SIG FPE where SIG stands again for signal and FPE means floating point error and SIG STOP is signal meaning stop and SIG CONT meaning signal continue. SIG SEG V is the signal usually sent in the event of some memory error such as when a process attempts to use an address and currently allocated in its address space. The name SEG V as in segmentation fault is actually a bit of a misnomer it's an archaic holdover from when memory systems were based around a scheme of segmentation rather than a system based around pages. More appropriately it would be named something like SIG PAGE F as in page fault instead we're stuck with SEG V for historical reasons. In any case when a process is sent the SIG SEG V signal the default action is to simply abort the process. However if a process registers a handler for that signal that is it registers a function to invoke when that signal is received then that function will run instead. In the case of a memory error a process probably should quit because it seems something has gone quite wrong but by registering a handler for this signal we can at least run some cleanup code before exiting say giving us a chance to preserve some important data or something. The SIG FPE signal the floating point error signal is sent to a process because that process is attempted to divide by 0. When your program tries to have the CPU divide something by 0 the CPU is going to throw a hardware exception and that's going to trigger the operating system to then send your process the SIG FPE signal. And again for this signal as in fact for most signals the default action is simply to abort your process to terminate it. Again however if you've registered a handler for this signal then instead that handler will run and if appropriate your handler very may well choose to terminate the process though nothing is forcing it to do so if you just let the handler return the process will continue from where it got interrupted. The SIG STOP signal as the name suggests is sent to a process to stop it the default action when a process receives SIG STOP is to go into a blocked state the process will then only unblock when it receives the SIG CONT signal In the case of SIG STOP and SIG CONT these signals are not sent in the event of an error condition they are sent explicitly from one process to another using the KILL system call KILL allows us to send any signal to a specified process the name KILL very misleadingly implies that signals always kill the processes which receive them but really that's not the case it just happens that the default action for most signals is to actually kill the process to terminate it. Still KILL would have been much more sensibly named something like SIG SIGNAL. It's just one example of a long UNIX tradition of stupid crazy names that we've been stuck with for 30 years Anyway the two most basic system calls associated with signals are first KILL and also SIGNAL which is used to register signal handlers So here for example we're invoking KILL to send the SIG STOP signal to the process 35 and the invocation of SIGNAL here is registering for the current process a handler for the signal SIG FPE and the handler is a function which we pass in here called FUNK Last thing we'll note about signals is that of course for security reasons you wouldn't want any process to be able to send any signal to any other process. Consequently unless a process has super user privileges it can only send signals to processes owned by the same user