 Unix was the name of an operating system created back in the early 1970s. Today, however, Unix refers to any operating system which imitates that original Unix. In the 1980s, the two most prevalent variants of Unix were called System 5, which was created by AT&T and BSD, which stands for Berkeley System Distribution, so called because it was created at Berkeley University. Today, the most widely used variants are Linux and Mac OS X and in distant third place are a few descendants of BSD, including free BSD and open BSD. As a matter of fact, Mac OS X is itself based upon a descendant of BSD called Darwin. Perhaps the biggest difference among these variants is that whereas Mac OS X is a proprietary operating system, that is, it's a commercial product which you must pay for for each copy, Linux and the BSD variants are all examples of what's called either open source software or free software. This means that the source code for these operating systems is really distributed such that anyone is allowed to copy and modify them. The other major difference is that whereas Mac OS X has now a quite significant share of the desktop market, Linux and BSD do not, Linux and BSD in contrast are very successful as server operating systems. In fact, Linux has the dominant market share even greater than Windows and Linux is also very widely used for small devices, including embeddable systems like say the computer in your car. When it comes to smartphones, the two major players now are Android and iOS. Android is a variant of Linux created by Google and iOS is a variant of Mac OS X used for the iPhone. So we have all these variants of Unix and none of them are exactly the same, which creates problems because ideally we would like to be able to create a program and then have it run on all variants of Unix. Basically what we need is some standard that defines what it means to be a Unix system. So in the late 80s and early 90s, operating system developers got together and created two such standards, one called POSIX, which stands for portable operating system interface for Unix and one called SUS or the single Unix specification. Today, most variants of Unix more or less conform to both of these standards, though there really is quite a bit of divergence in the details. These standards also fail to entirely solve the problem of portability because they don't really specify everything about a Unix system that are rather limited in their scope. So there are many features of today's Unix systems, which work totally different on different Unix systems, or in some cases exist only on some Unix systems, but not others. In the end, it is possible today to write some programs, which will run on any Unix system as long as you stay within the bounds of the two standards. As soon as your code uses platform-specific features, you're probably going to have to do some extra work to get it to run on other platforms. From the perspective of a program, the primary thing which defines an operating system is what system calls it makes available. System calls, as we discussed in an earlier unit, are functions in the operating system code, which programs can invoke using a special instruction. These functions are the primary means by which the operating system exposes functionality to programs so that programs can use features of the hardware, like say, read and write data on a storage device or send or receive data across the network. And again, recall that the reason these special functions can only be invoked with a special instruction is that normally when a process executes, it can't read and execute data that's part of the operating system kernel itself. Each process is supposed to run effectively confined to its own box, its own part of memory. The instruction to invoke a system call breaks out of that box, and the way it does that is that in the instruction, you specify the number of the system call you wish to invoke, and that causes the CPU to go and look up an address which corresponds to that number in a special table. What we did not discuss is that actually in most modern operating systems, the kernel code for these system calls and actually the table for the system calls itself are placed inside the address space of each process. Usually this is placed at the top of the address space and the stack starts immediately below it. These pages of the address space are normally marked such that the process itself cannot access them, only when the process invokes a system call via the special instruction does execution actually jump up to the kernel code. And when a system call is invoked, it uses the stack of the process to place a stack frame for that system call, just like with any other function. The purpose of all this is that it allows system calls to execute in the context of the process which invokes them, and that avoids a context switch, meaning we don't have to swap out the memory tables of the current process. We can just leave the memory table for the current process in memory for the duration. Another advantage of this arrangement is that it allows system calls to be naturally interrupted, that is suspended and then resumed later. When we suspend the process, it generally doesn't matter if it's running user code, that is code of the program itself, or if it's running kernel code, a system call. If system calls were to run outside the context of a process, that would make it trickier for the operating system to suspend a process while it's invoking a system call. In our earlier discussion of operating systems, we discussed how a process transitions between a few different states. Most obviously, a process can be running that is actually being currently executed by a CPU, or it can be waiting, meaning it can be waiting for the scheduler to put it into the running state. While running, a process can also be transitioned into the state of being blocked. While blocked, a process will not be scheduled, so it won't ever run again until it is unblocked and put back into the waiting state. There are a dozen ways in which a process may get blocked. The most common is from invoking certain system calls. For example, as we'll discuss shortly, the system call for reading from a file may block the process. The reason it does this is because most data sources from which we read files, namely storage devices like hard drives, such devices tend to be very slow, relative to the CPU. So there's nothing for the process or the system call to do while it's waiting for the data to be read off of that device. So the way the system call works is it tells the operating system, hey, I want that data off of that device, it then blocks the process and then it's the job of the operating system to once the data is ready, unblock the process. So that's the general pattern. A process gets blocked when it has to wait for something and then either the operating system or some other process will then signal that process to unblock it and try not to get confused that blocking effectively means waiting, but what we call waiting refers to waiting to be scheduled and ready to be scheduled. So a blocked process waits for some trigger and then it waits again in the so-called waiting state. So system calls effectively represent the functionality of the operating system exposed to programs. This includes system calls for, first of all, managing processes themselves, such as say from one process starting another process so that we can run another program. And then there are many system calls for dealing with files, creating files, deleting files, reading them, writing them, etc. And then UNIX has system calls for what it calls sockets. A socket represents one end of a network connection. So when you want to program to talk to another program on another system, you create a socket in your program and that socket communicates with the socket in the other program and that's the channel of communication. Linux also has system calls for what it calls signals. A signal is basically a notification of some event or some condition sent from the operating system or process to some process. Many of the signals sent by the operating system indicate some kind of error condition. Like for example, when your program causes a memory violation, that is it reads or writes some byte of memory it's not allowed to, recall from our earlier unit that that will trigger a hardware exception in the CPU causing the CPU to go and execute a pre-designated piece of operating system code and that operating system code will send a signal to our process indicating the error. And when the process receives that signal it gets interrupted and it will execute a function pre-designated to run for when that process receives that signal. UNIX also have many system calls for what's generically called inter-process communication meaning just some kind of mechanism for processes to communicate with each other. Actually network sockets are one form of inter-process communication because in fact when a process communicates over a socket the other program doesn't have to be on another system it can actually be another process on the same system. But for communication between processes actually running on the very same system we have some additional mechanisms which generally have the advantage of being more efficient. UNIX systems also have a surprising number of system calls just for dealing with what are called terminals. Terminals and command line shells are something we'll talk about in the next unit. Today's UNIXes also have system calls for what are called threads. When a process runs by default it has one thread of execution that is there's one code pointer there's one pointer pointing to what the current instruction is and there's one stack keeping track of all the functions that we've invoked. With multiple threads of execution you can effectively split a process into separate threads each thread having its own code pointer and its own stack. The simplest way to think about threads is that they are like separate processes which run independently and are scheduled independently but they share the same address space. So the data on the heap can be read and written by any thread in the process. All of the reasons you might have to make your program multi-threaded and all of the difficult problems that arise in multi-threaded programming we'll discuss in a later unit. Finally we have system calls for talking to IO devices that is everything from the system clock to the video adapter to storage devices. Though actually of course in the case of storage devices we generally read and write data to them in the form of files. And actually as we'll see UNIXes allow us to treat IO devices like files which admittedly is a very baffling thing to hear. What it really comes down to is that when it comes to reading and writing data from an IO device we can in most cases use the same system calls that we use to read and write files. So in that sense we can treat IO devices in UNIX like files. Now in this unit we're mainly going to focus on the system calls for managing processes and for reading and writing files and also a little bit about signals. We're not really going to cover anything about sockets or in a process communication or terminals or threads or even IO devices. Though as I mentioned there's some overlap between the system calls dealing with files and those dealing with IO devices. Now as I've said a system call is invoked using a special instruction. The question though is well if I'm not programming in assembly how do I invoke that instruction? Well in the case of the C language for example C compiles into machine code and in C we can actually write functions in assembly which we can then invoke from our C code as if it were like a C function. So most implementations of the C language will include along with just a compiler will include standard libraries that include these functions which invoke the system calls. So even though we haven't yet learned the C language here for example is what the read system call the system call which reads a file this is what it looks like in C. In the case of interpreted languages most interpreters are written in C or C++ and so to invoke a system call from your Python code we need our Python implementation our Python interpreter to provide for us some special function which ultimately invokes in C code that system call and because of semantic differences between the C language and say Python the arguments you provide for these functions don't necessarily correspond exactly to the arguments provided to the system call in C nor do they necessarily return exactly the same kind of value. The function provided to us by Python the wrapper function is bridging that semantic gap between the Python language and the C language. So the functions say in Python which wraps the system call read might be simplified to say have just one parameter instead of three. The purpose in this unit is to just familiarize you with the most important system calls and to understand what they do not necessarily how exactly to invoke them. As you'll see in a later unit the C language has no exception mechanism so there's a very different way we have to deal with errors basically when we invoke a function we have to check the return value for some kind of error condition so when you use standard library functions in C including these system call functions you need to learn how to check for and deal with errors for each particular function. In fact there are generally many things that might go wrong when you invoke a system call for example when you attempt to open a file so you can read or write that file what if the file doesn't exist well then the function the system call has to return some kind of error and you need to learn to check for those kinds of errors. In truth you should almost think of invoking a system call as more like making a request than making a demand. You're asking the operating system to do something and for various reasons depending upon what system call you invoked the operating system may or may not oblige your request. While the possibility of errors is in practice of course very important it's not something you can simply ignore for our purposes here we're just going to pretend that errors don't exist we're just going to lied over all of those details. As for the code examples in this unit they're all going to be kept exceedingly short and they're not actually going to be written in any proper programming language they're going to be written in something that looks like Python but don't imagine it's actual Python code this is just pseudo code for our purposes here. So let's start by again discussing processes. For every process currently in the system the operating system keeps a data structure that keeps track of all the things associated with that process and those things include first of all an address space that is to say the memory table which is loaded when that process is running but processes also include a few things which we haven't yet discussed including user IDs, file descriptors, what's called the environment which is basically a small amount of data associated with the process and each process has an associated current directory and a root directory. All of these are things which we'll explain in turn. Looking closer at address spaces we haven't previously mentioned that most processes include a section for what's called the uninitialized data and one for what's called the initialized data. These are sections of memory set aside at the start of a program for storing global variables the difference being that in the uninitialized section those global variables don't have any initial values whereas in initialized data section they do. When the operating system loads and executes an executable file it's the executable file which specifies how large these sections need to be and the reason we think of initialized globals is different from uninitialized globals is that for every initialized global that value it has to be somehow stored in the executable the initial value whereas with uninitialized globals there is no value so there's nothing to store in the executable. The executable just needs to make note of hey we need this much space for additional global variables. That's an important thing to understand. When you create an executable you're doing so from a compiled language like C and unlike in Python where global variables are sort of created in the course of the execution of the program and C at compilation time the number of global variables is fixed so however many global variables you put in your code the compiler knows and so it sets aside enough space in the executable. So these data sections have no need to either shrink or grow in the course of the program they're always fixed in size. As for the other sections in the process the code section which is actually usually called the text section the code is called the program text so to speak. The code section is fixed in size and the code is all loaded at the start of the program the same is true of the kernel code section. The stack section in contrast starts out empty at the start of the program and in the course of the program it will grow and shrink. The heap portion of the address space is all the space left in the middle but to use this space we have to explicitly allocate it from the operating system. We have to invoke a system call that says hey this part of my address space I want that mapped to actual addresses in memory so that I can use it. If your code attempts to use an address in a portion of the heap which has not been allocated that triggers a memory error. The CPU recall translates every address in an instruction to a corresponding physical address using the memory tables the current memory tables for that process and so when an address in an instruction doesn't map to anything in physical memory that triggers a CPU exception and the CPU runs some operating system code which then sends a signal to your process saying hey there was a memory error. To allocate memory in a process the process should invoke the M map system call and map short for memory map and what that does is it adds some number of pages into the process address space and maps those pages to actual addresses in physical memory. The M unmap system call does the opposite it deallocates memory pages from your address space such that they no longer resolve to actual addresses in physical memory and so you can no longer use them. So here for example we invoke M map and we pass in an argument of how many bytes we would like to allocate. Notice though we don't specify which bytes of memory we want because generally it's left up to the operating system to keep track of all of the chunks in the address space which are free and so it's the job of the operating system to find a chunk of 5,000 contiguous bytes somewhere in that address space and allocate it and then return the address of the first byte of that chunk. So what M map returns here is a newly allocated chunk of memory somewhere in your address space that has at least 5,000 bytes. Notice I say at least 5,000 because again memory is allocated in chunks of pages and pages are usually something like four kilobytes in size so if you allocate 5,000 bytes that probably allocated at least two pages of the address space so in fact you probably could get away with using up to about 8,000 bytes starting at that address but you shouldn't be making that kind of assumption. You requested 5,000 bytes so you shouldn't assume that the operating system is giving you any more than that. So once you have this allocated chunk and you know what the address of the first byte is then depending upon your language you can then manipulate all the bytes in that chunk you can read and write to them and then once you're done with that chunk of memory you no longer need you discard it you remove it from your address space with the M unmapped system call by passing to it the address of the first byte of that chunk you previously allocated. Understand that for every chunk of memory you're allocating the operating system is keeping track of those starting addresses the first bytes in those chunks so that later when you call M unmapped it can know which chunk of memory you're referring to. It may occur to you that the problem of allocating and deallocating memory from a fixed address space is really quite a tricky problem because what we want to avoid as much as possible is a scenario where we wish to allocate x number of bytes of memory and there's more than x number of bytes available is just that those bytes are not necessarily contiguous so we can't allocate a chunk of that size. The allocation request is going to fail in that instance. How to do allocation to avoid that situation as much as possible is actually one of the most studied problems in programming. It's not a trivial problem. What makes it especially difficult is that there's not necessarily a one size fits all solution. Different programs have different patterns of allocation and deallocation so what some programs do especially when you write a program say in C or C++ is that you may take the allocation problem into your own hands that is rather than letting the operating system make these decisions you can simply at the start of the program allocate one big chunk and then use your own allocation algorithm to hand out sub portions of that chunk. On the one hand this does mean that your process is grabbing a large chunk of memory which it isn't all necessarily using on the other hand it may enable your program to better use the memory which is allocated for it. Now a lot of students become unclear about why exactly they need to really give back memory at all because especially in trivial programs when you allocate memory why should you have to deallocate it before your program exits because when your program exits and the process terminates that memory is all given back to the operating system anyway so it's not really an issue. Despite this you really should always explicitly deallocate any memory that you allocate and even if it doesn't really make a difference in small trivial programs you really should get in the habit because what happens in larger programs especially programs that run for a long time or perhaps use a lot of memory if you allocate memory you never give back one it means you're likely at some point to simply run out of memory you're not going to have a more space in your address space to allocate more memory but even if that doesn't happen you still don't want to keep run memory that you're not actually using anymore because that means no one else can use that memory. Now actually on most modern operating systems we have a virtual memory system and so any memory that you're not using for a long time is probably just going to get swapped out onto a hard disk and so it's not really going to be stealing physical RAM from any other process or from it's from that same process but it still represents a waste of resources. Now all the languages we've already looked at are interpreted languages that use garbage collection that is we don't have to explicitly allocate any memory or deallocate any memory because it's always done for us. In Python say when you create an object it's up to the interpreter to make sure that there's some place in allocated memory to put it and it's up to the interpreter to keep track of that object such that when there are no more references pointing to that object it knows that chunk of memory is no longer used and so possibly can be deallocated. These days the only major languages where we explicitly allocate memory are aside from assembly of course C and C++. Virtually every other language these days does automatic memory allocation and deallocation for us. The way we create new processes in Unix is really quite surprising the only way to create a new process is for a process to copy itself that is to copy its entire address space to copy all of the other things associated with the process like the user IDs, the environment, and the file descriptors and so forth. This is done with the fork system call when a process invokes the fork system call a fork of that process a copy of that process is created when the fork system call returns both processes actually pick up precisely at that moment where the fork returns. The only difference in the two processes is what value gets returned from that fork system call. In the newly created process the fork the so-called child process the fork system call returns zero. In the original process the parent process the one which invoked fork in the first place the call to fork returns what's called the process ID of that child process and we'll discuss process IDs shortly as it sounds it's basically just a unique number identifying each process in the system so in this code example here we're invoking the fork system call and then we're testing what value that fork system call returns and if it's zero then we know that we are inside the new process the child process but otherwise we know we are inside the original process the parent process so this if else here has two branches the first of which will only run in the new process and the second of which will only run in the original process the parent now you may be concerned that a fork system call may be very expensive to perform because it will involve copying all of the data in one process over to the new copy of the process but that's not actually the case in older unixes that actually is what happened but in newer unixes we have virtual memory so what happens is that we only need to copy the memory table of the process not all the actual content in memory whereas the contents of a processes address space may be megabytes if not gigabytes in size and therefore take a long time if we wanted to copy the memory tables themselves for a process are really generally very small something on the order of kilobytes rather than megabytes or gigabytes so for example say this represents the address space of an existing process and the pages in memory as you see are mapped to portions of RAM and you may have some number of pages which are actually swapped out to the hard drive so they're not currently actually in RAM when we copy the memory table for the new fork its memory pages are all mapped to the precise same portions of actual physical RAM and pages on the hard drive so now say address 200 in both processes is pointing to the very same storage whether in RAM or on the hard drive so far everything's great as long as the two processes only read from their memory rather than write to it because we actually want the two processes to henceforth diverge such that when in the new fork we write to memory that change should only be seen in the new process not the original process or vice versa if you write in the original process you want that only to be seen in the original process not the new process to solve this problem all of the pages in the new process are marked as copy on write the copy on write flag indicates that as soon as either process attempts to write to that portion of memory that page of memory then that page in memory needs to be copied and the address table of the new process updated to point to that new page so here for example after the fork these two corresponding portions of the address spaces point to the same page in RAM and until either process attempts to write to that page that's just fine because hey the data hasn't changed why don't we just share the same copy as soon as either process attempts to write to an address in that page however that triggers an exception in the CPU because it detects it in the memory table that hey this page has been marked as copy on write and that then triggers the operating system to before allowing the right to go through to actually copy that page and then update the table of the new fork now that the fork has its own actual copy of that page it's okay for writes to go through so this copy on write scheme allows modern unixes to very cheaply fork new processes without having to copy a whole bunch of data at the outset in fact very typically when you fork a process only a very small number of pages ever get written to in the whole lifetime of the forked process so quite often the system only ends up having to copy a very small minority of all the pages so the fork system call creates a fork of an existing process but that brings along all of the already loaded code in that process so you're not really changing the program you're just splitting an existing program into two separately running copies to actually load a new program we use the exec system call which again surprisingly does not actually create a new process what the exec system call does is actually replace the current program in the process with a new program and when it does this the entire address space of the process gets discarded and effectively a new one is created first by loading in the new code from some executable file so after an exec your process now has a new code section a new uninitialized and initialized data sections no heap section yet nothing's been allocated yet and a totally empty stack and execution of this new program begins at some point designated in the executable file so when in unix you wish to start a new program very unintuitively what you do is you first copy your existing process and then in the fork of that process you call exec to load a new program such as here in this example we're exacting the executable file found at slash games slash paul after the exec most everything about that process remains the same except for the very big difference that well it's a totally new address space with a new program loaded the process however retains most of the other resources of that process such as the environment and the file descriptors which we haven't yet discussed if processes are only created by other processes well then where did the first process come from well when the unix system starts there's always a first process called the init process and from there all other processes descend effectively end up with this hierarchy of processes starting with a knit and then its children and in turn their children and their children and so forth and each process that's created is known by a unique id number a process id number or pid and init always has the pid of one and then any subsequent processes created from there have basically whatever's the next available pid number do understand that these process id numbers can be reused so for example if you have a process with the id 29 once that process terminates some subsequently created process might be given the id number 29 now processes can get terminated for a variety of reasons but when the process chooses to terminate itself it does so by invoking the exit system call and the reason for the underscore in front is to distinguish this from another function called exit in the c standard library but in any case when we invoke the exit system call we pass in a number called the exit code the exit code indicates to other programs what happened to this program why did it exit and by convention the exit code zero is used to indicate that this process terminated normally basically the program was over so it terminated itself other exit codes are generally used to indicate some kind of error something went wrong in the program and some programs will use specific codes to indicate specific errors so you look at the documentation for that program and say oh negative three means that the program terminated because it ran out of memory or something like that again there's no hard and fast rules about what these exit codes mean it's really up to each individual program to decide what they mean when a process terminates the process which can read its exit code is its parent the process from which it was forked and the most common way for the parent to do this is to invoke the wait system call when a process invokes the wait system call it goes into a blocked state until the specified other process its child process terminates at which point the wait system call then returns the exit code from that child so here again we have a process which is forking itself and then in the child the forked off process it uses the exact system call and hence abandons the current program and loads a new one the program in the executable file games slash pong meanwhile the parent process invokes the wait system call passing in the process ID of the child process recall that in the parent fork returns the ID of the new child the other fork so the parent sits in a blocked state until the fork the child terminates that unblocks the parent and the exit code from the child is returned and assigned to the variable code here so now the parent process assuming it knows what exit codes from that program are meant to indicate it knows what happens it knows if the program terminated successfully or if something went wrong what is confusingly called a processes environment it's really just a chunk of data it's some number of bytes with stuff in it there is an expectation however that the environment is in the form of ascii text where each line starts with a variable name and then an equal sign followed by a value and the value is really just any sequence of text characters it's every text character immediately after the equal sign but before the new line character so a typical environment might look something like this it has here seven different environment variables as they are so-called and notice that the variables by convention are in all capital letters the first here is called term and it has the value x term the second is shell and has the value slash bin slash bash and so forth the idea of the environment is that it's some kind of configuration data in the process itself which is passed down from one process to the next in fact the environment is always stored directly in the process itself somewhere on the heap and the address of its location is always stored as a global variable in the data section if you wish to read or edit the environment in the process the C standard library has a few functions for doing so and you should generally use these functions rather than try and edit the environment directly because if you edit it incorrectly you can easily screw up the format now again the idea of the environment is that it is handed down from one process to the next so when one process is forked from another the child receives a copy of the process in the state it was in upon the call to fork this happens naturally because the environment lives in the address space and when we fork then everything in the address space gets copied over however we also want the environment and a process to be preserved across a call to exec so one thing the exec system call actually has to do is copy the environment out of the address space to some temporary location before it wipes the address space and then copy the environment back in to the new address space without the special treatment the environment would get wiped every time we call exec processes not only have an associated process ID they also have an associated user ID that is a number which indicates the user which owns this process the idea behind user accounts is that a user account has an associated set of privileges and so when a process is owned by a user it has the privileges of that user and only of that user what that effectively means is that when you execute certain system calls those system calls may fail because the user which owns the process does not have the privilege to perform that action for example if the user which owns the process does not have the privilege to write to a certain file then any attempt to write to that file will fail in most unix systems the accounts on the system are all listed in the file called slash etsy slash password etsy is a standard directory which is sort of a grab bag of things well hence the name but it's mainly for configuration files the user account given the ID number zero is a special account called the root user or the super user and this user account is special because it's allowed to do anything it wants so when a process is running with the privileges of the super user account system calls will never fail for privilege reasons the super user is always privileged to do anything it wants now somewhat confusingly associated with each process is not just one user ID that is the user ID of the owner of the process but actually three different user IDs the so-called real ID is the ID of the owner of the process whereas the effective ID is the ID which actually determines what privileges this process has and then finally the saved ID is set by calls to exec to match the effective ID so the saved ID effectively keeps a record of whatever the effective ID was at the time of the last exec call each file and directory in contrast is associated with just one user ID the owner of that directory or file so here's how the three user IDs of a process can get changed first of all the exec system call may change the effective and saved IDs and it does this when the binary file the executable file being loaded has a special flag set on it called the set UID bit when this flag is on the exact executable file the effective and saved IDs get set to the user ID of the owner of that executable file so for example if i exec the executable slash game slash pong and that file has the set UID bit set and let's say it's owned by user 30 then the effective ID and the saved ID in the process will both get set to 30 the most common use for the set UID bit mechanism is that we want a program with non-superuser privileges to be able to run a program with super user privileges so if that program file that executable file is owned by the super user and as the set UID bit on then anytime that program is executed it has the privileges of the super user the processes user IDs can also be set more directly with the set E UID and set UID system calls the E UID system call only sets the effective user ID the limitation with both of these system calls is that for obvious security reasons only the root user can invoke these system calls to set a processes IDs to anything it wants normal processes processes without super user privileges can only set the effective ID to match the real ID or the saved ID obviously you wouldn't want to allow any process to give itself any privileges it wants because then the whole concept of privileges would become meaningless so consider what happens when the unix system starts when the unix system starts is just the one process the init process it's owned by the super user and it runs with super user privileges the init process then spawns a login process basically a process that just prompts a human to enter a user account name and the associated password to log in and again the login process also runs with super user privileges once you successfully log in the login process spawns a shell process that is some sort of interface process either a command line or graphical user interface and that shell is owned by your user account the user who just logged in and it runs with the limited privileges of your user account so how exactly do we get these three processes well again init is the first process run by the system so it's just started automatically but from there init forks itself and then calls exec to run the login program which upon a successful login will fork itself and then call set you ID to set the user IDs to that of the user that just logged in and then call exec to run the shell once the user has a shell they can then at the command line or at the graphical user interface spawn other programs which will be forks of that shell process itself and so those processes will inherit the same owner however any program which is exec that has the set you ID bit set it's going to run with different privileges it's going to run with the privileges of whatever user owned that executable and as mentioned most commonly this is used to run programs with super user privileges though that's not always the case it's actually quite common for some programs when installed on the system to actually create their own user account and then when that program is run it is run with the privileges of its own user account one thing this allows the program is to have some files that are all to itself that no one else can touch except the super user so just be clear that user accounts while we commonly think of them as associated with actual people they don't necessarily really have to be associated with anyone or in fact anything a user account is really just an arbitrary number representing some set of privileges in addition to user accounts unique systems also have a concept of user groups which are defined in a file called slash etsy slash group a user group as the name implies is simply an association of user accounts and these groups are non-exclusive so a single user account can belong to multiple groups though actually one of the groups associated with the user account is considered its primary group in addition files and directories are not just owned by one user they're also owned by one group and each process not only has three user IDs it has three group IDs the real group ID the effective group ID and the saved group ID and these IDs are all set basically just like with the user IDs except there's a set GID bit on executable files and the system calls to directly set these group IDs are called set E GID and set GID groups mainly exist so that we can designate a set of user accounts rather than just one user account to have privileges with a file or directory in truth groups are actually almost sort of a legacy feature of unix systems they're traditionally part of unix because in the early days unix was primarily used as a multi user system where you had one computer with many different people all logging in through terminals to the same system and so it was thought unix needed some sort of inbuilt mechanism to group users together but for various reasons that model of computing is really kind of out of date you typically just don't have unix systems where a whole bunch of different people are logging in to the same system everyone has their own system now changing the paradigm so the unix concept of user groups is just not as relevant as it once was associated with each file and each directory are actually nine different permissions these nine permissions are grouped into three so-called classes there's a user class a group class and an other class and for each class there is a read permission a write permission and an execute permission r for read w for write and x for execute and each one of these permissions is either on or off set or unset and so for example when the read permission in the user class is on when it's set that means that the user which owns that file or directory is allowed to read that file or directory which class of permissions applies to a process is determined by the simple rule if the user id of the file or directory matches the effective user id in the process then the user class applies if that's not the case but the group id of the file or directory matches the effective group id of the process then the group class applies and if that's not the case then it defaults to the other class so you first need to understand which class applies in your particular scenario but the question then is what exactly do these read write and execute permissions mean well in the case of files it's fairly obvious when you can read a file that means you can actually read the bytes of the file that is its contents the write permission means you can actually modify the bytes of the file and you can also expand the file in size or shrink it in size and the execute permission means that when you call an exec you can specify that file as the file to exec without this permission the call to exec would fail in the case of directories these permissions are a bit less intuitive the read permission means that you can see all the names of the files and directories which that directory contains the write permission means you can modify this listing you can remove or rename any of the files or directories in that directory or you can add new files and directories into that directory the execute permission on directories is the most confusing because it really doesn't have much to do with execution when you don't have execute permission on a directory that effectively means you can't use that directory in any file path any system call with that directory in the file path will fail so consider for example this file path if we do not have execute permission on the directory taft then we can't use this file path in any system call I actually have to add a caveat to that if the directory without execute permission is the last component of the file path if that is if the file path points to that directory itself then it's okay it's only when you try to use the directory in a file path to get it stuff inside that directory does it fail so for the file path slash adam slash taft the execute permission of taft itself is not relevant if we had read or write permission on the taft directory even if we don't have execute permission we can still read and modify that directory we just can't actually touch anything which that directory contains last thing to note here about file and directory permissions is that by convention when you see a listing of the content of a directory you'll see the nine permissions listed in this format where it's the first the user class the group class on the other class and where the letter is present that means the permission is on but where there's a dash instead that means it's off so say in the top example here slash adam slash Lincoln that file has the right permission disabled in all three classes also by convention the permissions for a directory will be preceded by the letter d to indicate that this is a directory