 All righty, welcome back to Operating Systems. So we are climbing up the difficulty hump again. So last few lectures were chill, now we're back on the upswing. So after this, this will be the hardest content of the course and like not this lecture but in the next week or so, then we're pretty much smooth sailing since then. So strap in and get ready, so threads. So before we first talk about threads, I think in the first lecture I mentioned the difference between concurrency and parallelism and that we think of them as different things. So in computing, they are not the same. So in concurrency, the goal of concurrency is making progress on multiple things, which means you can switch between things. So you can switch between two more things or in other words, you can get interrupted. So I can make progress on one thing, put it down, make progress on another thing. On the other hand, parallelism is about doing two things in the same time instant, two different things in the same time instant. So the goal for that is I want to do, run as fast as possible, do everything I can at the same time, go as quick and for things to run in parallel they have to be independent. So depending on the two tasks, two tasks can be maybe done concurrently, maybe done in parallelism. Let's see a silly real-life example. So here is a real-life situation to demonstrate the difference between them and the only assumption we have to make here is that we are not savages and we have like okay manners. Not terribly great, but like not terrible either. So you're sitting at a table for dinner, you have four options, you can either eat, you can drink, you can talk and you can gesture with your hands maybe you only need one hand. But there's this one weird caveat that you're so hungry that if you start eating you won't finish until you're done because you were in lectures all day or you had midterms or something like that, I don't know. So you should be able to pick two tasks here and see if they can be done in parallel or concurrently with each other. So for instance, can I, let's see, can I drink and talk in parallel if I am a normal polite functioning human adult? No. Can I do them concurrently? Yes, I can stop talking and then drink and then finish drinking, start talking and I can switch back and forth between them. So I can't do them in parallel but I can do them concurrently. What about gesturing and talking? Can I do them in parallel? Yes, can I do them concurrently? Yeah, okay. So what about, I don't know, eating and gesturing. Can I do them in parallel? Yes, can I do them concurrently? No, because I have that weird caveat. All right, what about, I don't know, that's pretty much all of them. All right, so does that make sense to everyone? Seems to make sense. Which can be done in parallel, which can be done concurrently. So here's the full list of them if you wanna know. So for eating, I can't do eating concurrently with anything else because of that condition but for talking and drinking too, I also can't do those concurrently because they're using, I guess, my mouth resource but I could do it in parallel with gesturing because that's a different resource I can use my hands and I think we got the rest of them. So any question of the difference between them? So in computing, the weird situation where you have this case where you can do things parallel and not concurrently generally doesn't exist in computing. Generally in computing, if you can do something in parallel you can do it concurrently as well. This is like only in weird situations. So, but it's important to know the difference between concurrency and parallelism. So any questions about that? Because it will be important. All right, good. So now into threads. So they're fun, they're like processes except everything is shared by default. So it has the exact same principles except they just share memory. So each thread has its own registers, program counter stack but all your threads are in the same address space as the process. So if you make a change in one thread it appears in the other thread. Unlike what happens when you have two independent processes if you make changes in one you don't see it in the other. In this case that's not true. The only thing that is independent to threads is their stack and anything you explicitly state using something called TLS or thread local storage. So that is just information that is unique to a thread. By default the only information that is unique to a thread is its stack, which will be important. Otherwise they wouldn't be able to really work. All right. So a single process can have multiple threads. So the threads live within a process and by default a process just executes code in its own address space while if you have multiple processes while if you have multiple threads well you can have multiple executions going on at once in the same address space and because of that well threads are going to be much much much faster to create and have less overhead in terms of memory and CPU time because whenever we create a new process we have to create new page tables. We have to create a new set of file descriptors. We have to do a whole bunch of stuff like that even just creating the new page tables as you have hopefully figured out in lab three is a lot of work. So if you are creating a new thread you don't have to do all that because everything is in the same address space. So they're much lighter weight. They by default share code, data, file descriptors, everything you just create a new set of virtual registers and a stack and boom you have a new thread. So assuming you even have one CPU threads are still nice because they will also express concurrency just like having multiple processes could also be executed concurrently switching really fast between them looks like they're running in parallel even though we're just switching really fast. So in the case of even a single threaded application it might be easier to program concurrently. So for instance if we were writing a web server and want to handle multiple connections at once well we could get a request essentially accept that connection and then create a thread to handle that connection and then that would be a new execution point we could switch between multiple threads that are handling different connections at once even on a single CPU we could switch really fast and then handle lots of web clients we can make the next Google or something like that handle thousands just each request gets its own thread and that's a real web server. So lighter weight than processes here's a handy dandy table to figure out what the difference between them are. So processes have independent code data in heap while with shred threads they are all shared because they are all in the same address space. Processes execute independently and a thread lives within a process so it does not have independent execution because well it can change some of the like if you use the heap or something like that those changes will be seen in multiple threads. It's not by default independent because they all have the same address space. The only thing that is independent in threads which is the same as in processes is each process has its own stack and registers and each thread has its own stack and registers which are independent towards each thread. Processes are expensive to create and context switch because remember we would have to set up the page tables and whenever we context switch between processes we have to do things like flush the TLB and flush the caches and switch the page tables and all that which is slow while with threads they are cheap to create all I have to do is create a new stack and a set of virtual registers and context switching is also quicker because I wouldn't have to flush the TLB between them so most of the things stay the same. The context switch just has to swap the registers and that's it. When a process exits it's completely removed from the operating system as soon as something actually waits on it while with threads whenever they get cleaned up their stack is removed from the process and that's all and the caveat here is that when a process dies all threads within it die as well so unlike if a process just ends if a child process ends it doesn't affect the parent process of all their independent threads not so much because they live within a process. So for this course we'll be using POSIX threads for Windows there's Win32 threads but we're gonna use Unix threads so these are available well these are also available on Windows but they're like the default in macOS and Linux so we will use that to use them you just include pthread.h.header file and whatever you compile and link there is a flag for pthreads don't worry in your labs all the flags are set up for you so you don't have to do this but if you do want to use pthreads in your own project this is what you have to do and all of them are documented in the man pages like the system call wrappers. So this is how we create a thread we call pthread create we give it a pointer to a thread structure and it will go ahead and initialize that for us then here we have a pointer to some attributes so we could populate that with all their functions and set some default attributes for a thread but if we give it null we get the defaults and we'll figure out what some of the defaults are a bit later today. The next argument looks ugly anyone want to give a shot of what the hell this is? Who can read C types? Yeah, a function pointer to what type of function? A function that doesn't take any arguments and doesn't return, takes an avoid pointer, yes and also what's it return? A void pointer, yeah so this funny thing declares a type that is a function that takes a void star or a void pointer which is just C's way if you haven't seen that before it's just C's way of saying this is a pointer to what I don't care. So whenever you see void star it's just a pointer to something and C doesn't care. So this declares a function that takes a single pointer argument and then returns a pointer argument and then the fourth argument is the value of that pointer to pass to the start routine whenever it starts executing. So for this it's unlike the wrapper functions that return negative one on failure and set error no. In this case all the pthread functions return zero if they are successful and otherwise they just directly return the error number and then the contents of that thread are undefined. So here is how you create a bit thread and again it's a bit different than processes, right because we would have a fork and then after the fork we would have an if to figure out if we're the parent and the child. If I were to do this using processes in the parent I would print in main and then in the child I would print in run while if I have threads I just create a new thread until I want to execute this run function and then at this point I won't know whether or not I'll continue executing main or the thread that will execute run is going to run. So I'm not sure what order they're going to go in just like when we had multiple processes. So in this case it would print off in run this one would return in main. What are some differences and are we missing anything compared to processes? So if I set this up using processes and just printed in the child and printed in the parent would I be a responsible adult or responsible parent? No, I didn't wait on it so I'm missing something here. So is there equivalent and yeah in threads there is a weight equivalent. Annoyingly it's called join. So there is a function called pthread join you give it a thread and then a void star star. So a pointer to a pointer for the return value and you can think of this as exactly the same as the weight call but instead of an int and then passing an int star for it to populate we create a void star and create a void star star for it to populate. So it's the same thing. Processes by default just return an int threads by default just return a void pointer other than that you can make the exact same substitution. So in this you say which thread to wait on and in this case there is no pthread equivalent of just wait on the first thread that executes you have to specifically say which thread you want to wait and wait for it to terminate and then return values the same as the weight status same thing return zero on success error number anyways and you are only allowed to call this on one time per thread multiple calls on the same thread leads to undefined behavior if you waited on the same process twice well the offering system is going to know that hey it's already terminated you waited on it twice but because everything's in the same address space the offering system knows a lot less information. Yep. Oh so pthread canceled is another thing we probably won't even get into in this course. I just put it there for information. Yeah so you can yeah canceling a thread is the I kind of like terminating it with a signal it's kind of equivalent to that but we will probably never see that. All right so let's go ahead and see our example because it's a bit strange so here was our example. So we declare in main we created a thread created a new thread told it to execute run at this point we don't know what's going to execute and in terms of terminology the easiest way of thinking about threads is whenever a process starts and executing main instead of just saying the process starts executing main usually what we say is the main thread starts so you can think of this you put everything in context of threads so by default whenever a process starts it starts with a main thread with only one thread that starts executing main so we would have a main thread that starts executing the main thread would create this thread variable then create a new thread so I would have the main thread and then a new thread that wants to execute run I don't know which one is going to execute next because it's up to the kernel to decide and in this case I might see in main and I might see in run so if I go ahead and execute that I should see something interesting. I see in main, oops, why is that? Yeah so it has to do with I'm missing the join so what's happening here is after I create the thread the main thread prints in main and then return zero which is equivalent of doing exit zero and that ends the process so because the process ends and all threads live within a process process is gone the thread that wanted to execute run doesn't exist anymore because the process is now terminated. Yeah, so in this case it is possible if I just like hammer it and get super lucky that's cheating. Yeah but yeah if I did a sleep 100 at the end it would also work, you're gonna sleep one. So if I did this it would also work but that's also not good because you cannot depend on that so this is like if you did that and you're like it works for me that is how you have bugs that last for seven years because you give that to someone that executes this on a toaster and then suddenly that sleep one isn't good enough and then they only see one thing and then they call you they'd be like you have to fix my code and then you have to fix your code and then you're like oh god what the hell and yeah debugging this stuff is awful so whenever you write this you wanna make sure you are guaranteed the outcome you want so in this case I could sleep but that's not guaranteed my guarantee is if I call pthread join on thread and I don't care about the argument so in this case I will always see both so if the main thread runs first it prints in main and then joins the other thread which means it will wait for the other thread to terminate at some point the other thread will execute print in run and then finish and then the main thread would go on call exit from the process so now in this case I should always see both messages and I'm not getting unlucky this time so we're also not guaranteed the order of them so at least in the last lecture I saw in run once first and then in one but we're not that lucky then question which process do all threads live in so threads just live within a process so by default when a process starts it has one thread and then if you create other threads in that process they live within that process so as you can see this will get fun if I have a process with multiple threads and then I fork and then that process makes multiple threads and da da da da da that's when this course gets real fun so once we combine everything together yeah is it blocking? yeah p thread join blocks yeah just like wait blocks p thread join will block but it will block the main thread all right any other questions about this lots of fun it's most of them most thread things aside from threads living in a process most of the things have like a one to one equivalent so let's get back so yeah here is our previous example that waited properly we just put a p thread join on it now also because we joined all those thread resources were cleaned up so here's a question is there an equivalent equivalent to orphan processes and zombie processes for threads so are there orphan threads or are there zombie threads hmm yeah okay yeah so there is an equivalent of zombie threads because well in this case if we didn't join technically we could be in the situation where this thread runs prints and then returns which is the same thing as terminating a thread and then because we don't wait on it its resources would still be it would still be using some type of resource maybe it's stacked maybe at least its return value or something that would be hanging around and we can only get rid of its resources whenever we call join and clean it up so yeah that would be the equivalent of a zombie thread but there is no orphan threads because to have an orphan you need a parent-child relationship right whenever we had an orphan that means that the parent process died before the terminated before the child in this case if a thread terminates before the other it doesn't matter because all the threads just live within a process there's no parent-child relationship there's nothing like that yeah alright so any questions about that so there's zombie equivalent of a zombie thread but not an orphan thread because well they all live within a process yep yeah so the question is well if there's no parent-child relationship could I join the other one and the answer to that is yeah any thread can join any other thread so if I wanted to I could have this thread join the main thread if I wanted to there's yeah there's no relationship any thread can join any other thread you're just constrained by that you can only call it on each thread once so that makes your life fun so here is the next equivalent so whatever we had to process and we wanted to terminate a process well we could call exit or return from main for pthreads there is an aptly name pthread exit so it's the same idea but instead of an int again it is a pointer and that will be the return value of your thread so whenever your start routine returns that's the equivalent of calling pthread exit just like when your main returned that's the equivalent of calling exit so there is a one to one mapping between the two things and just like yeah and just like whenever you no I already said that okay yeah exact equivalent so instead of exit it's pthread exit for thread so any questions about that? alright so let's see another fun example so you might get into the situation where you don't really care about the return value of a thread in which case you don't really want to join it and you would rather its resources just get cleaned up immediately whenever it term terminates so the default is to have joinable threads which means someone has to call join on them in order to clean up their resources but detached threads will clean up their resources whenever they terminate the only drawback is you cannot get a return value out of them because you can't join them because whenever they're terminated they just clean up automatically so these this is actually nice if you again like I said if you don't care about the return value if it's a detached thread it cannot be a zombie because it gets cleaned up automatically and to make a thread detached you just call pthread detach on it and then suddenly after that it's a detached thread so if you call pthread detach twice in a row or on an already detached thread that's undefined behavior because the see uh and they don't like you all right so let's see that so detached threads aren't joined so let's go back to that same example so here is that same example so the only difference is I after I create the thread I detached it immediately and then otherwise I have my main thread that wants to print in main and then the new thread that wants to print in run so if I go ahead and execute this I see in main so why did I see in main again yes same issue my main thread finished first it reached this return zero which is the same as exit which gets rid of the process so gets rid of the threads so before my solution was to join right here right and I made sure it finished so what the hell am I going to do now I can't join it it's detached what option do I have so this is where it is good to think of the main thread as just another thread so there is a nice property about the pthread threading library in which it keeps track of all the threads and if you just exit a thread it won't exit the process until there are no more threads remaining so in fact in main I should have just called pthread exit here then in this situation if the main thread runs first it would print in main and then call pthread exit so now the main thread is just finished but there's another thread that can run the one that was going to call run and the library is smart enough to know that hey well I shouldn't terminate the process yet and then this thread will eventually run at some time then it will return from null which is again like a pthread exit and then this library is going to know that oh there's no other threads I can just terminate the process now so if I go ahead and compile that I will get both messages every single time and you'll also implement this in lab four so in lab four you're making your own threads threading library so that will be fun it's actually much it's not that bad it's fun don't worry about all right any questions about this all right we're good good good good good good yeah oh so in this case yeah I could call detach here but in this case that would be calling detach twice on the same thread yeah yeah yeah thread can detach itself if you want because if you want to get a reference to yourself there's this pthread self so that will return you yourself so yeah you could detach yourself if you really wanted to all right any other questions about that concerns looks fun yeah threads make your life uh fun all right so let's go into some other examples then so let's first illustrate that they are slower and hopefully my laptop doesn't act weird so here in this if I want to see that hey creating threads is way faster than creating processes well in this case I will fork 50 000 times so in my code all I do is I would create have a for loop that goes 50 000 times I call fork check for errors if I'm the child process I immediately exit so I'm just creating a the child process doesn't do anything and then in the parent I just wait on the child so if I run that I can see how long it takes to create 50 000 processes this should take like six ish seconds seven 6.6 seconds so pretty pretty slow so the thread version of it does the same thing creates 50 000 threads in here I do pthread create and I set attributes here but we can ignore them for now and we will probably never use them then here I tell it to execute a start routine give it no argument and the run function just returns zero or I should have just wrote return null same thing because null is zero and then after I create the thread I check for errors and then I just join it and then I just do that 50 000 times so how much faster is our threads so it turns out there's a lot of waiting and my computer's out there we go so here the total time is four seconds but you can see like the user time is much quicker it's only 0.9 seconds as opposed to like 3.8 so they also use like only 32 percent of my CPU instead of 90 percent for processes so they are overall much much much much much faster so any questions about that that's fun stuff okay all right so whoops didn't see that all right so so you can use attributes that are used there to get and set thread variables so how you do that is you just create this attributes structure then you can initialize it using their function and then you can ask what the default parameters are so for instance you can ask what the default stack size is you could there's also setters if you want to set the stack size to something different for for example in this case all it does is print whatever the stack size is and if I run this on linux I can see that by default my stack size is 8 megabytes so if you want to fix your stack overflow error hey just give yourself more stack space and you can buy yourself a bit more time if you really want but with this you can also use the attributes to set the state to joinable or detach so you don't have to call the detach function so in here there's like one of the fields is called detach state you can make it pthread create joinable so that will create make sure that the thread is joinable you could pthread create detached and so you don't have to do the separate detach step it's kind of up to you but but generally you write a lot of code with that and it looks kind of ugly that's why they just created pthread detach so you don't have to deal with the attributes all right so we can compare them too so previously we had an example where we created multiple forks so do do do do so when we created multiple forks let's see we could have a for loop that creates the new process that gives us an id so each time through this loop i is one two three four and then in the new process it forks so because it forks there are exact copies at the time of the fork so the first time through id is going to be one then id is going to be two then it is going to be three so if we are the parent we just immediately return from this we're done creating the new process otherwise we check for errors and then in the child process we'll simulate doing some work so this just has a for loop that executes 10 times and it says what the process is doing what step of the iteration it's on and then sleeps for a bit so if we go ahead and run that we should see you know all of our four processes are kind of running maybe in parallel maybe concurrently but they're each doing step zero then each doing step one then two slightly different orders but all of the uh all the one two three the last part is going to be sequential within the same process but between processes we don't really know what the order is going to be so here we go all the things happen and that's lots of fun so we know how to do that the fun one is if we do the exact same thing using threads so if we do the same thing using threads the rest of the function can look the exact same so we create four threads pass in the id but because threads are not independent we have to be able to pass a variable to a thread and the only way we can do that is through a pointer so to pass some information to a thread I can't just depend on them being independent and copies at the time of the fork because they're not they're all in the same address space so here I malloc an integer so I get an int pointer then I write the value of the id to it and then I create a thread so I create a new thread give it no attributes give it a run and give it an argument which is the pointer I got from malloc so any questions about that looks a bit uglier than the process one because they're all in the same address space but this is how you pass things to the threads so is that okay just passing an int pointer we got from malloc all right so in the run well because voidstar is just C's way of polymorphism I guess and like an I don't care pointer well we know it is an actually an int pointer so we can cast the argument to an int pointer and then dereference it and read the value from it so we read it into id which is just a local variable so it's on this thread stack so it's independent to this thread so we get id in this variable id and then after that we don't need the memory anymore so we should free it so this where memory management gets funner because the main thread did malloc but and then passed the data to the child or not the child so the thread it created and then the thread it created has to call free so that is memory management gets funner with this but the rest of it just has a for loop that does the same thing except instead of printing process number and then what iteration it's on prints thread number and then what iteration it's on so if I run this I get the same thing it's consistent between a thread they seem to switch off a bit more but I don't know anything about the order between threads at all but it's the same equivalent so any questions about the difference between those two nice they seem to always work on this thing all right no questions we're all good cool all right so I think that's no questions all because we literally have nothing else all right yeah multiple fork yeah yeah it is so it's in that materials directory lectures 04 yeah the multiple fork one is before so I think yeah we might have executed it quickly and didn't really look at it but it's actually in the 04 yeah so yeah don't worry you have all the code from the lectures if you don't yell at me because you should all right so we're good it's Friday we can go home sweet all right I will summarize and so threads enable concurrency we explored threads related them to something we already know processes a lot of the things map one to one the only thing that doesn't map is that threads live within a process but other than that threads can be joinable which is the same thing as having to call weight on a process there's an equivalent exit equivalent to creating them but instead of fork it's p thread create the main thing to know threads are really lighter weight they're cheap to create and they share memory by default so if you want to instead of doing inter-process communication which can be slow you can just create a new thread and they all share all the information by default but it's a lot easier to make mistakes and that will be like the bane of this course don't worry we won't get into mistakes until next next lecture so then also one thing to know is that each process can have multiple threads whenever a process starts it has a single thread we like to call it the main thread just that's how we like to communicate so with that just remember pulling for you we're on this