 All right, welcome back to Operating Systems. I should turn this on. Here we go. All righty, welcome back. So today we're talking about threads, everyone's favorite subject they can't see. All right, great. So. All right, it just says threads on the top so I'll just yammer a bit. There we go. All right, so threads. This is getting back up to like the top of the hump of this course. So this is going to be probably our most difficult subject once we talk about how threads interact with processes. We'll find it to be very fascinating, very fun. There we go. So let's talk about what threads are. So first, at the beginning, I alluded to, well, concurrency and parallelism are different things in computer science and I didn't elaborate on that. Today I elaborate on that because it is important once we start talking about threads. So for us, concurrency and parallelism do not mean the same thing. Concurrency is about switching between tasks. So doing two or more things and making progress on two or more things, being able to switch back and forth between them, switch to it, make a little progress, switch back, make another bit of progress on the original thing, then swap back and forth. And the goal of this is to make progress on multiple things and we saw once we had processes, if we essentially switch really, really fast, it kind of looks like parallelism, looks like we're doing things at the same time. But we're not actually. Paralism actually means we are doing two things in the same time instant. So at the same point of time, two or more things are going on and the goal of that is to run as fast as possible if I have a multi-core machine. I want all those cores to be working at the same time. I don't want to just switch processes between all the cores. That just seems silly. So let us have a silly real-life example to illustrate the difference between concurrency and parallelism. So here's our real-life situation. You have to bear with me because you have to assume that you have manners and are not a savage. So you are sitting at a civilized table for dinner and you're allowed to do four things. You're allowed to eat, drink, talk and gesture with like one of your hands. And there's a silly caveat that you're so hungry that you won't stop until you finish. Why? Well, because I want to essentially not have concurrency. So if we pick two tasks, we can see if we can do those tasks together concurrently or do them in parallel. So for example, can I drink and talk in parallel? No. No, because I'm a civilized human being, right? I only have one mouth that's either being used for drinking or talking and not at the same time. Again, you could probably talk while you're drinking but you probably shouldn't. Again, this is like this whole civilized thing. Can we do those two tasks concurrently? So drinking and talking, can I do them concurrently? Yes, I can switch between them. So I can stop drinking, start talking and then whenever I feel like switching, I can stop talking, start drinking. So I can do those tasks concurrently but I can not do them in parallel. Can I talk and gesture concurrently? Yeah, at any point I could stop gesturing like right now and start talking. Could I do them in parallel? In fact, I'm doing it right now. I'm waving my hands and I'm talking so I could also do those in parallel. What about eating and drinking? Can I do those in parallel? No, can I do those concurrently? No, what about eating and gesturing? Can I do those in parallel? Yes, can I do those concurrently? No, because I have this weird caveat that I'm so hungry I won't stop. So I can't switch once I start eating. Why? I don't know why. That's just a caveat just to explain. All right, so do we, any questions about the difference between concurrency and parallelism? Hopefully that drives the point home. So here are all of the situations just in case. So between eating and then talking or drinking since they're essentially the same thing. Well, I can't do those in parallel and I can't do them concurrently. And this eating gesture, I can do them at the same time but I can't switch. So this is like a weird situation once you're dealing with computers because generally if you can do things concurrently or if you can do things in parallel you can do them concurrently. So that weird caveat about eating generally isn't true with computers but I threw it in here just to illustrate that they don't technically imply each other. And then for the rest of them any questions about the other ones? So it's essentially to do with my resource, right? I can't share my mouth, I only have one mouth. I can't do anything involving my mouth in parallel. All right, cool. So now about threads. So threads are fun. They're exactly like processes except by default they share memory so that assumption we had of one process not affecting another process. Well, that's not true in threads. Threads can affect each other and that's where the fun of the course begins if it wasn't fun already. So same principles otherwise as processes but by default they share memory. So what is independent for each thread? Mercifully is it has its own set of virtual registers which means it has its own program counter so it's at a different execution point in the program and mercifully it has its own stack. So that memory associated with that stack is independent towards a thread. Otherwise they share memory aside from registers and stack. If you wanted to have other memory that is only independent per thread you need to do something called TLS or thread local storage so that will allocate memory that is only visible to a thread. We won't deal with that in this course but some things will be allocated like that and mercifully like things that look like globals like error note. Well, it's stored in thread local memory so that's like a global that is independent per thread. That's really the only trick we need to know about thread local storage. So the difference between processes and threads are kind of the relationship. So a thread lives within a process and a process can have multiple threads. So by default whatever you create a new process it is running a single thread. We generally call it the main thread and we'll go over that and that main thread is running in the process's address space and if it's the only one running while it's independent it has its own virtual addresses and all that stuff. If we have multiple threads in the process well we have possible multiple executions in the same address space. So if I change a global in one thread it affects every other thread. Why I would want this? Well, threads are lighter weight so you can think about what it takes to create a new process. Whenever I want to create a new process I need to create a new process control block so I need to set, you know I need to copy a new set of virtual registers and now we also know from lab three hopefully you know from lab three we would have to set up new page tables for the new process, allocate some memory for it do all that fun stuff. So if you have threads you don't have to deal with page tables because you're using the same virtual memory so you don't have to do any of that and you're sharing file descriptors and all of that. So why would this be useful even with one CPU? Concurrency is actually a very useful thing so having multiple executions going on that you can switch between sometimes makes your life a lot easier when you're programming. So we saw that example when we had a web server it could only deal with one connection at a time because we only had our main process running that can only execute one thing at a time. Well even with one thread we could change our server so it looks something like this so like while true we could get a request that could accept the connection and then it could process that request in a new thread so then now I have two threads going on one can just sit here and wait for connections another thread can actually process connections and then I can just create a bunch of threads that process connections and let my operating system go ahead and switch between them really fast so it looks like I'm handling like thousands of connections at once even if I only have a single CPU. So they're lighter weight, processes are independent code data heap threads share all the code, the instructions, all the data, all the heap that's shared because they're in the same address space they have the same virtual memory. Processes completely independent even if you rewrite mess with file descriptors that is only unique to that process while since threads live within a process any change a thread makes makes a change the process and then it would affect every single thread. Thankfully processes and threads have their own stack and registers. Processes are a bit more expensive to create and context switch between because again we could argue about while it has to create a new virtual address space create all the page tables all of that and whenever I have to context switch remember we have to flush the TLB flush caches and do all that while threads all I have to do is create a new set of virtual registers a stack if I need to context switch between threads they're in the same address space so I don't need to clear the TLB or do anything like that then whenever you clean up a process well it's completely removed from the operating system when it exits whenever you clean up a thread all that gets deleted is its virtual registers and its stack and also there's this fun caveat that since threads live within a process if the process dies then all the threads are now gone because their home is gone I guess if you want to think of it that way so yeah so basically processes and threads are both stuff so technically threads are you can think of threads as just representing like the virtual registers and the stack so a process by default really we didn't talk about before but really it has a single thread when it starts right yeah so process just has a single thread when it starts so now we're just saying what that is and we can create multiple threads in a process instead of just one yeah so in this course we'll be using POSIX threads for Windows there's Win32 threads but we'll use Unix threads so this actually works on Linux and there's actually P threads for Windows 2 and that's generally what people use because screw Windows I actually I didn't say that Windows is great right nice alright so if you want to use this you include the pthread.h header file and then you add this pthread flag to your compile and link steps for all the labs where we'll use pthreads which won't be until lab 5 this will be done for you so this is only if you want to add it to your own programs because after this course you will have epiphanies and you will just start adding threads to all your programs because you want to make them really good really fast so here's how we use them so before when we wanted to create a new process what we use do we all remember what do we use to create a new process fork so the equivalent of fork for pthreads is a bit different it's called pthread create and in this case we have to create this pthread underscore t structure and that holds the virtual registers and everything that's supposed to represent a thread that you're not allowed to see so it will initialize the structure for you then its second argument is there's some attributes we can set about a thread we'll go over some of them today but for now we can just ignore it and if you want the default attributes you just set it to null then there is this anyone want to tell me what the hell this type signature means yeah yeah so this is a pointer to a function that has a single argument which is a void star if you haven't seen void star before you forget what that is that's basically C's way of saying it's a pointer I don't care what it is so you can just say it takes a pointer as an argument and then it returns a pointer so it just wants a function that takes a pointer returns a pointer then the last thing it wants is an argument whatever to pass to that function to actually run so it just wants a pointer and it would initialize it and or sorry it would copy it and pass it to the start routine so unlike the system call wrappers we've been using that return negative one and that set error no the pthread functions generally just return zero if they're successful and if there's an error they return an error number directly so just check if they return zero otherwise the error number is in the end here alright so let's go ahead and just dive into an example because I like code so here's our example I'll clean up that warning what this does will start at main it just creates a thread pthread underscore t type it calls it thread so that's allocated on the stack then it calls pthread create gives it the address of that structure so it can go ahead and initialize it for us we set the attributes to null and then say we want to start execute wanted to execute this run function we have at the top and then we will give it a null argument so just give it zero at the point right here to the offering system you don't know what thread is going to execute so we have our main thread that's going to execute continue on an execution and there's going to be a newly created thread that wants to start executing this run function so at this point it's the same thing as if we forked where we have two processes where we're not sure which one is going to execute next for threads it's the same idea now we have two threads and we're not sure which one is going to execute next so in this case the main thread could go first and then do a printf in main and then return zero or the new thread could run first print in run and then return null and then whenever it returns from this run function that thread essentially exits and we'll see what the equivalent of that is let's go ahead and run this because even the small exit so any questions about the small example yeah why do we have to return a pointer in run so that's just the way to make it flexible so for processes we return an int right that's supposed to signify an error message whoops so in this they just decide the most flexible thing since it's in C is just to return a pointer and then you can use that pointer for whatever you want okay sure so returning a pointer returning nothing yeah so return a pointer that's flexible so it doesn't it doesn't hinder however you use this library you could just have a pointer to whatever structure you want and it'll just pass it along that's a good question so yeah we have a good question of where do I access the return value so when a process exits how do you access its return value yeah or calling weight on it right yeah which it would be if you're creating a new process right so if I want to actually get that return value if this was a process I would have to wait on it well that actually leads us directly into this because if I run it guess what happens I just see in main hmm so I have two threads one should execute in run but that never happens hmm why is that yeah yeah the the main thread executes before the new thread even runs and remember returning from main is the same as exit is exiting from main so exit means like exit this process so if I exit this process and threads live within this process that new thread is also now disappeared so it never actually executes new thread so that actually leads us perfectly until this so there is oh yep yeah so let's let's try it a few times in main in main in main nope we're unlucky we could get lucky enough that it actually prints in run maybe and then in main but we're not that lucky it is not impossible no yeah well yeah it likely wouldn't because now we know it uses a completely fair scheduler right so it would have like it would have its time allotment still the original one so it wouldn't context switch to the other one um so yeah all right so that got brought up while if I want to get the return value or if I want to be a responsible so in this case threads aren't don't really have parent child relationships because they're all living within a process they're all people I guess is that yeah I guess there's no good other way to put that but there is an equivalent of weight for threads annoyingly it is called join so if I want to get the return value of a thread and also wait for that thread to finish I can use p thread join so p thread join just wants the thread to join on as an argument so this is like having weight PID and specifying a process ID to wait on there's no equivalent of p thread join with the first joining or the first exiting thread that's not a thing you have to essentially wait or join on a specific thread and then this is a pointer to a pointer which is the location that it's going to write the return values to from that thread so you can actually access it so looks a bit scary but between processes and threads there's pretty much a one-to-one mapping so for processes they return it in so if you want to read the int it you got out of it we used a int star and then wait wrote that value to that location for this it's the exact same thing but instead of an int it's a void star or a pointer so this is where it would write that pointer to and there is the same caveat although it's a bit more serious this time that you can only join a thread once so you can only join or wait on a process once and then it is completely deleted so if you try and wait on it again you will get an error with threads well that's not the case so you could try to join on it twice and the kernels or the operating system is not going to know so if you call join on the same thread twice that's undefined behavior and C can go ahead and reformat your computer do whatever so if I wanted to fix that previous example to always have it print in run and in main well I should probably after before exiting the program my main thread should go ahead and call pthread join so I will join that thread and I don't care about its return value so now if I compile and run this I should every single time see both messages so I see in run in main so if the main thread runs first it prints in main and then it joins the thread that it created and then this will just block until the new thread prints in main and then returns which means that thread is now done now because that thread is done join would return in the main thread and then it could go ahead and exit the process so now no matter how many times I run this it will always print both messages and I can't get it to flip so we are unlucky in section one I got it to flip one time so actually printed in run first yep yeah if I use if I don't use pthread join many thread can exit the process and then everything's gone yeah so we can call it the first the original thing that starts executing main we usually just call it the main thread so in this case originally we have the main thread executing this and then if the main thread continues on first it would join the other one so the main thread wouldn't call exit until the other one yep yep yeah so every thread needs its own pthread t if I wanted to I could make another one thread two do something like that and then the main thread if I want to see all the messages I should probably join it as well if I did that I have two in runs so now I have three threads yep so why is this just passed by value yeah so because it's writing to this and in the other one it's just reading from it so it doesn't yeah pthread t is like some well pthread t is actually just an identifier so it's just a number so it's it'll be associated with the struct and you'll essentially do the same thing in lab for when you create your own threads you'll just associate with them with a number so each thread will just get a number yep yeah if one thread exits the process process is now gone like you just want that oh geez yeah so if you do this it'll work right so it'll work because the main thread sleeps long enough but this isn't guaranteed so this only works because a second is a really long time and this is why things like this are hard to debug this is not guaranteed to always print all three right so if I ported this to a toaster or something like that and toaster was really slow or if I tried to run it on a literal potato well it might take longer than a second and to like context switch or do anything or do literally anything so you might port your application to a potato and then the potato users are like hey I don't see I only see one message instead of three please help and you have to fix it and debug it and it's a complete pain right so this is why and that's like why problems like this persist for like seven years so some bugs that look exactly like this have been around for a long time because they're literally impossible to debug so you should make sure that given how you write it it does exactly what you want whenever you write it no so joining just blocks it essentially does the same thing as weight does for processes yeah no they're not like forks so the main thread starts executing here and the main thread just goes line by line so it create a thread create thread two and then here after we create a new thread the main thread just keeps on going to the next line and unlike unlike unlike fork the new thread just is going to start here so when the thread the newly created thread reaches line seven and it returns that thread is now terminated just like your just like your processes no if one of the threads that terminates the process everything's dead but if the thread just terminates itself it's fine yeah sorry oh can I like in run can I wait on an on the main thread oh could I create a new thread and run yep yeah well okay this would be a bad idea but I could create a new thread so yeah what does this do yeah it's essentially like that fork bomb thing so every thread creates a new thread oh this isn't as bad because every thread just creates a new thread so only have like an extra thread at the time because the other one will terminate but it will be essentially an infinite loop of creating threads yeah yeah yeah if I do something like run two then I could do something like that and then I'd also probably need to p thread join thread two no so I could do something like that in which case whoops I seg fault it whoops figure that out later all right leave that there and figure that out all right because we gotta move on that's a fun seg fault all right so there is an equivalent of exit to exit a thread or terminate a thread so luckily it is called p thread exit so p thread exit is essentially the same as exit except it only terminates a thread so remember exit just takes an int which is like the return value of a process well p thread exit just takes a void star which is the return value of the thread so whenever you return from start routine whatever routine you give it let's go back so whenever you return from start routine which in this case I just called run returning from it is the equivalent of calling p thread exit just like returning from main is the equivalent of calling exit so they are completely equivalent so there's an implied p thread exit here whenever you make it to the end of the function you are executing from so that while I'm actually called p thread exit terminate the thread and again if this was a process and I return from main it's the same as calling exit which exits the process or terminates the process so with that we can talk about detach threads so why would this be a thing so let's think for a second so for threads is there equivalent of a zombie thread like there is a zombie process not for now not quite so let's uh so yeah in like this case if I create a thread in my main thread and I don't join it at all and it terminates well what would happen if I created a new process and then I didn't wait for it and it finished the child becomes a zombie so let's replace process with thread so is it possible that main thread creates a thread and then that thread terminates so it is possible that the main thread could create a thread and then that thread could terminate and I don't join it which is the same as I don't wait on it so it would be like a zombie thread so it would consume some resources probably have a stack allocated to it the virtual registers or at least the return value it can't clean up yet again you'll be doing this in lab four so there is a concept of a zombie thread is there a concept of an orphan thread yeah so a zombie means you have terminated but no one has acknowledged you yet and a orphan means you do not have your the parent process that originally created you is now dead yeah so yeah so there's no concept of an orphan because you live within the process so there's no parent child relationship doesn't doesn't matter what thread creates which you're allowed to wait on or you're allowed to join on any old thread so if I wanted to I could join the main thread from this run function if I really wanted to there's nothing stopping me from that so there is a concept of a zombie thread no concept of an orphan thread so in that case there is this fun thing called detached threads so joinable threads which are the default type of threads if you don't give it any attribute will make sure that there is such a thing as a zombie thread so it won't clean up the resources until you call join on it and then you can read its return value all that fun stuff and then it will release all of its resources so there is another option called detached threads and that exists because well what if I don't care about the return value of a thread if I don't care about the return value of a thread why would I bother having zombie threads around when if I don't care just have it clean itself up whenever it terminates and just yeah just have it clean itself up automatically whenever it terminates so it doesn't create any zombie threads and I'm not going to join it anyway so why bother so that's what a detached thread is so if you detach a thread it will release all of its resources as soon as it's terminates you will not create any zombie threads and for this you can only call pthread detach once on a thread if you call it twice uh see you can go ahead and blow up again because c is fun so in this case so let's go to it so you might notice it is called detached error because whenever I run this I see in main so why do I see only in main again because I haven't joined it okay how do I join it if it's detached I I don't wait so how do I fix this then any guesses sorry block so there is actually nothing preventing so the main thread is just a thread like any other thread so after I print in main well I should probably just exit the thread instead of exiting the process so I can do this so if I do this I'll see both of the messages again so why is this well the thread library is smart so even if you have detached threads it will not terminate the process if you terminate the thread until there are no threads remaining if there are no threads remaining well then I can terminate the process because threads are the only thing that execute code so then I don't need to do anything so in this case if I put a pthread exit here in main instead of just returning from main well I forced the main thread to exit so even if it executes first well creates a new thread detaches it prints in main and then it exits the main thread which the threading library is smart enough to know that there is still another thread to execute so I'm not going to terminate this process then it would wait for this thread to run it would print in rate in main or sorry in run then return which is the same as calling pthread exit and now if this thread also exits now I have no more threads left and it can just exit the process any questions about that sorry if I don't detach it then I could join it and not have to do this if I wanted to oh if I didn't detach oh if I so in this case if I don't detach it it doesn't do anything so detach just has it clean up its resources whenever it's terminated so if main runs first well the main thread is still going to exit and then the other thread is going to come here print in run and then exit and well the process is going to end anyways it's not even going to clean itself up probably the only difference would be that if after I create a thread the other thread runs first if it would print in run and then exit the thread and then it would clean up all of its resources and it would not be a zombie thread for any amount of time and then the main thread could go ahead and print in main that's the only difference yeah well pthread exit just terminates this thread that's it no this other thread this return and all from here is the same as calling pthread exit so it just terminates a thread no pthread exit just completely terminates a thread like exit terminates a process like exit is not a blocking system call it just means you don't exist anymore you can't execute anymore so same thing yep so why does returning from main call exit instead of pthread exit because the c developers have deemed it so because it was always like that so threads were introduced later and you can't change the behavior as soon as you have it c was invented in like the 70s or the 80s or 60s something like that so can't change it once you change it so that's why remember that system call that was called exit group well it was called exit group because they had to introduce that later because that meant terminate all the threads as well because the default exit system call only exits a thread so if you go back far enough exit termates a thread yeah sorry of using so detach detached is nice because it won't create any zombie thread so i don't have to join them in order to release their resources that's the only difference yeah so if they're not detached you always have to join them when they're done which depending on your program might be annoying or might be close to impossible if you create like thousands of threads keeping track of all of them and if you don't care about the return value anyways why bother right yeah so basically if we do not exit we may not get like with required code but it will still like no so detaching doesn't matter to whether or not a thread keeps on executing or not so detaching just is if it releases its resources whenever it terminates because i at any point i could call exit right if i call like if i get rid of this p thread exit i still the issue of the main thread terminates the process and everything's gone uh-huh yeah yeah you are not allowed to join a detached thread so your thread is either joinable or detached i believe you can make it joinable but that's like weird never been asked that before look at the documentation yep the p thread exit is just terminate a thread the process has to wait for all the threads so so in this if i uncomment or if i uncomment this p thread exits here will terminate the main thread whatever started executing main and that's it if i don't if i comment it out it calls exit terminates the process okay so i will go through this quick so here's how you use attributes you'll probably never ever use them because you only care about the tat detach and there's a function call for that so here's an example how to get the stack size i'll just blast through this quickly so usually the stack size is eight megabytes you can set it and if you want to set the state to joinable this is how it would look but by default they're already joinable so you don't really need this this is mostly for illustrative purposes and i don't need to go over it and you'll never use it so we can see multiple processes and multiple threads if you want i'll just throw show the thread example so in this example i'm going to just play with the variables just so you have it there so it sets it as joinable but by default all threads are joinable then it goes through this for loop of 50 000 so it creates 50 000 threads and then they'll create a new thread each time say it wants to execute run and then give it a null argument and then it would join it so it's just going to create threads over and over again in this case it does nothing so this is just to illustrate how long it takes so if i create 50 000 threads it takes like four seconds and not that much user time not that much cpu time or system time i also have an example where i create 50 000 new processes and wait on them so i can see how long that takes just to show that creating processes is a lot more intensive and i'm still waiting so yeah creating processes takes seven seconds takes four seconds of user time not like nothing takes a lot more system call time and waste a lot more resources so threads really really light um yeah another one so here is a multiple thread example if i wanted to create let's say i wanted to create four threads that each do a little bit of work and print off whatever their id is and what they're doing so in here i'll just write a little helper function that calls new thread in new thread if i want to pass an argument to a thread well i have to use pointers and declare some like allocate some heap space and then pass it a pointer to whatever it's pointing to so i can actually pass arguments so in this case i get space for an integer i write a value a dereference set write a value to it create a thread and then i give it this pointer as an argument so in the thread itself well it has to cast that void star to an int star because that's what it actually was and then it can read from that to get the id of it and because it used malloc well the main thread malloc and now it goes to the new thread so the new thread has to free that memory so i have to free free cross boundaries that's fun then in here i'm just simulating each thread doing a bit of work so it prints off its id and then what iteration of its loop it is on and the loop all these variables are on its stack so each thread would have its own independent i so within a thread it should be fairly consistent so if i run this i can see within a thread it will always print thread like for example thread one it would be one two or zero one two three four dot the dot but i don't know the order between threads or even yeah don't know the order between threads or sometimes the number isn't even the same between them so i do not have any guarantee whatsoever but this yeah this is how you would essentially split up work between four threads so any questions about this is this is kind of annoying because if i did the same thing using processes well every time through the loop here whenever i create if this was a process instead i would fork and then they would be copies at the time of the fork so i don't have to like allocate space or anything whatever id was whenever it got forked is what id is in the child so if it's got forked when it was one in the child it's one then i change it to two forked again it would be two same thing change it to three fork again it would be three so it's a little bit more of a pain to actually give independent information or to pass independent data to threads but if you want to share things it's really really fast all right got speedrun so that is compared oh yeah there we go so threads enable concurrency so even if you have a single cpu they represent concurrency they're lighter weight than processes shared memory by default each process can have multiple threads but initially a process only has a single thread called the main thread but processes just live within threads there's equivalents for anything so processes create wait exit well it's p-thread create p-thread join and p-thread exit so just remember pulling for you we're on this together