 Alrighty, welcome back to Operating System. So this is the second hump in the course. So we are beginning our journey on threads. So hopefully with all our practice with processes, starting threads won't be too bad, but it'll get hard again, and then we'll be over the hump, and then the rest of the course will be not as bad as this. So let's dive into threads, but before that, we have to talk about concurrency and parallelism and what the difference is between them. So I kind of alluded to it in like the first lecture that they aren't the same thing to us because reasons and the reasons will become apparent soon. That's annoying. All right, so they are not the same things. So to illustrate that, so concurrency when we talk about software is talking about switching between two or more things, which means you can get interrupted. You can stop progress on one thing and switch to something else. And the goal for concurrency is to make progress at multiple things and you can switch between them. Paralism, on the other hand, is running two or more things at the exact same time. So at the same time instant, two things are running, and in this case, they would be independent. That goal is to run as fast as possible, do as many things as you can at once. So here's some real life situations so we can figure out what we can do in parallel and what we can do concurrently. So assume you are sitting at a dinner table and you are not a savage and you are a polite human being. So at this dinner table, you can eat, drink, talk, and gesture with your hands. Say, just one hand is fine. But we have a weird caveat here where you're so hungry that if you start eating, you won't stop until you finish. So given that, what tasks can be done concurrently and what tasks can be done in parallel? So pick any two and say, whether or not we can do in parallel or concurrently. Yeah, yeah, so talking and gesturing, I can do in parallel, I could also do it concurrently because I could do them both at the same time and I could just switch between them at any given time. Between gesturing and eating, well, I have to eat with my mouth and if I start eating, I won't stop. So I can't do them concurrently because I can't switch from eating but I could do them in parallel because I could just use my one hand while I'm shoveling food with the other. Maybe we're somewhat of a savage and then let's see, between drink and talk, can we do those in parallel? Drink and talk, no, right? Yeah, so I can't drink and talk in parallel assuming we are not savages. Some of you might be able to like, but that's like doesn't count. So I can't drink and talk in parallel but I could do it concurrently. I could stop drinking and talk and then I could resume talking, resume drinking, switch back and forth between them. So I could do those not in parallel but I could do them concurrently and I think that's pretty much anything. So any questions about the difference between parallelism and concurrency or does that make sense? Yeah, pretty much anything you can do, well in this case, sorry, what'd you say? Anything you can do in, no. So in this case I can't like, because of that weird condition where I'm so hungry I can't stop, well I could do eat and gesture in parallel so I could shovel and gesture with my hand but I couldn't switch from eating because apparently I'm too hungry, I couldn't do it concurrently because I can't stop eating. Yeah, that's literally just because of that condition and in general for computing, if you can do something in parallel you can do it concurrently. Yeah, yeah, in general it's the case, this is just like a very contrived example to get that case just to say that it's not always implied but in software it's pretty much always implied. All right, good, all right. So here's all the things if you need to review them or if you're watching the slides or whatever. So like drinking and talking, I can't do them in parallel and I can't do them concurrently because of that condition for eating. Basically I can't do anything concurrently with eating because of that condition and my limiting resources I have like my hands and then my mouth. So I can't talk and drink at the same time or eat at the same time they all conflict with my mouth resource I guess so I can't do anything in parallel with that but I could do something with different resources like my hand and my mouth. So that's good, any questions about that? So this will become more important as we go on with this. So now we have to talk about threads. So luckily or unluckily they are pretty much exactly like processes except that they by default share all memory. So there's the same principle applies. They each thread has their own virtual registers. So that would include the program counter, its current stack and they also get their own unique stacks as part of setting up a thread but they're in the same address space. So if you modify a global variable it's modified in all threads which will be the bane of your existence going forward but we'll just go ahead and ignore that for now and that will be the tricky subject we get into later. So in here, if you want to have memory independent to a single thread you have to ask for it explicitly for something called TLS or thread local storage. So by default the stacks will be local to a thread so they'll be in thread local storage and later if you want any information any other information only local to a thread or independent between threads it has to be in thread local storage. So we won't really use that but by default your stack will be independent for each thread. The fun thing about this is processes can have multiple threads so a thread actually lives within a process. So by default a process just executes code in its own address space but once you have threads there are multiple, I guess virtual registers going around and multiple executions can take place in the same address space. So because of this, whenever we create a new thread we're just creating like a new virtual CPU you can think of it that way. We're not creating a whole new process we're not creating new virtual memory we're not creating new file descriptors we're not copying page tables we're not doing anything like that. So creating threads are way faster because we don't have to do any of the other information like it's only virtual registers not creating a new copy of all other information that are within a process. So because of that they by default share data but they're also way less expensive to create and less expensive in terms of offering system memory in terms of CPU time are the main things. So assuming you have one CPU threads can express something like concurrency so a process can appear like it's executing in multiple locations at once but it's not really the OS is just context switching between processes really, really fast might be able to execute concurrently. So what we could do is every time we have a new web request here so like if you were at the sockets example every time we have a new connection we can create a new thread that handles that connection and then the kernel can go ahead and schedule the threads to run in parallel or concurrently and it looks like lots of things are happening at once. So even with a single CPU all your threads running concurrently can look like they're running in parallel just like when we had multiple processes and we switch between them really, really fast it looked like all the processes were running at the same time. So they are lighter weight than processes so in processes right each process has independent code data heap because of virtual memory but in a thread everything is shared between all of the threads so they live within a process so they share code, they share data and they share the heap so there might be some conflicts if you're trying to modify memory in multiple threads. The process was kind of nice because it had independent execution so one process did not affect another process at all but because threads live within the executing process they affect one another so even if a thread modifies the file descriptors well suddenly in every other thread it's modified because when you modify file descriptors it's only tracked by the process and all the threads live within a process so any change you make affects all threads. So the only part that is the same is in a process it has its own stack and registers that's all independent and threads also have their own stacks and registers and that's the only part that is independent within threads. So processes are really expensive to create and to do the context switching for because they have to copy all virtual memory we have to set up page tables it would have to copy all the file descriptors set up new page tables, new virtual addresses all that stuff with threads it's really cheap to create them we just create new virtual registers and set aside a stack and context switching is also pretty quick because if we context switch between threads in the same process well guess what they're using the same page tables we don't have to flush the TLB we don't have to do any of that all the entries are still valid and it's just much quicker to do we don't have to yeah don't have to switch anything else when a process exits it gets completely removed from the offering system whenever it exits or terminates assuming someone calls weight on it and is a good parent but with a thread whenever a thread exits successfully all that happens is it's unique stack it's removed and that's it and the fun thing here is when a process dies all threads within it die as well yeah so if a context switches the context switching is still pretty quick and maybe you want to switch between tasks a lot to make it appear that they're running in parallel and sometimes it's just easier to program that way so if I want to like it'll be the only way that I could accept multiple connections or something like that or work on multiple files at the same time I could just have thread switch between them while it's doing something so yeah you will find threads very useful and we'll see some thread examples too so for this we'll be using POSIX threads so Windows there's a Win32 thread but we're gonna use Unix thread so this works if you're on macOS and if you're on Windows well you're not really developing anything terribly worthwhile okay that's probably a bit of an overstatement but most like server stuff and most heavy-duty software it's all running Unix anyways so we do P threads principles still applied just the APIs will be a different so to use P threads you just include the P thread header and whenever you compile you just add on the dash P thread flag to both the compiling and linking steps and it goes ahead and adds it for you for your, for anything you use the build files will already do that for you so all the P thread functions have documentation you can look it up in the man pages or on the internet or whatever so the API for that we'll get into some examples so to create a thread to create a process we use fork to create a thread we use something aptly named P thread create so it takes four arguments and the first one is the pointer to a thread this is a structure that you don't know the contents of but you just give it a pointer and you have this function initialize that structure for you so the attribute the second argument here is just some thread attributes for now you can just give it null and not set any attributes and accept the defaults we'll see some more details how to change this later but for the most part you can just accept the defaults and we will figure out what the defaults actually do the next argument is a start routine which looks like an ugly C type but if you read that all it wants is a function which takes a single argument takes a single pointer argument so in C a void star essentially just means a way to tell C it's a pointer to what I don't know it's just a pointer I don't care so this is a function called start routine that takes a single argument which is a pointer and returns a pointer so all those stars shouldn't look that scary it just needs a function to run whenever the thread starts and then the last argument here is a void star arg and that will be the value that is actually passed to that start routine function whenever it runs so this function returns zero if it's successful and unlike the system call wrappers that return negative one and then set error no this one just returns an error number directly so just check if this returns zero if it doesn't return zero it gives you the error number and if it gives you an error the contents of the thread are completely undefined yep start routine returns a pointer so avoid star yeah so you like read it inside out kind of so this is just a pointer to just a name start routine and because it's in brackets it means it's a function and this just is the arguments so just a single pointer and then returns a single pointer all right so that's how to create threads so let us get into an example and also see why they are heavier weight so go to code so here's no I don't want that one I want to create fork all right so here is our testing whenever we're creating new processes so we create a new process by calling fork so here I define a variable called or define something called end forks that is 50,000 and what I will do in this function is well I will loop 50,000 times fork and then check if there's an error if I am the child process I just immediately exit and otherwise in the parent I'll make sure to wait for the child to make sure I clean it up so I don't create a zombie process and then I just go through this loop again so every time through this loop I make a single child and I wait for it so this will create 50,000 processes correct all right so since this creates 50,000 processes I can see how long it takes to whoops to create 50,000 processes so 6.9 seconds nice so let's see what the thread version of that looks like so in the thread version here I create some attributes that we can ignore for now but it's the same variable so end threads is 50,000 so this loop executes 50,000 times it creates this pthread variable and then it calls pthread create gives it the address of that thread the attributes which we could have just set it to null and then the function to run which I just called run and then it gives it you can set the argument I'm not going to use it so I'm just going to set it as null so in here this will create a thread and that thread will go ahead start executing this function which just returns zero or just returns null same thing it just doesn't do anything and then there is an equivalent of wait which we'll get into later but you can just trust me that I did that so for that I'm creating 50,000 threads and if I time that and check it is four seconds which is a bit faster which should be a lot faster all right whatever it's still much faster it's still like almost two times as fast it should have been like a second but I guess my computer's struggling for some reason so yeah and here's another example of just how to create threads so it's a bit different than processes but hopefully it makes a bit more sense so this is like the minimal amount of code you need just create a main create a thread on your stack then you initialize it through pthread create give it the attribute so by default you can say null then give it which function to run and then what argument to pass it so in this case after I create the new thread it would start running run and then my original thread that was running main would just want to execute this line next so this is like the same situation where we made processes where we don't know which one is going to execute next so yeah so it's a bit different so then processes do do we like this better than fork or is specifying a function to run make more sense yeah yeah okay yeah it usually makes more sense to me and it's clear to read because I know exactly what the other thread is going to start executing with fork you just have to argue at the time of the fork a it splits off and then two of them can continue at the exact same point although before some other people preferred fork for some reason not quite sure about that so are there some differences or are we missing anything that we would have if we were doing this using a process instead the weight yes I was not a bad parent or sorry I am a bad parent so that's the main thing we were missing and there is a weight equivalent for threads and knowingly it is not called p thread weight it's called p thread join so how it works is it takes a thread to wait for if we were doing processes or a thread to join as an argument and then it takes this ugly thing which is a pointer to a pointer for the return val all this is going to do is write a value to that address you give it and that's the way to get a return value out of a process or read whatever it returned from its start routine so you can think of this as the same thing that's happening whenever you call weight and you give it the address of the W status this is the same thing you would create a pointer and then give this the address and then it would write the value of the pointer there so same thing happens threads luckily have pretty much exact equivalent with processes so for this it returns zero on success error number anyways and just like processes you can only call this one type like each thread you can only call this on each thread exactly once multiple calls on the same thread leads to complete undefined behavior because well with threads they're all in the same virtual memory space the kernel's not super managing them so it's hard to tell if it's already dead or not so like if you call weight on the same process multiple times well you can't because the kernel just cleans up that process and deletes it you just get an error directly with threads it's a bit more difficult you just get undefined behavior and you're left to your own devices for that so here is that same example fixed so in it I would just call pthread join and now the resources are cleaned up so if I go ahead and let's make that change yeah so you wait on a specific thread but we could do that with processes right with weight PID so you're forced to wait on a specific thread there's no like join on any thread here so it's like you only have weight PID with a specific process or thread yeah yeah for this example you wouldn't know what would happen first in main or run if I execute it well in fact because I didn't join here I don't even see in run so this time when I executed it well I created it this printed and before this thread actually ran return zero from main is the same as exit so it will have exited the process and everything's gone now so if I do this pthread join I have to give it the thread and then I don't care about the return value so here is the example where I join it properly and I should see both messages all the time because I'm actually waiting for the other thread to finish executing yeah so if you just did like this let's see if it will implicit me good enough yeah so you're not guaranteed like this is only happening because my computer is really fast most computers are really fast the second is an eternity so they'll both print by them but you're not guaranteed so for all this you want to be guaranteed something will happen then you know it will execute perfectly because if you just slept and your computer it worked for you and then you gave it to someone else and they're like it doesn't work for me because they're like executing it on some toaster or something like that guess whose problem it is to fix it yours and you will spend like two years of your life debugging that all right so yeah in this case I see every time I execute it oh there I got them in a different order woo so you're not guaranteed you always get them in the same order but you're guaranteed you always get both messages in this case so after pthread create I have two threads you can think of it like two processes you don't know which one is going to start executing next and it just so happened that this thread executed first in one case and then the thread that was executing main executed but normally you'll see in main in run but as you can see you can't depend on it and we'll see how to make this dependable and that's like the hard part of the course yeah so the question is so if there's if all threads die when a process dies is there no concept of orphan or zombie threads so would zombie threads still be a thing anyone want to think about that a little bit so could a thread finish executing and I essentially don't wait for it because that was the condition a zombie happened so whenever the function is done it means the thread is finished but we haven't waited on it yet or in this case joined it so if we did that for a process what's that but in this case that thread could finish and the process is still alive so if the overall process is still alive and this thread finishes then if we don't wait on that if that was a process would it be a zombie process it would be right so if you create a child the child terminates and you don't wait on it it's a zombie right so if it was a process so that would be a zombie process right so same thing to substitute the word process for thread and then wait for join so if I have the case where I create a new thread and it finishes so it finishes running its function that means it terminates and I don't join it that means I would have a zombie thread so for instance like it wouldn't waste that much space but each thread has its own stack and I have to join it so it can delete its stack and all that fun stuff and you will also figure this out because you will be implementing threads in Lab 4 so if it's not clear it will be so yeah you can create zombie threads but we cannot create orphan threads because as soon as and that's why because as soon as the process is done everything it's they all get deleted so they don't get reparented or anything because they just now don't exist so there is a concept of zombie threads there is no concept of zombie or sorry orphan threads no so just there's oh no there's no sub reaper for zombie threads you just have to you're the only one responsible for it so you have to join it so just like if your process keeps on running while you have children that are zombies the kernel is not going to do anything for you right you'll just hold on to the zombie as long as you exist and then they'll only get cleaned up once they get reparented but in this case they'll be zombies for as long as this process exists if this process no longer exists everything just gets deleted yeah yeah p-thread join will do all the cleanup for that one thread that is now gone or terminated Valgrind would not complain you might be able to typically there's nothing that will do that because it's hard to know your intention if you just wanted it to kind of die we'll see how to actually have automatic cleanup that is something we can do which is different than processes so yeah let's get into that because that that's a perfect segue but before that let's go with the other equivalent so in processes you can terminate a process by calling exit unsurprisingly you can terminate a thread by calling p-thread exit thankfully this one's named something we actually want so all it takes is a return value so by default main returns an int for threads by default they just return a pointer so this is like the exact same as exit with an int but because it is a thread it is exit with a pointer so that is the return value that's passed to something that calls p-thread join on it to order to capture its return value so also note here if your start routine finishes or if you return from your start routine that's the exact same thing as calling p-thread exit just like if you return from main that's the exact same thing as calling exit for a thread the equivalent is if I return from my start routine that will automatically do p-thread exit and it's implicitly called for you so now we can talk about detached threads so this is where we deviate from a process so by default threads are joinable which means you have to wait on them we have to be good well I guess we're not parents in this case but we have to wait on them we have to be responsible people I don't know a good word for it and then they can release the resources so if you don't care about the return value of threads and sometimes you will not well you can just save some time you can create a detached thread and that will make the thread automatically release its resources whenever it terminates the only drawback here is you cannot get its return value back but if you don't care about it anyways then just make a detached thread you don't have to wait for it and it just cleans up whenever it's finished you don't have you will not create zombie threads in this case so you can call pthread detach on a thread and it will create make the thread detached again it would return zero on success error number otherwise and if you called detached on an already detached thread so if you called detached twice that's undefined behavior again c can just go ahead and delete all your files on your machine so in this case a detached thread are not joined so if I run this code so let's see this example touch error so if I run this all I see is in main why is that the case yeah so in this case what would happen is whatever thread starts executing main generally you just call it the main thread so the main thread comes in creates a new thread structure creates a new thread that wants to run run with no arguments and then it detaches it so now we cannot join it and we don't know which one will execute next most of the time the main thread will go ahead print in main and then return zero which is like calling exit so that terminates the process and now all the threads are done the one that was about to run or to print in run just never executes so before all I could do was just to fix it I did a join here right but now it's detached so what can I do to make sure it always prints so what I can do is instead of calling or saying return zero which is the same as calling exit in the main thread what I should do is actually call p thread exit so in this case if I run it well the nice thing about p threads is by default it will not terminate your process using p thread exit unless it is the last thread running so if the last thread running calls p thread exit and there's no longer any threads running well nothing can execute so your process just gets ended nicely so if we compile this now and run it we see both messages because even if the main thread runs first it calls p thread exit instead of exit so the process still exists and there is a thread that wants to execute run so it would print f in run and then return null which is the same thing as p thread exit since it's the last thread when it calls p thread exit the library is smart enough to realize that oh nothing else can execute so I may as well end the process so in this case p thread exit is not kind of like weight wait what's coa weight so yeah p thread exit is more just like it's a bit different concept for threads as opposed to processes because if we terminate all of like the parent and sub children it doesn't matter because all the processes are completely independent so in this case this is something unique to threads yeah yeah p thread exits just exits like just terminates a thread and will not terminate the process unless it was the last thread standing in which case there's nothing else to run yeah same question okay so question is what is keeping track of what threads exist and the answer to that is it depends so threads will get into next lecture can either be like user space or kernel space so p threads in general the kernel keeps track of that so the kernel knows that you have the thread but it doesn't actually in this case it doesn't actually matter the library knows how many threads there are so the library just figures it out but yeah we'll get into the thread implementation a bit later yeah yeah yeah if I don't have p thread exit here and I just have this the concise way of saying it as well it creates a new thread I don't know which one is going to execute next it could be the situation where the main thread executes first and it prints in main and then it returns zero which just exits the process and then everything's gone yeah there there is a chance that I would see in run for the same reason like when I ran it I ran it like 10 times and then one time I saw in run first so if I had that same ordering I would see both messages I would just always see in run first yeah I either see in run then in main or I see in main by itself yeah so when I say main thread this bothers me so when I say main thread so by default whatever your process starts executing main we assume that it's a thread that's executing it because calling everything that's executing a thread is just way easier so whatever your process starts it by default starts with one thread called the main thread that starts executing main so that's what I'm talking about here so the main thread would start executing this code and then create a new thread so now I'd have a main thread and then the thread I created that wants to execute run yeah generally no one says main process they say main thread within a process yeah so there's no real child threads you just wait on a specific thread so you have to wait on a specific thread there's no concept of parent child because they're all in the same address space yeah yeah so in this case the main thread calls pthread join so the main thread will not continue until that thread I called thread is finite it terminates well kind of yeah pretty much because I'm waiting on another thread so in the detach example whenever I exited I actually killed like I actually terminated the main thread in this I just have the main thread wait for the other one yeah yeah yeah so if I have pthread exit this never happens like this is dead code because the main thread's now gone so no so in this case it's not possible that the main thread doesn't exit because remember returning this thread returning from here it's the same as it calling pthread exit so I could make it the case of like if you get super unlucky because there's nothing stopping this from just calling exit so if I do something silly like this this thread can just end the process and then everything's gone so if you have another thread that just calls exit yeah then that's your fault if you in here oh like just what like that yeah so there's a main thread so here it would just exit the main thread and now there's no other threads so it just exits the process so it would do the same thing just with more steps yeah yeah there's there's nothing stopping you from creating threads in the run there's nothing stopping you from forking in the run that yeah you can fork and create more threads or do it that's that's when things get fun so you might think that is a recipe for disaster it's actually not too bad so there are rules behind it yep so the attribute unused oh yeah so the question is what the hell is this thing so this is essentially what the hell they call a compiler attribute so this is basically me getting rid of a compiler warning so you know the warning like unused variable this is me acknowledging to the compiler I know it's unused don't warn me about it so this is just a compiler thing just so I don't see a warning message and that's just because I hate warning messages I don't want to see them so this is squelch the warning message because yeah I know it's unused it's fine yeah whatever that's a good habit to get into to max warning levels and make sure your code compiles without warnings it's a good tip I think I did that for like years um yeah so here is that example so whatever we throw p thread exit at the end if we detached the thread well we have to exit from the main thread do a p thread exit from the main thread instead of just returning zero because we need the process to still exist and this will always print the two lines and we don't know in what order they're going to be printed in so that was our fix all right so here's attributes for that third parameter also you can like use them to query information about about the threads so for instance if you wanted to get the stack size well you can use attributes to get the default stack size so here you just have an example that prints the stack size whenever you run this on unix or on linux I'll save running it due to time but the stack size is by default eight megabytes so if you go over eight megabytes right we talked about the guard page there'd be a guard page after that then you get stack overflow and all that fun stuff so in here you could also set the thread state to joinable if you want to be super explicit about it so you just take this attribute structure and you can use set detach straight and look it up and then you can create a joinable you can also have it so that you create threads that are detached so you could change that to p thread create detach and you don't have to do the detach one but typically that's why they give you the nice little helper function so let's see in this so whenever we had multiple forks we will probably run out of time so when we had multiple forks we had something like this so we could have a for loop that creates four new processes in that well I could call a new process function and in the new process all it would do is call fork and then if it is the parent it would just return and then the new process would execute the useful work so here is all the useful work that the child process is supposed to be doing and then it would eventually exit and terminate and everything could run at the same time and a multiple fork oh wait it's called so in this I create four processes and they all just essentially just count from one to zero all the way up to nine so if I did that same thing using threads oh I'm running out of time if I'm doing that same thing using threads it looks like this so the main part looks essentially exactly the same have a for loop and I create a new thread each time instead of here I would do a p-thread exit then in the new thread it looks a bit different because I want to pass an argument for it so whenever I'm doing multiple forks well I have a counter and whenever I fork both processes are exactly a clone at the time of the fork so I don't have to like make a new copy of the variable or whatever I just let virtual memory figure it out that loop counter will be different for each process because I fork it at a different time with threads they're in the same address space so I can't do that so if I want to pass things to a thread I have to use probably malloc I have to use pointers and all that fun stuff so here I declare an int pointer and I malloc the size of an int and I essentially write the id to that memory location then I create a new thread give it a run and then I pass that pointer to it so if I pass that pointer to it well now I can take that void pointer cast it to an int star because I know it is an int pointer and then I can dereference it to get the id out of it then after that I have to free it because I'm done with it it's the same case I malloc in the main thread pass it to the thread and now it essentially is the only one that owns it so it needs to free it then I would do the same thing count for that and if I run this it should essentially do the exact same thing does the exact same thing except the order is a bit different and yay all right so we can continue over this because we're out of time so just remember phone for you we're all in this together