 All righty, welcome back to operating systems. So today we get to talk about locking. So we kind of figured out what a mutex is. So today we will see some more advanced usage and kind of set us up for the next assignment. So as an aside, in some languages there might be something called a monitor and with object oriented programming developers wanted something easier to use in a mutex because with a mutex you can always lock it and forget to unlock it. If you have multiple ways to exit a method then you have to make sure you unlock at every rate before every return statement. So what some people did is they just say, hey, you can mark a method as monitored and automatically let the compiler handle the locking and unlocking for you. So the rule is an object can only have one thread active at a time in its monitored methods. Basically it means that the object gets a mutex as part of one of its fields, it's created for you so you don't have to manage it and then any method that is declared as monitored the compiler essentially inserts a lock call right at the beginning and then an unlock before every return statement. So if you've done Java, there's a synchronized keyword and that is an example of a monitor. So if I had a class that was supposed to represent, I don't know, someone's bank account I could have an in balance and then I would say the deposit and withdraw methods are synchronized. Therefore they won't have any data races because the compiler will essentially transform the calls into something like this. So this stop monitor would just be the mutex. So in the deposit method, since it's declared as synchronized, it would get a lock and then it would do its calculation and then it would get an unlock and then same with withdrawal and this would be one mutex per object. So you won't have any data races between with the balance of a single object. So that is what Java does for that. It's built into other languages, just depends. They'll all just be called a monitor. So we can go ahead and explore our last synchronization parameters. So there is something called condition variables. Although they're supposed to be a mechanism for keeping track of variables that might change and you want to wake up threads based off a condition. The implementation themselves, it's easier to just think of it as a queue. So there's this pthread conned underscore t. You can think of it as a queue and there's an init, a destroy and then there's a few other methods. So there is a signal. So that will explicitly wake up one thread that is currently waiting in the queue. Broadcast will wake up every single thread that is waiting in the queue and then this weight will put the thread in the queue and just block it until someone else calls signal or another thread called broadcast to wake it up. There's also a version of weight called time weight which will automatically unblock itself after a set amount of time you give it. So if you want to just make sure it gets up after some amount of time, you can use time weight. So for this, we'll just use weight and signal. So weight adds the thread to the queue, puts it to sleep and then signal will wake up a single thread in the queue. Broadcast wakes up everything. You might see broadcast once you get into GPU programming and things like that. That's basically what broadcast does. So all these condition variables which are basically a queue must be paired with a mutex and any calls to weight, the mutex must already be held. So you must have done a lock call on the mutex before you call weight. So you can only call weight if the mutex is locked by you. Signal and broadcast, you don't have to call it while you have the mutex. It doesn't matter, you're just waking up a thread. So why do you have to do this? Well, you can think of weight, it needs to do a few things. So one, it needs to add itself to the queue safely. So we don't have any data races and also it's probably protecting some variable that we're using to use as a condition. So we wanna make sure threads don't have any data races while they are changing that variable. So that's why it needs a mutex. And as part of that weight call, the weight call will atomically add this thread to the queue and also unlock the mutex so that another thread can make some progress. So you can have one mutex protecting multiple condition variables if you want. Generally you just pair it with one and keep track of some variable. So we'll just make it simple. We will just consider calls to weight and signal. So again, we should understand what weight does. So the thread calling weight will atomically do three steps. It will add itself to the queue for this condition variable. So you can think of that p thread con. I mean, I find it easier to just think of that as the queue itself. And then it will also unlock the mutex and then it will be blocked because it has put itself to sleep. So the thread calling weight, by default, if you just call weight, it's not going to wake back up until another thread calls signal on it. If another thread calls signal on it, well, it will be unblocked. It'll be eligible to run. And then the first thing it will do is try and acquire the lock before returning. So before you call weight, you will have the lock. As part of the implementation of weight, it will give up the lock and put itself to sleep. And then whenever it wakes back up again, before it returns, it will try and acquire the lock. So before and after, you have the lock. So you're guaranteed you have mutual exclusion. Yep. No, this is p thread con weight. To join? Yeah, so thread join, weights for that thread to terminate. This weight just puts this current thread to sleep. So we're definitely overusing the word weight because weight means the different things for processes too. And yeah, so we're not terribly creative. We don't have a lot of names for things. So there's like weight means different things if we're talking about processes versus these condition variables. So weight in this case just puts the thread to sleep, adds itself to the queue that's handled by this p thread con type. So if we go back and look at our producer consumer example, well, instead of using two semaphores, we could actually just use a condition variable. So our condition is going to be an int and filled. So we can just keep track of the number of filled slots. And we can have essentially a queue here for threads that are waiting for a filled slot and another queue here for threads that are weighing on an empty slot. So we can change around instead of using semaphores. We will have a mutex that's shared between both of those queues. So we lock it. That way we have no data races whenever we check this, that's annoying, whenever we check this end filled variable. So we'll just have a while. So while the number of filled slots is equal to n, which is going to be our buffer size. So everything is completely full. Then we're going to put this producer to sleep and throw itself in a queue that is waiting for an empty slot. So we'll just call it the has empty queue. And of course you have to give it the mutex. The other side of the coin is if the consumer is currently executing, while while the number of filled slots is equal to zero. So that means that everything is empty. Well, I can just put myself to sleep in this queue that is waiting for a filled slot and just put myself to sleep explicitly and I have a nice condition there. So back in the producer, if we fill a slot, well, we would increment the number of filled slots. So we just increased it again. This is in the critical section. So we don't have any data races with the number of filled slots. And then we could just signal has filled. So we could wake up exactly one consumer thread if there's a consumer thread waiting. If there is no thread in this has filled queue, then signal just does nothing. Then afterwards we unlock it and I could flip these two lines around if I really wanted to. Other side of this is in the consumer. Well, if it empties a slot, it can signal a producer thread that might be waiting with the has empty. And say we had a producer thread waiting, it would wake up right here on this line and then go try the condition again, make sure that every slot is not filled and then if not, well, it can keep on going, fill a slot, all that. So any questions about this? Yeah. So this condition variable, this p thread con, basically you can think of it as just a queue. So there is a queue for producer threads. So the producer threads go in this has empty queue. So weight essentially puts yourself to sleep and adds you to the queue. So in this, I check if all the slots are filled. If all the slots are filled, I put this producer thread to sleep in this has empty queue and I just put as many of them will go into this queue. And then in the producer, well, it will empty a slot and then signal that has empty queue, which will wake up exactly one thread. So we'll wake up one thread from the has empty queue. It will come back to life here, check the condition again, make sure that it's still not all full and then keep on going. Yeah. Yeah. So weight will just block until another thread will signal that queue. And signal just wakes up a single thread. Yeah. So the queue can have multiple threads asleep in it. Yeah. So that's a good question. And we call signal. Do we know what threads? So based off the implementation, it should just be in FIFO order. So what the order they actually went to sleep added themselves to the queue in. Yeah. So, well, silly answer is you have to because it's a parameter and other question and other reason is, well, essentially we need to ensure mutual exclusion because internally that weight function is adding this thread to the queue. We don't want any data races while it's adding itself to the queue. And so that's another reason why it would need a mutex and it also needs a reference to the mutex because we don't want to put that thread to sleep while it's holding the mutex, otherwise nothing else can do it. So this function right before it puts this thread to sleep, it unlocks it. So it needs it so that when it wakes back up, it tries to lock it again. And it'll only return from weight while it has the mutex. So it won't be the case where you call weight while you lock it and then you come back and then someone else can take the mutex because that would defeat mutual exclusion. Yep, so signal just wakes up a thread, it doesn't need any mutual exclusion at all. So just wakes up a single thread, that's it. So you can do it while you have the lock or while you don't have the lock, it doesn't really matter because the thread's just gonna wake up and then try and acquire the mutex, so. Yep, yeah, because we don't want, right? The problem we had with producers is when everything was full, we don't wanna overwrite any data, so that's what that's checking. So this is a bit more explicit than, remember we had our two center four case where we just did a center four weight and we initialized them, so. Is this easier to read than the center four case or worse? This is worse. Oh, we got thumbs up, so this is a mixed reaction for this one. So I think this one's kind of easier because with the center four case, it really depends on the initial value, too. And this, hey, I don't have an initial value, you don't have to argue it. Pretty, like it's more explicit what the conditions are. So, but again, personal taste. So let's make sure we understand it. So let's see this code and see what's the issue with this. So is there an interleaving of these two threads that will cause some bad outcome to happen? And some bad outcome would be, whoops, would be, if I'm using a condition variable, in here, I basically want to make sure that this condition is true, whoops. So I want to make sure the condition is true right here. So while I'm executing that line, that condition better be true, otherwise this whole thing is kind of a farce. So is there any way, any interleaving between thread one and two that will have thread one execute that line such that the condition is not true? So if thread, so initially, yes, let's also assume initially, whoops, initially our condition is equal to false. So you want thread two to execute first. So if thread two executes first, then what's going to happen is it's going to change the condition from false to true, and then thread one, well, if it executes, if we context switch back to thread one, it could grab the mutex, no problem. Check the while, so while not conditioned, well, it's currently true, so that's false, so it just goes here and it reaches this line while the condition's true, which is what we want. So that actually worked, and let's reset. So any other idea is how we can get something bad. So thread one executes first, so it grabs a lock, okay, and then does it keep going? So it checks the condition, so the condition's currently false, so it would hit the weight, and then add itself to the cue. So let's go conned. My cue just looks like thread one right now. So no other choice but to execute thread two. So if we execute thread two, it changes the condition from false to true, and then does a signal. So thread one is currently in the cue, so it would wake it up, and it would resume, it would wake up, try and grab the lock right now, no other thread has a lock, so it would go back, check the condition. Right now the condition's true, so it makes it to this line while the condition's true. So that's also good. Yeah, so in this case, we're actually good, but this is bad practice. So let's see, so what about if it looked something like this? Yeah, sorry. Going back here, yeah, so this is fine, but if we added another thread called thread three, thread three set the condition to false, then we might be in a situation where we are screwed. So it might be the case where we start executing thread one, while checks the condition. The condition is currently false, and then it would go put itself to sleep, and then thread two could set it to true, wake that thread up, and then before it even wakes up, thread three could set the condition to false, in which case that would also be fine. We would only have the, it would only be bad if what happens is thread two, thread two starts executing, sets the condition here, three set back. So the case that's really bad, everything's reset. So right now, if thread two executes first, what it can do is change the condition from false to true. So our condition is now true. If thread one executes, well, it could make it past this condition, make it to this line without executing it, and then thread three could run, because there's nothing to prevent it from running, even though thread one currently has the mutex, thread three doesn't use it, and it just sets the condition back to false. And then in which case, if we start executing again, then, well, we execute thread one, and at this line, the condition is false. So we don't have the problem if we only change it from false to true, but you can see if we make this more complicated and we add more threads that are changing the condition, well then, I should, what should I do to fix this? So I can ensure that this does not happen. Yeah, yeah, I want to add a mutex to be safe. I should probably just add a mutex to both of them. So I should, oops, and then same thing here, lock, and then unlock. So now, if I have this, no data raises, yeah. Yeah, so if thread one starts executing in this case, here we can reset everything, actually. So in this case, if thread one starts executing, well, it would grab the lock. No thread, even if we context switch it, they can't pass the lock. Check the condition, it's currently false, and then we add ourselves to the weight queue, and then as part of that, well, currently this thread goes to sleep, so thread one is on the queue, and then as part of that going to sleep, it also unlocks the mutex. So this is why we want to make sure that that's what one of the things that weight does. So since it has unlocked the mutex, now doesn't matter, either thread could go, and we're fine, no data raise. Sorry? No, so as part of the weight, it unlocks the mutex, well, first it adds itself to the queue, then it unlocks the mutex, and then it goes to sleep. So it can't execute anymore, so we still have mutual exclusion. So let's say thread two ran, and then change the condition from false, it got the lock while it was in the queue, changed the condition from false to true, and then signaled it, so now it woke up, so now it is currently not in the queue. So at this point, either thread two, or thread three could get the lock, or thread one could get the lock while it's returning from weight. So your question's about what happens if thread three goes? Yeah? Yep, well in this case thread two's done if that's the only thing it's gonna do. So if that's the only thing it's gonna do, then thread one is gonna wake back up, acquire the mutex, check the condition again, and it's false and just put self to sleep again, because while the condition's false, so it can't do anything anyways, because it wants to make sure that that only runs if the condition's true. So if it's not true and it never becomes true, well then it's just in the queue forever. Yeah? Yeah, in this example, but you could have in a realistic program, that multiple threads might be updating it to true or whatever, yeah. So if we don't have lock and unlock, so if we don't have these locks, so what could happen is, well the simplest case is thread one executes, grabs the mutex, and then let's say thread two executes, then changes the condition from false to true, then thread one executes, checks the condition, and it's true, so it passes, and then right before it executes a line where we want to make sure the condition is true, well thread three can execute, and then change it back from true to false, and then if thread two executed again, it would be running this code while the condition is false, which is not what we want. Yeah? Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. So we'll get, so another thing that could happen, so let's assume the wait call, or the lock and unlock calls are there, so it's false, we'll move thread one back to the beginning, thread two back to the beginning, whoops, and thread three back to the beginning. Oh no, I'm just resetting the whole question, sorry. So your question was, okay, so what happens if thread one executes first, grabs the mutex, conditions currently false, so it needs to wait, so it would put itself here in the queue, and then unlock the mutex, put itself to sleep, all that good stuff. So now you said what happens if thread two executes? That only does the condition. Only the condition equal to true. So it changes the condition to true, and that's it before it gets the signal, and then it unlocks the mutex, say, and before it gets the signal, we execute thread three, so then we execute thread three, it can grab the mutex, continue, we just change the condition back to false, so then thread three is done, and then at some point, the only thing that can execute is thread two, so it would signal, so it can't execute thread one because it's asleep, a sleep here, so because it's asleep here, it's in the queue, it can't run, so I have no other choice, but to execute thread two, and it wakes up thread one, and then they would check the condition again, it's false, and it would just put itself back in the queue, because it's false, yeah, yeah, so there is a, you can make that condition always false, if the condition's always false, then thread one, if it's waiting for that condition to be true, it'll just sit in the queue forever, if there are no threads currently running, just wake it up, so if you wanted to, you could, every time you change the condition, you could do a broadcast to wake up everything in the queue, or whatever, but generally, you'll just have threads going back and forth, like with the producer example, so any time it updates the variable, it just wakes everything up, or it just wakes up one thread, yeah, so broadcast will, so what Signal does is, wakes up a single thread in the queue, yeah, so it wakes it up, and then it's no longer in the queue, and broadcast wakes up every thread that is in the queue, yeah, yeah, and the queue is just everything that's asleep, so after broadcast empties the queue, everything's awake, all right, so we kind of answered our question, if we go to this, that has a mutex around everything, so all I did here is change the while to an, or yeah, change the while to an if, so right here in thread one, this was my only change, and you might think like, why is there a while there? Well, if I change the condition and it wakes up, if I wake it up only when it's true, shouldn't it always be true? Why the hell do I need an else? Well, or sorry, why do I need an if there instead of a while? Well, we saw that in the last example here, when we woke up thread one, but thread three executed, change it to false, before thread one resumed from the weight and change it to false, and that's the while loop there made it recheck it to make sure the condition was actually true, because well, even though you don't have any data races, you have to make sure that you have the most up to date value, so you have to make sure you recheck the value, so that's why you need a while there, so if I just change this to a if instead of a while, like with the previous example, it could be the case where the condition is false at this line because it wouldn't check it, right? So very important, if you do this, you have to be sure you have a while there and not an if, even though an if will probably work most of the time, won't work all the time. So any questions about that? We basically saw that in our example. All right, let's move back. So condition variables, pretty much serve a similar purpose to SAFORs, SAFORs are a special case of condition variables, so assume your condition was just an integer that you're keeping track of, well, now you have a queue that you manage, you can just go to sleep while the value is zero, and then whenever you post, you would just wake up a thread so you can signal every time you increment it, you can implement one in terms of the other, and you can also do the reverse if you want, but it can get a bit messy. Leave that for the computer scientist because messy, cool, messy means you probably can't read it. So for comp, the rule is basically if you have more complicated ordering between threads, most of the time condition variables are better for that. If it's fairly simple and fits into a center for, use a center for, and there's no real right answer, just try and do whatever provides better clarity. For more of this, this is mostly an intro for this course, so you will have more practice with it in other courses. Yeah, so the center for also has a queue, right? So if you wait on a center for another wait, so if it's zero, it puts itself to sleep, essentially it puts itself on a FIFO queue as well. So essentially it does the same thing, right? It's just a queue you can't see. So the difference is center for is just a value and just has the increment and decrement, all that, and that's it, that's all it can do, right? It just has those simple rules. If it's zero, puts itself to sleep and waits until something increments it so it can wake up and then decrement it, that's all it does. Condition variables, you have your own queue, you can manage it, do something more complicated than just incrementing and decrementing. Yep. So generally, so the question is, is there a way to pick a specific thread in the queue? And generally you wanna make sure everything is fair, so you want it in FIFO order, in which case you use their implementation. So if you really want to make sure that you pick a specific thread in the queue, well then you're doing your own thread management. Which, with pthreads you can put threads to sleep and wake them up if you want, it's just more complicated. So typically you just use whatever tool you actually need for the job, wow. All right. So next we need to talk about locking granularity. So this will be the topic of lab five. So that's the extent of your lock. So remember, you need locks to prevent data races, but if you do something silly, like you just lock the entire program and make it serial, then you don't have any data races, but you essentially made it serial with worse performance because now you added calls to locks and unlocks. So generally you want to divide the locks, maybe use multiple locks, make sure that anything that's independent uses a different lock, smaller sections. So for example, what about if you wanted to paralyze a hash table? Hmm, should I use multiple locks or only one lock? Well, you get to do that in lab five. So I will give you a hash table and then you get to paralyze it and make it go fast. Yep, yep, yep. So lab four or lab five will do performance testing for you. So you're only implementing, you just create mutexes and do lock and unlock calls, so it's about 10 lines of code, and but the drawback is you have to write about it. So yeah, prior to the midterm, but it's shorter. So some things to consider about locks. Oh yeah, and two, if you're wondering why Python performance is really bad, well essentially, at least up until recently, the whole Python interpreter, whatever you called into the interpreter, just had one giant lock that was called the global interpreter lock. So their fix to data races was just to lock the entire thing. So that's why if you've used Python, multi-threaded performance is absolutely atrocious because the whole Python interpreter is under one big lock. So there's been some efforts lately to actually remove that and try and divide the locks into smaller ones so you actually have good performance. So some things to consider about locks is, well, the overhead of the lock, so it's not free to lock and unlock. You also have to allocate memory to them, so if you create a billion locks, probably not good, that's gonna waste a lot of memory. You also want to make sure that with your locks, contention is a thing, so you have to argue about that. So if there are, what contention means is how many threads are using the same lock? How often do they fight over it? So the more contention you have, the worse your performance is gonna be because if you have multiple threads fighting over one lock, only one can execute at a time. So you wanna make sure your contention is fairly low and then the last thing is deadlocks, which we do not want to have. So some locking overheads, like I said, memory allocated with it, they take time to initialize and destroy them if you're creating new ones all the time and also the lock and unlock operations are not free. More locks you have, the greater the cost is going to be. So if you, like I said, if you have a billion locks, probably not good. So the last thing is deadlocks, which you absolutely do not want because that means no thread can actually make any progress, so the more locks you have, the more you have to worry about deadlocks. So for deadlocks, the conditions you have to have is well one, mutual exclusion, which if you're, we'll just argue about mutexes. So in that case, of course we have mutual exclusion if we're using mutexes. The second is hold and wait, so you have to have a mutex locked and try to acquire another. So I have to be holding a lock and I have to try and acquire another. The third is no preemption, which means I can't take a lock away. So after a thread has acquired the lock, I can't just brute force it and tell it to unlock it. I can't, you know, I can't rob it or anything. I can't just be like, hey, unlock that please because I need it. So you can't do that. And the fourth condition is circular wait. So you have to wait for a lock that is held by another process or thread. So for example, say we have two processors or two threads trying to acquire two locks. In the first thread I get locked, in the first thread I lock lock one and then lock lock two and then I unlock or release lock two and then unlock lock one. And then in thread two, I lock lock two and then get, or sorry, I lock lock two and then lock lock one. Well, I can get in a situation with executing these threads concurrently where I cannot make any progress anymore or in other words, deadlock. So if thread one executes first, it could lock the first lock and then we could context switch back over to thread two and then it could lock lock two. And now we're essentially at a stalemate where we can't make progress on either thread. So if thread two continues, it would try and lock lock one but it can't acquire it because thread one has it. And then if I tried to execute thread one, well, it tries to lock lock two which it can't make any progress because thread two has it. So does that make sense? Neither thread in this case can make progress. If all threads can't make any progress, that is what is called a deadlock. All right, so now we need to prevent these. So easiest way to prevent a deadlock is to ensure a set order of the locks. So remember in our conditions, we don't want any circular weights or and we also, yeah, we don't want any circular weights. So this breaks circular weights if we always acquire them in the same order. So if a thread tries to lock lock one and then lock lock two, no matter how many threads I have, I cannot deadlock because I can only acquire the second lock if I already have the first lock. So because of that, it's not possible for me to deadlock because I don't have that circular weight going on. So does that make sense? So no deadlocks because I broke circular weight. Yeah, so if every thread acquires the locks in the same order, I can't have that holding weight thing, right? Because in this case, they acquired them in a different order, so it was possible that you could get lock two and without having lock one. So in that case, I have circular weight, that's bad because I could have a situation where I'm waiting for another one. But if I always acquire them in the same order, the circular weight essentially goes back to the beginning, right? So if I have a set order and I only acquire them in that set order, it's not possible to have holding weight anymore. So that is one way to prevent a deadlock. So you have to be careful here if those are pointers and the locks can actually be switched. So generally, if you can find an order, like you could always go by like the lowest memory ID if you want, you could always, if there's something to identify these, you could go by the lowest ID, as long as you have some way of making some order between the locks. So another way to do it is using Trylock, which we have asked about why you would want that. Well, now here's the reason why you would want that. So this prevents holding weight. So in this case, I could try and lock lock two, and then I can have this while loop. So while I try to acquire the lock, if I go into this while loop, it means I have not acquired the lock successfully. So if I have not acquired the lock successfully, I wasn't able to get lock two. Well, then I'm going to break holding weight. So if I have the first lock and I try and acquire the second lock and I don't get it, I will give up the first lock. So I will unlock it. I could wait, yield, do something at this point, because at this point I don't hold any locks. So another thread could make progress. And then if I come back, well, I can acquire the first lock again. And then I would go back to the beginning of the while loop and then just try again. So then I try and acquire lock two. Hopefully this time I actually acquire it and then I could run this code while I have both locks. So I can get both locks. So this also will not deadlock because I broke holding weight. If I can't get both of the locks, I give up the lock I was holding. So this way I don't have a deadlock. So I could get unlucky where that keeps executing again and again, but I don't have a deadlock. So any questions about that? So those are the two main ways to avoid deadlocks. So we saw some more advanced locking today. So we have another tool to ensure order called condition variables, which we may or may not like. So generally they're clear for more complex conditions while signaling. If you can easily represent with a center four, you should probably use a center four. And then you will see that locking granularity matters, especially for performance. So like how many locks you have and how much code they protect, because essentially anything that is protected is running in serial. And also as a thing to make your life more annoying, as soon as you have more than one mutex, you have to prevent deadlocks. Yep. So lab four just mute, or lab, wait. This one? No, no, yeah. So don't use locks in lab four. You do have a data race, but you can fix it like in the common issues thing by just reading the newest value whenever you come back from a swap context. So you don't need to use mutexes or anything for lab four, but there is a case where you will have a data race. So just be sure because when you do a swap context, that new thread might change essentially your big array pointer. And when you come back, it might be a different value than it was before. So you just have to recompute that address when you come back. And that's one way to prevent data race because for us nothing can happen in parallel, it's just concurrency. So it's a bit easier. All right, so with that, I guess we're done. So just remember phone for you, we're on this together.