 All right. Good morning, everyone. So today, we will finish up our journey about all this multi-threaded stuff and finish up just talking about more general locking. And one thing you will be seeing in Lab 3 you have to implement. So first, just as a little aside, who here has programmed in Java before? OK, so some people have more than the other section, I guess. So there's this thing called monitors that are built into some languages. So if you have object-oriented programming, if you have mutexes, you have to remember to lock and unlock them. And sometimes that's deemed too difficult to use. So when they made object-oriented programming, they had a better idea. And they thought, let's create something called a monitor. So you can mark a method as monitored. And then the compiler will handle creating a lock for you for that individual object. And any methods that you have with that certain keyword, they will automatically have a lock at the very beginning and then unlock right before they return. So you don't ever have to remember to have any type of locking yourself. And you're guaranteed you will have mutual exclusion between all your synchronized methods because they're going to use the same lock. So basically, it's just one mutex per object created for you. Any method that you say is monitored has lock and unlock added for you. So in Java, if you happen to use Java, it's called the synchronized keyword. And that's the example of a monitor. So you could have a class that represents a bank account. And then you have a balance, which might be modified between different threads. So you would say that the deposit and withdrawal method are both synchronized so that you don't have any data races on your balance amount. So it would transform that code at the top there. The deposit would just have a lock whenever it's entered and then unlock before it returns. And same with the withdrawal. And they'll be using the same mutex pretty much. So that will make it so you don't have data races. So you don't have to argue about data races and all your synchronized methods. So that's just one shortcut that you might see if you happen to have the unfortunate, not really a privilege of having to use Java. All right, so let's talk about conditioned variables because this is, I guess, what you'll be implementing in Lab 3. And they're kind of nice to understand anyways. So they behave a little bit like Centiphores, although arguably that they're easier to understand. And they're basically like your own custom queue of threads if you know when to put them asleep and when you want to wake them back up again. So it's in the pthread library. So if you include pthreads, you can use mutexes, conditioned variables, threads, all that fun stuff. They have the init method that we're used to seeing, the destroy method that we're used to seeing, and then signal, which will wake up a thread on that queue. And we'll see how this works shortly. Then broadcast wakes up any threads on that queue that we don't really have to worry about. Then there is weight that takes the conditioned variable, and it must be paired with a mutex. And then similarly to how there's a try weight or try lock, there's a non-blocking version called time weight that will just return after a certain time out if it hasn't been woken up yet. But for the purposes of this, we'll just argue about weight and signal. So weight essentially adds this thread to the queue, and then signal wakes up exactly one thread that is sitting on the queue, and broadcast wakes up everything. So like I said, conditioned variables must be paired with a mutex. And unfortunately, the word weight is overloaded. So the conditioned variable weight is different than weight, which we saw before, which waits for a process to finish executing. Weight will put this thread to sleep, and when it calls weight, it must have the mutex already locked. And then signal and broadcast do not have to have the mutex locked. So why does it have to have the mutex locked? Well, you can think of it this way that weight needs to add itself to a queue safely. So it needs to not have any data races. So if it's already holding the lock, it can just go ahead and add itself to the queue and not have to worry about data races because it would be the only thread, because it has a mutex. And then it also needs to unlock that mutex before it goes to sleep so that another thread can go ahead and do some work that might change the condition. And again, we will see an example of this. And then whenever it returns from weight, that thread essentially is going to re-lock that mutex before it returns from weight. So when mutex can protect multiple condition variables, again, we'll see an example. And we'll transform the producer-consumer example we saw before to use condition variables instead of centerfors. But for the purposes of this course, we don't have to worry about broadcast or time weight. We can just get away with just using weight and signal. And unfortunately, yes, these are overloaded words. It's very unfortunate because weight is also the process weight and signals also the operating system sending you a signal. So unfortunately, names are reused a lot of the time. And you just have to keep track of what context they're under. So the weight call doesn't contain data races, so you can think of it happening atomically if you want. And to understand what weight does, again, this is condition variable weight, not weight, which is like waiting on a process to die. So whenever a thread calls weight, this will happen atomically. So it will add itself to the queue for a condition variable. So an easier way of reasoning and thinking about this is to just assume that the condition variable just represents a queue. So it would add itself to that queue and then unlock the mutex, all in one atomic operation. And then it would get blocked so that thread can't get scheduled again. And it's just going to wait for some other thread to call signal on it or wake it up. So it could call signal on it or it could call broadcast for the purposes of this. Let's just assume signal. So whenever the thread that originally called weight has another thread called signal and wakes it up, it will become unblocked so it can get scheduled again. And then the first thing it does before it returns from that weight, it's going to grab that mutex again. So that way you have a property where before the call to weight, it has the mutex locked. And then after the call to weight, it also has the mutex locked. So this is how we could transform our producer and consumer example to use a condition variable instead of a mutex. So instead of two center fours, I'm just going to have a int called which just keeps track of the number of filled slots instead of just using center fours to keep track of the number of empty and filled slots. I'll just keep track of the number of filled slots. And then I will create a mutex that I'm going to pair with it because I'll be modifying the number of filled slots. So I need to be sure I don't have any data races. So I'm going to do all of that while I have a mutex locked. Then I essentially could create two queues, a queue that is waiting for something that has a filled slot and then a queue that is waiting for something to have an empty slot. And then in the producer, I would lock my mutex. And then I would have this while loop that essentially while the number of filled slots is equal to n, which is like the buffer size. So all the slots are filled. So while all the slots are filled, I should put myself to sleep, which is this weight here. And essentially add myself to a queue that is waiting for an empty slot. So there could be one. There could be many threads. That queue is just the number of threads waiting on at least one empty slot. Then of course, after I filled the slot, I would increment the number of filled slots because I just filled a slot. And then I could signal any threads that are waiting for a filled slot, which would be the opposite thing you'd be waiting for. And then unlock my mutex. And then in the consumer, it's pretty much the kind of dual of that. So it would grab the mutex. And then its condition is it would wait while there are zero filled slots. So if there's nothing filled, I can't consume any data. It's all empty. So I would put myself to sleep and add myself to a queue that's waiting for at least one filled slot and with the mutex. And then I would empty a slot whenever I'm woken up, then subtract the number of filled slots because I just emptied one slot. And then I would signal any thread that is waiting on an empty slot. And then unlock the mutex. So does this kind of make sense, Ish, that this would work and do the same thing? OK, well, let's go to a simpler example. So let's take a look at this and see if we understand what is going on. This has a issue. So let's take a look at this code. So what is the issue with the following code? You have thread one that grabs the mutex, waiting on some condition, and then it waits, and then unlocks the mutex and thread two, sets the condition to true, and signals on the condition, and initially assumes that the conditions initially false. So let's look at this for like 30 seconds or a minute and then see if anyone can come up with some type of problem that might happen here. All right, anyone want to guess if something very bad that might happen here? Not you. So is there a situation? So if weight puts myself to sleep, is there any situation where I would put myself to sleep and I would never get signaled again? So I would never get woken up and essentially just sit there in a coma forever? Or if someone brave enough to tell me what happens if even thread one runs to completion then thread two runs to completion? Or are we all asleep? So if thread two runs to completion before thread one starts, so if we have this and thread two runs to completion, let's say, so if initially the condition is false, we essentially also want, at this point, in thread one, condition must be true if we reach here. So if thread two executes first, then it's going to flip the condition to true and this would be thread two. So it flipped the condition to true and then signal on that condition, which is fine because there's no thread waiting so it's not going to do anything. So it's essentially a no op. And then thread two's done executing. Then if thread one starts executing, it would lock the mutex, which is fine because initially it's unlocked right. So it would pass through the lock call and achieve the lock. So then it would check the condition in the while loop up here and then the condition is true. So it wouldn't go into the while loop. It would pass. Then at the green line there, the condition is true. It could do something using that assuming that condition is true. And then it would just unlock the mutex and then finish executing. They both finish executing. We're all good, right? No problem with that. So the more slightly interesting before we get into our problem is what happens if thread one executes the completion first? Well, what's going to happen is it's going to lock the mutex, which is fine because it's unlocked. So it's going to grab that mutex. So now it's locked. It's the only one with it. It would check the condition. The condition is false. So it would call weight on the condition with the mutex, dear god that's annoying outside, and essentially add itself to the queue. So if we wrote the queue to the side for our condition variable, we would now have thread one sitting here in the queue and it would be blocked because that's part of what weight does. So it's going to add itself to the queue, also give up that mutex, and then another thread can run. In this case, there's only thread two that can run. I don't have any other choice. So if thread two gets scheduled, it executes. It would change the condition over to true, which is good. And then signal on the condition. And then all that's going to do is make thread one in the queue be unblocked now. So now it can be scheduled to run. And now there doesn't have any other choice but to run. So it's unblocked now. And whenever it gets unblocked, before it can return from weight, it has to re-grab the mutex. So the mutex is currently unlocked because as part of the process of atomically adding yourself to the queue, it unlocked it. So it would grab the mutex again and then return from weight and just start executing wherever weight left off, also with the lock still. So it would check the condition again. Now it's true. So it wouldn't go into the while loop again. The condition would be true at the green line. And then it would just unlock the mutex, which is fine, because we had it locked. So in that case, we executed the green line with our condition. We were all good. Does this make sense to everyone? Hopefully? OK. Now what bad could happen? So again, here's our code. Yeah. I have a question. So as part of calling weight, you have to have the mutex. Yeah. When you call weight, you have to have a mutex locked. And there's kind of something going off that's related to data races, right? Our condition variable is just a variable. In one thread, we're reading from it. In another thread, we're writing. So what do we likely have in this scenario? A data race. Anyone tell me a bad interleaving of threads where the data race actually does something horrible? No, I'll wake this morning. Yeah, come on. All right, no one? All right, well, I'll start you off. So let's say thread 1 gets executed first. Locks and mutex. That's all good. That's all well and dandy. And then as part of the while loop, it's going to read that condition variable right. So let's just put in a read. So it's going to read that the condition is false. So it read the condition is false, so it's going to fall into the while loop. Everyone agree with that? So what happens now if it gets context switch right before it calls wait and thread 2 executes? Yeah, so it's good. If thread 2 starts executing now, it's going to change the condition equals to true, which is kind of fine. And then it's going to call signal on that condition. Well, thread 1 hasn't called wait. It's not sitting there in a queue. There's nothing in the queue, so signal just does nothing. There's nothing to wake up. There's nothing to unblock. It's fine. So now at this point, thread 2 is done. We have no choice but to switch back to thread 1. And then thread 1 would call wait on that condition with the mutex. And now our condition, queue, would look like. It would just have thread 1 in it, and it would be blocked. And now what do we? We can't execute thread 2. It's done. We can't execute thread 1. It's blocked. We're stuck, right? All right, everyone get that at this early hour? So that's another example of that lost wake-up we saw before. So you put yourself to sleep. Or sorry, you add yourself to the wait queue. And the thing that should have woken you up has already happened. And you just miss that wake-up, and you're essentially in a coma forever. Yeah, this is preemptive scheduling. So anything can happen at any point. So I'm just showing a possible preemption between the threads that cause something very bad, which, for data races and showing that bad things happen, there's like a lot of possibilities preemptions. But to be incorrect, you only have to show one is incorrect, and then your code is wrong. Yeah, so which one? So the question is, well, I have a lock. Shouldn't that prevent data races? So the answer is, only one thread even attempts to get the lock here. So if only one thread attempts to get the lock, it's just going to always get it. So it's as effective as just not having a lock. So if two threads try and call lock, only one will pass. But in this scenario, I only have one thread calling lock. So it'll never pass. So how could I fix this so that it doesn't do this? Sorry? So put some locks in thread two and thread one. There are locks in thread one. How do I do that? Someone want to save them of what I could do here to prevent this? Sure. Yeah, I need some locks in thread two, right? There's a data race essentially on that condition variable. So if I want, I can put a lock and unlock around it, right? So let's argue what would happen in this case. So even if I have the same interleaving of threads where thread one locks the mutex, now it's locked, and then reads that condition, which is still going to be false, now, whenever it switches to thread two at that point, the first thing it's going to do is try and lock that mutex. But thread one has already locked it, right? So it can't progress. It's just going to sit there, wait. Eventually, the OS is going to have no choice. The only way to make progress is to switch back to thread one, right? In this case, fails. Thread two has lock. So then if thread one, it switches back to thread one, then it would call wait on that condition using that mutex. So then it would add itself to the queue, thread one, and it would do this all atomically, so we don't have to worry about data races, and it would be blocked. And now as part of this, right, mutex is unlocked internally as part of that wait call. So then now the only way to make progress, we can't even schedule thread one now because it's sitting in this condition queue. So the only choice we have to make progress is to switch back to thread two, which was still executing that locked mutex call. But now, since thread one is blocked in this queue and the mutex is now free, it can go ahead, grab the mutex and then set the condition equals to true, right? And then it would unlock the mutex. So now the mutex is unlocked, and then it would signal on that condition, which would now make thread one, go from blocked to unblocked. Now the only way to make progress now, thread two, is done executing. So it has to switch back to thread one, and then thread one essentially returns from that wait, so it's going to recheck the condition because it's still in the while loop. It would read the condition, which is now true. So it would fall out of the while loop, do the green line, so now the condition's true while it's executing that green line. So that's good, that's what we wanted. And then it would unlock the mutex, and then we're done. Any questions about how that works? Yeah? Yeah, so as part of the wait call, what atomically is going to happen is it's going to add itself to the queue and unlock the mutex, all the steps here. So it's going to add itself to the queue for the condition variable, unlock the mutex, and then be blocked so it can't be scheduled again. And then whenever some thread calls signal, in this case there's only one thread sitting there, so it's going to wake up that thread. It would get unblocked so it can be scheduled now. And then before it returns from the wait, it will grab that mutex again so the mutex is locked when it returns. OK, so if we have that with lock and unlock, one question might be, well, do I really need that while loop? Because if some thread signaled me, it's because the condition's now true. And what the heck? Like, I know it's true. Why am I in a while loop just to check that it's true again? Well, basically if your code looks like this where you have an if condition, more likely than not there will be several threads swapping the value of that condition. So thread two could set it to true, and then signal that, hey, that condition's now true. You should wake up if you're waiting on it. But likely something like thread three would also set the condition to false or just some other thread in your program would likely set to false. So you could be in the scenario where thread one executes. So thread one could execute, grab the mutex, be locked, check the condition, which is false, wait. So it would add itself to the queue, unlock the mutex, and it can't be scheduled to run again. And then thread two could come in, grab the mutex because now it's unlocked, set the condition to true, unlock the mutex, and then signal that thread one to wake up, which is going to try and grab the lock. But thread three could have already been waiting for that lock before the signal. So thread three would grab the lock first ahead of thread one, set the condition back to false, and then unlock, and then whenever you return from wait here, you would see that condition as false. So if you just had an if, and it just dropped out, at this line, this line could execute while the condition is equal to false, which is exactly what you don't want, right? And also fun fact that this wait call, at least by the spec on some operating systems, can return even if the condition does not change. So it can just wake itself up just because it wants to make your life difficult. So just in all circumstances, you should always just do a while loop just to double check that the condition's actually true before you start executing anything, right? Cool. So no questions about that. Hopefully that will people up a little bit. All right, so condition variables kind of serve the same purpose as center fours. You can kind of think of center fours as a special case of condition variables. So you could just create your own integer with a value and then whenever you do something, you can increment it and then you don't have to do a signal. And then if you do, and then to implement your weight, you would just decrement it, but you first check if the condition, if it's equal to zero, you would just weight yourself, right? And then that way it behaves exactly the same as a center four. So condition variables are kind of just a more general solution to center fours and it's generally a bit easier to read if your conditions are very, very complex. But implementing a center four as a condition variable is easy, but you can actually do the reverse. There's papers on that. It just gets kind of messy and you would never want to read and debug something like that. So you generally, the two tools do the same thing, but the only difference is depending on what you're doing, one is much easier to read than the other one. So you will kind of get a feel for that as you go, you know, as you practice this stuff. So the next thing we can talk about is locking granularity and that's the extent of your locks. So between considering if you have a mutex, you know, all the code between lock and unlock is going to be essentially running in serial. So you want to keep that as small as possible. While also still preventing data races because that's your number one goal, you need to prevent data races or else your program's going to behave incorrectly. Although there are some cases where data races might be okay, but you really have to argue about that. But if you lock large sections of your program, that might be bad, you might want to divide the locks into smaller sections. For example, what if you want to paralyze your hash table in lab one? So who can do that now? Because you should be able to. So did anyone use like linear probing for collisions? Good, because if you use that, you're probably screwed. So if you just had, you know, if one of every slot of your hash table was just a list, you could have a mutex for each individual slot. And then that way, you know, you don't have any data races because you protect a slot, you know that any collisions will resolve to the same slot. So if you just run everything in the same slot in serial, then you won't have any data races, you will be fine. And then likely if your hash function distributes things well, it can essentially fully paralyze itself because the odds of hitting the same slot are pretty low. So you could actually pretty easily paralyze it. Well, if you have linear probing, where you know, if there's a collision, you go to the next slot and on and on and on again, you won't know how much of your hash table to lock. And you're essentially the safest thing to do is to just make the whole thing run in serial. So now you're getting into the situation where how you implement your algorithms depend, like how you implement your algorithms dictate how easy it is to paralyze it, which is going to be a major consideration, especially in today's day and age when you have multiple processors. So some things to consider about locks to is the overhead of the locks because they're not free. You have to create them. There's gonna be some costs to locking and unlocking them. And then even more majorly, there's a scenario of deadlocks that we'll look at. So overheads associated with locking and overhead is just any time you are wasting CPU on operations that don't contribute to actually running your application. So for example, for locks, there's gonna be some memory overhead. So if you create a billion locks, you can't create a billion locks for free. They have to be stored in memory somewhere. If you have a billion of them, even if they only take up a few bytes, they're still going to waste a lot of space. There's costs to initialize them and destroy them. There's also time required to do the lock and unlock calls. And depending on how big your critical section is, the time to even unlock and lock it might dominate your critical section. So in which case the cost of that would be significant and you might need to rethink what you're doing, right? And the more locks you have, the worse it's going to get because this is times the number of locks you have, right? So we haven't talked about deadlocks yet. So here we get to talk about deadlocks and the conditions for deadlocking. So the more locks you have, the more you have to worry about this. So there are four conditions you need to have for a deadlock. One is mutual exclusion, which if you have mutexes, of course it's going to happen because they're for mutual exclusion. Two, you have to have something called hold and wait. So you have a lock and try to acquire another. So you already hold a lock and you're trying to get another one. So you need at least two locks to deadlock. You also have to have no preemption, which means you can't take simple locks away. That doesn't mean you can't have, you can't context switch between threads still. The preemption is preemptible locks. So I can take a lock away. So like I can just make it unlocked if I want to. So to have a deadlock, you can't have this. And for simple mutexes, if you have the lock, nothing can take it away from you, unless it's yourself. And then four, you have to have a circular wait. So you're waiting for a lock held by another process or thread. So this is the simplest deadlock example that I can think of. So consider two processors or two threads, each getting two locks. So the code in thread one, it tries to get lock one and then get lock two. And then thread two, it tries to get lock two and then lock one. Well, since we're in the land of threading now, we have to argue about any preemptions between two threads resulting in bad behavior. So one bad preemption we could get is, thread one runs get lock one, it locks lock one, and then we get preempted to thread two, thread two locks lock two, and now has lock two. Then at that point, if it tries to get lock one, it can't because thread one has it. And then if we contact switch back over to thread one, it will try and get lock two, which it can't do because thread two has it. So we're at an impasse here, right? Each thread is trying to get the lock that the other one has while it's still holding another lock and neither can make progress. So does that make sense why that's a really bad thing that can happen? So neither thread can execute at this point where deadlocked our program can't make any progress anymore. So to prevent deadlocks, one of the simplest solutions is to just ensure some type of ordering between locks. So if both threads always got lock one, then lock two, then you won't have any issues, right? So you can only get lock two if you already have lock one. So it's kind of implied that if you can get lock two, you already have lock one. So if two threads race, it's whoever happens to get lock one will also get lock two because you can only get lock two while you have lock one, right? So if it's the same example where we have two threads, one's trying to get lock one and then lock two, whoever gets locked one first is going to also get lock two, right? So we won't have any deadlocks. So does that make sense? Okay, cool. So you have to be real careful about this too because generally you pass things by pointers. So if you have some code that looks like this, it doesn't necessarily that means lock one actually points to the same thing every time. So fun questions to ask is like pass this by pointers and then you actually switch the arguments around so that it actually looks like something like that and then that would deadlock, right? So even though the code looks like it acquires them in the same order, because of pointers and all that stuff, it might not. So that's fun. The other easy way if you don't want to ensure ordering between threads is to use a try lock because remember one of the conditions is hold and wait. So this is eliminating that condition necessary for deadlocking. So now this is finally where try lock comes into place. So instead, if you can't ensure order between the lock, you can do something like this. So you can lock lock one and then you can do a try lock on lock two. And what try lock will do is it will try and grab the lock and then tell you if it has it locked or if it has it unlocked. So in the case where it's not equal to zero, it means that it did not successfully grab the lock. So if it did not successfully grab lock two, it would unlock lock one, then wait, put itself to sleep, yield, do something here, then hopefully the thread that is waiting on lock one would be able to grab it now because we unlocked it. So we freed it up, we're not holding and waiting on it. So at this wait call right here, we have neither lock. And then next time we go through the loop, we try and lock the thread again and then see if we get lock two. Again, if we don't, we'll do the same thing. If we do get the lock, we'll fall out of the while loop and then be able to run our protected code here while we have both locks. And then of course we would unlock both lock. So this won't deadlock because we eliminated hold and wait. So if we can't get lock two while we have lock one, we will just give up lock one and essentially yield, hopefully have another process that was using lock two. It will be able to unlock it or do whatever it needs to do. So any questions on this? Makes sense. Cool, all right. Let's go look at our code two. So here's the producer consumer implemented in terms of condition variables. So I'll create a mutex for the condition variable and my condition is essentially the number of filled slots then have two cues, one for has empty and one for has filled. So I would just initialize them and then initially the number of filled slots is zero. So now my producer code, it would lock the mutex, check that number of filled slots. If it's equal to the buffer size, it needs to wait on an empty slot. So again, it would add itself to the queue using that mutex and then eventually, whatever it breaks out of this while loop, it will be active and the number of filled slots will not be equal to the buffer size. So there'll be at least one empty slot. So then in that case, I can fill the slot, increase the number of filled slots and then I would signal any threads that are waiting for a filled slot and then unlock the mutex. Right, and then same thing with the consumer where I would lock the mutex. The condition here is if the number of filled slots are zero then they're all empty. So I shouldn't empty anything because there's nothing to no data to get. So I would put myself into the has filled queue where I'm waiting for a filled slot then empty a slot, decrease the number of filled slots and then signal anything waiting for an empty slot and then I would unlock it. All right, that makes sense to everyone. So for this, we had two options, essentially. We could use a center four or we could use our condition variable. So here's our center four implementation again. So between the two, who thinks the center four one is clearer? No one? All right, then who thinks the condition variable one is clearer? One person, sweet. So they're equivalent. I guess for this, I would argue that the condition variable one is a bit clearer because you actually know the conditions. So the number of filled slots and then is what you're waiting on. You actually want to check if they're all empty or they're all full. So, and then you don't have to argue about the initial value of the center four because that actually determined correctness but I guess we're not terribly awake now. So we can just go ahead and think about this for later. So I guess I can just, yeah, I guess we can just end early and go take a nap I guess. So we explored some more advanced locking today. We have another tool to ensure order. Condition variables are generally clearer for more complex condition signaling. You have to implement them in lab three I believe so that'll be fun and that woke someone up at least. When you are trying to use locks to speed up your program, you're trying to paralyze things as much as you can. The locking granularity matters to how much you can paralyze your program. So you want to make it as small as possible but now in addition to preventing data races which you should do, you now have to prevent deadlocks otherwise, right? If your application deadlocks, it's not gonna make progress. So let's remember, I'm pulling for you, we're all in this together.