 Welcome back to 353. So today get to talk about more locking. Yay. So there are these things called monitors built into some languages. Who here has used Java? So have you done threads in Java? Could you do threads in Java? No? So you can. So in some languages there are these things called monitors. You know when they made object-oriented programming where literally everything is an object developers want something easier to use than like a mutex. So what you can do is mark a method as monitored and then let the compiler handle locking for you. But of course if you don't know how a mutex actually works and it actually uses a mutex under the hood and it's really easy to screw them up well they might get screwed up. So the rules for monitored methods is basically what the compiler will do is create one mutex per object and then in every method you marked as monitored it will do a lock call at the beginning and then an unlock call at the end. So you have mutual exclusion between methods for the same object. So in Java if you have the unfortunate task of having to write Java well there is a synchronized keyword and that is basically a example of a monitor it's just what Java calls it. So say I wanted to create a I don't know a class called an account and I had an integer balance so I was trying to run a bank. We'll get to run a bank and screw it up on next Tuesday but for now assume I just like I'm trying to run a bank I create a class that represents an account. I give it an int that's supposed to represent the balance of the account and then well if I have multiple threads and I write a method just called deposit that adds money to the account and a method called withdrawal that subtracts money from the account. I could have data races if I have multiple threads here right multiple threads could be calling deposit withdrawal we don't want to have some inconsistent result. So what I might do is I might just put a synchronized here and what that will do is a compiler will transform the code for us so it basically create a mutex in this class called monitor and it would transform the method calls to in deposit it would like lock this stop monitor and then do the actual adding to the balance and then it would unlock it at the end and for the withdrawal it would go ahead add it at the beginning and unlock at the end. So that way I wouldn't get into data races hopefully theoretically with this but we'll see later at the end of the lecture when we talk about deadlocks your code can have issues with this still. So there's also a another synchronization thing we can use called condition variables they're very badly named they behave kind of like centerfors basically they allow you to create your own custom queue of threads so I like thinking of them just as a queue and the name condition variables is because while they're supposed to predict a variable until it reaches a certain condition if it was centerfors the condition would be oh well the value has to be greater than zero in order for it to be able to subtract for it but if you don't want just that condition you can use condition variables to accomplish that for us. So their type is p thread con underscore t and while there's called a condition variable they have an int or they have a initialization function that has some attributes that we can ignore they have a destroy and annoyingly they have a method called signal has nothing to do with actual signals I get sent to processes it just has to do with what happens to that queue. So what signal will do this condition variable I like thinking of it as just a queue of threads or like a waiting queue so what that will do signal will just wake up the first thread in line and unblock it so it can go ahead and run again there is a function called broadcast that we can ignore but it will essentially unblock every thread that is currently asleep in the queue and then similarly to centerfors there is a wait and you might see here that it is paired with a mutex so how that works is it will atomically add itself to the queue and also the rule is you can only call wait while that thread has this mutex locked and it will add itself to the queue and then before it gets blocked it will go ahead and also unlock the mutex so that another thread can hopefully change the condition variable that it's waiting on which we'll see examples of it could be anything we want and make sure we don't have any data races with it then there's also a time wait just in case you get into that situation where you might you know block forever you could just put a timeout so that after a certain amount of time you will get unblocked so for the purposes of what we need to know aside from creating it and destroying it we'll just argue about wake that add wait which just adds this thread to the queue and then signal that wakes up a thread again don't have to worry about broadcast but that would wake up all threads so like I said has to be paired with a mutex so any calls to wait must already hold that mutex calls to signal or broadcast you don't have to have the mutex acquired when you call them doesn't really matter and why well we need to prevent data races so thinking of that condition variable as a queue well it needs to safely add itself to the queue and it also needs it as an argument because before it gets unblocked it will go ahead and unlock it before it blocks and goes to sleep and the other rule is whenever that thread gets unblocked it will only return from wait when it has reacquired the mutex so the mutex will be locked when you call it and then locked when you return from it so you don't have to worry about any weird locking conditions so we can have one mutex that protects multiple condition variables and like I said we will only consider calls to wait and signal so to understand again what wait does so doesn't contain any data races and it basically does three things whenever you call wait the current thread that is executing will add itself to the queue for that condition variable it will also unlock the mutex and it will also block so it will also go to sleep it now can no longer run so when eventually a thread that is in the queue gets like signaled like gets unblocked through a signal or broadcast then if it actually gets scheduled to run because again remember we're up to the scheduler so we don't know when it will actually execute it what it will do is it gets unblocked so it can run again we don't know when it can run and then it will try and lock the mutex again and wait will finally return when it gets the mutex so that is basically how wait works and if we wanted to we could rewrite that producer consumer example from last lecture using condition variables if I want you may or may not think this is easier than center fours but basically my condition variable I can just keep track of the number of filled slots and then I can have a queue that is waiting for a filled slot or a queue that is waiting for an empty slot and then while I would be reusing this mutex so this mutex would be protecting any data races to the number of filled slots variable and I can just go ahead check the value and make sure that it is actually valid so in the producer my condition is basically I only want to go to sleep when the number of filled slots is equal to the buffer size or capital n so I could that could be my condition so while everything is full I just want to put this thread to sleep I want to wait so I'm waiting I'm putting myself in this has empty queue so I'm saying okay I'm waiting until there is an empty slot and then I give it the address of the mutex so it can go ahead and unlock it and hopefully another consumer thread can actually execute if it gets woken up it would require the mutex and if the number of filled slots does not equal to n so it's not full it will go ahead and just fill a slot increment the number of filled slots because we just filled a slot still has the mutex so we don't have any data races and then it could go ahead signal any thread that's waiting for a filled slot and then unlock the mutex and then similarly for the consumer well we need to put it to sleep when the number of filled slots is zero so there's no data for it to get and then it can put itself in this has filled queue and then well eventually when it gets to run it would reduce the number of filled slots and then it could signal any producer threads that are waiting for an empty slot and then unlock the mutex so this easier or harder to read than the center for example easier right makes a bit more sense so I had some people argue that the center for was easier for this that one I don't know a bit more borderline I would argue that this is easier to actually read but whatever floats your boat I guess yeah so how the weight works so say that I don't know everything's filled here right so we have all filled slots so this producer thread needs to go to sleep because it's waiting for at least one empty slot right so it would put itself into a queue of threads that are waiting for empty slots and then it would also unlock the mutex and then actually go to sleep so I would have like just one thread in this has empty queue and because it unlocked the mutex well what could happen is a consumer thread could run right yeah yeah so it get blocked goes to sleep can't execute anymore yeah it's just in the middle of the wait call okay so we can have a smaller example where we can see if we have screwed up using one so let's take this example and I will give you a second with it whoops so the idea again with the condition variables is you are waiting for something to be true it doesn't have to like literally be a boolean value true in this case it is I will just assume that I have a global variable called condition and then initially it is false so the way I want to use my condition variable is okay well I will lock the mutex make sure there's no data races with it check so while the condition is false I will go to sleep because my intent is that I only run this line of code only run when so I only want to run this line of code if the condition is true so assuming I have two threads thread one and thread two thread one would start executing here thread two would start executing here is there any issues with the following code or aka is there any possibility that I can either run when that condition is false or I can go ahead and also maybe I don't know maybe a thread just goes to sleep forever give you a hint it's more likely that there is a possibility where a thread can go to sleep forever so so t1 is stuck in the wow loop so so let's assume t1 goes first so it would acquire the mutex so here I'll even put up here mutex and then which thread has it so mutex is currently held by thread one and then I could also write my condition variable called condition and then that could just be a queue of threads I'm keeping track of initially it would be empty so if thread one executes first it could acquire the mutex and then check the condition so while not condition well currently it's false so not condition is true so it would go into the body of the wow loop and then it would hit this weight so when it hits this weight it essentially would atomically add itself to the queue and then give up the mutex so it would look like that and then thread one can't run anymore right so thread ones put to sleep it is on that queue now if thread two runs okay well it changes that condition equal to true so this gets updated to true and then it would signal so it would wake up thread one and now thread one could execute again in this case thread two is done so if we choose thread one to execute again well as part of returning from this weight it will try and reacquire that mutex in this case no other thread holds it so it could grab the mutex that would go check the condition again the condition is not true so it's false so we break out of the wow loop and then this line of code runs while the condition is true no problem there that's what we wanted and then we unlock the mutex so we go back to this and everything worked no problem yeah so condition so condition is this boolean cond is the name of the queue yeah oops okay so here let's rewind so we can bring all the threads back uh so initially the condition is equal to false any other but is there a sequence of concurrent switching between thread one thread two that causes something bad to happen yeah yep yep so what you're saying so one thread has to execute first you're saying thread two executes first so what happens if thread two executes first well then it changes the condition from false to true whoops that is the wrong tool so changes the condition from false to true and then it would signal nothing to wake up so it doesn't do anything then thread one executes acquires the mutex checks the condition it's true so it jumps right here and it running this line of code while the condition is true all good unlocks the mutex nothing bad nothing bad happened there right anything else that could possibly go bad is it another one on discord that's oh that's close I think he might got bailed out by discord so what could happen that's bad is thread one executes first so thread one executes first goes ahead acquires the mutex check the condition so the condition is currently false so the next line of code it would run is putting itself on the queue but I can context switch over to thread two right now so at this exact point I context over to thread two thread two just changes the condition to true and then it would signal and then nothing to wake up so it's all good then we switch back to thread one thread one okay well it would hit this weight it would atomically put itself in the queue and also give up the mutex but now thread two is never going to wake it up again right so this is another example of lost wake up where that thread goes to sleep and never wakes up everyone agree with that that's bad all right fix it yeah yeah so my problem was well I had a mutex that was protecting a read to condition variable but I have a data raised because there's also a right that is not protected by a mutex so with that same mutex I should probably lock it here and then I can unlock it right after I update it or I could unlock it after I signal turns out it doesn't matter where do you want to put the unlock before or after after sure let's put it after so now if I just have that lock and unlock call what happens if we get the same unfortunate sequence of threads again so I'll move it back up oops so initially there are no threads in that con queue no thread has the mutex and the condition is equal to false so now what happened before is thread one executed first it would go ahead acquire the mutex so now it has the mutex that would check the condition okay so that's like a memory read of the condition it's false so that's not true so it would go into the body of the loop and then at this point right before it executed the con weight is where we had the issue before so now what happens if we context switch over to thread two so if we context switch over to thread two the first thing it's going to call is this lock of the mutex and can it make it by that no right only one thread can have a mutex at a time so it would get blocked it couldn't execute we have no other choice in this case but to context switch back over to thread one to make some progress and then what thread one would do it was would call weight so it would go ahead add itself to the queue and also unlock the mutex and now it's asleep it can't execute anymore the only thread we could execute is thread two so it would hit the lock call and it would acquire the mutex go ahead update the variable from false to true and then it would signal it so it would wake it up so now we have woken up thread one but thread one can't make any progress right because it would reacquire that mutex and currently thread two has a mutex so it's not going to return from weight until thread two actually unlocks the mutex so next thing thread two would do is thankfully unlock the mutex so it would look like this now if we chose the scheduler chose to execute thread one again okay well it would try and reacquire the mutex so it turns out nothing's holding it so it would acquire the mutex and now it is done with this weight function then it would go back up check the condition oh okay while the condition was true okay well great now I just execute this line that line's running while the condition is true so some of you might feel clever of like why the hell did you write this while why didn't you just write if does anyone have that feeling all right so we have one person with that feeling anyone want to tell them why they're wrong so I will give you in this case it could actually be an if that is fine but what about if I had three threads so I have three threads same idea in thread one I want condition should be true and initially I will have condition equal to false so I have that same thread but I just change the while to an if and then in thread two it's the corrected version so this will go ahead update the condition variable equal to true and then I also have thread three which just sets it to false and it doesn't have a signal here because well if it sets it to false the condition is waiting for while that until that value is to true so it wouldn't have changed it to true so now looks like everything's locked properly so I have no issues with the condition variable so if I just change that while to an if is there any possible interleaving of thread one two and three that could happen such that you know this line executes while the condition is false yeah okay so let me just write out a few variables here so you said we can context switch to thread three before it unlocks but before it signals okay so the suggestion is thread two executes first so it acquires the mutex and then updates the condition equal to true then unlocks so now thread three could run right so thread okay so thread one actually has to run first so let's rewind it all the way the beginning so bad thing that could happen thread one executes first so it acquires the mutex that is a okay no one has the mutex now looking at the other threads they both do a call to lock of the mutex so there's nothing else I could possibly run aside from thread one so here check the condition it's false so it would hit this weight as part of the weight adds itself to the queue oops and then also unlocks the mutex so now either thread two or thread three could run so you're saying thread two runs first right so thread two runs it acquires the mutex updates the condition to be true then it unlocks the mutex and now before it hits the signal we context switch over to thread three thread three acquires the mutex no problem changes the condition from true to false and then unlocks the mutex now at this point in time only thread two can execute next because thread one's blocked so next thing it's going to do is signal which is just going to wake up thread one and it is going to try and reacquire that lock to return from weight turns out thread two can't do anything thread one is the only one that can do anything so it would acquire the mutex and then executes this line of code and the condition is false whoo everyone good with that so how would i fix that the fix make this a while so there are cases like the previous one oh yep yeah okay so yeah what about if i just give these to the old swap Reno does that work all right so let's try with that swapping them and seeing what happens so the situation i was in where the bad was about to happen is that's basically the first thing so i'll rewind everything so let's rewind most of it thread one is just going to be sitting here in the queue and no one has the mutex now okay if i make that leap rewind everything thread one's just in the queue okay so let's see what could happen now so thread two runs next it acquires the mutex it updates that condition to be true and now it has no choice but to signal so it would signal and then we unblock thread two or thread one sorry so thread one can run can it run right now no because it would try and reacquire that mutex and thread two currently has it so couldn't execute but you know the only thing that could happen is thread two finally calls the unlock that we switched so now no thread has no thread has the mutex so right now either thread one or thread two thread three could execute right yeah yeah yeah so in this case so i might be okay if i make that switch if it happens that thread one executes next gets that mutex back and then now it's true so now we're good but we don't know if thread one or thread three is going to actually execute next so it might be the case and very likely this would probably work due to how the scheduler works but still technically has the issue and then these are one of the seven-year bugs that you will never ever fix so in this case i don't have to pick thread one to execute next it could be thread three which goes through and does the whole problem again where it switches it to false thread one requires the mutex and then runs that line while it's false oh okay so any questions about that yeah so if we got rid of thread two so if we got rid of thread two then it's not possible to actually execute this line because no thread is ever going to change it to true right and we only want this line to run while the condition's true yeah so if the condition started off as true then i could yeah but yeah so but if the condition started off as true and then all the threads only switch it to false i don't know what order they're going to be scheduled in so it's likely that that one thread is never going to be able to run or there's a situation where it might never run so with them typically you want to make sure that you eventually change that condition to be true otherwise it'll sit there forever um and yeah and other things like if i add a signal to here even though i don't change the condition equal to false not going to change the behavior it's just going to be inefficient so i just wake up a thread that would just wake up reacquire the lock read the condition again if it's actually in a while it just reads a false again and then goes back to sleep and then it's kind of just a waste of time but doesn't actually affect my correctness if i put a signal here if i put in however many signals i want if i just want to just throw some signals around for fun i am able to but like everything it'll just make your program go slower and it will just make things harder to read all right whoo all good with that all right so other fun things so we got that we got that so yeah like i said they whoops so they do serve a similar condition to center fours you can like center fours are basically a special case of condition variables where it's just protecting a value like it's just protecting an integer and it'll essentially call weight when the value is zero and then whenever a thread posts it would just signal so it would wake up a thread and then it would only increment if the current value is zero turns out you if you're like a computer science nerd that likes theory you can always implement one in terms of the other however the question of whether or not you can actually read your code after that is very questionable so in general if you have a like a complex condition that's more complicated than just an integer use condition variables it's going to be a lot lot easier to read so we'll have an example in the next one but once you actually like care about performance and not just correctness you have to consider like locking granularity so that's the extent of your locks or how big that critical section is because well you need locks to prevent data races but if you have a big chunk of code and it's protected by a lock only one thread can execute a time then suddenly your program is slow and if it's really really big you shouldn't have used a lock anyways just run with just a single core and it'll probably be faster so for like an example if i want to paralyze a hash table which you will do well should i just lock the whole table and then i don't have any data races well might be slow should i do something else should i only lock like individual elements should i think about something better to do might be something to consider if we actually care about performance so things to consider in real programs whenever you try to make your program not have data races there's going to be some overhead of using locks so if like the operation you're trying to protect is really really small like what we saw with just incrementing a integer well might be faster like take some time to do the locks has a queue take some time to do the unlock so that overhead might be more than you actually care about than just running it on a single thread might also have contention so you might also have like a lot of threads all fighting over the same lock in which case well they're going to be asleep a lot of the time and you're not going to have any parallelism and then the third one which we haven't got into yet which is the funnest subject is deadlocks so and like i said with overheads you have to also allocate some memory for a mutex so if you have billions of them even if they're only one byte going to take up a lot of space take some time to it initialize them whoops and destroy them takes time to lock takes time to unlock and the more locks you have the worse it's going to be but the most important thing and the thing that will definitely come up on the exam and you will definitely hit probably in your actual programs as soon as you have more than one mutex you might have the chance of having a deadlock so a deadlock is basically you'll get into a situation where a thread can no longer make progress anymore at least one thread sometime all threads and then your program looks like it just grinds to a halt but you could consider just like a single thread deadlocks if it can't make any progress anymore so there are some conditions you need to have for a deadlock so the first is mutual exclusion which we're just going to argue about deadlocks for mutexes so of course we have mutual exclusion second condition is hold and wait so you have to have acquired a lock and you are waiting for a different lock the third is no preemption so you just can't randomly take locks away from threads and then the fourth is a circular weight so you're waiting for a lock held by another process or another thread so these all have to be true in order to have a deadlock and it turns out we can break either the second condition or the fourth condition fairly easily and prevent deadlocks but first just to illustrate the example this is an example of a deadlock so we have two processors could be threads if we're sharing the lock between processes could be processes but say we have two threads the code that we write in one is to acquire lock one get lock two and then we release lock one and release lock or lock two and then lock one and the other thread we'd lock lock two then lock lock one and then release lock one release lock two so whoops typo so could get in the situation where we have thread one executes first it locks lock one and then we context switch over to thread two it locks lock two and now neither thread can make any progress so thread one has locked one thread two has locked two and if we chose thread what two to execute well it's waiting for lock one which is held by thread one if we chose thread one to execute it wants to get locked two that's held by thread two neither can make any progress and we are now screwed so that clear with everyone all right we're so in order to actually prevent them so to prevent a deadlock I can either ensure order so if I always acquire locks in the same order I don't have that circular weight problem so say lock one and lock two we're like global variables or something like that so they their address has didn't change well if I just have acquire lock one and then acquire lock two in that order doesn't matter I won't get into any bad situation so if I have two threads running this well thread one could go ahead and acquire lock one and then if we switch to thread two well the first thing it's going to do is try and acquire lock one not going to be able to get it so it's going to have to wait until thread one goes back executes acquires lock two and we can go ahead run our protected code and then we're all good so this code will not deadlock because well one way to think about is that you only get locked two if you already have lock one so the other way to do it is using that try lock that we haven't used before so the second condition is like hold and wait so if I don't know like if I can't ensure a absolute order of locks well this is another thing I can try so I could lock the first lock so I have lock one now and then I can do a try lock of lock two and then this will tell me whether or not I got the lock or not so it will return zero if I successfully got the lock otherwise it will return some value that is not zero if I did not get the lock well then I can make the assumption that another thread has acquired lock one before me and it might be waiting on like another thread has locked two so it might be waiting on lock one as well so what I'll do is I will just unlock lock one so I just give it up so I had lock one but I and then I try to get lock two if I don't get it I give up lock one and then I can wait for a bit I can yield I can do whatever I want here and then after a bit of time I can try and reacquire lock one again and then try to get lock two again if eventually I actually do acquire both locks then I can run this code knowing I have both locks acquired and then I can go ahead and unlock them both while I am done so questions about that yay now we saw a situation we can use try lock for so now we can wrap up if we don't have any other questions so now we explored some more advanced locking we'll have a larger example on Tuesday so do not forget that one because it will look somewhat like the lab and code you will actually write in your life so what we did today we have now another tool to ensure some type of order so that would be condition variables they're going to be a bunch a lot clearer for like more complex condition signaling but they're basically just a Q and then in terms of real world performance the locking granularity so how much code you have between lock and unlock is going to affect the performance of your program so you might want to have more locks that cover smaller pieces of code than just one lock that covers everything so if we've used python and thought it was really really slow when you're using threads with python it's basically because they use one gigantic lock to lock all of python because they didn't want to argue about every single thing and like make thousands of locks although recently they've changed it where python actually works with threads much better than it did before but if we have multiple locks then we might have dead locking situations and you have to prevent them so with that just remember falling for you we're all in this together