 All right, welcome back to Operating Systems. So I wish you well in your hell week. So hopefully it doesn't go too bad. This week should be relatively late. We'll have a review session on Thursday and then should be fine. All right, so today we get to talk more about locking. So in some languages there are something called a monitor and now you can understand how they actually work. So with object-oriented programming, developers thought that mutexes were too difficult to use. So what they do is allow you to mark a method as monitor and then let the compiler handle locking for you and any methods that are monitored, the compiler will make sure that an object can only have one thread active at a time, throw out all of its monitored methods. So if you are changing some variable in a monitored method, you would not have a data race while you are interacting between multiple threads because, well, only one monitored method is allowed to execute at a given time. Basically, this just means the compiler will create a mutex for you in the object and the compiler will insert a call to lock and unlock for you. So if you've used Java, if you've had that unfortunate experience, there is a synchronized keyword in Java and that is in fact a monitor. So if you were to create, I don't know, a class to represent someone's bank account and next lecture we will run our own bank, so we'll get to have more fun with this because this is probably, yeah, the Java solution is not that great, but anyways, if you were to operate a or represent a bank account, you might want to keep track of the current balance of that account and you might have a few methods like deposit, so I can deposit some amount that would increase the balance and I can withdraw some amount that decreases the balance. And if I want to make sure that I can safely use these methods through multiple threads and I don't want any data races on this balance variable because, well, we know that this plus or plus equals will do a load than a store, so we don't want a data race, mark them both as synchronized and essentially the compiler will transform them for you, so it will create a monitor which will just be a mutex as another field of the account and then at the beginning of the deposit method, it will insert a call to lock and before it returns, it will insert a call to unlock for you, so that whole method just becomes one big critical section and similarly, it would do that in withdrawal. It might be some nice thing here where if you have multiple return statements, it'll make sure that it unlocks it before any return statement, which if you do that in C, might get a bit hairy, but that's basically what a monitor is, so that is an option and if you have to use Java now, while everything we learned in this course, thankfully it applies to every single language you use because if they use threads and you know how threads work, well, then you know how data races work and how to fix them. So there are, as in the last synchronization primitive we will talk about is something called condition variables and they serve a similar function to centerfors, except they're a bit more explicit, so basically the way I like to think of condition variables are they're basically your own custom queue of threads, so it's in the pthread library, there's this init and destroy and there is a signal method which will wake up a current one thread that is currently waiting in a queue, broadcast that will wake up every thread and then this wait, so this is where the meat and potatoes of the condition variable comes in, so this wait function will put the current thread to sleep, put it on the queue and then you'll also notice that it also needs mutex, well, you need to have that mutex acquired before calling wait and it will make sure that it will atomically unlock it before going to sleep so that if it resumes again it can acquire the lock and it'll make sense whenever we see an example, so there's also a timed wait, so wait by default will just put this thread, put this thread to sleep, throw it in the queue and it's blocked, it can't execute until another thread calls signal and wakes it up or broadcast and wakes it up, if you don't want to depend on another thread calling signal or broadcast that's where this time wait comes into play, so this will automatically unblock you after a set specified amount of time, so again the wait function puts this thread to sleep, blocks it, adds it to the queue, you can think of this cond as just a queue and signal will wake up one thread that is currently waiting in the queue, broadcast wakes up all threads, so from looking at that the wait function has to be paired with the mutex, so anytime you call wait you must have that mutex locked, so it must be acquired, doesn't matter, you don't have to hold it when you do a call to signal or broadcast, why might you need to do that? Well you might think wait needs to safely add itself to the queue so we don't need any data races so we need mutual exclusion anyways and it also needs that reference to that mutex because before it goes to sleep it needs to unlock that mutex otherwise no other thread can make any progress that might hopefully change that variable whatever it's looking at and signal it to finally wake it up, so whatever it wakes up it will try and reacquire the mutex so that it is locked before and after, so you don't have to worry about it and it also you don't have to worry about data races or anything like that, so one mutex can protect multiple condition variables and for this course we're just kind of learning about it, other courses you'll be practicing it more but for this we'll just consider calls to wait and signal, so wait puts the current thread to sleep, unlocks the mutex and signal wakes up a single thread, so remember wait doesn't contain data races so you don't have to worry about the queue having a data race or anything like that, so wait performs three steps and they all happen atomically so you don't have to worry about them, so the thread calling wait the three steps it does is it adds itself to the queue for the condition variable then it unlocks the mutex and then it blocks so it can no longer be scheduled to run puts itself to sleep, then some other thread needs to call signal to wake up the thread that called wait and whenever the thread that called wait unblocks and starts executing then well it can now run and the first thing it will do before returning from the wait is try to lock the mutex again and wait will only return whenever it has the mutex, so before the call you have a locked mutex and after the call you have a locked mutex, so if we remember our producer consumer example that we used two center fours for last time well we can rewrite that using condition variables so I can I need a mutex to protect everything make sure I don't have any data races and I'll use them for both queues so my condition variable is something you create so in this case for my producer consumer my condition variable you can think of as just the number of filled slots so that is the number I care about in order to know whether or not I have to put a producer to sleep or a consumer to sleep so I'll have a queue that is waiting for a filled slot and then a queue that is waiting for an empty slot so we can change the code that in the producer well first we acquire the lock and then we can read the number of filled slots don't have any data races because anytime we access n filled we have the mutex acquired so only one thread can be in there at a time so here was our condition last time so we check if the number of filled slots is equal to capital N or the buffer size in which case I need a pursuit consumer to actually read some data before I can start producing more data so if all the slots are filled which is what this means I will put this current thread to sleep in this has empty queue so this thread will go to sleep and then unlock the mutex and then another thread can go ahead and execute hopefully it's a consumer thread so if a consumer thread executes well it would be able to acquire the mutex now it could check the number of filled slots and here the consumer remember the property we want that to have is that it won't try and read any slot that doesn't have any data in it so I can check the number of filled slots see if there are zero filled slots if there are then it can put the consumer thread to sleep and wait because it's waiting on a filled slot and then well let's say the consumer actually didn't put itself to sleep then it would just empty a slot decrement the number of filled slots so decrease it from two to one or one to zero whatever have you and then it would wake up since it just empty the slot it would wake up a single thread that is in the has empty queue which would be a producer so it could wake up a producer it could consume executing from weight but before that it would have to acquire the mutex so here could acquire the mutex check the number of filled slots now hopefully it's not the size of the array or buffer and then well it could fill a slot increase the number of filled slots so it just added it just filled a slot and then it would wake up any consumer threads waiting for a filled slot and then finally unlock it so same thing we had but is this more clear than the center four one better worse about the same somewhat better so it's a bit easier to argue about because remember as well you could have a bad bug in your center for example because you set the initial value wrong so this doesn't have an initial value I think this is more clear but at least for the case of producer consumer either one is fine it's just whatever you think is more clear so let's see if we know if we can exercise our knowledge about data races and fun things happening so in this code there is an issue I will give you a few minutes to figure out what I screwed up in this code or what bad situation that can happen so one thing that should definitely be true is since we're using a condition variable and say initially the value of the condition is equal to false the property I want that is if thread one is here then the condition must be true and it can't change so if thread one makes it past that while loop that just checks if the condition is false it should be the case that if thread one makes it past the while loop that condition should always be true so that condition not being true in this example is not the bad thing that can happen as a hint so a bad thing that could happen here is that thread one can put itself to sleep and never wake up so it's kind of an example that lost wake up problem we had before so let's give that a think and see if we have any bad context switching between thread one and thread two that causes thread one to essentially be deadlocked and never wake up again and we'll assume that you know thread one starts executing here and thread two starts executing here of course okay so not everyone may have heard that but that sounds reasonable so let's go over that quick so we will have thread one start executing it reaches the mutex lock currently no thread holds the mutex so it could pass the lock call and then I like writing to the side what the current what thread currently holds the mutex so right now thread one has the mutex so it could go ahead check this condition it's currently false so it would go inside the while loop and then it could context switch right here so we context switch right here and we start executing thread two we start executing thread two well it would change that variable from true to false nothing stopping it from doing that so now it's true now it would signal and then signal would cause it to wake up the queue is just going to be like a FIFO order of threads that are currently asleep so if I check the queue and I wake up the first thing on it my queue is currently empty so I don't wake up any thread so that signal did absolutely nothing and now we can context switch back over to thread one thread one would hit this weight it would put itself in the queue and then unlock the mutex and it can't execute anymore so it's just sitting there forever help so how would I fix this yeah so I should just put a mutex in thread two so before remember data races we want to make sure that we are making sure we have mutual exclusion even if we are just reading as long as one other thread is writing we have a data race and in this case well thread two is even writing so we definitely want to protect that so to fix it we would put a lock here to the mutex for the unlock it doesn't matter if I put it before or after the signal because whenever I wake up thread one it will try and require the lock anyways so it doesn't really matter so I can put it right here and then see if that bad thing can happen again so if I reset everything the condition is now back to false and there is no threads in the queue so now this time if thread one starts executing it acquires the mutex so thread one has the mutex check the condition it is currently false so it would go here now if we had the same situation where at this time we context switch to thread two well it would try and execute the lock of the mutex and it can't pass because thread one has it yeah no very good all right so thread two can't make any progress at this point because we have a call to lock and the mutex is currently locked by thread one so only thread one can make any progress anymore so now thread one would hit this weight as part of the weight it would add thread one to the queue and then also unlock the mutex so now it is asleep needs to wait for someone to signal thankfully thread two is going to go ahead lock the mutex so it can make some progress now so that is why it's really important that the weight actually has a reference to the mutex so now thread two goes ahead locks the mutex well so any thread that can execute anyways because thread one is asleep could change the condition from false to true and then unlock the mutex and then after it unlocks the mutex well it does a signal so that will wake up whatever the first thread is in that condition variable so now we would remove t1 from here as part of returning from weight it will try it will reacquire the mutex so currently no other thread has the mutex so thread one can get the mutex and then continue executing from weight in this case well it will go back up check the condition again the condition is currently true which is great and then I execute the next line and the condition is definitely true there so yay we have fixed it so anyone want to tell me if anything bad will happen if I do something silly like what about if I was like wow checking that variable again that was useless so I will just change this to a diff is that a cromulant thing to do yeah in this case not only change it from true to false once or sorry from false to true once so and if in this case mashley is fine in general though don't do that so let's see in the more general case so here I properly locked everything I just introduced a thread three so I introduced a thread three all it does is update the condition to false so it doesn't have a signal because the condition that I'd be waiting on is changing it to true so if I change it to false well guess what I don't have to wake it up because now it's false so in this case if I change that while or that yeah that while to an if let's see if we can make anything bad happen now so remember again at this highlighted line that condition better be true if I execute this line so is there a way given the code that is written here with three threads is there a way to execute that line while the condition is false yeah you could you can wake it up and it doesn't have to execute whenever you wake it up all right so let's see if we can do this in excruciating detail so currently I'll say that the condition is equal I'll even make it read condition is equal to false the mutex is currently held by no one and my condition q currently has nothing in it so what thread should execute first in order for something bad to happen all right thread one goes first so it would hit this mutex lock so no thread has a mutex it would get one in oops thread one so now because thread one has the mutex thread two and thread three cannot make any progress because while they would try and lock the mutex it's acquired by thread one right now thread one's the only one that can make any progress right now so we don't have to worry about any context switching right now because any context switch not going to make any progress so now it can check the condition so if not condition well yeah the condition is currently false so I would go in here and then execute this weight as part of the weight it would add thread one to the q unlock the mutex and now thread one is asleep it is well just asleep for now so now what can happen thread two executes so thread two executes well no thread has the mutex so it can pass by and then it would have the mutex now we don't have to worry about any concurrency because no other thread can make any progress thread three stuck at that lock so thread two two keeps on trucking along updates the condition from false to true and then does an unlock so now it gives up the mutex so we want thread two to continue more yeah so does it actually matter at this case not really so let's do the harder case where it actually makes it to signal because that might be a bit more confusing so if it makes it to signal well then thread one is eligible to run now so now we are up to luck because if thread one wakes up now and then acquires the mutex it would go ahead the conditions currently true so that highlighted line would be executing while the condition is true so we'd be all good but there's nothing saying that thread one is going to start executing it's either it's up to thread one or thread three so we don't know in this case we can be unlucky and we can say that thread three executes so thread three executes it acquires a lock and then thread three now has a lock so now thread one cannot return from wait because remember as part of returning from wait after it wakes up again it needs to reacquire that mutex so right now thread three has the mutex so thread one can't execute so here thread three could come along change the condition to false what a jerk and then unlock the mutex so now currently no thread holds the mutex if we pick thread one to execute well it has to reacquire the mutex because it's returning from a wait so it would acquire thread one and because we didn't put a while loop there well guess what it just continues along and executes this line while the condition is false and we have screwed ourselves so don't be cute with condition variables put your condition in a while loop even if we have the case like before where it just looks like this eventually you will change your code and that will not be true so just always put a while because that will always work so the fix we need to do here is again change the if to a while because it is possible that before we start executing that another thread changes the condition to be false so any questions about that well we speed running today all right so condition variables they serve a very similar purpose to center fours in fact the center four is actually a special case of a condition variable center four just cares about a value well if you have a mutex around it you can make sure you don't have any data race on it and then all you do is in the post you would have you would increment it and then signal so you wake up a thread and then in the decrement while you check the value if it's currently zero then you put yourself to sleep so you wait and then when you check the condition again make sure it's greater than zero if it is well then you can decrement it and then keep on going so you can implement one in terms of the other and you can also switch it the other way around but that's like a theoretical computer science thing because it gets really really really really messy and if you are writing code you should aim for clarity so for complex conditions use condition variables if it's just a simple number and the center four suffices use the center four all right so now we get to talk about walking granularity which is the extent of your locks and this is what will matter once you care about performance so if you lock a large section of your code like in that monitor example where just an entire function has a lock at the beginning and then unlock at the end well guess what you have turned your program essentially serial for the all those operations so if you have used python and use threads in python at least for old versions of python you might realize that they are dog slow well that is because python has something called a global interpreter lock so to solve data races what they did is just put one giant mutex around all of python and that's why it's slow it's been worked lately to actually remove that and add finer grain locks so it can actually go faster so it's always kind of a trade-off because having a lock and unlock well you don't have any data races anymore but your performance is also serial so why do you even have threads so ideally you divide the locks anything that can be independent where they don't interfere with another that they can run in parallel while instead of just having one lock you can have multiple locks and then you can lock a smaller section of code because remember that critical section is essentially running in serial so you want to make that as small as possible and you may want to use multiple locks to do that so for example if you were to paralyze a hash table hot cough cough hack hack guess what that's lab five so you get to paralyze a hash table so when you are trying to paralyze something well you don't want any data races and you also have to consider a few things so the overhead so how many locks you have lock calls aren't free there's another thing to consider called contention and that is basically how many threads are fighting over a lock at any given time so if you have a lot of threads fighting over a lock at the same time well if you've a thousand threads fighting over a lock well guess what only one gets to make progress 999 do not probably not something you want so you want to make sure that for performance you don't have that much contention and the last thing to consider is the fun one called deadlocks and that is a situation in which none of your threads can make progress anymore and your program just kind of looks like it's an infinite loop but your cpu usage is like 0% so it just kind of looks like it's just kind of dead so the locking overhead so this isn't too big of a deal but you do still have to consider it so locks take up some memory even if it's only a few bytes if you have a billion locks even if they take up one byte still a lot of room so you probably don't want to do that and we probably know they take up more because our mutex implementation we saw in the previous lectures was like two integers and a pointer to a queue and then you'd have some dynamic memory with that queue that would kind of depend on the amount of contention you have with how big it is initializing and destroying them takes some cpu time as well although it should be insignificant but again if you create a billion or a trillion locks it will not be insignificant and then there's time to acquire or lock and release and unlock the mutex so basic thing here is the more locks you have the greater this cost is going to be so the fun one is deadlocks so the more locks you have as well the more you have to worry about deadlocks so there are four conditions for deadlocking this is important and all four of them have to hold in order for a deadlock to happen so the first is mutual exclusion so if we're just arguing about mutexes well of course we have mutual exclusion second is hold and wait so that means you have to have a lock and try to acquire another the third is no preemption so that means i just can't take simple locks away so if one thread is holding a lock and trying to acquire another well i can't just steal the lock from that thread or force them to give it up so the last fourth condition is circular weight so you also have to wait for a lock that is held by another process or thread so here is a simple deadlock example so you have two threads in the first thread things have to execute in order so the first thing you do is try and get lock one or lock lock one and then you try and lock lock two and then in thread two you lock or get lock two and then lock lock one so a bad interleaving that could happen is thread one could start executing acquire lock one now thread one has lock one then we context switch over to thread two thread two locks lock two and now neither thread can make progress so now if thread two tries to execute well it tries to get lock one it can't get it because thread one has it and then if we try context switch over to thread one it would try and get lock two but it can't get lock two because thread two has it so neither thread can make progress and now we have a deadlock so does that make sense so neither thread can make progress anymore so easiest way to prevent a deadlock is to always acquire locks in the exact same order so if you can order locks somehow based off an id maybe their memory location or something like that as long as you have some type of ordering that is unique and doesn't change if you acquire locks in the exact same order every time you will not have a deadlock so that will break the condition of circular weight so that will break condition four because you essentially aren't allowed to acquire a lock that is lower than the number you currently have so in this case if we changed around the order so that we always acquire lock one and then lock two that means that if I have lock one then I can get lock two it's not the case that someone else has locked two and is once locked one so this will also not deadlock so I can only get lock two if I have lock one so if this is not possible or for some reason you decided to be clever or cute or you want to use try lock this is how you another way to prevent a deadlock using try lock so this will break that hold and wait condition so what this code does is well it tries to acquire lock one and then in this while loop it will try to acquire lock two and it does a try lock which remember is non-blocking and it will tell you whether or not you have acquired the lock so here I would try and get lock two and then if I don't acquire it I fall into the while loop and then I give up lock one so I give up lock one and then on this line I do not have any locks I do not have lock one anymore and I do not have lock two so I am currently not holding any locks so I could wait put myself to sleep yield do something hopefully I resolve the issue and then if I want to try again I reacquire lock one and then after this line I know I have lock one it would go up trying to acquire lock two again if I fail I give it up again but if I am successful well I can execute my protected code because now at this line I have I know I have lock one and lock two so this also will not deadlock so any questions about that all right speed running today so we're done so we export some more advanced locking today so we have another tool to ensure order called condition variables they are generally clearer the more complex your condition is but again if center fours are more readable use center fours so there's no hard and fast rule your reading code so it depends whatever you find more clear so once you get into locking and preventing data race the locking granularity is going to matter a lot so you want to minimize the length of the critical section and you also don't want too many locks you also don't want too few you'll probably find a Goldilocks number so you'll only find that through kind of experimentation which guess what you get to figure that out in lab five so that'll be fun and if you use more than lock than one lock which well as you might imagine you will probably have to use more than lock one lock in lab five in order to get good performance although you can see what happens well part of it is seeing what happens when you only have a single lock but once you have multiple then you might get into the case of deadlocks case of lab five hopefully you make a good decision and that is not possible but you still need to argue about it so easiest way to prevent deadlocks is always acquire locks in the same order if you need to so with that just remember pulling for you we're all in this together