 All righty, welcome back to operating system. So where we left off last time Which was yesterday we implemented a lock and it was crap didn't work So today we get to fix it So as an aside in some courses you can actually implement locks with minimal hardware So your hardware requirements can be as low as loads and stores are atomic and your CPU executes instructions in order and Some courses they'll let they'll walk you through Peterson's algorithm or lampards bakery algorithm Basically, this tries to simulate, you know taking a number and then they call numbers in order and that's the order you get Served it's kind of like a lock problem is that these are all computer Sciencey and they don't scale very well in the real world. No one actually uses this as a Proper lock implementation, so I will show you the proper way. We actually implement locks in today's day and age So let's assume we have this magical atomic function It is called compare and swap and again atomic means it either happens all at once or it doesn't happen There's no in-between so this magical Atomics function takes an address to a pointer and then two values an old value and new value and what it will do is it will try and Read the current value if it matches Whatever this old parameter is it will change that value to be new and it will do that Atomically and this compare and swap oops will always return the original value at that address so you're guaranteed if I use it like this if I do a compare and swap and Use the pointer to that integer and then give it the old value as 0 and the new value as 1 Well, this function will only return 0 if it if it successfully changed the 0 to a 1 Otherwise if the current value is 1 it will just return 1 So this way we can implement a lock So if the initial value is 0 if one thread calls lock Well, it would do a compare and swap on that Its current value is 0 So it would change the 0 to a 1 Atomically and then return 0 since it returns 0. Well while 0 that is other words for false So it would fall out of this while loop and continue so that one thread would make it through that lock call Now if another thread tries to call lock well the current value of that L is currently 1 So compare and swap will return 1 so 1 is otherwise true So this while loop will just keep on going over and over again And this way we only allow one thread to make it past lock at any one given time assuming that this Compare and swap function is atomic So everyone agree with that So this is an implementation of a lock. It works only one thread can make it past lock that lock function at any given time Yeah, this works. So This is now safe Right, so it works as a lock only one thread makes it through at a time but it still has that problem while if we only have one core on our machine and Another thread has locked so the current value is 1 well The other thread is just going to execute this while loop over and over and over and over again And it's not going to make any progress. So we're just wasting some time So this is a proper implementation of a lock. It just has that problem where it has that busy weight So that lock is a valid implementation And it's given a special name because it's actually kind of useful it it's called a spin lock Why is it called a spin lock? Well, they call it that because of that while loop that constantly keeps on retrying it over and over again and This compare and swap function that I used here That is the atomic instruction that supported on all your CPUs on x86 They happen to call it comp exchange or Why I don't know they want to shorten it for some reason. What's it supposed to stand for well It's supposed to stand for compare and exchange compare and exchange compare and swap mean the same thing So again, this is an implementation of a lock it works, but still has that busy weight problem So again like if you have a unit processor system So a system with only a single core on it If you can't get the lock you should be smart enough to realize that while you shouldn't retry or anything Just yield put yourself to the back of the queue. Let the kernel schedule another process or another thread Hopefully that one calls unlock and then you can proceed so if you have however, have a Machine with multiple cores on it you might actually want to use a spin lock because every time you read While some other thread because it could have executed in parallel might actually Unlock that mutex so it might be worthwhile to just try again Depending because as we'll see the implementation of a mutex is much more complicated than this This is essentially one line for lock one line for unlock. That's it. Can't really get much simpler than that So any questions? Nope, we're good. We got a lock. It is a spin lock. All right so Here is what would happen if we tried to make it a bit more efficient. So We will change that while loop to do wall compare and swap So if compare and swap returns one that means some other thread has the lock So we should just yield So if we do this well now we have a new issue and it is something called a thundering herd because You might be in a situation where one thread has a lock then eight threads are trying to acquire that lock and Essentially all those eight threads will just fight for each other as soon as you unlock that so That's kind of a waste of time So if you have eight threads all fighting for the same lock and it's a mutually exclusive lock You know that only one thread is going to be able to get it any way So why would you let all eight threads fight for it? Only one thread should try. Yep Yeah, so the problem is Exactly exactly that so if I have eight threads well, they all yield and they'll come back. So say another thread unlocks this So we have eight threads that are yielding What one thread unlocks a lock and then all eight threads are going to check So the first one is going to acquire the lock and then pass But all the other threads are just going to check again Seven of them are going to fail and you already know seven of them are going to fail your wasting time Waking them up and having them yield again, right? Yeah, so say you have 10,000 threads Yeah, so if I have 10,000 threads, I don't want Like if I unlock it, I only know one of those 10,000 is going to get the lock. I don't want all 9999 of them to just try again. They're gonna fail. They'll get a yield again. Why bother? So we want to fix that problem so The other problem with the thundering herd is well They'll all try again and you're not guaranteed which one of those threads is actually going to acquire that lock So if eight threads were waiting for that lock Well, all eight of those threads should be able to acquire a lock before some new thread tries to acquire a lock You want to be fair maybe form an orderly line so you can reason about it So that's why we say FIFO is okay. That's just first in first out. That's a normal cue That's you know going to McDonald's or something like that So here is our solution. So instead of that We can if we fail to acquire the lock will add some pseudo code here We'll just add ourselves to a weight cue for this specific lock And then we will put ourselves to sleep sleep is a bit different than yield So sleep will just block this thread from executing until another thread calls wake up and wakes it back up again So it will block until another thread wakes it up Assume this is you. I don't know during finals week. You go to sleep. Someone needs to wake you up So probably a bad example, but anyways, so in the unlock How we change that is while we change the value from a one to a zero and then we could check If there is a thread in the weight cue so say there's 10,000 we just wake up whatever is at the beginning. That's it so This is a good try But there's two issues from this because well, there's probably some data races here So there are two issues and we should be able to argue about them one is lost wake up so that means is there a situation where One thread has the lock already and another thread tries to acquire it Is there a situation where it will just put itself to sleep and then it won't get woken up again So essentially it will be in a coma. Let's say so just goes to sleep Nothing ever wakes it up. Is that a possible situation and another situation that might happen is Creatively named the wrong thread gets a lock, which means is it possible that there is a thread waiting for a lock and It should be next in line But is it possible that another thread can acquire that lock before the thread that's patiently waiting gets it so let's think of this and then see if you can come up with a Order of threads executing on a single core So just by context switching just by having concurrency Is there a situation where I will have a lost wake up or some other thread will steal a lock from a thread that is currently in the waiting queue and If we can come up with a situation that means that our implementation is incorrect No matter how unlikely those context switches may be if it is at least feasibly possible It means that your program has an error in it Even if you run it 10,000 times and it seems to work. Well, you're not guaranteed it will always work. Yep If you keep getting like No, so if a thread So say thread one has a lock and thread two tries to call lock It will check the value the value will be one So it will go into the while loop and then it adds itself to the queue and then puts itself to sleep Yeah, so unless something wakes it up No, unless something wakes it up. So sleep just blocks until someone wakes you up Yep set L to the first thing in the queue So if I want to hand the lock to the next thread, I would Need to change my implementation of it. Keep that in mind because we'll do that for a better implementation But given how it's written out, is there a situation where I will get, you know, the lost wake up or the wrong thread kind of butts ahead So here let us go there. So say we have thread one that has a lock So the lock is currently one. So you said thread one starts executing and Changes the lock from a One to a zero Okay You can contact switch so we could So right now thread two is just calling lock. It's not blocked at all So in this situation, well if thread two starts executing now Well, it would call compare and swap The current value is zero So it would change it from zero to one and then it passes. So it makes it through a lock, which is fine So that that seems to work. That's not a problem. So let's reset this Nope, so thread two is just calling lock. It's not in the queue yet. Okay, so let's first think about this lost wake-up So is it possible? So if thread one has the lock and it's calling unlock and thread two is calling lock is it possible that thread two will put itself to sleep and Thread one will never call wake up on it. Yep. So Here we'll just assume concurrency so nothing happens at the same time either thread two executes or thread one executes So which one do you want to run first? Doesn't work. Yeah Yeah, so to explain that again, let's see so first initially we could have thread two executes if Thread two executes while currently thread one has a lock the value of it is one So thread two would execute compare and swap What it would it return? So it always returns the value pointed to it So it would turn one because it didn't change it from a zero to a one because thread one has the lock So it would go into the while loop So at this point we could contact switch. So now we could contact switch back over to thread one thread one can set the value from one back to zero and Then if thread one continues executing it will check. Is there any threads in the weight queue? Is there currently any threads in the weight queue? No, so I'm done my job here. It's done. So thread two just finishes the unlock Now if I come back to thread two Well, now it would add itself to the weight queue. So now it's part of the weight queue now It puts itself to sleep Yeah Yeah, so compare when thread two calls compare and swap because Here we said that thread one has the lock the current value of the lock is one and thread one changed it So it should not be able to acquire it until thread one calls unlock All right. Yep When does what so thread two? Got into the weight queue in this situation Right after thread one finished unlocking it So here we can back up again. So in this situation If we reset everything again We're essentially in a situation where thread two executed first It called compare and swap it failed because thread one has the lock. So it would Want to start executing inside the while loop now we could contact switch before it adds itself to the queue So we contact switch back over to thread one it unlocks the lock it would check the queue there's nothing in it because thread two was about to add itself to the queue, but we contact switched away from it because unlucky for him and Then we finished and then if we come back to thread two thread two will just resume So we'll add itself to the queue put itself to sleep and there's no one to wake it up It's going to miss its final exam and fail the course Which our final exam is 9 30 if you haven't seen that yet. So yay Yeah, it's scheduled for December 8th at 9 30 9 30 a.m. Yeah, so I will be up at like 4 a.m. That day. Yay Yep So you yeah, you can't make oh like the whole body of the while loop So yeah, I guess I could solve it if I made the whole thing like I'd have to make that whole function atomic that whole lock atomic Right, so if I just like put a mutex around this I still have that problem because it could not acquire the mutex until Right, so I still have a slight issue right What about this situation where the wrong thread gets the lock? So Let's change our scenario a little bit So we can start off where thread one Has lock it's calling unlock. So if it starts executing it would start executing here Let's assume thread two is in the So thread two is in the weight queue It should be the next thread that gets the lock as soon as thread one unlocks it, right? So let's assume we have a thread three calls Lock so Is there an ordering by just switching between these threads such that Thread three will acquire the lock before thread two gets it. We all agree with that that one ended fast so Let's go over that real quick if you didn't hear it. So what we could do is initially Thread one has the lock. So the value of it is one We could start executing thread one So thread one could change the value of the lock from a one to a zero and Then we could get unlucky and then context switch immediately to thread three In which case if we context switch now immediately to thread three, well, it will do this compare and swap What's the value of the lock? Zero so it will change so compare and swap will change the value from a zero to a one To indicate that this thread has the lock. So thread three now has a lock So it has the lock if threat if for some reason if thread at this point thread one continued executing Well, it would check the queue. Oh, there's a thread in the queue. Okay, great I will wake up thread two and then I will finish So when thread two wakes up It would resume right after the sleep So it would go back to the while loop and then try this compare and swap again and then oh, no the value is already one because Thread three came in swoop my lock damn thread three So it would see that the value is now one Then add itself back to the weight queue sad day and then put itself back to sleep again So we so everyone agree that that is also an issue thread Three butted ahead in line thread two was patiently waiting there in the weight queue like a good thread I don't know why I'm persona-fying a thread But anyways, it was waiting there thread three came in butted ahead Just rude so any questions with that So let us solve these issues So in the slides there's an example for lost wake up Which is what we went over and then also an example for the wrong thread getting the lab Although or thread although I swift swap the names of thread one and thread two Doesn't matter as long as you are consistent So here is our fix and it looks ugly looking at this just looks like a Big block of code, but there is a trick to reading this So let's go here So the trick to reading this is well before in our first attempt at doing this We essentially had a lock so the value we were using as our mutex and We also were maintaining a queue. So the new thing here is this guard So this guard we are going to use that as a spin lock. So remember Way back here At the beginning we said that this was a valid implementation of a lock. It just kind of Spun it would try over and over again, but this worked to ensure mutual exclusion So if we switch back to the big ugly thing Well, this guard we are using as a spin lock and we will only use that spin lock within this lock and unlock So this big ugly thing Well, if we wanted to rewrite that as something that's more manageable This is essentially a lock of guard Then if we went through if we set guard to zero Well, that's essentially the same thing as an unlock To guard So remember everything between the lock and the unlock That's the critical section only one thread can do that at a time And we have an if branch here We want to make sure that whenever we lock it lock the spin lock we eventually unlock it That's why we have an unlock in each of the branches of this if statement Now if I go back and read this unlock well, the first thing it does as well is It will try to lock the guard and then at the end This line is the same as unlocking the guard So that this is kind of like your idea where I should make the body of that while Atomic essentially right So let's see if we fixed this lost wake-up example So lost wake-up what happened was well while thread one was trying to unlock the mutex It was done halfway and then thread two woke up or sorry. We context switch to thread two After it read it knew that it was already locked and that's how we got into that bad situation So now in this case We would have thread one and thread two One of those goes first and because the first thing it does is lock the guard as soon as one of them goes through The other you can't context switch to the other thread. I mean you can but it's not going to pass the lock So what would happen? Well thread one could start executing it could lock the guard So it could acquire the spin lock then it would check If there's anything on the queue if the queue is empty All I would do is change the lock. So the lock is our original value It would just set it to zero So it would change it from a one to a zero and then just finish Otherwise if the queue is not empty, this is your idea here. So that was a good idea I will just transfer the mutex to the next thread So all I have to do is just wake up the next thread if I don't have any data racers or anything the value of the lock is still One and I just woke it up. So it's going to resume somewhere It's going to resume after it fell asleep and just finish. It's going to immediately finish the lock so Thread one will eventually unlock that guard or unlock that spin lock and then thread two can execute So in this case, it could acquire the guard. It would check the value of the lock You'll notice here. We don't need compare and swap because well if we have a spin lock We don't have any data racers because only one thread can be doing this at a time So we don't need compare and swap anymore So we could check the value of the lock if it's zero well We change it to one because we now have acquired it then we can just unlock the guard and then We finish the lock call and keep on going otherwise Well, we could add ourselves to the weight queue if we Were if the current value of the lock is one Add ourselves to the weight queue and then unlock the guard and then put ourselves to sleep and Then if threat if we put ourselves to sleep then if thread one tried to execute After we put ourselves to sleep and put ourselves on the queue. Well, it could get the guard check the queue then Well, there's something in the queue. So it would go into this else Then it should just wake up thread two and then it should be able to resume with the lock So is there any issues with this or does this work? Data races are like a fun puzzle. You get to solve every time you program now Yeah, so a guard is a spin lock to ensure that Essentially to ensure that our lock and unlock functions Have mutual exclusion. So only one of them will execute at a given time Because our problem before was we started executing it We got like halfway through it then we swapped to another thread and that's where our issues came from So the idea is yeah, we use a lock so we can implement our lock. So we Get rid of data races while we're implementing our lock. Yeah Yeah, so this Lock so it should solve the thundering herd problem So whenever you unlock only one thread tries to wake up and acquire it And it should also solve the problem that we saw before was it should also solve lost wake up so that shouldn't happen anymore and The wrong thread getting the lock also should not happen anymore. So we want to solve all three of those so oh, so is this more efficient than a spin lock and Depends so if you have a single core machine and you want to be fair and all that so fairness costs time So if you want to be fair you have to do something like this So fair means you essentially acquire the lock in the same order you tried to like no other thread butts ahead of you Or anything like that. So spin locks don't guarantee that spin locks also have the thundering herd problem but Depends on your application if your critical section is really really small a spin lock might be fine But if your critical section is really big well if you have a spin lock It's gonna waste a lot of time just retrying over and over again So you might not want to use that in which case this would be way more efficient again situation if there's like a Thousand threads all sharing this one lock and all trying to lock it at the same time This is going to be a lot better because well, it's only gonna wake up one thread at a time The rest are just gonna go to sleep block. They can't execute. Yeah So you won't have a thundering herd at the guard each thread. So say one thread has the lock Then ten thousand come in to try and acquire it those ten thousand threads are all going to put themselves on the queue And then go to sleep Yeah, but if another thread tries to call lock while another one is in the middle of it It has to wait right just happens serially. So we don't have any data races But yeah, there's a lot of threads trying to call lock at the same time But the code in here doesn't do that much adding yourself to a queue. That's a few memory writes And that's it. We're just Doing a few memory writes. That's it That's probably a lot better than just having ten thousand threads each trying to read read that value over and over and over again Then just doing like a few memory writes once Yeah, so spin locks are totally appropriate if your critical section is very small So you might even want to use spin locks sometimes like the answer is it depends So sometimes spin locks might be better if your critical section small Sometimes mutexes might be better if your critical section is bigger Also depends on how many threads are trying to acquire it. So there's lots of trade-offs. That's why we have different different implementations All right, so is there a problem with this implementation a subtle one and I will give you a hint What is happening to the garden unlock? What do you mean? So after thread wake-up is done, it would just execute the next line and set it to zero If it continued, yeah wake up just unlocks that thread doesn't context switch. I Mean you may context switch, but if you context switch to it, I mean it just gets the lock That's it. It would just resume. It doesn't unlock it or anything Yeah No, so it's subtle and the hint is the arrows where the threads are currently at pretty much demonstrate it Yeah Yeah, so I could context switch. So let's say thread two thread one has the lock and thread two goes ahead and tries to get it while It would get the guard Put itself on the queue and then unlock and then it could context switch right now before it puts itself to sleep And then thread one could start executing right before it puts itself to sleep It would do compare it would get the guard and then check the queue Oh, there's something in the queue and then it would try and wake up thread two, which is currently not asleep So this is a very subtle data race, but it's also not that big of an issue because Well, you didn't have a data race with the queue. So thread one If you're a thread one right now, you actually know thread two the next thing it's going to do is put itself to sleep So if you wanted to fix this If you just detect that. Whoa, I'm trying to wake up a thread. That's not asleep yet All you have to do is just retry it until it eventually puts itself to sleep because you know It's going to so I only omitted that from the slide just because it makes it way uglier than it currently is but this actually has a very subtle issue, but You could detect it. So if you try and wake it up and it's not asleep yet Just yield and wait for it to go to sleep and then you can successfully wake it up Or if you're implementing this thread library yourself, you could go ahead You could cancel the next sleep of it or do whatever mechanism you want But you know what the next thread is going to do so if you wanted to In your lab four, I don't make you write thread sleep and thread wake up But if you really want to I took it out. But if you really want to you can So feel free to implement that yourself so last thing So remember What causes data race? So this is going back to that discussion of oh, should I use a spin lock? Should I use a mutex? Again, there's a series of trade-offs. So going on with that series of trade-offs remember the definition of a data race So two concurrent accesses to the same variable And at least one of them is a write which implies that we can have as many readers as we want and we don't have any data race so You wouldn't even if you only read You only you don't even need a mutex only reading is fine value doesn't change Nice and safe So you may be in the situation where you write an application You want to prevent data races But writes are fairly infrequent and you have a lot of threads trying to read so ideally You would want You know, there's eight threads You want all them to read in parallel all at the same time because that's safe And then as soon as one tries to write you want to prevent any of the other ones from writing If they're infrequent, that's pretty good Otherwise if you had that scenario right now all you have in your arsenal is a spin lock or a mutex In that situation Well, no matter if a thread is reading or writing only one thread can execute at any given time Probably really bad for performance so There is another type of lock called a read write lock. So This a lot. This is a special type of lock that has two As it kind of implies it has two types of locking calls There is a lock for writing and then a lock for reading And the difference between them is if you acquire the lock for writing You should be the only thread that has that lock and there should be no other threads trying to read that variable because again We don't want any data races The other side is well if a thread has the lock for reading While other threads should be able to acquire that lock for reading. They should be able to execute Concurrently or in parallel likely you only care about parallelism because of performance So ideally you want them all executing at the same time So this is how we would implement a read write lock So in our implementation for a read write lock We could keep track of the number of readers And we could just have a generic lock to represent our guard This could be a mutex or a spin lock. In this case, it's the same idea we had when we were implementing mutexes So we could just use a spin lock because we're only using it internally And then here is the lock that we're using to represent our mutual exclusion So it could just be something like a mutex So the easy implementation for it is Well in the write lock we want to guarantee that only one thread has the lock So we just use that mutex directly. So for a write lock Just lock the mutex and then for a write unlock Just unlock the mutex simple the more The harder situation is for the read call. So let's go through it So it would let's say it has a spin lock. So it acquires a spin lock called guard And the idea behind this guard is we want to prevent data races on this number of readers variable because we're going to keep track Essentially, we're going to keep track of how many threads are sharing the mutex for reading So we want to guarantee that there's no data races when we increment this Because we saw when we had that plus plus count example with multiple threads You don't the value can get inconsistent. We don't want data races So here we would increment the number of readers If initially we increment it from zero to a one It means we are the first thread that tried to acquire the lock for reading So we would just try to lock the mutex directly If we can lock the mutex directly We just unlock the guard because we're done modifying the number of readers and we can just continue Now if another thread tries to Tries to acquire the lock for reading It could acquire the guard Set the number of readers from one to two and then it wouldn't go in this if because they're all sharing that mutex And then it just unlocks the guard We could have any number of readers it could increment from two to four four to five Five to six do to do to do then eventually In the unlock what that will do is decrease Acquire the guard so we don't have any data races with the number of readers Decrement the number of readers because now this thread is not going to have that lock for reading Then it would check If it has decremented it from one to zero it means it's the last reader So it should give up the mutex And then it would give up the mutex and then unlock it. So this way If we had two threads that are called that have the lock open for reading While they are sharing that mutex if a writer comes along It's going to sit here at this lock call until eventually there are no more readers anymore And then the readers unlock that mutex and then the then we can have a single writer So any questions about that? Yeah, so it's kind of like I know if I gave you a key to a lecture hall or something like that It's like the rule the first person that comes in can use the key unlock it And then let as many people in as you want If you want that lecture room to be secure the rule is the last person that leaves like locks the door when they're done So it's the same kind of situation. So I'm essentially yeah, I have one mutex and either the readers One of the readers acquires it and then shares it with all the rest of them until no threads are reading anymore Or a single writer gets it But because I'm sharing that mutex between This I'm using the same mutex here Here and here between for the right Lock and the read lock. So I'll either have a bunch of readers or one writer And that's it Which is safe, right? So I can have as many threads reading as I want, but I should only have one thread writing Yeah, so if Say even one thread has the mutex like acquired the mutex as part of this read lock Well, if someone tries to call right lock, well, it will try and lock the mutex that The reader already has So it'll just sit there until it can acquire it eventually Yeah, yeah, so like that lock is just like a normal mutex. So it would just be blocked until eventually The reader reaches this and unlocks that mutex Yep Yeah, so there is a slight problem here because all the readers are essentially sharing that lock Well, if you have a lot of readers They might this might unlock might never happen right, so That is a problem for a realistic read write lock So this is just a simple implementation that has an issue The smart developers that actually implemented the proper read write lock has have thought of that so Ideally you would want it if you request the lock for reading No new readers or for writing Any new readers that come in should have to wait until you get it for writing Yeah, yeah in this case There is this lock it would be a mutex and it's shared between the readers and the writers Yeah guard could be a mutex in this case. It could also be a spin lock It's kind of small. So like it's only protecting an increment Which is pretty fast. So spin lock would probably be more appropriate But it could be a mutex if you really wanted it to be Yeah, well, so Sorry in a natural implementation, you can use a spin lock or mutex. They both have a lock So there's a p thread spin lock. There's a p thread mutex. There's also a p thread read write lock So luckily you don't have to implement that but this is basically the gist of it that gives you the idea So any questions about that? All right, we are good. So again Summaring up all this locking stuff. So We want to prevent data races. How do we prevent data races? Well, we put any code that might be susceptible to a data race in a critical section Such that only one thread can execute it at a time And we should know what data races to concurrent accesses to the same memory location With at least one of them being right So our mutex or spin locks are the most straightforward locks They only ensure one thread is in that critical section at any given time If we want to implement them ourselves So we had to implement a spin lock Then we use a spin lock to implement our mutex and then we could you know implement read write locks on top of mutexes and spin locks But to implement that spin lock you need some hardware support So we need that compare and swap instruction for You would need some if you're implementing a mutex You would need some kernel support for like wake-up notification for threads So you need to be able to put a thread to sleep and be able to wake it up So you would need to implement that so that's missing from your lab for but if you wanted to you could implement it and there's also The case that if we know we are reading a lot through multiple threads and writing infrequently Well, we should probably use something like a read write lock for performance because All the threads reading at once perfectly safe to do So just remember pulling for you. We're on this together