 All right, everybody. Let's get started. So this time, we're going to build on the ideas from last time about primitives for concurrent programming. So last time, Anthony introduced atomic instructions that are executed robustly without interference from other threads. And we saw atomic reads and writes last time. We're going to look at some more abstractions for synchronization that are more powerful than the ones you saw. We'll also look at ways of concurrent programming that are more abstract than manipulating individual primitives. All right. So we also looked at last time at using interrupts as a mechanism for controlling access to certain resources that you don't want to be shared. So the trouble with using interrupts, even in a single threaded implementation is that there are drastic restrictions on the machine. And you can't let users use them routinely because they'll interfere with the operating system's function, among other things. And there's even bigger problems on multiprocessors. The only reason interrupts work on a uniprocessor is because masking interrupts prevents other threads from running. When you turn interrupts off, the OS can't get control back and run a different thread. It's really stopping those other threads from running, which is what is protecting critical sections and critical resources. So it just doesn't work on a multiprocessor. If you disable interrupts, the running thread may not get interrupted, but the other threads running on other cores just keep running. So those can mess up your state. So an alternative is richer atomic instruction sequences. So, and we'll look at those next, and these basically perform both reads and writes in a single operation. The idea of that is it allows you to figure out if you have control of something, and in the same action, prevent somebody else getting control of it. So they require some amount of hardware support. They're not really implemented in single instructions on the machine because they require a lot of state to be synchronized together. So there is native support on x86 architectures in the MIPS machine that you're using for the course. There's only partial support. They do have some special instructions, but they actually only work on values that are in the cache, not values that are sitting in memory. And multiprocessors, it's considerably more complicated because in order to make sure that everybody else sees the same, sees the update that you did, you've got to propagate changes through all of the caches as well as main memory. So these operations though, that once they are supported, and they are supported in most languages and OSes now, they work across both uniprocessors and multiprocessors. So here are some examples, some common examples. Test and set is the most common one. And the idea is that you can, in the same action, read and write a specific location. So we have a memory address here. This action allows you to get a result, first of all, the result that was there before, set it to one, in this case, for test and set, and then return the result. And although it's shown as separate instructions, it's implemented atomically, so nobody else can interfere with the state in between the actions. Whoops, on MIPS process, Whoops, on MIPS processes, sorry. This is actually implemented with a special instruction sequence using linked instructions. And other architectures use other ways, but the net effect is it just runs without interference. x86 has a related instruction called swap, which takes a value in a register, effectively swaps it with a value in memory. So it returns again what was in the memory, and it allows you to set an arbitrary value other than one into that memory location. So again, these give you the ability to basically grab a lock, change the value in memory, know that you have it first before somebody else can get it. And then there are more complicated operations like compare and swap, which have a condition in them that allow you to check a particular memory location, update it only if it holds a certain value, and then set it to another value. All right, so that first primitive test and set, which allowed you to do the read, and then the write in one operation allows you to implement a pretty simple kind of lock. So if you have a variable value whose value zero represents free, remember test and set's gonna set it to one when you execute it, and it will return the zero, the first time you do that when it is zero. So that while is gonna stop. The first execution of while gives you a zero. So you continue through there, and return from the acquire, and what's the value of one? Excuse me, what's the value of value? Circular statement, it's one, right? So once you've acquired it, if somebody else tries to acquire it, it's already one. So what happens when it's one? What happens when somebody else tries to acquire it? What's that? Yeah, they spin because test and set, once it is one, test and set's gonna keep returning one, so they're gonna be sitting there in the while loop waiting for something to happen. All right, all release has to do is set it to zero, which is the free state, and then the other threads that are busy waiting looking at the value one will be able to continue. All right, and in fact, the first one that executes this instruction once the value's released is going to get, effectively get the lock, because only one of these will get the new value of zero. Does that make sense? So only one of those, once it's released, even if several of them are spinning on the same value, the first one that's able to manage, that manages to execute the test and set instruction is gonna set it to one, get the value zero and continue. All right, so is it clear why that implements the lock? All right. So, yeah, we just went through this, so if the lock is free, we read zero, set the value one, now we own the lock, so it returns zero, we get back, throw the control. If the lock is busy, test and set, reads one and sets the value equal to one, no change, returns one, and the while loop continues. And finally, once we set a value of zero and the release function, then it allows others to get the lock. So the main problem with that is that it's relying on busy waiting for synchronization. Positive things are that it's not relying on interrupts and user code can use this. So, you know, we can give this primitive to users because it's safe, doesn't interfere with the operation of other primitives. And it'll work on a multiprocessor assuming test and set works on a multiprocessor. The negatives are that it's inefficient, it's relying on the CPU to spin in order to execute the wait operation. So that waiting thread, because it's always ready to run, can take cycles away from threads that are doing actually more work, including the one that's doing the work that's gonna eventually release the lock. And also, it's susceptible to priority inversion if that waiting thread has higher priority. And that's exactly what happened in the Mars rover Pathfinder fault. You recall that was the one where the software reset itself because it got stuck. It ran for too long. All right, so semaphores and monitors, which we'll see in a moment. Because of the way they're constructed, they can wrap large, long lasting operations and therefore you can end up waiting a very long time. So that's not really acceptable. So what we'd like to do is find a solution that doesn't require busy waiting. That's always our goal. And it's in particular your goal for the homeworks and exams because we grade those rather harshly if you don't do that. All right, so let's look at a better execution of a lock using test and set. So recall our goal is to avoid the busy waiting. So we wanna somehow break open the lock acquire function in a way that will allow us to get a lock but still give up the processor in case we don't get it. So what we do is introduce a secondary lock variable called guard. The value that's the actual lock is no longer atomically being updated. You see we test it here and then we potentially update it later on. So there's a gap, which is whenever you have a gap between a test and an update, there's potential for the state of it to change in between here and here. So the purpose of the other lock in red is to basically protect this action and effectively make it atomic by guiding it. And you have to pause here. There's an if statement. So if you imagine this is effectively wrapping a lock around the busy update, you sort of acquire the lock here, give it up here along one path of the if statement. You also have to give it up here if you execute the other path of the if statement. But otherwise it's more or less a conventional lock. Does that make sense? So we've done a second order lock on the code that's testing our real lock and that allows us along one of the branches, the one where we are busy and we wanna wait to actually execute a sleep instead of just simply spinning. All right, does that make sense? So just at high level, ignoring the guard for a moment, we simply test if the lock is in a busy state and then we put the thread on the weight queue. The current thread that's running gets queued and then we go to sleep. In other words, we give up, suspend this particular thread and give up control back to the OS. Is that clear? So hopefully that's clear enough why we do it this way. Release, we have to do the opposite operation which is basically, all we have to do is set value to three but we also have to check the weighting queue of threads before we do that. So if there is a thread that's ready to run, we take it off the weight queue, put it on the ready queue. All right, if you recall, we don't normally run threads, we simply take them from waiting, put them on ready so that the operating system can troll when they actually run. All right, and then we've got our guard in red, our second order guard in red wrapped around the actual updates, yeah. Well, the reason we need guard is that because we don't set busy until down here, if you imagine another thread tried to acquire the lock, imagine a first thread comes down here, they see that the value of value is not busy, so therefore they proceed down here and set it to busy and effectively take the lock. In between here and here, another thread might come along and execute the same piece of code, might try to acquire the lock and because this hasn't happened yet, if it does the test in between here and here, it's gonna see that it's still not busy. So the second thread would believe that it's acquired the lock but it actually hasn't or rather they both would have acquired a lock effectively and gotten into trouble very quickly, yeah. Yes, it is. So the difference is, yes, it definitely is. We can't entirely eliminate busy waiting but what we can do is constrain ourselves to only busy wait on a lock that only involves short term operations. So the updates along either of these paths here, the actions are fast and time bounded. So when we grab the guard, if the value actually is busy, we put the thread on the wait queue which takes a fixed amount of time and we simultaneously go to sleep and release the guard lock. That's another bounded time operation. And similarly if we're just going down here, we're just updating one value and then releasing the guard lock. So in other words, whatever that is, it takes a fixed amount of time, 10 microseconds or something, yeah. Yeah, yes, yeah. No, you're exactly right that we're still spinning on that but the point is that when somebody else is using guard, they're only gonna run for a short amount of time. That's the trick here. They're only running long enough to do a sort of a protected update to the real lock. So that's gonna be fast. Similarly here, this involves checking taking a thread off a queue and placing it into a different queue. So that's all constant time actions which are fast. So yeah, yeah, the actual lock is value, yeah. Guard is access control so that you don't inappropriately update busy. Yeah, I think you're next. If there are multiple what? Multiple, well, I'm not sure where you, do you mean there's only one sleep associated, I'm sorry, no, that's not quite right. There's only gonna be one instance of the lock on this particular value. Other threads might be waiting but yeah, so you'll take one of those threads off. No, just one at a time. Yes. Well, it's not atomic but it's a bounded amount of time. Some reasonable amount of time, yeah. Well, wait is the operation that we just defined which is just some code that's, this kind of wait, where is it here? This kind of wait is just a piece of code that we just defined that's continually doing a check. So that's very simple. Sleep is involves taking the thread, putting it on the thread control block, excuse me, creating a thread control block and putting it in a queue. So it involves taking all of the registers, the program counter and so on from the running thread, saving them away. So it's a context switch or rather it's strictly speaking, it's a context switch into or half a context switch in two pieces. We first of all, basically take the thread control block, save it somewhere. The sleep piece is just basically stopping the thread, giving up the PCB. All right. Okay, I'm gonna just go ahead one slide and then revisit the questions in case you still have questions. Cause I had one slide that was sort of the analogy which might help. So basically back to the fridge example from last time. If we naively locked the fridge, we might have to wait a long time, right? Somebody might have gone to get milk and be back who knows when. So in the meantime, nobody has access to it. And in fact, if it's implemented with a simple test it'll evolve spinning. So the analogy for this second order lock is it's like a lock box. So you have synchronized updates to the lock state. In other words, you can quickly open the lock box, take the key out, close it or open the lock box, see the key's not there and close it. That's the second order part of the lock. But then the first order lock which is say the key itself, somebody might take that for a long time and own it. All right, so still questions. Okay, maybe the back. So you're saying there's an interrupt somewhere, where do you say again, after there? Oh, okay, that's a very good question. All right, let me think about that. Goodness, all right. Oh, that's a good question. Yeah, that's kind of bad actually. That would be something I think, all right. Well, I think the second order, a better version of this probably wouldn't allow you to do that. So that, by the way though. Okay, it's very reasonable to block interrupts from happening here because here what we're trying to do is not so much protect other threads from running as actually making, protect this code from being made to run a long time. So I think that's a very good question and probably it would be a good idea to added no interrupts here. Especially because this is basically OS level, there's obviously an OS level system call type operation because it's manipulating the busy and ready queues. Yeah, yeah. All right, well let's, no, that's a good question anyway, let's keep moving. Okay, so, all right, so just in to revisit and contrast with the enable interrupts from last time. If you recall, we looked at this templated version which is in using interrupts instead and there's a very close analogy between the version that does use interrupts and the version that uses the test and set. So basically disabling interrupts is analogous to capturing the second order guard lock and then re-enabling the interrupts is analogous to clearing the guard. So intuitively you'd like to clear it here so that you give back control while you're sleeping. There's a little bit of trickiness involved exactly when you do it, when you do that and we're gonna see that in a second. Anyway, similarly on the release side, here the interrupts are playing the role of grabbing the second order lock so that you're protecting that release code. Okay. All right, let's see. Okay, so there's an issue with this method as we've described it. You know, we've got this slightly funky piece of code here that says go to sleep and guard equal zero which the semantics is that you're executing those operations atomically so that they're not separated in time. There's two problems that can arise if you separate them. The first one's kind of obvious. What would happen if we executed the clearing of the guard after sleeping? Yeah, that's right and we don't know when that would happen. That's exactly what happened to me last night so I had to submit a research paper. Basically I had to finish writing a paper and go to sleep. Those were my tasks for yesterday so I finished the paper and went to sleep. Obviously it works better than if I'd gone to sleep and then thought about finishing the paper because it had a midnight deadline. Right, so it's elementary that you don't wanna sleep before executing the clearing of the second order lock. What about the other way? Well it'd be the problem if we cleared that lock and then went to sleep. These are always gnarly questions, aren't they? So what can go wrong with a multi-threaded situation like this? Well it's pretty clear the things that can go wrong involve things that happen between here and here because this thread effectively has the lock they're probably not gonna come from another thread trying to acquire a lock. They're most likely to come from here because this is the thread that's currently running that won't block here. Yeah, go ahead. Well it's actually slightly worse than that because notice that we actually put the thread on this particular thread, we already added it to the ready queue. So in fact, yeah, so depending on how this is implemented, this is either gonna find that thread, the thread of this guy here who's trying to acquire the lock. The original thread that has the lock, find that guy in the queue, put him in the ready queue, try to start him, he may actually start and then he's immediately told to go back to sleep. So that's pretty bad because we've basically killed that thread. Most likely if you try to start running a thread that's already running, probably a lot of worse things can happen. So it's clearly something you wanna avoid. Trying to reload the saved thread may also override its registers in incompatible ways. So it's critical that we actually implement this the way it's originally shown where we're doing the sleep and the release of that second order lock at the same time. Yeah, should the guard equal zero and release? Let me think. All right, let's see. Gonna wait queue, wait shouldn't ready queue. Place on ready queue, all right. No, this thread's still running though. I mean, all right, this is perhaps a little bit confusing, but see this thread is not gonna suspend or anything. It's just gonna exit. So I think this is fine. All right, okay. So just to summarize some of these ideas. Sort of version zero was trying to implement locks using interrupts. So we try to acquire a lock by disabling interrupts, release it by enabling interrupts. Lots of problems with that. On the other hand though, we could, we did describe an extension to that where we have an explicit lock represented by value and instead just use the disabling of interrupts to protect the update to value. Basically make sure the tests and the updates are compatible and similarly on the release side. And then our second iteration was to use the spin locking test and set action on the lock to implement acquire and release. So this was a better version because it didn't require us to manipulate interrupts. On the other hand, it has that one big weakness which is it's implementing the weight by spinning. So our final iteration is this one which is adding a second order guard that's basically making sure that updates to the real lock are effectively not quite atomic but time bounded so that we won't have to worry about extended weighting on the locks. Well, this is a different branch, remember? Fair enough, yeah, it's gonna basically just exit, yep. Well, it's just gonna return because there's no code to execute. Yeah, that seems right actually. Oh, that's a good point. Yeah, that's, all right. So yeah, I know he's right. I mean, it's a kind of trivial observation that it was already one. So I know this is weird, but yeah, basically you just, you don't have to set it because it's already one. In a certain sense, yeah, I mean, it's weird, isn't it? I mean, it's, yeah, that's kind of a sneaky coding trick but yeah, good observation. So yeah, I mean, logically, yes, you wanna set it to one but it just happens that in this branch it's already one. All right, so let's, so very quickly before we take a break. So at high level, we're trying to move from these low level primitives involving loads, door operations, interrupts, and so on. Atomic memory updates, up to high level primitives which we're gonna talk about in a second, that will considerably simplify our updates and the concurrent programs and the goal is to get thread safe program in the end that multiple threads can run safely without interfering with each other. All right, so we're gonna take a break. I would like for when we come back we have to do a little simulation of a concurrent programming, of a concurrent program which is always challenging. So I need three volunteers, preferably people in the front to help me out and you just have to execute a single semaphore. So can I get three volunteers? And we're busy waiting on the volunteers. Okay, one? Okay, one, two, three, okay, awesome. All right, so we'll take a five minute break and resume when we come back. So all you have to, all you have to do. All right, so let's continue. All right, so we're gonna talk now about some of the higher level primitives for concurrent programming. One of the most important ones is semaphores which was developed as part of the DER system for concurrent programming. Still widely used though. So they're a kind of a generalized lock implemented with a test and set primitive defined by Dijkstra in the 60s and was the main synchronizing primitives in the original UNIX. So a semaphore is a non-negative integer value that supports two operations. P, which is an atomic operation that either weights or decrements by one. So it waits until the value becomes positive because you can't decrement zero. And as soon as it does become positive, it decrements by one back to a smaller value. And it's similar to UNIX weight which we'll look at more later on. The other operation is V which is atomic operation that increments the semaphore for one by one and wakes up any of the weighting threads that are stuck on a P. All right, so notice that it's a kind of inter-process communication actually. So it has similarities to UNIX signal which is often used for that reason. Okay, and both the names are Dutch so they're not particularly easy to pronounce or remember. But P, maybe you can remember, it's roughly to probe or test. And V is verhaken, which means to increment. So okay, so semaphores are like integers except they don't have negative values. Only those two operations. You don't have any kind of non-destructive reads, I suppose, or non-changing reads, non-mutating reads. And it's critical that the operations are atomic. Okay, so, but they do allow any number of threads to execute the operations and any number of threads to sort of be woken up. Sorry, go ahead. No, no, it never becomes negative. If it's zero, the caller's gonna wait until it becomes positive. So the P is a potentially blocking operation, the V never blocks. All right, so to use a railway analogy and to see how the semaphore part works, if you imagine we have a semaphore that's set to two and someone executes a P, they're basically getting a resource. Semaphores typically control access to resources. The resources here are these two sightings on the railroad track. There were two available. When you execute a P, you get one of them. When somebody else executes a P, they get the other one. If somebody else executes a P, the value's zero, so they just have to wait. As soon as someone, one of the first two trains executes a V, then they clear that resource, becomes available again. And actually, well, you already thought, the train that was waiting when it executed P, it became stuck. When the other train moved off, it moved on because its own P cleared. Does that make sense? So all right, so we executed three P's and now two of the trains have the resources. There's no free values. But third one comes along. And he's gonna, sorry, the clearing operation involved a V, so we did PPPV. And now there's a third train came along executing another V. All right, so there are two uses of semaphores. One of them is for mutual exclusion. And that typically involves setting the semaphore to one and allowing it only to move between zero and one. So that's effectively a mutex. It's usually used in pairs like this in one piece of code to protect a critical section. A rather different way of using semaphores is to allow communication between two different threads. So, and we just saw how that might work is that one thread can execute a P operation and it's going to get stuck if the resource is not available. A second thread can tell the first thread that it's okay to proceed by executing a V on that semaphore. And we'll see rather complicated ways of doing that later. So a common example of this is thread joining. So in a thread join, you have a master program, main program that started a secondary thread, and it wants to wait for the secondary thread to finish. So the secondary thread finishing is a join, involves a join between the two threads if you want to think of them sort of coming together and only the main thread continuing. The join executed by the main program would just be a P operation on the semaphore. And the semaphore starting at zero, so this is simply going to block at that point. The other thread that's running, when it's ready to finish, it just has to execute a V operation which will increment the thread but also effectively communicate with the first thread that this incremented therefore, this one suddenly decrements and returns to zero and continues, yeah. So there, all right, so if multiple threads try to grab it, you're gonna end up with a Q of thread. So yeah, so one of them is gonna, whichever one is first in that Q is going to grab the thread once it becomes free rather they're going to, so you can basically, what's the word, serialize all of this. The thread is getting incremented here. One of the threads that's waiting is gonna see it incremented and then decrement it. So in effect, that's the one that's grabbed, the lock and can continue. It's similar to test and set, yeah. If there are multiple threads that have executed this on the semaphore and are waiting, only one of them will continue. All right, so in effect, that the operation is a signal back to the first thread. All right, so let's look at executing producer-consumer pattern with semaphores. All right, question? Well, I mean, typically the, if the main program spawned a thread, it would have created the semaphore before it started the other thread. So it would pass the semaphore as an argument to the other thread. That would allow it to execute, both the master thread and the subordinate thread would have access to the same semaphore so they could signal each other through that. All right, go ahead. All right, so that's an implementation question. But, yeah, it's from the code's point of view, the code stops there, it doesn't continue. It may sleep, it may be spinning. But that's an implementation issue. It's transparent to the program. If it's well-executed, it should not be spinning. All right, so our producer-consumer pattern is implementing a Koch machine. And the producer is the person that fills up the machine with Koch cans. The consumer is a consumer of Koch that's taking Kochs out of the machine unless it's empty. So we wanna first define the constraints on this example. And there are two scheduling constraints and then a sort of a consistency constraint. The first scheduling constraint is that there has to be something, the consumer must wait for the producer to fill up some slots, okay, so it can't be empty. The producer similarly must wait until there's room to put more Kochs in. Both of those are scheduling constraints that involve both producer and consumer. And finally, only one thread should be manipulating the data structures for this pattern. So that's a mutual exclusion constraint. So typically to execute these constraints, we wanna use a separate semaphore for each one. That'll keep the code pretty clean. And in fact, typically we'll have a pair of semaphore operations associated with each of these. All right, so we have semaphores, full slots and empty slots. Those are actually count semaphores that count the number of empty or full slots. The semaphore mutex is a binary semaphore that only says whether or not a particular thread has control of the critical section. So let's set things up. So semaphore full slots is going to start off at zero because assuming the machine starts off empty and the empty slots will be initialized to the maximum capacity of the machine. Mutex is going to be initialized to one which is the free state for a semaphore. So the producer first gonna test if there are empty slots. Remember the producer is trying to fill the machine so they wanna test if there are actually any free slots in the machine. If there's zero, it's gonna wait. That's what the P operation does. Then it's going to try to grab the lock on the state of the machine, which is the mutex. So it does another P. In this case though, we've initialized the mutex to three or one, so it will succeed. So it'll be able to get into the enqueuing code and finally when it's done, it'll be able to simply increment the mutex which frees it up for somebody else. The last thing it has to do then is increment the count of full slots because we've just added in effect a new coke to the machine. All right, so on the other side, the consumer first of all tests if there's any cokes in the machine. So that only fails if full slots is zero. And if it is zero or doesn't fail, it'll block if it's zero and wait until there's a coke. As soon as the producer puts a coke into the machine, that's full slots to something other than zero, then the consumer can proceed. They grab the mutex and similar to the second order lock we saw before, this mutex only protects a very fast piece of code. So in terms of this mutex here, it's fast, both operations are fast, so they normally won't cause delays of any significance. Once the consumer has grabbed a coke, notice they will have, when they grab it here, they've actually already decremented the count of full slots. So the only other thing to do is to increase the number of empty slots and finally return the item. So one quick question is why do you think we have both full slots and empty slots because that's redundant? Yeah, yeah, that's most of an answer. Anyone else? Well, remember that we've got either of these actors might be blocking, right? Either of them might get stuck. And the full slots and empty slots here, there's signaling style semaphores, right? So we want to have a way of both the producer signaling the consumer and the consumer signaling the producer. So that's why there are the two semaphores. Because the P test is really receiving a signal, the V test is sending a signal. If we didn't have both of these, there wouldn't be a way for one or other of the actors to know when to wait or rather be signaled that waiting is over. All right, so there are the signals. Okay. All right, so what I'd like to do now is actually try to simulate this sequence of instructions which works although it's somewhat fragile. So first we'll simulate it running correctly and we'll make a slight tweak to the code and see what happens. And so I'm going to use help from three volunteers. So basically volunteer number one is going to implement full slots which initializes to zero. That one, awesome, yep. And volunteer two is implementing mutex and volunteer three, awesome. All right, empty slots is initialized to two. All right. Okay, so all right, so I'm going to run the producer. So first I'm going to do an empty slots dot P. What's that going to do? Good, yep. So I decremented the number of empty slots. Mutex dot P. Yep, NQ. So I've successfully NQ'd and now mutex dot V. Release the lock, good. And full slots dot V. Nice, nice flourish there. All right, let's go back. All right, so I'm going to go through one more time. Empty slots dot P, yep. Mutex dot P, mutex dot V. Full slots dot V. All right, so now look, we've actually filled up the machine with two codes. There's no empty slots anymore. So we might as well grab a code, don't you think? So all right, full slots dot P. Full slots, yep. No, no, no, no, helping the other semaphore, please. All right, mutex dot P, mutex dot V. All right, empty slots dot V, return item. Okay, good. All right, I'm going to do one more produce. Empty slots dot P, mutex dot P, mutex dot V. Full slots dot V. All right, so it seems to all work and we can try different combinations that will keep working. But if I'm a naive programmer, oh yeah, there we go, signaling again. All right, if I'm a naive programmer, maybe I might have put the mutex P before empty slots. It looks similar, but what's wrong with that? Does anyone see what's wrong? Well, let's try. I mean, producer, okay, so I'm gonna put a code in there. So I do mutex dot P, yep. Empty slots dot P, what's gonna happen? Nothing, right? So this thread's gonna stall, waiting for some empty slots to become available. All right, so we're stuck on the producer side. What about the consumer? So now the consumer does come along and they're gonna grab a code, so we should be okay. So they do full slots dot P, mutex dot P, mutex dot P. No, no, mutex dot P. All right, mutex dot P. All right, no, I'm stuck, right? So what happened was the locking of the mutex happened too soon and we got stuck, we got a deadlock. All right, thanks guys, let's give them a hand. All right, so the broken version didn't work. So on the bright side, semaphores are a huge step up because if we tried to do this with the atomic loads and stores, we'd have a much more complex problem. And remember the solution from last time was also asymmetric, even for that very simple case. Yeah, go ahead. Well, it's pretty close to test and set. The only difference with the semaphore is that it's an integer, so you can basically count resources and keep track of them. So we used it that way here. But it's, I mean, the binary semaphore is a test and set, it really is, and it would be implemented in a similar way. So the one thing is though that it has this generalization. I mean, it's binary, it happens to be the zero and the one are flipped, but it's otherwise equivalent. But it's a slight generalization because of the N, you can actually use it to manage N resources in a concurrent setting. So it does do a bit more. And it's often more usable in multiprocess settings in multiprocess settings to have the ability to signal N different things. Yeah, question? Yeah, that was a deadlock. Yes. All right. Good. All right, so on the bad side, we seem to be missing a bit of structure when we use the semaphores this way. This seems to be something essential about the matching P and V for the mutex and not wrapping them around some other mutex operation. Okay? So there seems to be some structure missing. And also we might be overloading the concept a little bit by using them both for scheduling and also for mutual exclusion. So, and probably most serious of all, it's really hard to analyze complicated programs that have a multitude of semaphores. So the next step is to go up one level of abstraction and use the idea of a monitor. So a monitor is typically defined as an object, a high level instance of a class and whose methods are implemented with mutual exclusion. So it's a kind of a class that has a single variable that's a lock that's used for all of the methods. Or if you like, you can think of it as being a class whose methods are all synchronized on this. So only one method can execute at a time. And it's very easy to implement these in Java. If you take a class and put the synchronized keyword, do public synchronized method name on all the methods that becomes a monitor. So condition variables are a related idea that have two operations. In the book, they're just defined with two operations, X dot weight and X dot signal. And the idea is to basically wait on some condition that's in a different thread. And especially to allow multiple running threads to wait on the same resource. And they'll end up being queued to do that waiting for calls to X dot signal, which is freeing up that resource. Each time somebody calls X dot signal, one of the threads that's waiting in the queue is gonna get freed up and allowed to run. So that's a general way of allowing internally to the monitor, it's allowing threads to go to sleep without getting into a deadlock situation. So here's the monitor, it encapsulates some data, instance variables of the class, some operations or methods that are synchronized and some setup code. And because it supports these monitor variables associated with each monitor variable is a queue of the people that have said X dot weight on that variable. And finally, X dot signal will cause, somewhere internally in this code it usually happens, will cause those threads to proceed. Okay, so the monitor is providing mutual exclusion to the data. And we're gonna look at some lower level implementation using locks, but you can also in some cases use higher level. When you're typically programming you'll probably use the higher level primitive. And the condition variable supports a queue of threads waiting for something inside of a critical section inside of the monitor. So the idea is to allow the thread to go to sleep if it is waiting on something inside the critical section by atomically releasing the lock. Very similar to what we've been doing up to this point. All right, so here's a synchronized queue. So the monitor is this special synchronized class. It has a single instance of its lock and a data structure that we wanna protect. So when you add to the queue, you first acquire the lock associated with this particular queue. So you're assuming all the methods are locked with the same lock. No other method can run at that point. Then you actually do the data update and release the lock. And then similarly to dequeue something you acquire the same lock, dequeue it and then release the lock and actually return the item. So that's the core of the monitor idea. It's pretty simple, but it's also not really using the condition variables yet. So, and basically if we have the risk is that we'll actually end up spinning on one of those locks. So yeah, so we'd like to not have the consumer, you know, say removing from queue spinning if there's nothing for them to consume. All right, so the conditional variable and in the book they just have the basic condition variables with weight and signal. Idea being that, you know, threads that have used X dot weight get queued up waiting for somebody else to do X dot signal. And the signals cause the weighting threads to continue in the order that they're waiting on the queue. And we're just going to tweak the weight a little bit by adding a lock variable. The idea is that we're going to make sure that when we execute a weight inside some critical section that we also free up at the same operation the lock that it might be holding. So that's going to allow, that's going to avoid us getting into bad deadlock situations again. And finally, some systems in addition to signal have a broadcast primitive. Again, this wasn't in the book but it's a primitive that's, for instance, implemented in notify all or it's equivalent to notify all in Java. And that will wake up a whole bunch of threads. All right, so here's the monetary example with condition variables. So similar before the new stuff is in red. Now we have this condition variable data ready. And add to queue. So the idea is that the data ready condition is saying that data is ready. So something that's in the queue ready for a consumer to consume. So the producer, once they put something in there once they've been queued an item they can actually signal any potential consumers that the data's ready. And then on the consumer side the consumer checks is a QMP. Let's see, yeah, that's just a test. We'll come back to why that's a wild statement in a second. But if the queue is empty they wait on that data ready condition variable. Now, because we have the lock releasing version this weight can safely be implemented by sleeping. If we try to implement this with a sleep without the lock we basically would have frozen the lock and caused the problem, right? So it's important if we do any sort of sleep operation in a protected section like this that we somehow make plans to make sure that the lock is released. So that's what this sort of extended version of weight does. So all right, so all we did was we added the variable made sure that there are signals when there's new data. So in other words the threads here that are sleeping will be woken up as soon as the producer puts something in the queue, is that making sense? So there's a signal coming from a producer to a consumer once they have pushed something into the queue. Somebody's waiting here, that means that the queue's empty so as soon as one thing goes into the queue they'll be ready to take it out again. All right, so last important point is that the way that we implement this test here is going to depend on exactly how we implement the signaling and weighting. So, okay. We could have done this with an if test instead of a while test, right? It's at least in conventional code, you would expect that if we wait for a while until the queue is ready as soon as the queue is ready should have something in it, right? So we shouldn't need to go back and test again but in fact sometimes we do and it depends on how ready, excuse me, how a signal and weight are implemented. Okay, and there's two widely used approaches. One of them is horse style and one of them is messer style. So, in the whole monitor there's kind of an instantaneous transfer of control between the acquirer and somebody else. So, the person that's, let's see, the signaler actually effectively transfers their thread of execution to the waiter and causes them to run immediately. All right, once the waiter's given up their lock, they give their, the process of the CPU back to the signaler as soon as they execute the critical section or if they have to wait again like here. So, let's step through that. All right, so, we acquire a lock and wanna do a data ready signal. So, this is a lock within the producer. Effectively the producer transfers lock, control of the lock and the CPU to this guy who's waiting over here, who's a consumer. So, they were waiting, they must have come in and found the queue empty and also found that data was not ready, so they've been waiting here. They would have freed up the lock, so they don't have the lock, right? When they executed the statement earlier, they don't have it. The whole monitor transfers control of the lock to them when it executes and it also transfers the program counter to them so that they start running and resume execution from here and as soon as they release the lock, then the processor goes back to the signaler here or potentially if there was some other weight in here, it would also go back. So, in this case anyway, it looks like this. So, we resume execution here, run down to the lock releasing code and the program counter and the lock because it's been released goes back to the signaler. We have to give it back because it still in some sense owns the lock. It's kind of loaned the lock for a little while to the receiver so the receiver could continue but it needs to get it back. So, once it gets it back, it has the CPU and it has the lock so it can continue as normal until it gets to the release, all right? So, that's one style of doing things, only one style. All right, so, Mesa Monitor is a bit more laid back, I suppose. So, the signaler keeps the lock and the processor and instead the effect of the signal is to put the waiter into a ready queue because they would have been suspended. They would not be ready after hitting this weight. They would because they're unable to run but the effect of the signal should be to make them ready to run. But the Mesa style doesn't try to do that instantaneously, it just instead makes this thread runnable by putting it into the ready queue, yeah. So, the practical import is that there isn't a need to check the condition again after the weight. So, most modern operating systems do things this way. Yeah, question. Yeah, I was just thinking that too, actually. So, let's see if we can think why. Well, let's see. I think that's probably true. I think it's a good question. Yeah, I think basically the queued process would need to have a lock associated with it. So, that's an interesting question because that potentially, let me just check one more thing. Yeah, I think that's true because this thread would have released the lock by the time the second thread is being queued but another thread might have grabbed it in the meantime. So, I think you're right. I think it would involve also adding a lock to that queued process. Any other questions? Yeah. Well, no, nobody's, well, no, we're not busy waiting. In fact, we're trying to avoid, we're trying to do an even more asynchronous. No, wait a minute, maybe I'm not getting this. You're saying, is this busy waiting? Is that your question? No, it shouldn't be because it's actually, because it's a wait that's got an automatic lock release, it normally would have gone to sleep at that point. Right? All right, so that's a good question. So, you're saying, yeah. Yeah, well, no, okay. So, I'm a little confused because I'm not seeing what the problem is. I guess it is actually checking, right? It's gonna reimplement that test, right? So, it should be fine. Somebody could have, the queue state could have changed but it's in a while loop but it's basically gonna go back here. So, maybe if you're asking, does it continue down this way? I said that it did, but you're right. It doesn't necessarily continue straight from there. It should go back and do the while test. Yeah, no, so you're right. So, I oversimplified it. Yeah, question. Well, the trouble is, I mean, lots of things may be going on in other threads. You're saying, well, you could, yeah, no, that's true but the semantics of this particular producer consumer is that the consumer is a blocking call that's supposed to either give you something or wait until something's available and give it to you. That maybe that wasn't clear but that's the goal of that particular consumer. So, it always returns an object. It doesn't have the ability to return null. So, since it has to wait, it's gotta execute a wait on the queue to become empty. And yeah, I think there's an issue of somehow making sure that it gets locked in the second case. So, we'll work on that and fix it. Any other questions? All right, we'll see you next time.