 All right. Let's get started. All right, so we have two new topics for today. Quiet please. So two new topics for today. The first one is programming languages and how they support concurrency. And the second one is deadlocks, which are a very important class of concurrency problems to understand and try to prevent. So we'll start off by reviewing the main topic from last time, which was the reader's writer's design. And then we'll go through some of the common programming languages and how they support synchronization. And then finally, we'll talk about contention for resources and how it can lead to deadlock and how to recognize that and hopefully avoid it. So from last time, we looked at the reader's writer's problem and the model is to consider some kind of shared writable storage like a database and users that are trying to read and write from that store. So the readers don't change what's in the store. So multiple readers can access it at the same time. The writer, since they're modifying the content, we want to give them exclusive access. So question is, is single lock on the database sufficient? It wasn't the model we looked at last time. We wanted to basically do something a little bit more complicated that would allow many readers at the same time and only one writer. So the correctness constraints were readers can access the database when there are no writers. Writers can access when there's nobody, neither reader nor writer. Only one thread, only one writer should be manipulating the state variables, whether they're a reader or a writer. Okay, so here's the basic structure of a solution. Here's the reader piece. So the reader waits until there are no writers, accesses the database and then checks out. The writer is waiting until basically nobody else is trying to access, accesses the database and checks out. So that's the high order and the only sort of tweak to that is to give writers a bit of an extra priority. So in order to manage the access, we keep track of the state of the readers and writers using these state variables. The count of active readers, the count of waiting readers and similarly active writers and waiting writers. And we have some condition variables as well. Here's the reader code again. All right, so we saw this last time and at high level we're using the locks to manage access to these state variables but we're actually releasing the lock before we access the database. Then re-acquiring it afterwards, changing some of the state variables and then releasing again. Why do we do that? Someone quickly remind us why we did it that way. Well, it's not really a Mesa versus a whore monitor question, it's a question about allowing access, allowing the other threads to figure out what's happening. So it makes sense, so what we're actually locking here is the state variables. So we want to allow a reader to go and endure a read and another reader to actually get through here and do a read at the same time. What's that? Yeah, exactly, the database is not changing so we don't need to lock it. And specifically we want to allow other readers to get through all of this stuff without getting stuck. So we want to synchronously update the state of readers and writers, all right? So we may have updated a writer, sorry, excuse me. We may have updated some state here but we want to allow the other readers to go through and check and possibly wait if they need to but if they don't need to wait, we want to give them access to the database. All right, so writer code is kind of similar. You know, we went in detail into all of these conditions last time so I won't revisit that but at high level we have a similar pattern which is, you know, the writers come in. In this case though, there's a big difference between last time, between this time and last time which is what happens to a second writer here. What's gonna happen when a second writer comes in? Yeah, right, if there's somebody actively writing this variable is gonna be greater than zero so this test is going to succeed which will keep you in the loop and will keep you waiting until somebody signals you that that resource is free. All right, so this is protecting access to these state variables but still allowing a writer through, in this case only the first one and once they go through and update the number of active writers at the end, decreasing that then that will allow other writers through. All right, so let's see. All right, okay, so we had some detailed discussion of broadcast and signal at the end. Why do we broadcast? So what's happening down here, let's see. We have, if we have waiting writers greater than zero, yeah, yes, yeah, we wanna let all of the potentially waiting writers and readers know that the state is now accessible and we first checked actually if there are waiting writers and signal those first actually, why do we do that? Yeah, intuitively you wanna give writers priority in case they're ready to do an update to the data so you'd rather that happen first before somebody reading the data and getting stale data although they could get more up-to-date data, right? So that priority is important. All right, so next topic is language support for synchronization. So locks have been in the C language since the beginning and a lock in C is a very simple object. You can acquire it and release it and that means that, excuse me, you're responsible for taking care of all of the exceptional conditions that might occur. So that typically doesn't involve throwing exceptions, they're not part of the language by default so you have to test error conditions, explicitly release the lock and then return an error code. So it's not completely trivial in a complex program but it's somewhat tractable because you can eyeball the code and try to figure out all the code paths and make sure there's no exit from the SQL to a lock that might escape without unlocking. C++ is more challenging because you do have exceptions in the language. So exceptions create a lot of new exit paths because you can throw an exception potentially anywhere in the code. So, all right, so yeah, here's a little routine, acquires a lock, this function here can throw an exception somewhere in its body, that will actually skip past the code that's following it in the block there and basically leave the lock still locked. So that's very dangerous, all right. So there are a couple of ways of dealing with this. One way is sort of following the C design which is to try to catch, to try to find all the code paths that can exit which would include exceptional exits. So that requires you to catch the exception that might be thrown and then release the lock and then also have a lock release in the normal code path. So there's a bunch of disadvantages of this. It's certainly better than not catching the exception. One gnarly issue with C++ is it doesn't have an ancestor exception class that basically specializes to all the exceptions. So there's no way to catch a universal exception object here. And this piece of syntax is literally three dots. It's not a placeholder for something. It means catch anything, it's a specific piece of syntax. But since it doesn't know the type of that thing, you don't actually get a handle on the thing that's being thrown. So it does allow you to release the lock. On the other hand, when you throw, you're gonna throw an unknown exception. So this is kind of bad because it loses the state that you had about what was wrong. So there's a different approach which is recommended by Jan Stroustrop which is using the support that's built into the language for unwinding the stack when there is an exception. So in C++ you have constructors and destructors for objects. And for variables that are allocated on the stack for automatic variables, the constructor is called when the variable is created and the destructor is called when you exit from that block or stack frame. So that allows you to let the unwinding code within C++ deal with locks. So a recommended motif is to have a sort of a primitive mutex object here, and underline and then create a constructor method for the lock class. This is a constructor which basically initializes its internal reference variable to some explicitly created lock, then acquires it and finally in the destructor releases it. So you're relying on the fact that that destructor is called automatically. So now if you wanna protect some code in here, this is a local variable declaration. It creates an automatic variable that's on the stack. And that variable is automatically cleared off the stack. The whole stack frame is cleared and the destructor is called at that time. So that will automatically release the lock. And the unwinding and the destructor is gonna be called whether that's a normal exit or an exceptional exit. So if an exception's thrown, the exception handler is gonna unwind every stack frame and if there's anything allocated in any of those frames it frees it or it calls its destructor. So any locks that are along that path will be cleared. Yeah, question? Well, it's gonna be called whenever the variable for an automatic variable, whenever the variable goes out of is basically destroyed which means going out of scope for a block scoped variable like this one. So the critical section here is, so you can put a block anywhere in code just in closing it with parentheses. And for the instructions in there, any variables declared inside of the block are in scope but as soon as you leave the scope it has to destroy everything that was in that scope. Yeah, oh, I see if you, well that's a good question. All right, so if you have code that's possibly involving the objects that you're trying to protect in the destructors. Yeah, I think, yeah, I think you can go wrong but normally you would be doing explicit manipulation of the object state in here and then in a certain sense the destructors are being called right there. You know, the obvious thing is not to put code in the destructors that's manipulating those, you know, what should be protected variables. I think it should be fine. All right, so in Java actually there's two really different families of synchronization primitives, both low level and high level ones. And the low level ones look like the ones we've seen so far. We have basically a lock class, supports locking and unlocking and we have conditioned variables that support waiting which is called await and signaling through a signal method. So these really are very similar to what we've seen so far. The new set of primitives are the synchronized primitives, synchronized keyword and the corresponding, so the new primitives involve implicit locks and implicit conditioned variables. One of each of which is associated with every object in Java. So those allow you to basically apply a lock that applies to one object while you manipulate the object and have it automatically be released when you exit a code block. Similarly, there are conditioned variables which are accessed through wait and notify methods that are implicitly attached to the object that's currently holding the lock. So these work apparently without any explicit connection between them. The connections are all implicit. It sounds complicated, but it's usually intuitive and very simple. All right, so let's look quickly at a synchronized queue using the explicit synchronization primitives in Java. So first of all, we create a lock. We use a re-entrant lock class most of the time that's what you want. Non-re-entrant locks are usually a really bad idea. And we can create a conditioned variable that you notice is being called as a method call on the lock. So in this way, the conditioned variables associated with a monitor which is basically the lock. All right, so, and then finally, there's some internal state which in this case is a linked list of integers. So the enqueuing pattern should look pretty familiar. We want to first of all acquire a lock, add an item to the queue, and then go to sleep. And basically signal anyone that might already be waiting to see if there's an item in the queue. And we have a nice extra feature in Java which is a finally keyword which substitutes for normal and abnormal exits from a try block. So if the code executes normally here, we're gonna pass into the finally block. If the code has an exception somewhere, it's also caught by the finally block. And this is executed. So finally gets executed always, including if there was an explicit catch in there, the catch would get executed if it did catch the exception, but the finally block would also get executed. So it provides a fail-safe guarantee that that lock's gonna get released. On the other end, when we dequeue an item, let's see. So again, we first of all check if the item is, sorry, we check if the queue is empty or not, acquire the lock first, check if the queue is empty. If it is empty, then we're gonna wait for a while until somebody signals us. Otherwise, we're gonna remove the item from the queue and finally abnormally we're gonna exit and release the lock. Yeah, that's a good question. No, maybe it's too much detail, do you think? Or, so yeah, a little bit too much detail for DesignDoc. All right, so that's the low-level version of synchronization in Java. Java also has high-level synchronization. In fact, Java had high-level synchronization primitives before the low-level ones were added. And Java generally, if you don't know, was a language that was entirely motivated by the idea of avoiding common programmer errors. So the developers of Java looked at the most common errors that programmers made in languages like C++ and tried to engineer the language to avoid them. And initially, they only supported implicit locks and condition variables. And they're following very much the KISS principle. So what is the KISS principle, somebody? Keep it simple, student. In this class, it's keep it simple student. All right, so we wanna make sure that the code is easy to understand by looking at it and we wanna minimize complexity. If there's a simple way to do things, we're gonna favor that way. So there are reasons for using explicit locks, plenty of reasons, but you should have a good reason if you use them. All right, there's a real issue with code maintainability these days for a lot of projects. Last time we talked about teams and how different members of the team can contribute. Sometimes people can also play a kind of an oversight role without necessarily, or a checking role without necessarily doing the code. But coding in the large is really a many-person activity that extends across time and space, which means that often the people that are developing code never actually spoke to the original authors of the code. So again, it's important to think about lifetime and complexity. So what is the typical lifetime of a piece of code? It's a lot bigger than most people think. If you look at most pieces of software, most of the open source projects, it's in the decades. And that's a lot longer than the original developers anticipated. The developers of the operating system, Mark, who were working in the 90s probably didn't think they were programming for iPhone, program the operating system of iPhones or Macs. But that's what happens. So anyway, so it's important to anticipate and try to minimize complexity and try to be explicit about assumptions that you're making. It's interesting to look at concurrency bugs. So I'm briefly gonna mention some things that people have observed about concurrency bugs. So one thing is that concurrency bugs seem to be one of three main types. The most common type is atomicity violations, which means not protecting some resource that should be protected. The second group is order violations, doing things out of order, perhaps reading, perhaps not waiting for something to be ready before you try to access it. And we already saw a number of order constraints in several of the pieces of code that we looked at. But in general, they're quite complex. The last one is deadlocks, which we're gonna look at in detail in the second half of this lecture. So that's what people run into. And it's interesting that there's a sort of a duality between the type one and the type three. So atomicity means sequences of steps that should be atomic that need to be executed together for correctness and not being executed together. So you're in a sense under protecting. Perhaps there's a critical section that's too small or there's protection that's missing entirely. Type three on the other hand is a kind of a deadlocks, as we'll see often result from over-protecting, reserving too many resources or locking too much code, causing resources to become unavailable to other threads and leading to eventually to a deadlock problem. And unfortunately, people have found that fixes to deadlock problems are often involved creating atomicity problems because people see a deadlock and they start trying to release and back off the locks and end up actually exposing further problems. The good news is that most non-deadlock bugs involve only one variable, surprisingly. And 90% of the deadlock problems involve only two threads and two resources. And some deadlocks involve only one resource and one thread. And I'm gonna come back to that because it's very counter-intuitive but it's actually simple to fix but it's a problem that shows up often. All right, and finally, some not so good news. Can currency bugs seem to be a small fraction of all reported bugs, maybe about 3% but consume a large fraction of debugging time, perhaps closer to 30% and the typical time to fix bugs is in the days instead of hours. So all of this is part of hopefully strong motivation for you to be careful about using concurrency, make the uses of concurrency as transparent as simple as you can. All right, so let's look now at the high level primitives that Java provides and we'll rely on the fact that objects in Java without any action by the programmer have associated implicit locks with them. So every object has a single lock that is locked implicitly when you use a synchronized block or method. And the example's on the next slide. So synchronized, the synchronized keywords acquires a lock and releases a lock around some section that's protected by the synchronized keyword. So for a synchronized method if you put synchronized in the preamble of a method declaration here, then basically everything in the method is synchronized by the implicit lock associated with the main class, an instance of the main class. So in other words, it's this object if you wanted to think of it in terms of language. So the locks are quiet at the start of the method released at the end. You don't have to worry about naming it though. It's always the one that's implicitly associated with this. The block-structured version of synchronized is a block that can occur anywhere in a routine and it involves the implicit lock associated with a different object. So that can be any instance of a class in Java and you basically acquire a lock on its implicit lock at the beginning of the code and then you exit it automatically at the end. Again, it'll take care of exceptions if an exception arises in that block. So a consequence of this construction is if you wanted to replicate a synchronized method using the synchronized primitive, it's essentially the same as putting synchronized of this right at the start of the method declaration and the closing parenthesis right before the method executes. Okay, is that clear? Yeah. Well, it's locking an implicit lock on the object. So if every method is synchronized, then only one method's going to be able to access the object. But yeah. Well, the non-synchronized method can access the object. It doesn't, yeah, it'll just go straight through, yeah. So if you wanna implement a monitor, you're supposed to synchronize every method to make sure that only one method's executing at a time. Yeah. No, it's not an issue. Java doesn't have pointers, but it has references and it's normally passing references to objects. So the same bank instance, if you take the variable of type bank and you copy it somewhere or pass it to a routine, the two different contexts have the same reference, the same object, the same lock. Yeah, it'll be the same. Yeah. All right, yeah, all right. And so Java, the virtual machine's taking care of allocating and releasing the lock on normal and abnormal exits, which is very nice. Okay. All right, so in addition to the implicit lock, there's a single associated implicit condition variable associated with it. And that allows you to implement waiting and signaling or notifying very easily. There's three versions. There's a wait with no arguments and a wait with a timeout and even a wait with both a millisecond and a nanosecond parameter for the timeout. So they're probably completely meaningless in most runtime environments. But what the timeout does is cause the method to wait, but then wake up after the timeout. So it'll wake up either when it's notified or if the timeout expires. Just in case you're, you know, if you're not completely confident that something's gonna signal it to wake up, it will wake up by itself. Okay, the signaling function on the implicit condition variable is called notify. And there's a corresponding broadcast version which is notify all which will wake up all of the waiting threads on that condition variable. All right. Yeah, and so, you know, among the uses of the timeouts are dealing with exceptions that might kill the signaler. And if it does, then you can still, the correctly running threads will continue via the timeout. Okay. So some gnarly issues with this is that the VMs are not very consistent in Java. Most elements of Java are extremely stringently compliant with the standard, but scheduling is not, unfortunately, because for efficiency reasons, it tends to inherit the scheduling framework of the operating system. And some versions of Java virtual machine, the ones on Solaris, for instance, for instance, don't have preemption. So you have to be very careful in terms of threads sometimes becoming starved, essentially, not being able to run, even though in principle they could. Okay. So using the high-level primitives, the synchronized queue is quite a bit simpler. So this is the whole thing now. We have, by wrapping the synchronized keyword around the enqueue function, the locks both acquired and then released at the end of the routine, we can notify other threads when we add something in case they're waiting for the queue to become non-empty. The other synchronized queue wraps dequeue. We again wait to see if there's something ready to be read, if not, we wait. And we finally remove it. We have to catch an exception here, and we actually, this time, catch it and just return a kind of an error code, return a valid value in case the interception, in case the routine is interrupted. So any other exception is going to be automatically caught outside of here and it will cause that lock to be released again. All right, well anyway, so this version's a lot simpler. And the last thing I'll talk about briefly is a programming metaphor, a paradigm that's showing up in some newer languages that avoid synchronization issues entirely called actors. And it's in the Scala language, for instance, which is a newer functional language that uses the Java virtual machine. It's a very interesting compromise between new languages with very little library support and more conservative languages like Java. But anyway, actors are a perhaps elegant, high level abstraction for concurrent programming. So far we've been struggling with concurrency issues that are caused conceptually because we have different threads that are trying to concurrently access different objects with state in an object-oriented framework. So if these are the objects in yellow and these are some methods of each object, in concurrent programming we allow many threads potentially to access the methods and modify the state of these variables in more or less any fashion. And in order to enforce consistency we have to do synchronization. So Scala avoids that by associating the program or process activity with each significant block of state, so each actor now is going up from Java objects to actors. Each object has a single thread and in fact there's only ever one thread running the code inside of an actor so there's no concurrency issue. And actors interact by sending and receiving messages normally without any notion of blocking. And in fact, rather than function calling it's a message passing paradigm which means when you ask something to happen if you want a return value you actually have to wait for and receive a return message. So the main functional part of an actor is a loop which is sending and receiving messages and synchronization is happening invisibly in the implementation of the while loop which basically has a synchronous queue. So all actors can pass messages to each other through those synchronized queues without the programmer having to worry about it. So it allows you to create some simple, well, simple programs to do fairly complicated things. So here's an actor that implements a bank statement that allows you to do deposits, withdrawals, calculating interest, computing a balance, has a single internal variable and you send messages to it to basically cause these things to happen. And here's a simple actor that also this is the message passing syntax here. It sends a message to the main bank account asking it to add a certain amount of interest. So, and that sleeps for a while. So this program basically creates an account and updates it periodically. And you notice there's no synchronization even though there is shared state which can be accessed by many different actors but the message queues guarantee that it's consistent. So anyway, it's worth knowing there's, so actors can't do everything but they are an interesting paradigm that's seeing increasing use in industry for concurrent programming. All right, so all right, we're gonna move on now to deadlock, are there any questions about the programming paradigms? All right, so now let's look at contention and deadlock and what we can do about it. So resources, they're the passive entities that programs use in order to function. Things like disk drives, IO devices, memory and so on. And they come in two flavors, the preemptible resources, commonly the CPU which can be scheduled and there are non-preemptible resources such as disk space, printers and so on which normally only one process that's accessed to at a time. And so resources may or may not require exclusive access from the processes or they may be shareable. So read-only objects are typically shareable and objects who state you're modifying such as printers are normally not shareable. All right, and the operating system has a duty of managing access to and protecting the resources. Across users. Okay, so there's two ways to get into trouble with multiple threads running. The first of them is starvation where threads are waiting and for some reason unable to run. So starvation is a simpler problem where threads may not be able to run because they lack priority. Sometimes because, well, if the operating system's not preemptive, they may also be stuck because nobody's given up control. Deadlocks are a more serious problem which is that threads end up waiting on resources that they can't access because some other thread has reserved them and is not gonna give them up. It inevitably involves some kind of cycle and normally at least two resources although people have managed to find ways to deadlock with one resource and one thread. Normally involves at least two though. And the resource is owned by one thread which is waiting for a second resource which is owned by another thread which is waiting for the first resource. So it's a surprisingly common problem as you saw in that list of top concurrency bugs. It's one of the top three. So deadlock implies starvation. It implies that certain threads are unable to run but not vice versa. And starvation can end in a variety of ways sometimes changing a scheduling policy but deadlock cannot change unless there's some more drastic external interventions such as killing one of the threads. Okay. Very often probably, well actually probably almost always deadlock is non-deterministic and it requires specific timing in the acquisition of resources. And we saw an example of this before actually when we were discussing semaphores for the first time. If you recall, P is the probe function which decrements the resource and weights if it's zero and V is the increment function. And if you put these out of order and if in addition they execute in the wrong order you get a deadlock. So basically A is grabbing the first mutex, B is grabbing the second one, now A can't acquire the second one and B can't acquire the first one. So you get the loop structure that we just saw. On the other hand, if thread B is able to execute a little bit faster than thread A and acquire both resources, then it just runs to conclusion and then thread A will start. And in fact, there's multiple ways in which this can evolve. All right, so because of this dependence of multiple resources there's often not a simple way to sort of break the problem down and solve it separately. You have to deal with the whole cyclic structure of the deadlock. So intuitively we can use a couple of metaphors for this. Deadlock is very similar to gridlock on a highway or a freeway. So gridlock arises when you get different vehicles trying to use the same piece of road at the same time and there's not enough road for the people like a one-way bridge is an example. So in that case you have to resolve the deadlock by somebody giving up the resource that they've partially acquired. So a more realistic example comes from routing on multi-core processors. In a multi-core processor you have some kind of physical wired interconnect network and you try to route messages from one processor to another and you want that routing to be extremely fast, meaning you wanna basically get a wired connection from the source to the destination without buffering in the middle. So somehow that requires you, if you wanna send from here to here you have to reserve both the wires along the way. And as you can see from this picture it's potentially very easy to get gridlock. In fact with a hardware processor where things are happening on a very tight schedule deadlock's very likely because each core here might first reserve the first link that it needs to get to its destination. And so you'd have a synchronous acquisition of the very first resource and then when you try to get the second one you'll get a deadlock. So sorry we'll come back to that one later but you can see that there's a real potential of deadlock here. Another situation that Amore perhaps a more humorous version of this situation is the dining philosophers problem. So in this situation you imagine five philosophers sitting down at a table in a Chinese restaurant and there are only five chopsticks. And each philosopher intuitively grabs for two chopsticks but in the worst case only acquires one of them. Say the left one. So again you get the cycle of each actor owning one resource and trying to acquire another one. All right so those examples give us some of the elements of deadlock so let's try to define them more precisely. The first requirement is that there's some kind of mutual exclusion in operation. So the resources are such that only one of the processes can use them at a given time. And secondly there's at least there are several hold and wait states where one process is holding some resource and trying to acquire another. The holding is preventing some other process from acquiring that resource and the wait is causing that process to stall and not continue and potentially blocking other actors that are trying to acquire its own resource. And sorry a third criterion is that there's no preemption meaning that all of the actors along this chain have the same priority. So in other words an actor can't simply grab a resource that some other actor is holding because that would break this cycle. So there's an assumption that nobody is getting priority and able to sort of break the cycle and move ahead. And the last condition is that there should be a circular weight. So sorry we have a thread T1 that's waiting for a resource that's held by T2 and then another thread T2 that's waiting for a resource held by T3 and so on. The last thread is holding resource that's held by T1. So that cycle is propagating this waiting action back and back and back and all the way around in the cycle. All right, so we can analyze the situation a little bit more generally using a resource allocation graph. So in the resource allocation graph we have a set of threads T1 through TN and a set of resource types R1 through RM. And for each resource type we can have multiple instances of the resource. So the instances are shown inside of the square blocks there. So there's three instances of resource type R2 and one instance of resource type R1. And the threads are going to request and then use and then release their resources in some fashion. And we can represent all of those actions by the threads using a graph. All right, and so again this entire set of objects is partitioned into the threads and the resources. And we have a graph between these objects from threads to resources which are the request edges and then assignment edge is going back from resources to threads. So a request arrow means that this threads trying to acquire resource RJ and the assignment edge says that RJ is already owned by the thread. Okay, so here's an example resource allocation graph. All right, so what does this mean? So this arrow here means that this resource is held by T2, T1 is requesting that resource. Here we've got two instances of resource type R3 being held by two different threads. So is there any deadlock there, anybody? There's sort of a cycle, but is it a problem? Well there is an undirected cycle, is that a problem? Well which of these threads is waiting on something? Well T2 is potentially waiting on T1, right? Because T1 is currently listed as the owner of that resource, but on the other hand T1's waiting on, well it's only uniquely assigned to that instance of resource R3. So it's not really waiting, it can just grab R3 and run and finish. So a requirement of the deadlock is that the cycle should be directed so that every thread's being blocked by something. This thread's not being blocked by any other thread. All right, and what about this one? This is, all right. Who thinks there's a cycle here, is a directed cycle here, right? If I take this arrow away, there's a directed cycle. But is it deadlock? All right, I heard a yes, go ahead, right, right. Yeah, potentially, yeah, there's two instances of R3, so in fact, actually it's somewhat complicated, but yeah, it's not a clear deadlock. Depending on the order in which things happen, if T2 or T1 is able to execute, it basically clears the cycle and this process can continue. All right, and actually no, I'm sorry, I got this wrong. There is a problem here because of that, yeah, there is a directed cycle. Everything is basically pinned by a preceding processor, sorry about that. Here we have the cycle that doesn't comprise deadlock because this second resource is actually consumed by T4. So, all right, sorry about that. Take a quick look at this and I'll try to work through this a bit more carefully. So the trouble here is that these two resources are potentially, one of these is owned by T3 so there's potentially a cycle through there, which can cause deadlock. Any questions? So we're gonna, rather than going through this example, we're gonna do an algorithm for detecting deadlock so I think we should probably wait till that. All right, let me see. So request edge is a directed edge from T1. All right, so T1 is requesting R3. Yeah, you're right actually. All right, so T1 is requesting R3. And yeah, all right, thank you. So that makes it clearer. So yeah, T1 is requesting R3 and both instances of the R3 resource are being held by different processes. All right, so yeah, good comment. All right, so yeah, it is actually, it's much simpler to explain now because yeah, we see these two resources being held here. This T1 here is trying to acquire one which is being held by T2 and so on. All right, so that's where the cycle comes in. All right, so, okay, so we are gonna go through and do a fairly careful procedure for detecting deadlock which will solve this case and other cases. So, some administrative here, the project one design documents are due tomorrow at one minute before midnight so there's no slip days on the project design documents and it's helpful if you post anonymously to post non-anonymously to Piazzo. You can post anonymously if you want to but it helps us to understand your issues if you get posted non-anonymously and it also allows you to get credit for class participation in Piazzo. So it's better if you can do that. Well if you, yeah, all right. So all right, well, so I assume that you could create a pseudonym if you wanted to but all right, so let's take a five minute break and we'll finish up and look at the some deadlock detection algorithms. All right, let's get started again. So I'm gonna try to get through the rest of the material quickly so let's just try to stay in sync and see what we can do. So there are several methods for handling deadlock. A simple method is just let it happen and try to recover. So that assumes that you have a way of detecting deadlock and once threads become locked that's usually straightforward. Finding a cycle of circular weights is fairly easy. There's a Java call that you can use within a program or if you want JVisualVM which is a utility that's in the JDK, does anyone run that? It's very useful for profiling but among other things it allows you to examine the threads in a live running program without the overhead that you get with using the debugger which sometimes causes strange deadlock situations not to happen. So you can wait for deadlock and then examine the deadlock using this routine. You can forcibly preempt resources once you recognize it. There are a variety of deadlock prevention systems that attempt to make sure that deadlock just doesn't happen. Those typically involve monitoring resource requests i.e. lock acquisitions and deny those in case they might lead to deadlock. And finally you can ignore the problem and just allow the programmer to try to deal with it and that's the approach taken in UNIX. All right so you can look for a simple cyclic loops on individual resources, a more general deadlock detection algorithm use sets of resources and so we assume that we have a vector x of numbers of resources of each type. So free resources x is a vector of those numbers. A request is a vector of requests, again a number of objects that you're requesting of each type and finally the allocation is the number of objects of each type that you're allocated currently. So here we have two resources and I guess four threads so you'd have a vector of length two with account of the number of resources of each type that you are trying to acquire. And the routine basically tries to simulate those threads acquiring and then releasing the resources in some order and if you can find an ordering in which everything holds it determines that there isn't deadlock because if there is an ordering the threads themselves will find it by executing. So the basic cycle is for every node that hasn't yet finished look at their request see if it's less than or equal to the available resources at that time. If it is you simulate them accumulating the resources and then releasing them. So you remove that node from the list of unfinished nodes increment the set of available resources by what you had allocated that to that node. So that's simulating releasing those resources. And you set a flag to say that you're not done yet but you basically go back to the do loop and reset that flag done flag is just testing if there's any work to do in the unfinished list. Okay, so once there's nothing left in the unfinished list if you weren't I'm sorry if you exit with the unfinished list not empty that means that there were some nodes that weren't able to acquire their resources therefore you have deadlock. All right, so here's our collection of objects again. So there are two resources. So each vector is a vector of two elements. So this is the count of resource number one that thread one is requesting and this is the count of resource number two that thread one is requesting. All right, and in the beginning we've allocated thread T one one copy of resource number two. So that's the arrow there. So in other words, this arrow is the allocation and this request is the request vector there. All right, and similar the other arrows are represented in these request and allocation vectors. Available resources is zero because you can see all of their resource instances are currently held by something and unfinished basically everything's running currently none of the threads have finished executing. So we can simulate by working through this routine the request. So we pick a node which is gonna be unfinished so we'll get T one first. T one's request here one zero. Is it less than the available resources? Well, there's nothing actually available right now. So that's gonna fail and we'll skip to the next node. Okay, so it's gonna be T two. T two's requesting nothing. So that actually is less than the available request. So T two, in other words, T two has this resource here. So it's able to run and finish and depart from the queue. And as it does that it's gonna free up its currently allocated resources here. So a lot T two is one zero. Those go back into the available list there and we continue. So T two is out of the picture. We're gonna look at the next node which is T three. T three is requesting an instance of R two and R two is not currently available. So we'll go back and try again. We'll try T four. And anyway, you get the idea. T two, T four is not requesting anything. It already has its resource. So it's gonna depart from the picture. And similarly, as we continue through back to T one, it has the resource that it needs. So it's gonna exit the picture. Finally, we're gonna get back to T three which had the resource that needed and it's gonna exit. So through the simulation, we were able to show that all of the threads were able to get their resources and finish. And in the process, free them up for other threads. And that's how this whole process, the whole picture was able to clear. Okay, so any questions about that algorithm? So it's an effect of simulation of the whole process of threads acquiring resources. We're not guaranteeing that things happen in the same order. But what we're showing is that there's some order for which the processes can acquire what they need in complete. And if that's true, then they won't be deadlocked. All right, so what about some techniques for preventing deadlock? One approach is to somehow present very large numbers of copies of the resources. So for instance, most operating systems give you a huge number of file handles. Any kind of object that's cheap, that consumes a small amount of real resources is easy to make close to infinite. And that will often prevent deadlock situations by basically always allowing threads to acquire more instances of each type. But of course, things like printers, you can't do that. Things like routing paths on a multiprocessor network, you can't do that with. You could require that there's no sharing of resources. Something like that is done in the active framework. But there are some pretty severe restrictions of that. We talked about that earlier in the class. Another approach, maybe that's more promising, is by not allowing waiting, perhaps having a return message or callback approach that basically stops the caller from blocking and just waiting for the resource and allows them to do other work. So that actually is a more realistic approach. But again, it doesn't work for everything. So, and as we drill down a little bit, there are also techniques that are closer to that simulation of resource acquisition that we can use dynamically to predict if deadlock can occur and again prevent acquisition of resources that might lead to deadlock before it happens. So here's an extreme example is deciding not to leave home because anyone might be on the highways and freeways and therefore might cause you to be prevented from going where you wanna go. That's far too conservative. A realistic approach that is actually used is ordering the priority of requests. So we already argued that in order to have gridlock, we have to have a circular sequence of dependencies with no preemption and if you enforce a certain type of ordering, you can prevent those cycles. So a simple way is to use geometry because cycles, for instance, on the processor grid have X and Y orientations. So let's go back to that problem. So again, the problem was to deal with the circular dependency of resource allocation where we have say four processors that are trying to route some streams of bits around the multi-processor grid connection network and each processor is trying to communicate with the processor around the corner. So they require two links to stream the bits around but you've got four processors each requesting two links in this example and with a simple synchronous request policy where each node requests the first link first, you can see that you get a deadlock situation here which is like gridlock. So the solution is to use a dimension ordering because in a geometric cycle like this, there have to be X links and Y links. Dimension ordering gives priority to one of the directions, typically the X direction and says, all right, this request here is starting with an X route first. It's now requesting the Y route but the initial request was for X and therefore the second request will actually preempt the guy that's holding the second link here. So the first request is getting both links because their initial direction was in X. Similarly for the request down here, this request from the corner guy here preempts this other guy and so you get two of the nodes getting their entire path as a result of preemption. So that allows those two nodes, the two corner nodes to route their packets through the network. When they're finished, the other two nodes can get their request fulfilled and route. So that's an example of X, Y routing and people have realized that actually this is a constraint on the turning directions and it's often expressed in terms of east to south turns or west to north turns and you can actually use the turn direction as a constraint and basically give priority to these turns over these ones. So all right, so any questions about the routing preemption? Yeah, no, I think that's a very good comment. I mean, yes, you essentially have to have an a priori knowledge of what the requests might be in some way of creating a total order on them which can be quite hard, yeah. Yeah, yeah, if it's a store and forward network that's not too bad, although the cost of buffering here is very high and but for a process of switching network you're trying to get these packets across extremely fast and if you have to buffer your doubling the time. In fact, this processor, it's often gonna start working on the bits before all of them have arrived for a lot of operations. So, no, this is really a switching network so you need to switch all the way through. All right, so the last topic is building on this idea for deadlock detection with multiple resources and kind of making it dynamic so that it can be used as a potential deadlock detection algorithm. So, the idea is to require a bit of extra information from the threads about what resources they might use in the worst case and then making sure that your available resources are at least enough to fulfill the requests by other threads. So, in other words, we're talking about a single thread that's trying to proceed. It requests some resources. There's a certain number of available resources and basically this test says that after you give those resources to this requesting thread, everybody else, what's left is at least enough for the largest remaining thread. Okay, all right, so that's a pretty heavy requirement. The banker's algorithm is a more modest algorithm that requires a sort of mini simulation of the style that we just did a few slides ago whenever a new request comes in. All right, so when a new request does come in, you grant it if you can find an ordering of the threads grabbing using releasing the resources that doesn't cause deadlock. So in other words, you do a full simulation of the kind that we did a few slides ago. And this guarantees that the system can't deadlock because you've proved that there's some execution of threads, the least one thread that can proceed at each step acquire what it needs run and complete. And so the threads may not do exactly that, but in the worst case, they'll do what the simulation simulates. And so it allows the maximum resource needs of all the threads to be greater than the total resources. So it's in other words, more not as conservative as the first one. All right, so before you grant the request to the thread, you pretend that it's been granted and then run the deadlock algorithm on the resulting request graph, right? So that's a piece of, this is sort of transformation we did to the previous code, but here's the routine by itself. So again, we go through each node that's not finished in the process of resource allocation graph and make sure that the maximum request of that node because we don't know what it's actually gonna request minus the current allocation is less than the available resources. All right, so this is somewhat different from what we had before where it was a static allocation graph. Here we're saying we have an upper bound on what this thread might require as it's running and we'll let it run as long as its maximum request minus what we've allocated is still gonna be available at least for some thread, okay? So it's a little bit more dynamic than the previous version. The previous version was using for this block here what each thread was actually currently using. Here we're saying we don't know what it's going to use, but as long as its maximum request minus what it currently has is within the available set of resources, then it can clearly run so we can simulate it running and remove it from the queue of things that still have to finish, free up its resources and then declare that you should continue through the loop. So in other words, it's exactly the algorithm as before except that it uses kind of an upper bound on resources instead of the actual resources allocated. All right, is that intuitive? Okay, so, all right. So applying that to the dining philosophers problem. So we recall we had five, effectively five threads and five chopsticks, we wanna allocate them. So there are some situations where we can examine what the philosophers actually did in the process and determine that we won't get deadlock. If either when somebody grabs for a chopstick, it's not the last chopstick or it is the last chopstick, but somebody will have two of them afterwards. All right? So we can let, we're not in risk of deadlock when the first few philosophers grab a chopstick. We're not in a bad situation yet. And in particular, if a certain philosopher, so we let this thing run a little bit if some philosopher grabs two chopsticks, we're actually, we're golden because they're gonna just finish and then eventually you'll get to eat, might be much later, but you know, they can complete their task, free up the resources and everything will be fine. But if let's say four of the philosophers have run grabbed a chopstick, then that's the stage where the algorithm may declare that there could be deadlock, right? Because if it is the last chopstick, then you will have a circular resource allocation graph. So the bankers algorithm will determine that you shouldn't give the last chopstick to the last philosopher if there's already a cycle or partial cycle. All right? But if it's not the last chopstick, then you keep going. So you let the algorithm run because it may all by itself avoid the deadlock. Is that intuitive? That's not a complete simulation, but it's perhaps enough of the essence of the method that you can see how it works. Yep. No, it's simulating the, I suppose the operating system, granting the resource requests. Yes, it's sort of last line of defense that's saying you let whatever piece of code you're running, you don't really know what the piece of code might be doing as far as allocating the resources. But when you can determine that this, because you have some visibility through the resource request, the locks and the condition variables that you can determine, actually just through the locks, I'm sorry, but you can determine that there's a potential cycle and then you can stop, just avoid that allocation. That's a fair question, although I think one saving grace is, again, if you look at the empirical studies of concurrency problems, the most common problems are happening with two resources and two threads, so in which case this is probably fine. Was there another question? Yeah, no, I mean, we're giving you probably artificially complex examples. You know, the frequency of complexity of these examples goes down exponentially because it's very unusual. In the philosophers case, the probability of everybody grabbing just one chopstick at the same time is very low. If you do it, naive simulation of this kind of situation, it actually won't deadlock very often, even with sort of fairly consistent execution of all of the threads. So as the cycles become bigger, the probability of actually deadlocking them becomes quite small. So most likely in the simulation, you'll actually fairly quickly exit from the routine. Like each resource will, you'll basically determine that each thread can complete and finish, and then you're done. All right, so, okay. And the last thing is if you have, they will generalize to k-handled philosophers if that's the last chopstick, but nobody would have k of second to last and no one would have k-1. So there's a sort of a generalization of that one. All right, so let's finish because we're a little bit over time. So, all right, so we talked about, initially we talked about software support for concurrency. We spent a good amount of our time on starvation versus deadlock. Starvation means some setup where threads are waiting, but not necessarily wedged. Deadlock is a non-continuable state, requires external intervention with a circular dependency chain, requires all of these four conditions. And then we talked about ways of both detecting deadlock and then both statically, but also dynamically and hopefully potentially preventing it by not allocating resources that might lead to deadlock. And that was the bankers algorithm. All right, thank you.