 Hi, hello and welcome back to program analysis. In this fourth and last video of this lecture on analyzing concurrent programs, we will look into exploring different interleavings that the execution of a program may actually trigger. And this is a problem that is kind of orthogonal to the problems that we've looked at in the previous videos of this lecture, because in the previous videos, we were always assuming that we somehow triggered the right interleaving when executing the program. And now we will look at one approach for hopefully triggering the right interleaving a little faster. Again this is based on the paper, so if you're interested in more details, please have a look at this paper down here. So the underlying challenge that we are addressing here is the problem of scheduling non-determinism. That's a fact that you've already seen with one of the examples that I've shown earlier. It's the fact that a single program executed with a single input may have many different interleavings and therefore also many different behaviors. Now in practice there's some scheduler that decides these interleavings non-deterministically, which means if you execute a program multiple times, you will get different interleavings. And many of the concurrency bugs have the nasty property that only some interleavings actually expose the bug. And others just execute correctly, which has given these bugs the nickname hyzen bugs because they kind of disappear when you try to look for them. So the challenge is how to actually explore different interleavings in a hopefully systematic way and how to then also detect buggy interleavings. So one way to explore interleavings in a systematic way is an approach called chess, which we will have a look at now. So chess is a user mode scheduler that controls all scheduling non-determinism of a program. So you essentially give chess a program that has multiple threats and then it's controlling in which order the different statements and operations of the concurrent threats are executing. Chess makes two very interesting guarantees. The first one is that every program run takes a new threat interleaving. That means if you execute a program three times, you will have triggered three different behaviors or at least three different interleavings that potentially lead to different visible behavior. The second property that chess guarantees is that it can reproduce the interleaving of every run. So this is very important because otherwise you have these hyzen bugs that you maybe observe once, but then you cannot observe again because you can't reproduce the interleaving. But with chess, if you have seen an interesting run, for example a run that exposes a bug, you can reproduce this run and debug it in more detail. So the way this works is through systematic but non-exhaustive exploration, which basically means that chess is looking at the set of possible interleavings and then systematically tries one after the other without really attempting to try all of them because for realistic programs there are more than you can reasonably try in the time that's typically available for testing. So to reason about the different interleavings that the execution of a program may have, we need some kind of representation of this space of interleavings. And the way this search space can be represented very nicely is as a tree of interleavings, where basically every path through this tree, so from the root to some leaf node represents one possible schedule that a program is taking. So nodes in this tree represent scheduling decisions, so basically every point in the program where this thread or that thread may execute next is represented as a node in this tree. And then the edges correspond to decisions that are taking, so from every node there are at least two outgoing edges that represent that you either do this or do that. And then if you have the tree built up this way, then every path from the root down to a leaf represents one possible schedule and one possible interleaving that the program may take. So as an example to illustrate this tree of interleavings, let's just have a look again at this example that we've seen earlier on in this lecture, where we have this piece of code that starts with some initialization of variables and then has two concurrently running threads that each are executing two statements. A slightly more abstract way to look at this example is by just saying that we have one thread T1, which is executing a statement one followed by a statement two. Doesn't really matter what these statements are doing. And then we have statement T2, which also has two statements one and two, which maybe usually are different from what thread T1 is executing. Now using this abstract notation, we can try to write down the scheduling tree of this little program. And we're starting here with the root where we just called the root 0, 0. And the notation here is the following. So for every node, I'm having two values such that the first value is the last instruction that has been executed by a thread T1. And the second value is the last instruction that has been executed by a thread T2. So 0, 0 essentially means we have not yet executed any of the instructions of any of these threads, but are literally at the beginning of the concurrent part of the execution of this program. So when the program is in this state, 0, 0, there are two options, what could happen next? And each of these options is represented as one outgoing edge out of our node. So one option is that we start by executing the first instruction of thread one, which is represented as 1, 0, because then the first execution of the first statement in T1 has been executed, but nothing yet in T2. And the other option is that we start with thread two, which is represented as 0, 1. Now after having executed the first instruction in thread one, what we could do next is to also execute the second instruction. And then once we've done this, there's only one option left, namely to execute the first instruction of T2 that we haven't yet executed. And then the one outstanding instruction, which is the second instruction of T2. What we could have done here, instead of continuing in thread one, is of course to also switch to thread two. If we do this, we end up in this state 1, 1, because now we've executed the first instruction of each of the two threads. And in that state, we again have two options. One is to go back to thread one and execute its second instruction, after which we then will go back to thread two. Or we stay in thread two and execute its second instruction, and then go back to thread one to again end up in state 2, 2. And on the right side of the tree, you can do essentially the same, just the inverse, where at each node you basically have to decide among all the possible instructions that are executed next. And at the end, you always end up in state 2, 2, because at the end we always must have executed all statements of all threads. So as you've seen for this tiny program that has just two threads, that each just have two instructions, it's possible to write down the entire tree of interleavings. But let's now have a look at the space of these interleavings for more realistic programs. So in general, we have not just two threads, but we have n threads. So thread one to thread n. And let's just to simplify things a little bit, each of these threads has k instructions. So in every thread we have k different instructions or statements that need to be scheduled. Now the number of interleavings that you get in this general setup is in the order of n to the power of n times k, which means it's exponential in both n and k, which does not really sound good. And in practice also is not good, because typically you have a lot of instructions per thread, so typically more than hundreds. And a relatively small number of threads, so typically less than 10, but still both together leads to the situation that it's practically impossible to explore all interleavings, in particular because of k, which is getting pretty large pretty quickly, giving us so many interleavings that you just can't explore all of them. So now in order to still be practical, chess implements a pretty clever idea. This is called preemption bounding. So the basic idea is to limit the number of times that the scheduler switches from one thread to another. So each of these switches is called a preemption, and the basic idea is to limit this number of preemptions to a small number c. And if you do this and think again about how many schedules the scheduler then has to explore, you'll find out that this isn't the order of what you can see here. And if you look at this term, you see that it's still exponential in c and in n, but not anymore in k. And as we've seen on the previous slide, k, this number of instructions per thread is what is really going high in real programs. Now this idea of preemption bounding is based on the empirical observation that most concurrent c-related bugs can actually be triggered with few typically less than two preemption. So you do not have to go back and forth between threads that often, you just have to go back and forth at the right points in time and are still able to trigger most concurrency bugs. Now of course this is a empirical observation and this whole idea of preemption bounding is just a heuristic. So it's of course possible that you're missing some concurrency bugs if you limit the number of preemptions to a c that is too small. So this idea of chess has been implemented through binary instrumentation. So basically taking an existing binary program that has concurrent behavior and then it's instrumented in order to control the scheduling decision that happened when the program is executing. And then this implementation has been applied to a couple of midsize and larger systems up to 175,000 lines of code in which the tool was able to find a total of 27 bugs. That's a pretty nice result and beyond the ability to find bugs that you may not find if you just randomly repeat the execution of this program. So stress testing is that once you have a failure detected it's easy to reproduce and debug it because you know which interleaving has triggered this failure simply because the scheduler has controlled the interleavings and then also knows how to trigger the same interleaving again. Finally, let me say that there are also other ways of controlling the scheduling decisions, of course. One, and that is surprisingly effective in practice is to just randomly delay some of the concurrency related operations. So for example, if a threat is trying to acquire a lock you can just randomly delay it a little bit hoping to trigger a different interleaving by doing this and more often than not this actually turns out to trigger behavior that is able to detect some bugs. Then there are different pieces of work that use various heuristics in order to influence the scheduler. For example, based on known bug patterns where you know that if you influence the scheduler in this way or that way it's more likely to trigger a bug and in a sense chess can be seen as one instance of this heuristics-based idea. Another one is to use a programmer annotation. So if a programmer explicitly says that maybe here we should wait a little to see if the program still behaves correctly then you can use this as a way to control the schedule. And finally there's a pretty nice way called active testing which works in two phases. Phase one is to find potential bugs using some other kind of analysis for example aesthetic analysis of the code and phase two is then biasing the scheduler knowing about these potential bugs to what confirming these bugs in an actual execution. So for example if we know that two memory accesses maybe in a data race then maybe we should wait before one of the accesses until the other one is ready and then see if by doing this we can trigger some interesting behavior that hopefully exposes some bugs. Alright and this is the end of part four of this lecture on analyzing concurrent programs. I hope you now have a better idea of how we can actually control the interleavings that happen while the program is executing so that we can trigger interesting bugs faster and maybe even reproduce the interleavings that are triggering these bugs. Thank you very much for listening and see you around.