 Hi everyone, it's Jeff Chalon and this is the second screencast for assignment one. So in the first one we set up our environment, that was pretty fast. You guys are used to how to do that now after assignment zero. So we're going to start looking, this is the first assignment where you get to write some code and an important part of writing the code that you need to write for this assignment is understanding the code that's already there. So in this screencast I'm going to show you some of the code and a great starting point is to look at the implementation of semaphores. So let's get started. So let's look at the existing semaphore implementation. Now when you're looking at code in OS 161, it's always very helpful, so I could start reading the semaphore code here and here's a couple of the functions and here's exactly how they're implemented. However, it's very helpful to look at the includes. So in this case I'm going to look at include sync.h and the reason for that is David has put really nice comments here in the include files and in some way C header files define the interface that's implemented by the .c file itself. And if you understand the interface, you may have a great, you either have a great starting point for understanding the code or in some cases you don't even need to understand the code and look at how the code works. And I'll show you an example of that when we start looking at the wait channels. So what does this tell me about semaphore? So let's look at the sync.h file. It indicates the name field is just for debugging. These are the fields inside the semaphore, so the semaphore has a count. It also has an associated wait channel and spin lock and I'll show you in a minute about how those get used. There are two operations that initialize and destroy a semaphore. It's pretty common. And then these two operations define the semaphore interface. So a semaphore has a P operation and a V operation. And as promised, the comments here do a nice job of explaining what those operations do. P and V are historical parts of the, you know, historical ways of naming these functions. They go date back to Dijkstra, who was the one who came up with the idea of a semaphore in a paper, a very seminal paper. However, they're going to be a little bit hard to understand. And the Dutch words here, the funny thing about them is that they are kind of made up as well. So you might think these are real Dutch words. I think one of them is sort of a made up word. But I usually like to think of these as just up and down. So this is down, this is up. What does V do? So a semaphore maintains a count internally. V increments that count. These operations are both atomic. And that's all V does. V will never block. That's important to understand about V or an up, is that V cannot block. The P is a little bit more complicated. So the P will decrement the count. However, the semaphore semantics indicate that the count cannot fall below zero. So if I'm performing a down that would drop the count below zero, I need to wait until somebody calls V. And this is what makes the semaphore a synchronization primitive. And this is what allows us to use it to coordinate between multiple threads. Because if one thread calls P on a semaphore that has a count of zero, it will block until somebody else calls V on that same semaphore. Okay, so here's our interface. And again, if I just wanted to use a semaphore, this would probably be enough for me to understand what the interface is and how the semaphore gets used. However, you guys actually need to implement some of these, including your own locks and condition variables. And so let's actually look at the code that implements P and V for a semaphore. Let's start with V because it's a little simpler. So what does this code do? The first thing it does is assert that the semaphore is not null. This is really good programming practice. We encourage you to use the K assert features aggressively to test things about the world that should be true. So if a V gets called on a null semaphore, that's a problem with some other part of your code that you want to find as quickly as possible. And by asserting that the semaphore is not null, this allows you to make sure that you see as quickly as possible if you called V in a way with an uninitialized semaphore or something like that. So this is really good programming practice. You'll see it down here, too, where after I increment the count, I K assert that the count is greater than zero. In a lot of cases, K asserts can seem kind of obvious. Well, look, I have a count value and I just incremented it and now I'm asserting that it's greater than zero and the count itself is unsigned. So why would that be there? Again, the count might have been overwritten by something and might end up ticking on a value that is interpreted as negative. So this is good programming practice. You don't know what else is happening and in the best case, this K assert just does nothing. In the better case, it helps you find some problem with the code. The semaphore defines a spin lock and that spin lock is used to protect its internal state and also required by the weight channel interface. So the internal state of the semaphore is the count and both V and P, when they manipulate the count, hold the semaphore. And so here's my critical section, sorry, hold the spin lock. So here's my critical section that's defined by the spin lock. Starts here, goes down to there. So it's these lines, spin lock acquirer, spin lock release. Remember, spin locks are used to protect short critical sections that do not sleep. So in this case, all I'm doing here is bumping a count and calling this function weight channel, wake one, which we'll explore in a second, and neither of these operations can sleep. So this is a safe place to use a spin lock as opposed to a lock that might sleep. OK, and what do I do while I hold that spin lock? Well, I increase the count. That's the core thing, one of the core things that V is supposed to do. And I also need to wake up any threads that are waiting for that count to increase. So I call this function here, w-chan wake one. And you'll see here that I pass at the weight channel that's associated with the semaphore and the spin lock that's associated with the semaphore. OK, so let's try to, so this is, again, a good way to start is to read the synchronization formulas that we've given you and some of the other test code and then kind of work backwards from there. So we've come to this function that we don't understand how it works. So let's try to find out. And I would encourage you, again, to start in the header files. So this is the header files that defines a weight channel. Similar to the semaphore, it has a create and destroy function. And here's the rest of the interface. So it has a call that allows me to determine whether it's empty. And then these are sort of the core pieces down here. So there is functionality that allows me to sleep on a weight channel and then wake up one or all of the threads that are sleeping on that weight channel. So the weight channel represents a group of threads that are all waiting for the same thing. In this case, the weight channel for a semaphore defines the group of threads that are waiting for the semaphore's count to go from zero to one so that they can complete their P. So I can go to sleep on the semaphore. That means I'm waiting for something to happen. And if I am the thread that has caused that thing to happen, I can wake up one or all the threads that are waiting on that semaphore. So you'll see over here, if I'm being the semaphore, I'm increasing the count. So if that count has gone from zero to one, then I am the thread that is going to wake up other threads that are waiting. So I'm gonna call W Chan wake one. Now note here that V doesn't check whether or not the semaphore count is one, it just calls W Chan wake one. And that's safe to do. It's possible that nobody is waiting on this semaphore right now. But it's safe to call W Chan wake one. Now why would I call wake one versus wake all? It would also be safe to call wake all here and wake up all the threads that are waiting on the semaphore. But I know that I've only increased the count by one. That's the semantics of V. And so no matter how many threads are waiting for the semaphore to go from zero to one after calling P, only one of them will be able to proceed. And by calling weight channel wake one, I ensure that I only wake up one thread and that's the thread that gets to continue. If I call weight channel wake all, what would happen is that all the threads that are waiting for the semaphore would wake up. One of them would win and get to complete its P and the others would go back to sleep. Okay, so that's V. Prior to the spin lock, bump the count, wake up anybody who's waiting on the semaphore. Okay, so what does P do? The P's a little bit more complicated because P will block. So similar to V, I have a couple of asserts here. Make sure the semaphore is not null. And then down here I assert that I'm not in an interrupt handler. And the reason for this is that when I'm in an interrupt handler, I can't do anything that could potentially sleep. And you'll see down here I'm about to call potentially weight channel sleep. And so I assert that I'm not in an interrupt handler. Now similar to V, I use a spin lock for two things. I use the spin lock that's part of the semaphore for two things. One is to protect the count. So you'll see here when I manipulate the count, I'm holding the spin lock. That's because the count is part of the shared data structure. It's internal state to the semaphore. I also use these spin lock to get onto the weight channel safely in the case when I need to sleep. So what does this piece of code do? It says if while the count is zero, I'm going to sleep. So I call weight channel sleep. And one of the things you'll notice over here is part of the weight channel interface is that calling all the weight channel functions requires me to pass in a spin lock. And in certain cases, the weight channel will just check to make sure I hold the spin lock. So if I call weight channel wake one or weight channel wake all, all it does is make sure that I hold the spin lock because that's part of how I use the weight channel correctly. However, if I call weight channel sleep, this function actually manipulates the spin lock. So what does it say it does? It says the associated lock must be locked. It will be unlocked while sleeping and relocked upon return. So what's going to happen? Let's say that I loop through this piece of code several times. So I've acquired the spin lock. The count is zero. I come down here and I go to sleep. So in the process of going to sleep, I'm dropping the lock. That's important. If I held the lock, nobody else would be able to do anything to the count and I would sleep forever. So if I held the lock through this sleep, no V would ever be able to proceed. And so the count can never increase from zero to one, which is what I'm waiting for. So weight channel sleep drops the lock, goes to sleep on the weight channel, and then when it's awakened by a V, it grabs the lock again. And that's what allows me to safely check the count again up here. And that's what allows me to drop out of this loop holding the spin lock. So every time I check the count value, I have the spin lock because it's reacquired after weight channel sleep. And then down here at the bottom, if I break out of the loop, if I'm the thread that gets to break out of a loop, if I'm awakened and the count has increased, then I know that I hold the spin lock right here. So what does the rest of this do? I K-assert that the count is greater than zero. That's important because I'm about to decrement it. So remember, the semaphore count cannot fall below zero. And then I release the spin lock that's part of the semaphore and I'm done. So these two things together, and you should sort of walk through a couple of examples of what would happen. Let's say I have 10 threads that I'll call P and what's gonna happen? So let's say that I initialize the count to zero, they're all gonna come through here, they're all gonna hit this condition and they're all gonna go to sleep. And they'll sleep there until one thread comes through and calls V. Or what that thread will do is it'll bump the count, it will wake up one thread and then it'll release the spin lock. So as soon as it wakes up one thread off the weight channel, that thread is going to start to run. So remember, as part of waking up, weight chance sleep will try to grab the spin lock. So that thread actually won't continue until this thread releases the spin lock. At that point, it'll come up to the top of the loop, the semaphore count will have increased to one, it'll drop out, decrement the count back to zero, release the spin lock. And then all the other threads that we're waiting for that semaphore will continue to sleep. So here's an introduction to the semaphore code. The next thing we'll do is look a little bit about how the semaphore is actually to get used in some of our test code.