 All right, welcome back to operating systems. So today, you should have what you need to start Lab 4. After today, you can start Lab 4 and save like 20 plus hours. So I will show you how to use UContext in Qs. So before we get into that, first, let's complete our diagram of a process. So here is what a process looks like with four threads. So each individual thread would have its own registers and hopefully independent stack. All four of the stacks in this case live in virtual memory. And what else is in virtual memory? Well, the heap and all global variables. Then other than that, besides threads and virtual memory, the process just has a table of open file descriptors as part of it, which we will get to whenever we get to file systems. But this is our more or less complete picture of a process with four threads. So any questions about this? All the threads live within the process before we just had registers in the stack. But now, we can include them as part of the thread. And by default, there would only be one thread instead of, in this example, four. All right, we're good, we're good, we're good, all right. So all this code is available to you that I'm doing today. You might, if you haven't looked at, the other examples, you might want to look at these examples. So they're in that materials repository. This is clickable. And specifically today is lecture 18. So that's where you'll find all of your code examples. So again, after this, I've updated description of lab four and stuff. So it will walk you through how the test infrastructure works, how to write your own tests. And then after explaining a really basic test, it explains a more involved test. So hopefully everything will make much more sense than lab three. So let's get into how to use a U context. So what the hell is a U context? So a U context, basically it's just a structure of like all the registers of your thread. And it gives you a bunch of nice helper functions to go ahead and save all the registers for you and restore all the registers for you. So if you are implementing your own user threads, which guess what you're doing in lab four, this will do most of the heavy lifting for you. So it is very good to understand how these functions work. So let's walk through the code. So at the beginning, I just define a bunch of includes and say what I use from each of them for you. We only really care about this U context today and these are the functions you are going to have to use. All the other ones are just helping me and doing things to make your life a bit easier. Like this Valgrind stack register. Turns out Valgrind does not like what we are about to do. So you need to be placated somehow. All right, sirens are really close, nice. So I created this, are we in trouble? No, okay, we're good. I'm hidden for another day. All right, so there is this die function. So it just saves error know that I've been doing pretty much this whole lecture, prints an error message and then otherwise exits with whatever the error know originally was. And for lab four, you should not have any fatal errors. This should only happen for debugging purposes only. Or if something really goes wrong like you've ran out of memory. So I define a function for you called new stack. You do not have to read it, but if you were to read it, well it basically just gets some virtual memory of length this stack size and your stack is always going to just be this size. It's just a default size that we want to use. So it will check for error messages for you and then it will register with Valgrind. So you can check your code for memory leaks using Valgrind. You need to, turns out Valgrind does not like when you use other stacks and start executing stuff. So without this, Valgrind will give you like pages and pages of errors that aren't real errors. So just use the new stack function then you can use Valgrind all you want. Then if we have a new stack, we should have a delete stack. So here's a delete stack that does pretty much exactly what it should do. You don't have to read the details of it. Then here, so for this program we're going to create three threads. We're going to create thread zero, thread one and thread two. So we're going to create three UContext variables and the UContext structure is basically a structure that has all of the registers in it and saves you lots of hassle with trying to deal with saving all the registers and writing pages of code. That really doesn't do that much interesting. So after that, I declare globals for two stacks, a stack for thread one and a stack for thread two. I do not declare a stack for thread zero because I'm going to just use whatever the stack is that started executing main, like the main kernel thread stack. I'll just use that for thread zero. Yeah, sorry. So char is just like a byte. So this is just a pointer to a byte so that will represent the start of the stack. So char pointer or void pointer, doesn't really matter pointer to I don't know or pointer to char which if you go at the definition should be a pointer to a byte. Same thing just turns out that the types that the libraries we're going to use want to be char and char stars, but it's just a pointer. It represents a stack. So it could be void. They just decided to be a char pointer for some reason. Don't ask me why, but don't worry. Just use char star and it's just an address and it won't give you any compiler warnings. All right, so here is our main. So we'll start off by doing a get context call. So what get context will do is whatever thread is currently executing right now, it will save all of its registers to this structure. So it will populate it with the values of all the registers of whatever is currently executing. And it will also play with the program counter back such that that if we try to restore that state again, t zero u context will resume here. So it will resume like it just called get context and keep on going. So the next thing we will do is ignore the comments for now is the counterpart to get context is set context. Or if you want to think of it in a different way, sets context will restore the registers from whatever this structure is. So if I do a set context here to thread zero context, well, it will restore it to whatever the last call to get context was, which in this case was here. So goes here, prints or goes here, then execute set context, then goes here, execute set context, goes here, execute set context. I probably did something really bad, right? So let's do a print statement. Oh, no. So if I do this, this looks pretty bad, right? What do we think will happen? Probably just print, oh no, over and over and over again, which is a fun way to infinite loop ourselves. So there we go, prints, oh no, over and over again, because it's just going to set context and then restoring whatever these values for all the registers were, which brings us back to here, and then prints and no, does the same thing goes over and over again. It's a fun way to give ourselves an infinite loop. So doesn't this sound exciting already? Yeah, all right, groans, perfect. All right, so let's do something fun here just to see if we have any questions. So create a variable called x, then we do that get context. Let's do something fun. Anyone want to hazard a guess as to what will happen here? So print oh no one, and then just oh no one over and over and over again. Oh no one over and over again, we all agree. Whoops. Did I not compare, I don't think I saved it. Hit capital W, whoops, ah, stop. Oops. Anyone care to explain that? So what happened? Hmm, yeah, so if I do int x, where does x live? The stack, right? So here, basically what this is gonna do is bump the stack up four and then use that memory for something, for x. So bump the stack up by four and then write the value two, or sorry, write the value zeroed there. I don't know why the hell I said two. And then it's gonna save all the registers to this context, including the stack pointer. So the stack pointer is also going to be safe to whatever it is. And then here, I increment x, which is like, hey, I'm going to load it from memory and then increment it in a register and then I'm going to write that value to memory. So this would change the value of x on the stack, which lives just in virtual memory somewhere from zero to one. Then we print off whatever the value of x is, which in this case is one. Then we do a set context, which goes back here and stack pointer just gets restored. In fact, it doesn't change at all. It's still pointing to the same thing and we don't even really care that much. So x lives in the same place, so its value was still one. So we increment it, it goes from one to two, then we print two, then set context again, increment it from two to three, that the da keeps on going like this forever. Sorry, any questions about that? Yep. Well, yeah, if you put it in the stack, or sorry, if you just put on a register and didn't use a stack, then sure. But if you, no one should use the register keyword unless you know what the hell you're doing. You could declare it as register in, oh god. All right, geez, why? There. Not a fix, do not use a keyword register at all unless you really need to. I guess you could use it for this. Anyway, yeah, well, so if I declare a variable in C, it just lives in the stack, so the C standard has to respect that. So is there a way I could make this always print one with just moving a single line of code? So what happens if I do that? So should I have it increment over and over again or will it always print one? Hopefully one, let's see. So that time it print always one. Why? Well, at the beginning, we didn't do anything. We saved whatever the stack pointer was, which had nothing. Then here it would have grown the stack by four bytes and then wrote that value zero there, then incremented it from zero to one, printed one, and then then it set context back. So the stack pointer got reset back to its original spot. Then it would have created space for an x or an int, so it would have used the same memory, wrote zero, so there still would have been a one there, so it would have written a zero and then incremented it from zero to one, then printed one, done it again and again, yep. So in this case, this will not overflow the stack because it keeps on resetting, allocating four, resetting, allocating four, resetting, allocating four. So this way, as long as this would go on forever, it would not run out of stack space. Because it's doing the same thing every time. Yeah, the stack pointer doesn't really matter. That same value is there anyways. It's just in this case, we're manipulating the stack pointer, but it would be the same thing if we just moved in x up here. It just wouldn't mess with the stack pointer, but memory would still be the same. All right, so that's fun. So let's see what else we can do. So I can delete that, let's get rid of that. So let's see how to create new threads that start executing something we actually want. So let's create thread one. So we allocate a stack for it, then to initialize that you context structure, even if you're going to overwrite some things, you still have to call get context to overwrite it. So we call get context initialize t1 context. So right now, if we did a set context, it would just return right back here using the original stack. But here we can play with some of the fields. So in order to change the stack for this thread, we can modify the UC stack field and it has two other subfields. So there's SS underscore SP. So that is the initial stack pointer. So you initialize it to the start of the stack, which is whatever you get from new stack. And then you have to tell it how big the stack is. And then after that, you can do a make context call. So make context will properly figure out what register represents the stack pointer depending on your specific hardware. So you could just do this on a Mac even if you wanted or an ARM processor. And then it will initialize this you context. So it will set those fields, fix them up, and it will also change the program counter such that it starts executing this function you give it. So in this case, I'm going to tell it to execute t1 run. And then I can also give that function additional arguments. If I want, it kind of gets kind of ugly. So I can say how many arguments I want to pass it and you are only allowed to pass integers. So here I will pass a single integer just to show you how it works. And I will pass a single integer with the value 42. And I have to do this ugly cast because C does not like this. So now if I do this, if I do a set context to t1, it will, it has been reinitialized here as part of this make context. So it has its own stack and it wants to execute t1 run. So it would swap everything over, start executing this function with a new stack. So here it should say, hooray, got arg zero and then print off whatever the argument was, which hopefully should be 42. So everyone agree with that? Hopefully, if I didn't lie to you, it will do any thoughts of, it will do something really weird after the printf, like what happens after it reaches here? Who knows what it does? All right. So this case looks like it just kind of ends. So we see we got our argument, it was 42, and then it reached to the end of this function. So by default, if a new thread from make context returns from the function it executes, it will just end your process straight up, which will not be good if you want to have other threads start running. So there are two ways to essentially combat this. You could have this do some type of cleanup at the end, or there is a special field in the U context called UC link. And what this will do is change that behavior. So if you set UC link, it will essentially do a set context whenever this returns. So to whatever UC link is. So in this case, if I set UC link to T zero context, it will do a set context there instead of ending the process. So that is all I did is I set, sorry, I set this line. UC link is thread zero context. And remember the thread zero context, well, currently that would be the last time that called get context and that would be back here. So if I run this, what should probably happen? Probably fairly bad things. I infinite loop myself again, but now I did it through two threads. That's fun. So what happened there? Well, I initialize get context for thread zero here. So if I do a set context on thread zero U context, it would resume here. Then here I create a new thread or a stack for thread one, do to do. I set it's UC link to go back to thread zero U context. I tell it to execute thread one. And then I do a set context to thread one. So now it switches over the stack and everything, starts executing this function prints. And because UC link gets set, well, it does a set context to thread zero, which goes back to here. So it just recreates everything, goes back, do to do infinite loop again, but with more steps. Isn't that exciting? All right, so that's exciting. Let's see, what else can we do? Yes. Yes. Okay, so let's see the rest of this program. So let's get rid of the set context. So let's instead of doing a set context or anything, let's make a less ugly one. So we'll just create another thread called threads two, give it a new stack, initialize it with get context, set up the UC stack parameters, then we will do a make context. So T two U context will run T two run, and we give it no arguments. So now, if we uncomment this, what should happen? Well, set context should go, it should start executing thread one, print off, and in this case, it should just end here because I did not set UC link. But if I did set UC link to go to thread two, well, at the end of this, it's like saying set context to thread two, then we would start executing this function. It would print off thread two should now be done, switch back to thread zero. So it would delete thread one stack because now that thread has terminated, we can safely delete it stack. Then we set context back to thread zero, which, oh no, goes here, which probably looks bad. So if I do that, I should have our third infinite loop of the day, that's even longer. Yay. All right, so what I would like to happen is if you're doing threading libraries, you probably want thread zero to resume from wherever it did it set context from, right? You probably want to come back immediately, not come back to the very beginning. So there is this better function called swap context, and this will save you a headache. So I could have told you to essentially implement this yourself because what it does is it does a git context of t zero context. So does a git context of its first argument, and then it also does a set context of its thread one argument. So it will essentially save the current progress that it is currently at into this u context and then do a set u context of this. So if we do a set context on thread zero, it looks like it resumes right here. And this is a nice little helper function. Technically, you don't really need it. I could have been mean and not given it to you and let you waste like five hours implementing it yourself. So basically this swap switches from t zero to t one, or does it go the other way? So it does a git context of thread zero, the first argument. So it saves that and then restores this u context. So it switches to that and then if it comes back, the t zero u context would resume back here. So in this case, if I do a swap context, t zero should resume here. So I set context over to t one. So it should go ahead print here. Then this function exits. And because I set uc link to thread two, thread two should start executing this, which deletes thread one stack. And then we set context back to thread zero, which should resume here. Then it should print main is back in town and then delete thread two stack because it is now terminated as well. And then it should return from main, which would exit the process. So in this case, I should actually see three print lines and nothing bad should happen. It should terminate successfully. Yeah. Oh, so I mentioned uc link here. Okay, so yeah. So any questions about how that works? Sorry. Yeah, so well, these are threads. Yeah, so any questions about that looks good. All right, so fairly, well, without swap context, it's fairly easy to infinite loop yourself. Thankfully you get to use it. So hopefully you don't infinite loop yourself, but if you do, you'll get into this situation. So this does all the heavy lifting for you for lab four. You basically just have to maintain a queue of threads and then do some stuff. So you can switch between them, make an exit, make a join. You're just adding stuff on top of it, but all the heavy lifting is done by ucontext for you. So, so make context. So make context, you can, first you have to initialize this ucontext through a get context and then you're allowed to change things. So I can change the stack and then make context. All it really does is change the program counter such that if you do a set context, it will start executing that function and it will do some things to make sure arguments get passed to it as well. All right, so we're good. We can figure out how to use a list. All right, so here is a quick way of using a link list so you don't have to implement your own. I highly suggest you do not implement your own because if you screw it up, especially with threads, you'll be debugging for like another 10 hours, which is more than you should spend on the lab. So use this. So this is the first time you should be having to use a link list in the labs. If you have not, you probably shouldn't have used the link list, so here we go. So there is a sysqh and it has a bunch of fun things in it that will let you create link lists. One of the link lists name is this tail queue and it's a weird name. All it is is a doubly linked list. So how it works, the queue you will need in lab for is just a ready queue or a waiting queue. So it's just what threads want to execute. I do not suggest you like having a link list of thread control blocks or something like that. All you need is the IDs of the threads, whoops. I hate this microphone placement. So all you need is an ID of the threads. So just have a link list of ID and then this tail queue entry macro, it needs as an argument the name of the struct it lives in and then just any old name that it will reuse for other macros, you just have to be consistent. You can call it whatever you want. This case I call it pointers because essentially, because this is a doubly linked list, it will initialize a previous and a next pointer for you without you having to make it, without you having to use pointers at all. The next macro is this tail queue head. So it wants a name of a structure that it will create for you and then it wants to know what this list will hold and it's just the name of the struct of whatever the elements of the list are. So after that, it will create a struct called whatever this is. So in this case, I just called it list head that you can create variables are and of and it represents a list. So here I'm creating a list called list head as a global variable. Then the nice thing about it is it has macros for iteration too. So if I wanted to print all the elements of a list, I could just print off the list, then create a pointer to a list entry and then I use this tail queue for each macro that will do a for loop for me. I don't have to, you know, go to the next pointer, go to the next pointer, check if it's null exit. Don't have to do any of that. Just use tail queue for each. It needs a pointer that it updates and then the address of the list. So in this case, it's just the address of that global I defined and then it just needs a name of the pointers that I used up at the top. Then after that, every iteration of the loop, this element will just be the next element. So it would print off the first element and then the second and the third did the done until it's over. So don't have to play with pointers. Again, don't play with pointers, especially in this lab, you will probably mess it up. So if you want, there's other ways to access elements of a list. So there's tail queue last. So you just give it the list and then the name, the structure of the list and it will figure out how to get the last element of the list for you. So if we wanted to print off the ID of the last element, this is how we would do it. We should probably check that in the case, the list is empty, this would just be null. Okay, so let's go through main and see how to use it. So to initialize your list, you just use tail queue init and then give it the address of the list. And then here, I'm just asserting that my new list is empty. So you can go ahead and use that. Might be useful for you in lab four, if this is supposed to represent a queue of threads to run. If there are no threads to run and some thread just died, probably means there's nothing to execute, which probably means you should terminate the process. So let's see how to add things to it. So I will declare a entry on my stack. Obviously this will not work for your code because while you're probably going to have to share that between all the threads, so you're probably going to have to malloc this. But in my case, I just put on the stack just because it's easier for me. So here I set the ID to one, then there's tail queue insert tail. So this will put it at the end of the list. You give it the address of the list, the address of the entry to insert, and the name of the pointers that you used in the macro way above. So here I print off that I inserted it, then print the contents of the list. Hopefully it's just one. Then I create a new entry for ID two, insert it at the tail. So I insert it in the back. You can go ahead and use the man pages. So man, tail, queue and knit. And it will tell you all the other functions you can do in this. So you might find some of these useful. So for instance, if I wanted to, I could insert it at the beginning of the list with insert head. I could see whatever the thing at the beginning of the list is with tail, queue, head. There's all these functions for you. Do not write your own pointer manipulation. So here I say insert two. If I print the list, hopefully it has the elements one and two in that order because it should be at the back. Then I'll also print last element should be two. Then here I can use remove. So I give it the address of the list, give it the address of the element I want to remove. And again, that name. So I removed one. So hopefully when I print the list, whoops, that is the wrong example. It is called tail queue example. So here I inserted one, my list has one. Inserted two, my list has one, two. And the last element is one or two. Then I removed one. So my list just has element one in it. So fairly straightforward how to use this. Any questions? Use this. This also makes it easy that if you wanted to change it to like a singly link list for whatever reason, there is, I forget what the name, I think it's called S list queue or something like that or S list. I think it's just called S list. So if you want to change this to be a singly link list, just change tail queue to S list and boom. You don't have to change anything else. It's now a singly link list because of the magic of macros. So that would make change this so that it only has a next pointer that it makes it really easy to change the implementations if you want. Yeah. It's called tail queue. Why is it called tail queue? Tail means you can access the back of it. So normally singly link list is just the front and then you have a point of the front. Call the tail, I don't know why they called it the tail queue. That's my best guess. All right, any questions about that? Again, use this and also that S list I talked about. Well, you will have to read it for lab five anyways. So you should probably just get used to using this because you'll have to read it eventually anyways. All right, any questions what we did so far? Other than that, I'm going to ask you a question and test your knowledge. No, good? Sure, like the conference. All right, so here is basically an exam question that was, I forget what year it was. So what this question does is create some global variables so it creates a global int called I, what creative names we have and initializes it to zero. There's also two U context. So there's U A, which is initialized to execute thread the function thread A. So U A wants to start executing here and then it also initializes U B, whoops. Then also initializes U B that wants to start executing there. And then this question says U one kernel thread that does a call to set context, which again shouldn't have a space here to thread A. And then it asks you what happens, what do I see printed or just what happens in general. So I will leave that with you for like a minute, have a good think about what the hell this does and let's see if we are correct. All right, we have any good guesses? All right, do we agree with that? So the argument, we will see A one, B two, A three and then B terminates and it's done. Yeah. For a B end, it should start at that line. You see link was not set. Which is to be D should be one, no? Yeah, D is one. All right. And if we switch back to A, it would get that it's condition gets different. D is not one. Okay, is there any other thoughts up to what it does? Or are we all agreeing? Okay, so, all right. Well, let's just go through it to see what happens. So thread A starts executing and has its own stack. So in D here would declare AD on thread A stack. It would be a local variable. So here I will just put off and it would initialize it to zero. So now thread A would check the condition, the while loop. So the current value of I is zero. Zero is less than three, at least in this world. Then it will increment B here and increment I. So the global variable I will change from zero to one and then it will execute the print F. So it will print A one and then it will change the zero to a zero because it sets the value for D and then it does a get context call. So when it does a get context call, if we do a set context of UA, it will resume right here. So next thing it does is check the value of D. Current value of D is zero. So it would go ahead and set D to one and that's it's D which is currently on its stack. So then after that it does a set context to UB. So we start executing the purple one over here and it would have its own stack. So the first thing thread B does is declare AD on its stack and it writes a value of one to it. Then we'll check the while condition on this side. So in this case, what is the value of I? One. So the value of I is currently one. So one is less than three. So it will increment I from one to two. There's where that two comes in and then B will print B two. Then thread B will change its value of D from one to one and then call get context on UB. So UB will move here. Then it checks the current value of D. So the current value of D is one. So it would go into this if and then it would change the value of D on its stack from one to zero and then it would do a set context to UA which would change the stack back to A stack and it starts executing right here. So if we switch back to it, the value, it doesn't get context, doesn't save the values on the stack at all like we saw before. So the current value of D is what? The current, so the current value of D is one. So we would not go into this if branch. So it would go here then go check the condition of the while loop again. So what is the current value of I? Two, two is less than three. So it would increment I from two to three. We would print off A three. Then it would set its D back to zero and then do a get context call which doesn't move it at all. Check the value of D which is now of course zero and then it would change it from A zero to A one and then do a set context to UB. So UB would resume. So what is the current value of UB's D on its stack? Zero, so zero is not equal to one. So it will go back, check the condition of the while loop is three less than three? No, it is not. So it would exit here or it would drop out of the while loop, exit return from thread B and assuming you see link is not set it just terminates the process and we're done. And spoiler alert, this kind of looks like what would happen in swap context. So this is basically swap context because first time I call it, I call it get context and then do a set context and then whenever it comes back to me it just skips over. So this is, if you had to implement swap context yourself guess what, this is pretty much it which kind of mean if I just like don't show you this and say just do it. All right, any questions about that? Yeah, like when we're executing you threat, oh here? So it would switch the stack back over to thread A stack and it would resume executing here because this is the last place that UA called get context, right? So the I gets incremented to three whenever, so thread B would have incremented from one to two and then swap back over to A and then A would have not gone into here it would have went up to here and then I is two so it would have went into this while loop and then it would have incremented from two to three here and then print and then we would have got this print here and then it would have played with it then did another get context call which doesn't really change anything and then set context back to thread B which exits out of the while loop. All good, no questions? All right, cool, then we're done. All right, so just remember, pulling for you, we're on this together.