 Let's let's get started So how is that body doing in terms of assignments? Which is quite a working on what's that? All of them How many of you have already passed the file only test? Not all of them. I mean the best ones open really right close nobody This is how this is not how it works I mean you are supposed to get together and get a set of files is called working before you move on to process this cause Otherwise, there is there is really no way to test it To run all the test cases you have to have the files is already. Otherwise use a program cannot print Yeah, so you have to get a file is called working before you do process this cause Anyways Yeah, I think so Yeah, not all of them all of them, but I try to upload every slice Yeah, okay, I'll try to upload all of them. I have them. Yeah, okay, so before we talk about Today's process is called. Let's do a first recap of what we talked about last time. So we Go over the file handle design. So what information should go into the file handle basically? The what a flex the user used when user call open. What's the offset? What's the reference count? So why do we need a reference count? Because fire handle could be potentially shared by multiple threads, right? So you need to maintain that account so that when close you know when to destroy that fire handle and what else the Vinode pointer to point it to the actual file and also the The lock that protects the access to this fire handle, right? So that's the fire handle design and we also talk about the file table design File table is basically a just a map between the file descriptor, which is an integer to the fire handle So you can choose whatever that structure can meet the requirements that Giving a fire descriptor get a fire handle or found available fire descriptor And then we talked about what's already there for you to do the files is cause we talk about the Vinode and VFS so we both Vinode and VFS have some helper functions or macros to help you do the actual File read or file write or open the file or initialize a Vinode, right? And we then talk about the UIO Which you will be using in sysread and syswrite to actually perform the the actual read or write operation and Then we talked about a special file, which is cause console and There are three special fire descriptors associated with the console which is SDD in, SD out and SDD error And we talked about how to initialize them and we talked about when to initialize them And finally we go over the each of the file syscalls and Most of them are really easy if you know which on the line on the line call you need to use For example in sysopen you do whatever you need to do to the argument checking the fire handle maintenance Bookkeeping then you just call VFS open to do all the hard work and It says similarly for read and write where you just call viewp read or viewp viewp write to perform an actual read and write And I hope you guys have some idea of all of this so any questions of the contents of from last recitation Okay So when you so as I I was saying a Generous suggestion for this assignment is that you and your partner get together to get a file syscall working Before you move on to process these calls. So I saw a lot of Groups that submit a work one partner do the The file syscalls another one do the processes cause it won't work because You cannot test any processes cause unless you have all the files is already So one most fundamental thing is that to run all the test you should you are expected to see some Print from the user program, right those print is actually Write syscall to the console STD out file So if you don't have the sys write and sys open all those console initializer Initializer initialization ready. You cannot run any of the test So you should get a file syscall working first then working on the process is cause and here is how you can know You can sure you can be sure that you have a pretty solid file syscall First we have a test a program called file only test as a name suggest It will only test the file syscalls It won't test any of the fork and exact V way PRD and exit none of them So this test will only test your sys open sys read sys write LSEC and sys close So this is the first test. It should be pass when you When you're doing the file syscalls, then there is another General test program called better call which as a name indicate We will give you all kinds of bad arguments to kind of stress test your argument checking so you want to make sure you pass these two tests before you do the process is cause and I will show you what's the correct output of this text of this test So now I mean the in the kernel manual the first test I want to do is the test of being file and test Right, so you will see the output like this is saying it's opening a file is writing to that file is reading that file And then it's close that file So this this test test Program won't want to return to the manual as you can see that it says spinning in case of exit doesn't work So it won't call the exit as the normal see Program will do instead. It just just use a while one loop to avoid calling sys exit because you may not Mean you may not have access already at this point so this is the correct output of the final test and another test or is Test being better call right better call has another sub manual which you can choose Which of the cities call you want to test on so The the one related to the file is called in this assignment is C for open here and D for read E for write F for close and J for L C I think that's on and W for do to right you can run them individually for example around C Unfortunately, I think I don't I'm not present It is yeah, yes, it is so you should test that why it doesn't work. Anyway, let me run another kernel I have so many curves this Okay, so I run C for testing open It will pass in all kinds of better user pointers like now the kernel pointer or invalid pointer So now you can see why you want to use a copy in instead of just if the pointer equals Now I return some error if the pointer is in kernel space I return some error because there are so many cases where the pointer can go wrong You don't want to check against every of every one of them. So just use copy in which will test all the invalid pointers so and then for read for What was that other one? Right so You can see here it will possibly all kinds of bad FDs or passing or FD that has already been closed and All that so you can also run the test For all the six calls in some two once you have all of them all of them ready Including the process six calls. So for example, I can run this It would basically test all the six calls. Let's see Yeah, the test web PID exact V and all that So here is this is another test program called better call You want to pass both the file only test and also better call that related to to the file six calls Before you continue to the process six calls Any questions about the test program to test the user pointers because I saw a lot of you doing So, okay, I want to check invalid the user pointers So I check if the final pointer is now I return there some error code if it's larger than 0x at a million I return some error, but there are other cases where this this pointer can go wrong Which doesn't the following any of your special cases? So you want to handle all the invalid the pointer cases using copying sdr copying or copying str So that way the copying or copy copy str will handle all the invalid the user pointers Right instead of you check No, you need to cut first I look at a space. Yeah Yeah, and then our questions, okay So today we're gonna we're gonna most of full focus on the process six calls So we'll first talk about the process structure Then we will go over the process is called six calls, which is fork exact V web PID and exit So In this particular OS in OS one six one Each process will only contain one thread as Jeff talked about in the lecture the other operating system or more generally a process can have multiple threads Right, but in this particular OS one process can we only have one thread So about the process structure you can reuse whatever It is whatever is already there for the thread, right? So basically we in this particular OS we use a concept of process and a thread interchangable Right the process is a thread and thread is a process So the structure thread, which I think you are all very familiar with already have all kinds of process data, right? to make it running as a normal thread and what else do you need to add it to it to Help you manage all those processes. So first of all, of course, you need to edit the file table, right? when you do the fastest cause a more One of the first questions you want to ask yourself is that is that where do you put the five tables, right? Our file table should be per process Best you cannot just have one system wide file table, right? this is where you want to put the file tables in the thread structure and Then because we have later on we have with a PID and exit, right? You want to establish the relationship of the parent thread and child thread, right? So that parent thread can wait on the child thread. That's why you want to Keep the parent PID in it. First of all, you need a PID for each thread or for each process, right? So each process have a process ID and it also have a parent process ID Which indicate who created this process, right? And then you know again for what a PID and exit you want to include the information That whether this process or thread has exceeded if it has what's the exit code? So later on if some of us if some other thread or where PID on this right, you can just collect the exit code and Finally you want to have some synchronization Mechanisms going in in the thread structure so that say process a use fork process to create Excuse me to create process B Then before B call exit process a call way PID on B in that case You should block process a until the B exit, right? You should So you should have some some primitive to help you achieve that purpose, right? We have The primitives you have you have done in assignment one Lock CV semaphore you can choose whichever is using is suitable So any other questions about the process structure? What else what information you need to keep or what other what more extra information you need to add to the thread structure Anything missing here? Yeah, I don't think so so we basically cover all the information you needed to add it to the thread structure and And the process table so process table So you When when do the code reading questions? You would already know that the threads are organized as a link list in the per CPU based the wrong queue, right? Each spew has a link list which we wish is all the thread that are belong to that CPU and Here the process table is another kind of desk structure that help us to do other Bookkeeping many for web PID and exit So process table is very similar to File table in the sense that process table is also a mapping from the PID to the actual process structure In this case the process structure is just your thread structure, right? so When you when you allocate a thread in in fork you want to assign a PID So you need that you need a way to get an available or get the next available PID right, that's one of the functionality functionalities of the process table and In what a PID you you user give you a PID so given that a PID you want to find that on the line process structure or the thread structure that associated with as with that PID so you can either query the exit status code or get the exit or Sorry, either check the exit status or collect the exit Code from data thread right so you need a mapping between the PID and the process structure and again similar to file table you may choose whichever data structure that can Provide that a kind of functionality here. What's the most intuitive option of the data structure? Another array right So you basically you can have a ray of pointers Which in which the index is the PID and the pointers are pointing to the actual thread You may learn all kinds of fancy death structures in this desktop course and but at the end of day you will find that race is the most powerful one and So next we go over all the syscalls first of all fork So here is a piece of code that user will use fork So use the right code like this. I first call fork I get the return value and then I check the return value if it's positive meaning that I am Still this thread a says both says right a call fork So if the return value is positive we're still in thread a I mean the parent thread if it's zero Indicating that whatever the thread is is the child thread say thread B Otherwise if it's negative then something's wrong something wrong so the the semantics of fork is Is very is very simple right so you have one thread is in the process of execution Then at some point this radical fork now after fork suddenly you have to thread right the first is due the original thread But the second is identical to the first thread except for one thing which is the return value of the fork So the second thread will still feels like I guess come I just complete the fork syscall But the return value different from the one in the parent is zero right actually For close almost exactly The current process except for two piece of information, right? the first is PID or PPR and P PID which is trivial because First of all is there are two thread each thread or each process should have its own PID, right? Then the P PID is different because say it's right a call fork and create the thread B Of course, right B's P PID will be straight A, right? So it is P PID will be whichever thread whichever thread that created thread a right? so this is one information that one or the one difference of these two thread and Another difference is the return value of the fork although it may appear like both ready just to call fork and But then their return values are different So this is the semantics of fork any questions on that I mean it's maybe a little bit tricky to understand at first, but once You get more and more of it you come used to it Okay, so let's first take a look the exactly part So we said that fork will create an almost exactly same process with the current process So to copy a process what do you what do you need to copy? Right, so you need you need to first look at what's inside a thread structure or process structure And it just copy everything inside it right more particularly here in this OS 161 case Each process have has its own address base Which you can so how to copy the address base? What's that? Yes copy right which has been has been provided to you and That is copy belong to the dump VM that we have been using and in some is really you Maybe want to change that but but for now Let's just assume as copy do the magic stuff for it for us, right? We call as copy we get a complete identical address base with the current thread And then file table right this one is edited to by you So you are should have some idea of how to copy the file tables Right, so we said that file table is typically provided to each process Unless that process called fork In which case the two processes will share the same file table Right, so by by copying the file table. Are we actually copying the content of the file table? Say I have array which is belong to which is array in which is the file table of thread a After this fork should I have one array or two array? We still one right because the these file tables are supposed to shared between the A and B That's what we mean by a Copy the fact that we're not not actually copying the content of the file table We just make more faulty scooters pointing to the same file table entry Right, this is where you want to play with the reference count because now you have multiple thread or multiple Processes pointing to that file table entry Right, so so how would you copy the file table? I mean in the English So you want to go over the file table right because you don't know which one is used which one is not Right, if you you go to the file table and see if you have an entry there if you have an entry there you want to Sorry, I was saying that before there was only one array I was talking to the talking about the file handles. You still have two arrays of pointers Right The actual file handle will be only there will be only one copy of it But you may have multiple pointers pointing to that file handle, right? so you end up having two array of file table pointers and Some of them will may point to the same file handles right, so Go back to what are we saying to about how to copy the file table? You go over the file table entries of the original thread if it points to some file handle You just also let the new thread file table also pointed to it and in the meantime You will have to increment the reference count to indicate that multiple thread or multiple processes are referencing the same file table entry right and Finally a thread has a context by context. I mean so what is what is the context here? What's that? registers right so when when the original thread which is read a core fork, it's just it's a sis call Sis call is kind of an interrupt so whenever there is an interrupt the the kernel interrupt handle will always first freeze the Context of the current thread by saving all the registers in a destructure or call check frame, right? you want the Charles read has exactly almost exactly the same trend frame as the parents read Right, so somehow in the sis fork you should copy the transfer and I will get to We'll get to how to do it later. I think I Don't know if I have slides for that. But anyway so this other parts that we need to do in sis fork and you you You have you notice that they are much much more information in stress threat Structure thread right so who will copy them? I mean the the wish CPU it belongs to and what's the interruption level and all that information How do you copy that in that information? We just talk about the information we added or we need to do in sis fork What's the main body of? Create another thread Somebody says right fork. Yes. So thread fork will copy everything everything else right, so We copy this three part and thread fork copy the others so together. We're copying the entire process or thread um into Okay So now we have been we have talked about the exactly part where we copy all the stuff of the current thread To the new thread then so let's move on to the almost the part almost I mean that is there's something different Right, so as we mentioned the PID and PP ID is different Which is trivial right every time you allocate a new thread or I look at a new process You always find a new PID to it and you since you are handling the fog You know the parent and child relationship, so you can also set the child's PP ID correctly And the second part is the return value, right? So so parent Will return the child's PID while the child will just return zero Right, how do you achieve that effect? Yeah, actually I should add an animation to it. I shouldn't just give everything up to you at once, but anyway So any any ideas on how to How to trick the transfer to make the child things that the fork system called returns zero Yeah, so we talked about the MIPS return value convention That the one last time I think So the return value is solely based on what's stored in V zero register, right? If you start zero in it, then the user level pro program will think the Cisco returns zero, right? And what is next? So let's go over to the big picture of CIS fork, right? So in CIS fork, you know that You know what you need so you know finally we'll cause this thread fork to copy most of the stuff Right, but so one need to copy before that as we said in this part You need to copy the address space. I think is it handled in thread fork. I think so, right? So address space already be handled in thread fork the file table right the context context which means the Trip frame, so how do you copy the traffic? What's that? It's copy. No traffic You can yeah, yes, so traffic is nothing but a big big chunk of memory that contains the value of all the registers, right? So To copy the traffic you need a new place to store the traffic Which you can use came a lock to allocate at the kernel buffer that will contain the traffic Then you just use mem copy or you can just directly assign the strap structure value Small trick in C, but anyways is equivalent to a memory copy to that reference Then you pass the trip from to the child thread So the child's ready can get the trip from of the parent and also take the return value before the child thread Return through user mode. So that's about it. I mean Cis fork Any questions on the Cis fork Cisco? Oh, yes, of course when you copy the file table Yeah, actually we each thread has an array of file handle pointers Right, that that is your file table by copying the file table. I mean for each file handle We for each file for each file handle. We don't copy the content of the file handle Right, we only have one copy of the file handle and multiple threader can have pointers pointing to the same file handle All right, that's how you copy the file tables. Any other questions? Next is exact V Okay, so it what's the exact V are supposed to do All right, if you look at the manual page, you will find the exact V are supposed to Launch a new program and they replace everything in the current thread basically just get a new thread running But in this do in the same thread container, but with a whole different executable All right, so last time I think we demonstrate a case where you can use fork and exact V to write a small shell Right, so how would you how would you do that? I mean, so what's the what's the first thing to do when you run a shell? Say we have a shell thread, right the first thing you want to do is get some input, right? What user have typed I? Suppose you get a string that string represent what's the program? What's the arguments and all that then what they do? Do you want to just call exactly to run whatever the program is to replace yourself? You want to create a new thread right? You want you run whatever the program in the new thread right you do that by using fork which will create Identical copy of the of the shells right now you have two shells right or one of them is parent One of them is child and then you can differentiate them using the return value of the four Now in the in the child's right you call exact V Right, which where you replace yourself with whatever the new program is and you run the new program in the new thread Right in the in the parent thread you wet PID for that thread to wait for the user program complete right, this is how you can use fork and exact to write a simple shell and So the most trickiest part of the exact V because most of most of the functionalities is very very similar to run Program right you need to open the open the binary file you need to load load elf to into memory and Finally, you need to enter you enter user user process Right, so the main part is very similar to run program So keep that in mind and when you run when you write exist exact V try to Mimic the behavior of run program, but plus one thing which is argument of passing Right, so as I said, this is the most trickiest part of exact V So let's go over the an example of how to deal with argument passing Right, so this is the prototype of exact V here We we have two arguments the first argument is character pointer Which points to the name of the program and a second is an array of pointers each of them pointing to some arguments Right So here is what it looks like in currently user space before the user because before the thread called exact V Right, we have so first of all forget all the pointers here. We have Four strings the first string is a program man Which it has been our test a second string the full was 30 is os my six one and the finally We have bar so we have four strings here now. What's the what's the pointers? Right, so as I said the program are supposed to pointing to the program name in this case is a test being our test right, then the second argument is a Pointer to pointers which is a pointer to this Array of pointers here and Each each of them are pointing to the first the second third and fourth argument So you will notice that in this case the first pointer in the arcs array are pointing to the same string as the program right, so that's kind of The behavior of this minimal shell of OS one single that's that may not always the case So it does anybody know the busy box to busy box, but basically it's a multi-core Binary, so you have this one binary called busy box Right, then you have a bunch of symbol links to that binary You can have ours you have a cat and and everything right, but eventually you will You will you only have one binary and that binary will behave like whatever it links to it If you call errors, it will just a list of the current directory, but the actual binary is still that busy box That's achieved by so inside a busy box It will check the first argument which is the program name and if it's errors I do errors if it's some other cat I do cat it's called multi-core binary So this is kind of one trick you can do With arguments, but anyway, so in the conventions the first argument is always the program name Right, so this is what looks like before we call exact V. That's clear, right? Yeah What what do you mean? Okay, it's something else It's You will you will not encounter that case so you don't need to check that so it No, you should you just copy whatever is there, but when you when you Open the binary you always use the first argument, which is a program You can get you are guaranteed guaranteed that that program points to some actual binary You can run with and what the arguments you don't need to care about that's the that's the business of the actual program Yeah, so the the overall flow of exact V is like this So the first thing you want to do is copying all the arguments into you into the current space And why we want to do that? What's that? Yeah, because later on we need to destroy the current address space and create a new one Right if you don't copy the arguments out from the user space when you when you call is destroy You basically lose everything right? So you want to get a hand of all the arguments before you destroy you play with the address space Then you do whatever the wrong program does you call VFS open on the program You call a lot of elf on the on whatever the argument you need to use and then finally Before you enter new user process You need to arrange the arguments so that the new new user process Can't get all those arguments right? You cannot just pass the pointer the current buffer pointer to the user process because the user process Don't have access to that Right, so you need to copy out all the arguments to the new user space Right this copy in the first step and the third step copy out is the most difficult part of the exact V So Suppose so we know that every new user thread or every new user program We will start at the main function right the main function takes two arguments Which is the first is RTC meaning how much arguments are there and the second is ARGV which is pointing to the actual array of pointers which which is in 10 points pointing to the actual strings So if you look at the interface of enter new enter new process Where where is that to find trap to see The all the enter new process. Okay. I thought so this function takes the first argument is ARGC right which is corresponding to the ARGC in users main function and a second argument is the ARGV pointer, this should be a user pointer or the value of it should point into the user space And finally we have a step point to an entry point which helps the enter new process does its job But basically the first of two arguments Corresponding to what to the user's main function, right? And the user expect the ARGV in this particular format, which is very similar to the exact vc is called So ARGC is nothing but an integer indicating how many how many arguments are there? ARGV are supposed to be a Pointer to a pointer array Which and each entry of them should pointing to some strings or character character arrays Right, so this is what the user expect to have when we end when the user's program gets gets running right, so it's your job as a kernel or as the CIS XCV writer to Organize the arguments in this particular format so that the new user program can get all those arguments without any problem right, so this is what it expect when user program gets running and So a natural question to ask is where do you store all those arguments? You cannot store them in kernel buffer because user don't have access to the kernel So where do you need to store them? So first question is kernel space or user space of course user space and the second question is where you use a space? Stag right so user space where I have what have code segment data segment heap and a stack, right? So the heap Haven't that doesn't exist as its point because user haven't called my luck or whatever right the The fact is the only place that you know and for sure was the Where it is, right? So in this MIPS architecture the user stack always starting from OS OX at the medium which is a user address best top and grow downwards right, so So this is the only place where you know for sure where it is and you know that user won't By putting the arguments in here You won't interfere with the user as long as you update the stack pointer So user state user spread will just continue using the stack growing downwards. It will not Try to access the memory above the stack pointer, right? If you put the arguments in here, you know that you have a good place to use It's user space and user won't try to overwrite the the the memory So As you can see here, do we really care about where these strings are? I mean, do we want to align the strings in some way to organize them? No, right? So what the only thing matters here is this pointer Array as long as we have this pointer array are ready, right? Then we don't care. We don't really care where we place all those strings As because we know that the first pointer we always point into the first argument no matter where the argument is right, so what do you? Care most about is how to get these pointer array Correct as long as have you had you have this correct user will have access to all to all of those arguments right, so this is one possible arrangement of the arguments right starting from there We have user stack top, which is the absolute value of all x at a medium So this is zero. This is all I mean oh x at a medium these two gigabytes space is user space virtual address space and We start cop so we start this array zero one two Three somewhere in stack in this particular Figure I put in the in here, but you can put it put it anywhere and and Each of them pointing to some strings in again in user stack Again here I put from top down four OS one six one bar and the first it has been our test You can arrange it whatever way you want, but What it really matters is this the order of this array pointer array, right as well as as well as you have this array Ready or correct user can already de-referencing these pointers to get the strings right, then you So if you think about it How would you actually do it? How do you actually copy the arguments? So Say you already have this you already have copied all the arguments into user space when you call enter new process What's these deck pointer value? This is it should be somewhere Here or below here, right? You don't want to Set a stack pointer to be somewhere here I'll in that a case when users program gets running the stack pointer will overwrite this region Right, you want to put the stack pointer somewhere below or equals to this? bottom of this of this part of this part, right then what's the RGV as As the this figure says RGV should be here, right? So if you associate this this figure with the previous one You find that RGV should point into the start of the array Right, so this is one possible case so Again, how do you know how many space you need? To store all these all those arguments So in this particular case we have one two three four arguments Right the total space you need to store these arguments in the user stack is the four pointers plus whatever the string occupies Right, you may need to do some alignment for each string for example this OS one six one is five characters plus one zero Character at the end you have to in total six characters, right? You may want to Align up to add characters to make sure everything is aligned by four. That's but but that's Kind of a minor thing So here you need in total that many space to store all these arguments, right? Once you get that number What you can get you can get where this is Right, what's what's the bottom where to start copying? Then you can just then you get when you once you get this address you can just to copy first copy the The arrays or set of the value to it then copy all the strings upwards Right, so you need to do some calculation to figure out of how many space you need Then you just reserve this much space from user stack and do copy this arguments in it Any questions on this? No, everybody understand how it works What's that? Alignment I think so. Yeah, so in because because in MIPS we require every pointer the value is aligned by four so if you have An array pointer that is not aligned by four, I think that will cause the some segment of segment of fault Yeah, so it's better to align everything. So finally we have exit and a web PID Oh So these two you want to write them together because they're kind of the Coordination function that when reliable relies on each other. So in exit The is pretty simple. Are you sort of first store the exit code? Whatever the exit code user passing you just saw that in the structure Then you indicate that is this right has exited by setting some flag on the straight structure Then you just call stress exit to just clean up everything in web PID So you you you first want to check if the child has exited Right, if it has not then you need to figure out a way to put the current thread into sleep And then if it has exited you just collect the exit code and This is how you can test the Process systems first of all you want to run the shell So there are two ways to run a test of program, right? One is using P test being something to run it that way is using the wrong program Once you have exit exit V ready, you want to use your exit V instead of wrong program. So you first run shell I will show you how to As for shell now you are in shell you can use in just the direct parts of the Fog test this is one test you don't have to do P something you can just directly type the binary name and Yeah, I run up memory I think and For fork you want to test both for test and a fork bomb in case of fork bomb It shouldn't print any error message other than that it says there is no memory if your implementation is good and for exact V you want to use the ARC test to Test it the artist is basically does nothing but just to print all those arguments So if you type this and you see that text ARC tests are printing a b c d f g h You know your exact V is good And of course when you need to run better call for all the arguments to make sure your argument checking is good any questions about That V Y PID and exit. What's that? It's different person. Yeah Okay, so that's that'll be it good luck with the assignment