 All right, everybody, let's get started on time so we can finish on time. OK. Quiet, please. All right, so today's topic is threads, processes, and the address spaces that they make use of, and how the address space defines the differences between these concepts. So we'll get to that in a moment just to quickly recap what we did last time. We talked quite a bit about virtual machines, the two main varieties of virtual machine, the system VMs, which present a virtual copy of an entire operating system, and it's virtually like sitting at a machine. And there are products like VMware, the dominant player in this marketplace, parallels desktop for Mac, and then a collection of open source virtual machines, which are part of the Zen project. And we also talked about process virtual machines, which are a more focused idea and canonically include the Java virtual machine, where they present one set of APIs for really running one language. They sit on top of the main operating system, but they can provide an even higher level of abstraction to the programmer and even more portability because they're standardized through many layers of abstraction and functionality. So those are the ideas from last time. With system virtual machines, they have a number of advantages. You restrict crashes and problems to just one of those virtual machines. The other ones that are running concurrently on the physical machine shouldn't be affected. So there's a protection and safety advantage to those virtual machines. That can considerably help debugging and testing, especially if you're working on things like kernel code or drivers, which can potentially crash the machine. So any other advantages you can think of, let's say, any cost advantages of this approach? The reason this virtual machine marketplace took off actually probably have more to do with economic arguments more than functional arguments. There are two big ones. Anyone think of some economic advantages to the virtual machine approach? Yeah. You can rent them. That's close. Yeah, that's close to the answer I'm looking for. Or anyone else? It's renting is like a business model, yeah. What's an advantage if you have to administer a lot of machines? Yeah, exactly. You can make exact copies of machines which make them vastly easier to administer. That's a huge advantage. So virtual infrastructures dramatically drop the cost of maintenance for many enterprises. And there's maybe one other one. It's probably the most important one. It's a bit subtle, though. Yeah? Go ahead. Yeah, that's probably true. You get more thorough physical machine utilization if there's more people working on using those virtual machines. The last one, though, is actually primarily the ability to keep legacy code running. There are a lot of enterprise systems and people software systems which are quite ancient. And enterprises become dependent on those. And they often become obsolete, or at least the hardware. The operating system versions that they run on become obsolete. So anyway, a lot of the VM industry got started by providing companies the ability to buy new hardware to stop having to support ancient machines but allow them to run legacy virtual machines to keep this software going. All right, so the course this semester is going to use nachos again, which is this very simplified high-level operating system written in Java now. It was originally written in C++. But it provides the key elements of an operating system kernel and including an entire virtual machine that can run binary code. So it's more or less the complete machine-plus operating system that's been virtualized. Runs in Java, so it's highly portable. And we'll use it at least for the first two projects in the course, so you get a real picture from inside of what an operating system looks like. So also last time, we talked about the main roles of an operating system. And we had three metaphors. One was the policeman, traffic cop, designed to make people not do harm to themselves or others. So that manages resources, in particular, protects resources, stops people from using the same resource at the same time, typically, resolves conflicts when people are requesting the same resource and hopefully prevents errors. OK, then there's a facilitator function which is providing useful services like the network stack, the file system that everyone needs, and going up for the GUI and windowing system that helps people develop their applications hopefully quicker. Yeah, so because of these high-level interfaces should make programming easier, faster, or less error-prone by providing more of the system support, widely used support software in part of this operating system. The one I like is the operating system as government. So if you think of governments at their best, they often marshal very large amounts of infrastructure, freeways, bridges, and so on, which is what cloud operating systems allow you to do. Get quickly a lot of machines running at once, running your application, and then just give them back. You don't have to maintain them all the time. And a consequence of this, this one of standard interfaces is that you get leverage also from other people's applications because the operating system forces you or nudges you towards using standard interfaces. One in particular I have in mind. Anyone think of a good example of this in Unix, say? Yeah? Yeah, standard idea. I was thinking of pipes and we're almost, it's almost the same thing, yeah. Standard in, standard out, which gets leveraged into pipes if you interact with them in a simple enough way. So that allows you to compose many applications, lots of ones that other people wrote as well, to augment your own application. All right. So a very brief history of time in operating systems. All right. So computers started out in an era, they were heavily bolstered by military applications before and around the Second World War. So at the time, the hardware was very expensive and humans were relatively cheap. They were never cheap compared to the hardware they were. And then hardware became cheaper. We got transistors first and then integrated circuits. Relatively the humans became much more expensive and actually their wages did go up. So we had personal computers, workstations and human-oriented graphical user interfaces. And finally now we're in the situation where the hardware is incredibly cheap, humans relatively much more expensive and computers are everywhere and networking is everywhere as well. Interestingly, as an example of this, this is an early sort of control room for a computer and the people in there doing a mixture of data entry, mundane data entry, monitoring the system to keep it running. Very early computers had failures every day, vowel failures every day. So somebody had to spend most of the time isolating and plugging in new vacuum tubes. They got a lot better, but still right around the Second World War, there's a famous quote supposedly from the head of IBM saying that I think there's a world market for maybe five computers. And actually he was right in a way. 10, about 13 years later when he actually passed on, a lot of people called him the world's greatest salesman. And by that time he sold about 10 computers. But he did a remarkable job of setting expectations and then exceeding them. But anyway, at that time, at that time people really didn't need a lot of computers, but obviously his legacy was a strong one. He created a nice growth pattern, his son took over and added a lot of zeros to that 10 computers. So, and we moved into the personal computer era driven a lot by innovations from Xerox PARC. The emphasis on actually early text processing and business processing and documents, creating documents with words and pictures, which was a radical idea for ordinary people to put graphics in their documents. And it worked, it successfully created an industry, not for Xerox, unfortunately, but for others. And finally, here we are today, moving all the way from dozens of people who care taking this monolithic computer like it was a little baby in an ICU or something, all the way to, and here we have like once this admin for every dozen computers and now we have computers in everything and virtually no maintenance, except for robot pets, which once again require feeding and nurturing and a family to keep them happy. So, all right, and some of the consequences of these changes, we've moved from batch processing in the early systems where people were only running really large jobs anyway, so the throughput was limited so the latency wasn't a big deal, so they tended to run them in batch. As people tried to get these systems to be used by more people and they moved to multi-programming and time-sharing where there was maybe only one machine physically, but it was time-shared in small chunks that created the sensation of interactivity among many users. And then in the 70s, the Graphical User Interface came along and really started to make things move. All right, and so these in higher level features were migrating down into smaller machines. And yeah, in some ways, though the operating systems have become now more like mainframe, operating systems used to be, and they take hundreds to thousands of people years to develop. Yeah, so, and yeah, go ahead. What's the people here? Oh, that's just like one person working for a year. It used to be a man year, right, I'll say that. So the archeology of OS is really interesting and literally you can do OS archeology in a lot of cases. You can go off and look at line by line of code and operating systems and you'd often find them going back generations of older operating systems. So yeah, because it costs so many person years of time to develop an operating system from scratch, doesn't happen very often. We have a lot of operating systems now, but most of them have a substantial lineage back to earlier versions. So one of probably the most important thread, you can trace all the way back to a system called Multix, which maybe people don't know. It was one of those systems that was very influential back in the 60s, was built in the 60s, but it was many ways ahead of its time. But a lot of the concepts that we'll talk about in the first few weeks of the course more or less were in modern form in Multix. So we're still running on a time-sharing system running on one machine, but had many modern ideas and was enormously influential on Unix, the first Unix from AT&T. In particular, the two main developers of Unix, Dennis Ritchie and Ken Thompson were both earlier on part of that Multix project. In a lot of ways, AT&T Unix was intended to fix the problems with Multix. So Multix had a lot of good ideas, but it was ahead of its time. It was considered too big and bloated. It was 135 kilobytes of RAM, and at the time that was about a fifth of the memory of the system it was running on. So that was considered very wasteful and ridiculous. Early versions of Unix were heavily pruned from that Multix design. Oh, by the way, what else did Ritchie do? C, yeah, what did Thompson do? He did B, so. This is a recurring story in this history, which is every great system usually had a system before that got most things right, but whenever you innovate a lot, nobody gets everything right, so a few things go wrong, and that system doesn't really succeed. So Multix was a failure, but it had so many good ideas, and then the developers of Unix got to look at that and say, well, this is really cool, and these bits will just throw them away. For instance, Multix had a sort of essentially a entirely virtualized memory file system, so if you didn't worry about reading files, you just wrote everything to memory, and the system tried to cache it to disk when it needed, and that was just not very good. But anyway, later on, the Unix developers fixed those problems, and it became a great system. Pretty much the same story with B and C, like B was pretty good, and then C was really the one that took off because it was a fix of the problems with B. And from AT&T Unix, the really sort of pinnacle or milestone in Unix was Berkeley Standard Distribution Unix, which was mostly done here, and took ideas from AT&T Unix, but also made an open-source system that people could really develop and understand and build on, and that was the root of most of the commercial versions of Unix through the 80s and beyond. So yeah, so getting things almost right. So Multix was a partly academic project, it had MIT, people, GE, and Bell Labs, so really they thought a lot about multiprocessing, how to isolate processes, got it mostly right. Okay, another very influential system that came along somewhat later was Mark, which is the first microkernel operating system that was developed at CMU. And Mark at the time was also considered a failure. In this case, the microkernel architecture, you should remember from the readings, the idea is just to get the absolute minimum set of OS functions into the kernel that's running in kernel mode. So typically it's just the threading memory management and some inter-process communication. So everything else like file system and especially device drivers can live above that level and still interact with the OS in the most sort of simple way and also but still have protection from each other. So Mark was pretty good, but it was a little bit inefficient. That extra level of sort of virtualization was in the machines at the time quite drastic. Also it was a research project, probably never really got optimized enough. So anyway, so Mark was the microkernel and typically you'd have some other, the rest of the operating system you could get from anywhere. An interesting combination was to use basically the BSD API and some of the code and just layer that on top of the Mark kernel and that was the genesis of next step. So what was next step, by the way? Yeah. Yeah, so Steve Jobs took a long sabbatical from Apple, long involuntary sabbatical in the 90s and he did some pretty important things then. So he started company called Next. Also Pixar and probably most important of all met Johnny Ive who did all the design for Apple when he came back. And next step was a very innovative machine that was really more of a technical workstation and it had a lot of clever ideas and some that were just not good ideas. Everything was innovative but for instance it had an optical disk drive instead of a hard disk which proved to be just not very good. So on the other hand, there were so many good things that has at the time by far the best user interface development system. Very elegant, basically WYSIWYG user interface development system which became part, essentially became the, whatever it is, part of Cocoa and Apple. So Apple also has one of the best UI development interfaces. So a whole lot of stuff from Next that was adopted by Apple pretty recently though. I mean, this is all stuff that was added in OS X. Earlier versions of Apple had much more of an old fashioned legacy operating system that it evolved from very early Macs, kind of like Windows. But anyway, recent, the most recent Apple OS and also the iPhone OS have this elegant Mac-based kernel now called Darwin. And yeah, it's allowed really a pretty, not just to create, I don't know, a pretty efficient machine but a very elegant machine to develop on. So here maybe the critical design was Next Step, again which failed but really if you look at it there was so many pieces of Next Step that are in the OS X and iOS environments that you really have to give credit to that system. And of course, when Google developed Android operating system, they did it pretty fast and they were able to do it fast by taking one of the many Linux distributions that just seemed to be about the right complexity for an embedded system and made it the core of Android and then added there a sort of Java, customized Java virtual machine on top to create Android. Yeah, and so Windows has got a kind of somewhat, well, somewhat complicated history. You know, way back, you know, it started off life as a text-based very simple operating system. Interestingly, MS-DOS, the system that IBM almost made their standard operating system instead of DOS was called CPM. But the creator of CPM couldn't agree to some intellectual property rights with IBM so they didn't buy this. But what happened was that Microsoft just bought a product called, a company and a product called Q-DOS which it turned out, this developer basically bought the CPM manual and implemented everything in his own way which doesn't constitute, luckily, plagiarism. So Microsoft was able to sell IBM a version of CPM that wasn't the original one. And from there, you know, they gradually added a windowing system, really, this is, although it's a windowing system, it's a fairly small increment on DOS. From an OS point of view, they're really primitive, both of them. What happened, the significant step for design in the Windows legacy is probably NT which was designed from scratch to be a real operating system on the current architecture of the time and it, again, used almost microkernel design, not quite, NT's considered a hybrid architecture where it had a small kernel but a number of services that did run in kernel mode, so not quite as heavily stratified as a microkernel architecture but it did explicitly borrow some ideas and I think some code from Mark. And, yeah, and you could look at today's Windows and it's still got a Windows NT directory with a lot of services from that time. And this also gave them a server quality operating system that had some real protection and could really run many different services and many different users. And from there, you know, the lineage continued, not really, I don't know, maybe not a huge amount of development happening from that point on. Obviously, the look and feel changed a lot. The size of the operating system changed enormously. But I'm not sure, I think the big steps, again, were going into NT. Yeah, any comments, by the way? Any other tidbits about any of these systems that people know about? Okay, well, Linux is a bit of a black sheep, I suppose. I mean, it was really developed from scratch. May have borrowed a bit from some of the other open distributions but it really was driven by Linux Torval's initial sort of small kernel that he wrote himself and then helped by a lot of other developers. And of course that's had a huge lineage of other systems that use many of the same elements. Okay, so today we wanna talk about processes and threads mostly and how they're sort of addressed, how they share the same physical memory and how they cooperate and communicate. All right, so the simpler concept is a thread. And a thread is a unit of execution. It's kind of like a virtual machine but it doesn't have its own memory. Has its own stack and registers but it shares memory with other threads. So it runs in an address space and unlike a process, it doesn't own that address space. So here's a simplified execution picture that we could borrow from 61A. So over here's the main memory and there's the processor with some registers. So we have general purpose registers and then a program counter at the bottom. Okay, so the cycle starts out by, the program counter should be pointing at one of these locations that actually contains code, maybe here, so it's gonna fetch the contents of memory at that location. Pull it in, pull it into the processor, decode it, figure out addresses and destinations and the actual instructions to be performed and then perform them. Sometimes on registers, sometimes back on main memory and then write the results back and then update the PC either incremented or load it with a branch instruction result. Okay, and loop. So to just quickly simulate that, here's the program counter pointing at the first instruction. So that instruction we go through once instruction zero is supposed to go in there, update some registers and keep going. All right, so okay. Well, okay, any questions on that? So yeah, so that's a simple sequential single threaded program. So in uniprogramming, there is one thread at a time. That was true of MS-DOS and early Macintosh. Actually, Macintosh had very limited thread support for a long time, much longer than Windows. And gradually, starting with Windows 3.1, there was some legitimate support for preemptive threading, although it still had a compatibility mode, 16-bit mode that was not preemptive. Preemptive means that the OS can interrupt a running process without the running process giving up control. So MS-DOS, actually it had GUI, sort of GUI applications with different typeable fields, but actually it had to scan them sequentially. It created sometimes the impression of being multi-threaded but it was not multi-threaded. So uniprogramming is in some ways easier, although you may have to do a lot of polling, which means effectively keeping several processes running by hand. It does, though, get rid of all of the tricky concurrency problems. So you will only, you'll be the only one accessing that field on the screen, you'll be the only one accessing the disk at any given time. Does it make sense for personal computers? What do you think? No, not anymore. I mean once upon a time people were used to typing text and it didn't matter so much, but as soon as people get used to graphical interfaces, there's many elements on the screen that they want to be able to use. System has to keep all of those alive and in a reasonable way. It has to be modular as well so that you know, you can add new widgets on the screen and they should be responsive and the operating system clearly can't sort of be doing all of that in one thread. Okay, so multi-programming. So Multix was the very influential system that really did that in a modern way and it was followed on by Unix and Linux, evolved by them. Who knows what OS2 is? Well, it was a competitor of Windows that IBM developed. An interesting corollary of the long lineage of a lot of the successful operating systems is that there's some others that didn't have a long lineage that are also not around anymore. So IBM invested a lot of people years in the system and it was just not quite competitive with never really was able to keep up with Windows. Anyway, let's see. Yeah, so anything modern though is multi-threaded and has true preemptive multi-threading. Yeah, well multi-tasking is a slightly different idea than multi-programming. We're gonna focus on multi-programming right now. Well, so multi-programming as an abstraction means that you want to present some kind of virtual machine abstraction to the programmer. The system has to then be doing some of the traffic cop functions in order to manage resources. So it has to arbitrate between shared resources and it also has to protect applications from riding into each other's memory. Yeah. No, I didn't mean to say that. DOS, yeah. No, it really has more to do with the, well, at the time there were no multi-core systems. What multi-core Intel architecture? Yeah. Processes, okay. There really weren't, no, I mean the kernel would sort of run around. It would jump into the file system. So yeah, it didn't have a lot to do except wait for your input, maybe run an application that used the file system but it'd be doing one of those things at a time. Just was the nature of the system. If it was running a spreadsheet, that was the closest it would come but it would have to sort of check. You know, I suppose check the GUI if it was doing a big calculation. Actually, I don't know how it did that. It probably had to, people did do non-preemptive schedulers which basically meant any long running piece of code had to periodically call a sort of a yield function. Yeah. Mostly, I mean the thread is supposed to be, it is a program but it includes the state of the program including at least the registers and stack in order to, you know, it has to be enough that you could save it and restore it. That's the essential part of the thread. Yeah. Not exactly. Well, is it multi-threaded programming? I guess that's, yeah, I think that's close enough, yeah. All right. Anything else? So we want to, let's see, present enough abstractions to the programmer that they can write code to a simple interface. The two challenges you get are concurrency which means that very often when you do a multi-threaded program you're dividing the work of the program into multiple threads in order to typically run faster. So almost surely the subdivision of the problem sooner or later you'll have to collect results together and sometimes you need to communicate periodically for the whole calculation. So threads that are doing a common problem have to be communicating and that raises the communication challenge, the concurrency challenge. In addition, sometimes they may be trying to access the same database or file system, so they also need to sort of keep out of each other's way that's all part of concurrency, yeah. It's not, well, okay, it can in effect give you better CPU utilization because it means that the processor is doing something more of the time. So that assumes that it's able to wait on something that it couldn't proceed with anyway. Obviously the GUI is a good example because people generate events really slowly. You generate at most hundreds of events a second if you're moving the mouse with a normal mouse. So the multi-programming in that case means even if you only had one processor it can fairly easily keep up with events from the user interface. At the same time it could be pushing bytes from some calculation into a buffer for a DMA area for disk and that copy won't necessarily consume the whole CPU. In fact CPUs are usually much faster than memory and in fact this modern CPUs will often do a DMA command which is just telling the hardware to do the copy itself. So after saying do the copy the CPU is free again so it can go back and execute another little fragment of a task. So actually it clearly doesn't, it's not like the CPU can't do more work than one CPU but it can be busy more of the time, yeah. It's, there might be more than one answer to that. I mean people definitely have developed abstractions like OpenMP that try to make the threading invisible to the programmer. So they define things like a parallel for loop and how that actually gets executed is not really deterministic until it runs and the system itself divides, decides how many different threads to use and what size of blocks. So people do often try to hide that complexity and let me think, yeah there's probably a bunch of normal processes that you can run that will be, what's the word? Not transparently, the opposite of that. It'll be opaquely hidden to you how many threads they're actually running. Okay so process is a unit of resource allocation and execution so the process in contrast to the thread owns its own address space and in fact it often owns the other system resources of a machine like file descriptors, other IO devices and file context. So a process may or may not have multiple threads in it. So it's somewhat like, I guess a reasonable metaphor is that you can think of a process as like a household. You know a household is typically in one house which is like the memory space. The threads are like the occupants or roommates of the household so there may be more than one of them they have to cooperate, they have to try to stay about each other's way and not trash each other's space and so on. But the whole collection of those things forms a household which sometimes you want to think of as a coherent entity. So why processes? Well they provide in some ways the cleanest, most extensible way of supporting a lot of different services and functions in an operating system. They provide the most protection and security for different applications. And on the other hand the flip side is that they're slower almost always they're like 30 times more, you know typical operating systems, 20 to 30 times more slow to create and they're also substantially slower to use. The context switches are typically maybe four or five times more expensive for a process compared to a thread. So they have these extra affordances but they cost for that. All right, yeah and application code often will comprise at least one process, sometimes a few more than one. So a spreadsheet probably these days has multiple processes, some doing user interface stuff, some potentially talking to databases and so on. So let's see, problem of concurrency involves resources. You know we have a typically single CPU, maybe a couple shared RAM on one computer and particular IO devices. The multi-programming API though for processes makes them feel like they have unique access, exclusive access to those resources. And the OS is then charged with making the mapping from those virtual interfaces to the physical hardware and trying to keep everything consistent and straight for the different processes, yeah. The idea is that from the programmers point of view, they'll use an abstraction like open to open a file and the operating system will look and say, well there's somebody else using that file right now. Depending on the particular kind of command that the program issued, they'll either block the second open until the first one's finished or there are also non-blocking calls which return straight away and say no, you can't access this right now to allow the user code to do something else. So I suppose the non-blocking calls in a sense perhaps uncover a little bit of what's going on. So you're not supposed to think about other, what other processes are running, but you do need to be prepared to deal with the case that you can't access all the resources when you want to. So these kind of problems don't have simple magic answers. So for that reason, there are different types of calls, blocking and non-blocking calls especially so that the application developer can do the most appropriate thing in that context. Sometimes it's best to wait, sometimes you wanna forget that operation now and come back to it later. So both things are possible. Okay, so we have the virtual machine abstraction and you can multiplex somehow, keep those multiple machines running. Oh yeah, and the perhaps earliest clean description of the system was in that additional reading that's posted on the website which was a very early multiprocessing system, multiprogramming system which did in a few thousand lines what had required millions of lines of code in the OS IBM 360 system and the millions of lines of code had thousands of bugs which the small version didn't. Okay, so let's look more explicitly at how we do some time slicing for multiprogramming. So let's just take a simple case which is a single processor and we wanna provide the illusion and experience of multiple processors. So we wanna, you know, we have shared memory, we wanna create those virtual processors up there. Okay, so we use time multiplexing. So the idea is that the physical process is gonna execute little chunks of code for each of the virtual processors in sequence and it'll be busy all of the time and hopefully those chunks are close together enough that the virtual machines, especially if there's a user attached and then we'll have the sensation that they're interacting by themselves with one operating system. So the virtual CPUs need to hold their program state a stack pointer so they can do function calls, they can keep track of their own series of function calls and registers. Okay, and so this is the thread abstraction, by the way, since they're sharing memory, these constitute threads rather than processes. And for that reason, when we save state, we only need to worry about state that relates to the stack pointer, registers, and program counter. We don't need to worry about state that might relate to some kind of mapped memory block that would be required if it were a process. So, yeah, so context switching saves the state that's essential to the thread each time and then you load the state that's in the state and then you load the state that's relevant to the next thread that's going to run. So the switches can occur with a timer. That's the preemptive part, typically. That's for operating systems that reliably preempt the user code. Sometimes the application code will yield, which means I don't have anything to do now. I'm not gonna waste CPU time just in a crazy while loop sometimes happens, but the right thing to do is call yield, which is a system call that goes back to the operating system and allows it to allocate the resources to a runnable thread. Sometimes our operations will interrupt the processor and say some data's ready to be read and there's a few others too. Okay, so yeah, memory is the same, IO devices are the same. So some consequences though that we have good sharing makes possible typically high bandwidth communication, lots of communication between threads. On the other hand, because of that reason, if threads go wrong, there's a much higher likelihood of damage and the threads can share instructions as well. They often, they usually are actually, the programs for different threads is typically in the same block of memory. Also that's bad for protection though. And yeah, well, do you think you should be able to overwrite OS functions? Presumably not, so what does that mean, but how do you make sure, yeah? So that's a good question. Let me think about that. So yeah, I suspect that has to do with the level at which each thread is running. There's certain code that only should be running by say kernel mode or higher protection mode threads. That's a good question. I don't think that's a very complete answer, so yeah, I'll have to come back to that one. Anyway, should threads overwrite OS functions? Obviously not, but what does that imply? How do we make sure that happens? Yeah, you clearly need some parts of memory that are holding any kind of operating system or higher protection level code not to be writable, so you need to protect specific areas of memory. Okay, so some places where this model shows up, embedded software, because there's typically so many devices being run, the processor often has limited resources in terms of its ability to physically protect memory very often. And then also some earlier legacy systems that similarly had either, the early systems often used the simplest instruction set of Intel processors which didn't support more elaborate protection. Okay, so things are, I guess at the other extreme, when you have a multi-core and or multi-CPU architecture, then you'd like to allow threads to use not just a single processor, but as many as you have available. So that's supporting multi-threading, hardware multi-threading and hyper-threading. Yeah, so in order for that to work, or at least the simplest model, yeah, in order to make that work, the different thread processors, cores, or whatever they are for GPUs, they're actually much more fine-grained than cores, but it's a requirement that each thread unit contain the essential elements of a thread, so each distinct thread processor has to have a set of registers that its own set and a stack pointer that points to a unique area of memory that's its own stack. All right, so that's the minimum requirements. So in this diagram, the first column shows you what's happening in the single processor time-slicing example that we looked at before in a different representation. Here's a version with an additional thread shown in green, and the second column shows you the two threads processing available blocks of executable code, and then finally, hyper-threading is actually running, it's a little bit hard to see this, but hyper-threading the idea is that this is one physical core, but a hyper-threaded architecture actually provides additional thread state to each runnable core, so that it can run multiple threads at once. So it's almost like a, it is like an explicit thread processor, but there's more thread processors than cores. So in the Intel architecture, there's twice as many, and I guess they replicate the minimum thread hardware, which is the register set and program counter. So it's not quite as, it doesn't quite double the speed because you still just have the one core executing these two threads, but it can potentially give you better utilization of that CPU by allowing you to quickly switch to the other thread whenever there's a weight operation. At the level of hyper-threading, the weight states are incredibly fast. So for instance, the core may have issued a memory fetch operation, and it may simply be waiting for the memory data to come back. That gives it plenty of time to switch to the other hyper-thread and do something else, say complete an arithmetic operation. So hyper-threading can give you speed-ups, but it's a very mixed bag, actually. If you have memory-intensive code, you'll get typically almost no speed-up because you'll switch from issuing and waiting for one memory instruction to issuing and waiting for another one. So whether you actually get these speed-ups is pretty subjective. And it turns out in the latest Intel architecture, they're migrating away from hyper-threading. It does, the flip side is that sort of keeps the processor busy, but the cost of doing these swaps is fairly high and it also often causes some cache thrashing. So actually, the silver one architecture of Intel won't have hyper-threading. Okay. Yeah, so non-linear speed-up meaning less than linear. And yeah, there's a reading though describing the original hyper-threading. It has been in Intel architecture processes though for quite a while. And it does give you speed-ups and people do notice a performance. But these days though, processors are increasingly hitting a power limit anyway. And because of the rise of laptops and mobiles, people are more sensitive to the power efficiency. Yeah, question? Okay, go ahead. Yeah. Well, that's kind of a executable blocks of code, I suppose. So it's, or instructions that are potentially executable at that next step. So, yeah, so yeah. So there's a sequence of, each of these is showing an actual execution of an instruction. So there's holes here because there are sometimes weights and dependencies, some of the instructions effectively take multiple cycles and so on. Yeah. Well, I'm hesitant to say that just because there's, is that a reasonable way to put it? I mean, even, but superscalar processes are really complicated these days. They're a large amount of the circums, not just the pipeline. It's actually branch prediction. It's predicting what's going to be runnable at a given time. That's a reasonable statement. I think it might be a little bit simpler than what's really going on now. Just because, you know, if you look at the chip, there's more, something like half of the area of the chip now is not caches and not CPU. It's sort of, it is predictive code trying to anticipate what's actually going to be runnable soon and get resources ready for that. So, yeah. I suppose that is sort of a, it's a pipeline, but it's just much more non-linear than a simple sequence of executable steps. Yeah. Yes. Yeah, that's accurate. Okay. So let's just do a couple more things so we can break. I'll try to go a bit quicker, actually. So we need to implement memory protection to make sure that, you know, certain critical sections of memory, say kernel code, and in fact, probably user code as well as protected. So the threads are not writing to code sections. Usually need to protect access to IO devices at a minimum to make sure that it's exclusive. And yeah, so, and the processor access needs to be managed by the CPU, which is usually easy, but it doesn't require preempting, you know, and in the diagram before, though, preempting is almost redundant in that diagram because the processor has the ability to just execute instructions that are available. Okay, there's really much less sort of coupling between the instructions in some program of one user. All right, and some more, in order to make sure that the processor is able to switch continually, you can have a timer, but you have to protect that timer from errant applications changing its rate and preventing it from being able to issue the interrupts on time. Okay, so we, you know, memory addresses are pretty big. It's four gigabytes on a 32-bit processor, but most processors now have 64-bit addressing. So, oh yeah, and here's a typical layout of the areas of memory. Stack's gonna be coming down in the opposite direction from heap so that you allow the maximum growth of both of those, and text is typically the program area and then data, constant data sits here. Actually, no, not just constant data. Any fixed size data can be allocated there. Dynamically allocated data has to be on the heap. All right, so various things can happen when you execute a low-level read or write. Sometimes it won't actually address a physical memory or anything, you can trap or just do nothing. It should mostly act like regular memory if it is a physical memory address. It might be an area to which you have no access and that would again be a trap or a no-op. Sometimes it'll be a DMA access or a memory, a map-dio address, and that will be doing something significant and a number of those will cause traps if they're inappropriate. So, in order to be able to implement processes, remember they have to have their own view of the memory space and it has to be unique to each process. So that requires something and here it's a translation map from physical addresses, physical blocks of memory or ranges of memory into a virtual contiguous address space for the processor, excuse me, for the process. So you want for program two that's running as a process, you want the code data heap and stack areas to look like the picture on the previous slide. You want it to look like that even though physically it looks something like this and there's another process that's accessing nearby areas of memory. They have to be particularly careful with the heap and stack areas. This one's done a sort of challenging thing and split them but anyway. Yeah, so from this perspective, those two different processes are seeing a unique virtual machine, their own virtual machine, but the physical machine is looking like the picture in the middle. All right, so, okay, let's just a few administrative details and we'll take a break. So last time we mentioned we're using the piazza instead of a news group. So please make sure that you've joined up with the piazza and feel free to ask questions. We're just getting a few questions now. It'll ramp up, I'm sure, when the projects kick in but don't be shy about asking. In fact, because we have a participation grade posting on piazza is one of the most reliable ways of getting credit for participation. So don't be shy. We have some additional account forms if you didn't already get one last time. So you will need those for submission. And I guess, yeah, we're processing the wait list. We'll try to clear it actually as soon as possible so the wait list should be done by Wednesday. Here are some suggested resources if you don't know Java because we will rely on it very heavily. So you really need to be fluent with Java just not familiar with it. So make sure that you are in that shape when, by the time the projects skip around. We are planning some pop quizzes, there'll be more about that next time. We just need to get on the same page with Anthony. So critical thing, make sure that you do your project sign up soon. We're hoping every group can sign up for three slots at least so that we can optimize the schedule for everybody and that's due by Thursday. Thursday midnight. You'll need your account form to do that so we can put the numbers in. And yeah, please make sure when you're entering a group you've got all of the IDs correct so nobody gets left out or misplaced. Okay, new section assignments. And okay, sorry, you're right. Here are the sections and this is a bit misleading. You'll be attending sections this weekend next week. Here are the times so for this week you can pick any section and just pick one that's the most convenient. By next week we'll have people assigned and you'll be in a section with your group at the scheduled time. Okay, yeah, so we had some details about the projects last time so just to remind you there is a section this week. Groups will be assigned though this week and then next week it'll be a discussion section with your group and that will be true from now on. And we have a closed laptop policy in the sections and ideally in the lectures as well. All right, so let's take a break now for five minutes, have a stretch, get some air. Okay, let's try to finish up. So running a little bit late so I'll try to get through this quickly. So let's get started right away. Okay, so we're talking about process and threads. The processes are a more heavyweight abstraction and intuitively we think of them as a sequential stream in its own address space or at least maybe sometimes multithreaded in its own address space. It includes an execution stream, the state of some registers. It also has different from a thread, copies of resources such as main memory and IO devices. So what this means is typically when you start, yeah, I think you can see what's coming. When you start a heavyweight process you're starting a single thread. It may end up spawning multiple threads but what the operating system launches is a single thread, yeah. Yeah, that was a picture that we saw a bit earlier where you wanna create for each processor what looks like its own address and typically will be contiguous addresses for its different code and text segments. Yeah, so it looks like a virtual address space that's contiguous that's actually mapped often non-contiguously onto physical memory. Okay, so in order to be able to do the switching of processes we have to write something called a process control block and that's a snapshot of the execution and protection environment. So in the simplest picture there's only one process control block active at a time. So it includes the process state, process number, program counters, the registers, memory limits. So that's basically the mapping onto the memory and then some files and any other IO information. And so the operating systems has a number of these typically it's trying to manage and so it's gonna allocate in the simplest case just one of those at a time. It's gonna make one of those active. And it may have prioritization which means giving the more important process as higher priority process is more time. It also may have various access control attributes on the IO objects that it will also be obeying the rules of those access controls, yeah. No, this is a process. So we're only talking about one type of process. The process has basically like a virtual machine it has a copy of the memory and a copy of the IO devices. Yeah, there's just one process. There's just more detail about the actual information that the OS has to maintain in order to keep processes running. Oops, did I miss something there? Yeah, memory mapping, okay, that's fine. So let's say we have two processes running like this one and this one. Then the operating system is doing the work in the middle which includes things like interrupting. We'll see a little animation in a second but it's gonna basically manage some kind of interrupt that stops that first process from executing. Then it saves the state of that process into PCB zero somewhere. Then it's gonna load the control block for some other process and put that, well, loading that effectively brings that process in it'll then hand off a control to the PC of that process which will run for a while until there's some other interrupt or system call and then you do the opposite to get the other thread back. So yeah, so quite a bit of overhead. That's quite a lot of work for the operating system to do compared to thread switching. Also, that's typically, this is more explicitly a software process typically whereas the OS can often handle a lot of the thread switching, yeah? Oh, Lord. That's a hard question. I can tell you in absolute numbers, I know it's like 30 times the thread switching typically but no, I couldn't tell you instruct, I mean, yeah, I wouldn't attempt that for instructions. Quite a few, I mean, let me think, do I know in order of microseconds anyway? Sometimes even milliseconds, Windows sometimes milliseconds. All right, so anyway, here's an interrupt on the first process. The state saving happens. We load the other process control block, handoff control, runs for a while. Then we save that state back again and run to the other original thread, the original process, okay. All right, so some important things about the state of a process. Here's a canonical picture of the states that a process goes into. Normally it starts off new, it's created with all of its runnable state. It requests access to be run by the operating system and the first thing it does is put in a ready state which doesn't mean it's running yet. It means that it has everything it needs to run but it's not running yet. Then when the resources are available, the operating system keeps a queue of ready processes and from there based on priority and availability it will actually run them. So they'll run until there's some interrupt or perhaps they have to wait on something but whenever they stop, they go back to ready, they don't go back to running because the operating system should be in charge of deciding what actually runs. Okay, so created, waits first, then yeah, after it's been waiting for a while then when there's an opportunity, an available core, then it runs until there's something to stop or wait for, goes back to ready and then the OS will start it again when there's an opportunity. So back and forth between running and ready is the picture and that way you'll have, you can keep your process of busy, it also won't thrash by trying to run too many processes at the same time. Okay, and finally, finally, yeah, all right. Gets terminated at some point, hopefully with a good answer. So a process is, we've already talked about this, a process is not equivalent to a program. The program is just specifying the sequence of instructions. The process date includes also a kind of a snapshot of the machine state like registers memory and so on. So a thread includes just the code plus PC plus registers plus stack, the process adds that plus memory and IO. Yeah, I think that's this we can move on from here. Yeah, so process is often collaborate and for that reason we have to worry about communication. So, and already last time we talked about the two main types of ways of communicating between processes or threads. One is shared memory. The other one is what? Shared memory and what's the other one, yeah? Message passing, which pipes are a special case? Yeah, so message passing, send and receive, pipes have more limits on how and when, but yeah, so message passing can either be local to a machine or it can work across a network. So for shared memory communication between processes, we have a similar kind of memory map as before, except that this time we have a special area of memory that is visible to both processes and that's their communication space. So yeah, so that allows much lower overhead communication but adds the complexity of the application program now having to figure out when the rights are complete, who has access to which part of that area and so on. Very often processes, but the majority of the time though processes communicate with messages instead of shared memory. It's a lot easier to set up and usually much less error-prone. So yeah, so canonically, a message passing system has send and receive operations. It does require that they have some kind of communication methods, sometimes sockets. Sockets are very elegant because they support both local and remote communication and some kind of send receive primitives. Yeah, so let's see, we talked about threads as being lightweight processes. Though processes still work with a single address space and we have no protection between threads. They're sometimes called lightweight processes but as we've seen, there's actually a lot more differences, especially in safety and performance. So finally, to sort of separate threads from that of a process, you can think of a process of having a thread part which is the state that's other than the memory in the IO. And yeah, heavyweight processes are the ones that have that sort of one main thread that starts up. All right. So processes though can be running multiple threads. So here's a simple single threaded process here and here you can see, should be starting to look familiar now, that they share the code, the data and the files. Threads do, but they have a separate set of registers in stack in order to be able to run independently of each other. So they'll communicate as needed through that shared state at the top. All right, so some examples of multi-threaded programming. Programs, most embedded systems are heavily multi-threaded because they need to respond to many different sensors and stimuli. So yeah, there's a single program often written. There's not like separate programs that need to be loaded. So there's simplicity in the programming, but on the other hand, there's a lot of complexity and concurrency in that program. On the other hand, operating systems have to have the ability to take a piece of code that's unknown and manage the resources that it uses. So yeah, last thing I want to just mention is database servers, which are a really important example of concurrency, the complexity of concurrency being hidden from users. So you can almost think of databases as providing something like a very macroscopic shared memory abstraction. And they manage it in clever ways such that clients are only able to read and write certain parts of the shared state. And they do it without, even though they're doing it using separate threads, they hide that from the programmer as well. All right, so, okay. Last graphic. So we can classify whether or not systems have one or many different address spaces. So when it's multiprocessed as many address spaces and whether there are multiple threads supported per process. And very quickly, we had the early operating systems were only one process, one thread. You know, UNIX supports many address spaces. Typically, only one thread though per address space. That was the heavy weight process we were talking about. Embedded systems though, encourage the use of multiple threads within processes in order to support rapid context switching in full use of the processor. And then now we have some more modern operating systems that support both many processes and many threads within them. All right, so quickly, here's a summary of what we talked about. Threads, the fundamental simplest unit of concurrency, but also the most challenging to protect and keep running accurately. They run in one address space. We talked about how concurrencies achieved through time slicing either with one processor or many. And we talked about protecting access to memory by mapping resources, mapping the virtual memory to different parts of physical memory for process isolation. And yeah, I'll leave those last comments for now. All right, good. We'll see you on Wednesday.