 So I'm going to talk to you guys today about the threads. Doing threads in Ruby, doing threads generally, thread synchronization, any problems with thread programming. Here's a little example that I actually used to start out the thread in Ruby. You create an instance of the thread class with thread.new, pass it a block. Inside of the block is the value code for thread. And that's pretty much how you create a thread. That's good. So that's what you need to know, right? It's everything you're just going to know about threads, right? Well, there is also another little problem, which is race conditions. His threads, unlike processes, are unlike processes in this thread, they're independent threads of execution. But unlike processes, they actually share a memory with all the other threads in the program. So you have to make sure that you don't have two threads modifying the same piece of memory, modifying the same object at the same time. If that does happen, you have what's called a race condition. And at least 90% of doing thread programming is avoiding race conditions. So we'll skip these slides and come back later. So race conditions. Two threads modifying the same object at the same time will cause a race condition less and less. It's one of the very few objects which are designed to be used for multiple threads. In the root standard libraries, there's exactly four types of objects that can be used to save this for multiple threads at the same time. There's our mutex, q, size, q, and condition variable. If you're not using one of those, you're going to have a problem. See, like right here, this is an example of race condition. There's going to be a problem here eventually. We've got this shared object, a, that's being used by both of these threads. Now, we imagine this array contains a few names. And so what you want to end up with is the names near each other, presumably. So John and Jones would be right next to each other, saying the same thing, right next to each other. But if you do it this way, without any kind of synchronization there, you might well end up with this situation where the first and last names are not actually adjacent. And that could happen if you had the context which right at this point, after the first variable, Christian's first name, and of course, the first and last name. It was kind of a toy example, but it's a good one to show a different synchronization method of how to use those to resolve this problem. So let's see. Well, before I do that, let me say a few things about threads in Ruby generally. Ruby threads, now, unfortunately, they're not really that good. 1.8 would be a green thread, which is the city interpreter at its own notion of threads independent of the operating system threads. And that's usually green thread implementations tend not to be as good as you get in properties as well. 1.9 Ruby's actually using the native threads API, but from the user perspective, it's much the same situation. With 1.8, because it had main threads, you ended up only using one CPU on multi-core systems. It didn't have the ability to actually use one on core. 1.9 is using native threads, so you'd think you'd be able to use multiple cores, but it's not, because the interpreter itself is still not re-entry. And so there's this big lock, which every thread has required before it can do anything. And the pretension for that lock ensures that no more than one thread can run at once. So a lot of what I'm going to say applies to go through the 1.9 and 1.9. When you use your perspective, unless you're maybe writing a C extension, it looks pretty much the same. 1.9 threads might perform a little better. Now, Python has the same problem. They also have this global lock. It's been that way for a long time, Python. In fact, I think over 10 years ago, someone tried to get rid of it, and they managed to interpret the work without the global lock. But the result was so much slower, even so that change got backed out. So it's not going to be an easy thing to fix. I think we are going to get there eventually. Question? I'm sure it's basic, but what exactly is T1.join doing there? Oh, the join. So what that means is that whatever thread calls join will wait until the other thread is finished, whether it's joined or finished. And then if that thread, whatever the value of glass expression value that that thread was, will be returned by joining, which is kind of a nice little indication mechanism sometime. So it's a blocker waiting for the thread to end? It's waiting for the thread to end, yes. The other thing that the join is useful for, if the thread is joining ends up normally, if it raises an exception and that got raised up past the highest level on the thread, that exception will get propagated to the caller and call it join. I'll talk about that a little bit later. Thank you. So threads on Ruby run MRI and it sucks, because you can't use multiple cores. You can use JRuby, which has native threads and doesn't have a lock problem. So you can't actually use multiple cores with threads on JRuby. This is one of the big selling points of JRuby. Another possibility if you really need to use multiple cores is to use bossa instead of threads. Where threads will help you is not for CPU bound problems, but if you have an IO bound problem or network bound problem specifically, using threads will make that better. Because the global lock does actually get released by threads that's waiting on some kind of IO operation. Any idea on their other miscellaneous uses? It's like if you're waiting for an external process to finish or something like that, it's more or less an IO bound problem. I don't know what's next. So this is a race condition. And there are various ways of solving race conditions. I'm going to talk about a number of mechanisms. First, mechanisms I'm going to talk about are low level mechanisms, which aren't generally recommended for every age. And I'm going to just go over them real briefly. There's critical sections. This is an example of using critical sections. Critical section stops all of the threads until you end the critical section. And another mechanism that exists in some systems, Ruby doesn't actually have this, but I think it should. It's thread-sustaining thread-sustaining. Where, instead of stopping all other threads, it's a different implementation system. Instead of stopping all other threads, you just stop the one other thread that you know is going to be getting the same resource. There's atomic access. This is usually reserved for assembly language programmers. So you can do it and see if you know what you're doing and get away with it. You can actually get away with it in Ruby. I'm not really recommended. But basically, the insight I have at atomic access is that the instructions are not applicable. So an instruction will continue executing, but if an interrupt comes in the middle of a CPU instruction, and then the interrupt won't actually get handled until the instruction has retired. And since thread switches are always one way or another based off of some interrupt coming in, it's usually a timer enough to be at something else like a keyboard interrupt. That means that if you can get the synchronization work that you need to do, cram down into a single CPU instruction to use that as a synchronization mechanism. So here's some examples of some things you can do and see that usually end up being atomic, but sometimes it depends on the platform. This last one actually isn't a single atomic operation. It's two atomic operations. One would be one. And also, it turns out, are actually atomic in Ruby as well, but for a different reason. So one trick that I've used sometimes in the past is you can have an integer counter variable, and you can actually write to that variable. You can increment that variable in multiple threads without synchronization, and get away with it. We'll end up with the right counter again. But it's not exactly kosher to be that. And it probably doesn't work, but in jail, Ruby might not work in the future. Another synchronization mechanism that you see sometimes in Ruby is sleep. I talked about critical sections and so forth. So I'm not recommended for ordinary use. Sleep is not recommended at all. But nevertheless, you do see it used. The idea with sleep is one thread just goes to sleep for some amount of time. And hopefully, whatever other thread is running will finish doing whatever it was. So first, the thread is sleeping. Now, when you're designing reliable software, you don't want to have the group, hopefully, anywhere in your behavior description. Yeah, any questions? I understand why this is a bad practice. But why is the critical section not recommended? Why is the critical section not recommended? There's a number of reasons. One reason has to do with latency. So you're stopping all of the thread in your critical section. One of the other ones could be doing something important. There's a portability issue. Critical sections don't work on all systems. In fact, I just went real briefly through the example of mutations I had for this. But on a multi-core system, implementing critical sections is kind of tricky. And a lot of operating systems like Linux, for instance, don't actually have critical sections. It's not part of the positive TPI. There's some other reasons. What are the other reasons? I can't. Before we get too far away from atomic stuff, too, I have a... Yes. And in the past, I've used the system called move of the file. And that has, in my experience, been universally atomic across at least Solaris and Linux systems. I'm just curious if you have any knowledge of whether that would work in there? So the question was if the system called move is atomic. I believe it would be, you know, I haven't looked into that. It's got a vice-pattern, I realize. Well, that's a pretty good question. It probably says something about that in the positive TPI description for move. So I'm guessing that the kernel guarantees at least the appearance of atomicity when you call move. So that if you're moving multiple files into the same directory, they'll end up at different points in the directory. They'll end up with different... It's a lot of use the same durant, basically. So you won't... And that would be important for file system consistency to get it end up with bad files and bad ones in case. So presumably, they've made that safe operation. Any other questions? All right, let's see. I think I'm going to go on at this point. So I don't use sleep. There's another mechanism that she sometimes sees used for synchronization, signals. These could be like regular unique signals using the signal API, but it could be other things that are not standard unique signals that resemble them in a lot of layers. But either way, you run out of the same problem. Signals are like these ephemeral messages that are sent between threads or between processes. Now, for an inter-process communication mechanism, signals are actually decent for some things. They're limited because there's only like 32 signals or something. And on some systems, you also get this, like, addition of signal number. You get the signal parameter. It's just like a 32 bit word. You can set anything you want. But that's not a very broad channel for communication. You can't cram a lot of information with that. And most of those 32 or 64 signals, whatever it is, are reserved for various system uses. But really, the big problem with signals is that they're ephemeral. If the guy who's receiving the signal isn't in the right state at the time when the signal comes in to handle the signal properly appropriately, the signal will just get dropped. It'll get lost. And the information that was sent is just darn. And that can be a problem. I actually used to work on a system which used signals extensively as its primary means of inter-thread communication. And it was not good. We could eventually make it work with a lot of jumping through poops to ensure that signals all get delivered at the right time so that nobody was happy with it. So signals should be avoided as an inter-thread communication mechanism for possible. There are some cases where they might be desired if nothing else would work. Now, when it comes to signals in Ruby, Ruby actually takes a kind of a nice approach to signals in some ways. Ruby forces all signals to be delivered to the main thread. And this is a good thing because this means you don't have to install any signal handlers in any thread other than the main thread. You don't have like race conditions where signal could come in before you install the handler or signal comes in and gets delivered to whatever thread is running at that time, which leads you to possibly inconsistent signal handling. It's just a whole lot easier this way. But this does mean that you can't actually use them for inter-thread communication. And it would kind of actually be nice if there were like a thread signal feature in Ruby as opposed to the process. So let's see. So I talked about some low-level, not recommended communication mechanisms. And I'm going to talk about some high-level mechanisms. First of all, these are semaphores. Semaphores are a pretty important mechanism. Semaphores are basically a lot like signals, except that they have state. They remember if they've been sent or not. So they don't have that forgetting problem that you get with signals. So there's like three important methods that semaphore class has. As this signal class would say, whatever the event the semaphore represents has happened. This is an example implementation. I emphasize it's pseudocode, you could use the idea of how it could do it. There's a weight method, which says, I'm going to just block me until whatever this signal is gets sent. There's also a broadcast method, which is like the signal, except it wakes up all the threads that are waiting on the semaphore. Now here's how you could use a semaphore to solve the example-width condition I was showing earlier. So basically, the second thread weight on a semaphore, which is the first thread of signals when it's done with the share of resource. There's also a variant of the semaphore called the cabin semaphore, which would be signal multiple times. And it will remember that it's been signal so many times, and then if you signal it four times, then four threads can come along, wait for that semaphore, and it won't block. But the fifth blade will block. Now, there isn't, unfortunately, a semaphore class in Ruby. It used to be the case in older versions of 1.8. I don't think this is true for the current releases. But you could kind of twist the Lutex and use it as a semaphore. But this is a nice communication mechanism to have. It's one that I didn't really relate to very well. So it's something I hope that will be added to Ruby. One of the current Ruby summer code projects, which happens to be the one I'm mentoring, the student is, among other things, implementing semaphore class. Hopefully that will get accepted into GMI at some point. So there's two use cases for signals. I tend to demonstrate in the signaling use case for semaphore. One thread signals to another thread, then it's out. There's another use case, which is mutual exclusion, where you use a semaphore to detect some common shared data structure from semaphain's access. And that use is so important, so common, that we have a special type of semaphore just for mutual exclusion, which we call the Mutex. So Mutex is like a semaphore, pretty much. Instead of signal and weight, you have lock and unlock. Lock is the equivalent of weight. Signal is the equivalent of, you know, unlock is the equivalent of signal. And the idea with a semaphore, and with a Mutex, let's see, there's some examples of how you could implement it. Here's how you use it, and this would be the normal way to solve this particular race condition. You have a Mutex that you create before you create threads, and then each thread locks the Mutex before it touches this shared variable A, and unlocks it afterwards, and this ensures that there won't be any kind of weirdness in the sequence of operations that get done on that. Mutex is in some implementations, has some additional features, beyond what you see in the normal semaphore. Only one of these is actually implemented in Ruby at the moment. So right now, Ruby has an error checking mechanism where it verifies that the same thread unlocks Mutex as the one that locked it originally. Because if you don't, if that's not happening, you probably have a bug somewhere in code. This is also unfortunately the feature that prevents you from using Mutex like the signaling semaphore by a well. There are some other nice features that some operating systems or environments have for Mutexes, Ruby does not, or absolutely added at some point in the future. So for instance, there's deletion safety. So if you have a thread which dies suddenly, maybe it ran into an error condition, and it just got raised all the way at the top of the thread. If it's holding any Mutexes at that point, you probably want those Mutexes to be released. Otherwise, they're never gonna get unlocked. And anybody else who tries to use those Mutexes will just end up blocking forever, deadlocking forever. Another nice feature to have is recursive access. Sometimes it can be convenient to have a thread lock a particular Mutex multiple times. And then the system will know not to unlock it until not to actually unlock the Mutexes. So it's actually the same number of unlocks have been followed. And then the other feature for, it's nice to have for Mutexes is priority inheritance, which is a solution to the problem of priority burden. I'll be talking about that a little bit later. There's a couple more synchronization activities to cover these condition variables. This is the last, no, it's not the last one. This is another mechanism that's present in the Ruby standard library. I don't relate well to the condition variables, they make it, they make it confused. So like here's an, I believe I actually took this code and printed out of the Ruby standard library, and cut it down to a fit on the slide. Condition variables basically like a signaling sound board is used to signal an event has happened. Except there's also this other Mutex that's involved. And it's kind of hard to see why you want the Mutex. It's starting to grow on me a little bit. Basically, you use this if there's some signal that you need to wait for while you're holding a Mutex. And you want that Mutex to get unlocked while you're waiting, and then locked again when the wait is over. That's when you need to use a condition variable. I don't run into cases where I want this that often, but it could be nice for that situation. Here's how you could use it to solve this race condition. Is this a lot like the semaphore COVID-19 showed? I've got this other Mutex here, which apparently exists only to be passed, wait for no good reason. Okay, now finally we have queues. We all know what a queue is from our CS 101 class, I'm sure. There's a special type of queue. Here's an example of a queue implementation for the threat communication. There's a special type of queue that's used for Q and Q among threats where if you try to take something from the queue when it's empty, the wait and your threat will block waiting for someone else to push something onto the queue. That's nice to have. And there are some styles of threat programming which emphasize queues extensively even to the exclusion of Mutexes. Now there's some people who say, I don't really like Mutexes, I'd rather have queues. So if anybody's ever heard of Actors, that's a queue-oriented style of threat programming. And I don't think you should go to that extent, but there's nice to know about these. So here's an example of how you could use it to solve a race condition. Basically, you put this shared variable on a queue and the threats have to take it off the queue before they use it and put it back on when they're done. All right, how am I doing on time? How much time do I have left? Let's see, two, three, I have a few minutes. Okay. So there are some pitfalls to doing threat programming specifically in Ruby, there's a couple and there's a couple that generally run on two. So one of them, exception that's raised in a threat so by default, cause that threat to just die silently without any kind of error message or actress or anything. And you won't know that anything, what that has happened. This has bitten me so many times. So it's a good idea generally to do something to deal with this problem, because otherwise stuff will go wrong and you'll just be like, whoa, scratch head, what happened? So one way to deal with this, you put an exception handler up at the top of it, right? Another way is there's this a board on exception method that you can set for your threads, it's an attribute actually, so you set that to true and that means that when an exception comes along it'll print something out to standard error, standard usual stack trace. And then another thing you can do is to make sure to always call join on all the threads you create. It's not always convenient to do that, but if you do do that then that means that whatever exception caused your threat to die will actually be re-raised at the point where the join call is called. Which is one of the nice things about join. So another of the thread pitfalls for Ruby has to do with the main thread. When you start Ruby process up, there's one thread which is called main thread that has some special characteristics. One of them is that when it dies, either by calling exit or implicitly by falling off the end of the program, all of your other threads die at the same time. Even if there's still running, doing stuff, all of the threads, the whole process just goes away. So you have to put usually something in the end of your main thread to either put it to sleep forever or join all the other threads or something. Or this will be a problem for you. There's some general pitfalls for the program in any environment where you see your job or whatever. I wanted them as race conditions, but talk about that a lot. I can solve those with a lot of mechanisms. Another is deadlocks. A deadlock happens when you've got typically two threads which are both waiting on the other thread to do something before they can continue. Since both of them are blocked, neither one is gonna signal the other guy, both of them basically are just gonna sleep forever, never do anything. In practice, I haven't run into deadlocks all that often. It do happen sometimes. And they are something you have to think about when you're designing a program. But they're, it's like maybe a tenth as common of race conditions. Now these two problems are kind of symmetrically related. Race condition is the result of under synchronization in your design. A deadlock is the result of over synchronization. And there's one technique that I've heard of that sometimes people use for dealing with deadlocks. Which is, if you have multiple mutexes that you have to acquire, make sure you always acquire them in the same order. And that helps avoid the case where you've gotten one of the mutexes and then somebody else got the other one. Other than that, I don't know any general mechanisms for dealing with those. Okay, so another problem to sometimes run into is priority inversion. Priority inversion, this only happens when you have threads that go to different priorities. So here's a picture of priority inversion in action. If you have threads, three different threads with all the different priorities. Initially the low priority thread is running. It gets some lock. And then it gets preempted by the meeting part of the thread which wakes up. And then that gets preempted by the high priority thread. And then it tries to get that same lock that now it can't because the low priority thread is still holding it. So the high priority thread goes to sleep waiting for the lock to get released. It goes to sleep and the next highest priority thread is the meeting priority thread. It goes along the lines for a while. So basically what you have happening here is this high priority thread is waiting for the meeting priority thread to do something. So it's waiting for a lower priority thread which is something you're not supposed to have happen. Thread priorities. Eventually the meeting priority thread presumably to finish and then the low priority thread will release the lock and finally the high priority thread gets to run. But that wasn't what you wanted to have happen. So the solution to this is something called priority inheritance where if you have a high priority thread waiting on a mutex which is being held by a low priority thread, the low priority thread gets a temporary boost to its priority at the same level as the high priority thread and that ensures that the situation can't happen. So the final pitfall for threat programming is performance. Now we think about threads as something used to increase your performance in your system, right? Use multiple cores on those languages that support it or do things in the background while you're waiting for IO. So why do we have these performance, why would there be performance problem 12? You have multiple threads, you have context switches and in certain, depending on circumstances context switches can be relatively expensive. Not so much the saving or storing of registers on the stack and that kind of thing. But the fact that when you switch from one thread to another, you're pretty much guaranteed that you're gonna have a pipeline reload. You're pretty much guaranteed that whatever was in the data cache is not gonna be useful for a new thread. Whatever was in the instruction cache is not gonna be useful. So all of those things have to be reloaded. And that can add up to a hit, especially if you're doing a lot of context switches. So context. Two minutes. Two minutes, thank you. Context switches are something you wanna avoid or possible. And they do show up a lot in designs that are retext heavy, which is why some people prefer to have cues. So what you can do about that, there's a number of different techniques. There's some special types of new texts that you can use. For instance, there's a read, write lock. You'll notice if you do a lot of thread programming that a lot of times you're blocking a new text just in order to read something from some global data structure. And it's actually safe to have multiple readers meeting from a shared data structure at the same time. You just don't wanna have a reader overlapping with a writer or a writer overlapping with another writer. So a read, write, lock allows you to have... Multiple readers actually use the data structure at the same time, but only one writer. There's another thing called a spin lock, which is useful for systems where you actually can use multiple cores. So when you try to acquire a lock, instead of going to sleep and causing context switch, you're just going to busy waiting on the lock. And if you know that you're not gonna be waiting for a long, and that can be more efficient than actually doing a context switch. Another technique is to spread out the mutex usage. If you've got one big mutex protected, one big data structure, which is used a lot, there could be a lot of tension on that mutex. And you can save, prevent some of that contention by making instead multiple mutex spaces to protect parts of the data structure instead. Now it depends on your application that may or may not actually be useful or possible rather, but it is a technique that you can use. So that's about all the material I've got. So at this point, does anybody have any more questions? No questions. Okay, thank you very much, everybody.