 I'll come back everyone. This is Brian. We're going to talk about thread pools. So what is a thread pool? Well, it's reusing existing threads because creating threads are expensive. And let me explain just a little bit here. So when you create a thread, it's very simple in Python. But what happens under the hood is a lot of calls to the operating system. The operating system has to allocate CPU, RAM, all this other great stuff that you need. And it gets very expensive in terms of well, computing power. Now, think of this in terms of you have thousands, and I mean, thousands of things you want to throw onto a thread. Your computer's not going to handle that many threads. Actually, most computer operating systems will cap the number of threads a program can run. So eventually, your program will crash if you try to create too many threads. Don't worry, that number is absurdly large, more than you're probably going to use. So a thread pool looks something like this. You have a bunch of work producers and these could be classes, these could be network sockets, keyboard input, any sort of work, anything that's going to be created. That becomes work that sits in a job queue. Think of this like a list or a tuple or a dictionary or something like that. And then you have a predetermined number of red pool workers, or these are individual threads that sit in a pool. Think of a pool like a well, a list of threads very similar to what we did in the previous video. You are going to have a set number. The difference here is, as these threads finish their work, they become available for more work. So you can then basically just create a factory where you're pushing work into this thread pool and work gets completed by whatever worker that's free and the work stops and waits for a free worker. It's actually incredibly cool. Let's dive in and take a look. Okay, first thing we need to do is imports. I love importing things. So we're going to import logging because we're going to be well logging things. And we're going to import the redding. We're going to import that entire module just to show that you can do it. It really doesn't have some severe impact on your program, even though it is kind of poor form to do it this way. And we can say from concurrent dot futures, import the thread pool executor. What in the heck is this? Okay, so we're saying from the concurrence module futures, we're going to import a thread pool executor concurrent when you hear the word concurrency. Really, this is a high level version of threading. It's a way of doing threading with hiding all the ugly thread details. Future basically means we're going to work with a value, but we don't know what that value is, we're going to get that value in the future, which is why it's called a future. And then we're going to import the thread pool executor. This is a very fancy term for saying hey Python create a thread pool and execute some code. Now, I think somebody correct me if I'm wrong, you have to have Python 3.2 or higher in order for this line to work. So if you start getting a unknown can't find it blah, blah, blah. That's what's going on. You need a newer version of Python. And then we're going to go ahead and say import time because we're going to stop time using the sleep function. And when I say stop time, we're stopping the current thread. Remember these threads are going to run in a pool of threads. And we're going to last but not least import random because we're working with the random number generator. Now that we've got all our imports in, we need to make a test function. This is the function that we're going to run on a thread. So we're going to say def test, and we want an item that we're going to be handing to this. From here, I want to get a number that we're going to pause this thread and I want it to be random. So I'm going to say s equals random, brand range, and we want between one and 10 seconds. And I'm going to say time that sleep, or whatever that number is, effectively putting the current thread to sleep. Now this is not abundantly helpful because it doesn't really tell us what's going on. So we're going to use logging. Now we haven't configured logging yet, we're going to do that in the main function. So leap of faith, let's just assume that if we're running this function, logging has been configured. So info, let's go ahead and say thread. And now we want to know what item we're on, because each thread is going to be a different item. We want to know the ID of the thread and this is not something we assign to it, it's something the operating system tells Python and Python tells us. So this is going to be different on every single app, every single platform, every single operating system and even in specific versions. So do not get attached to this, it's not going to hang around long. So I'm going to say threading. We want to get underscore ID ENT short for identity. So we want to get the current threads identity. So get a dent, return a non zero integer that uniquely identifies the current thread amongst other threads that exist simultaneously. This is basically how the operating system determines what the thread actually is. We also want kind of like a user friendly way of doing it. So we're going to get the thread name. The name doesn't always exist. But because we're running this in a thread pool, it should be there. So I'm going to say current and underscore red dot name. In case you're wondering, yes, the current thread does actually hand you a thread object. So return the current thread object corresponding with the caller's thread control. Oh, man, that gets super confusing. But basically what you need to understand here is we're getting an ID that the operating system uses to determine what it is and we're getting the name which is basically a friendly version of the ID. I shouldn't say it's a friendly version of the ID because they're two totally different things. The name's not guaranteed to be there, but the ID pretty much is. All right, now let's go ahead and just grab this. And I want to say, probably should fix this or we're going to have a bad time later on. There we go. Sleeping for and then I want to know how many seconds we're actually going to sleep for. Then we're going to go ahead and put that to sleep. And then I want to know when we're actually finished here. So I'm going to say whatever the item is and let's go ahead and say finish. So looking at our test function, it's abundantly obvious this does well, nothing but just kicks out a bunch of logs and then goes to sleep and then lets us know when it's done sleeping. And that's fine. We're really just simulating activity, but now we're going to make this work in a pool. So let's go down and we're going to make a main function. Go ahead and fill this in. We've talked about all this before. So if this looks really confusing, I humbly suggest you just rewind the playlist. Go back and watch previous videos, but we're basically saying if we're running in main, meaning if Python's running this directly, go ahead and run the main function. This function we're going to go ahead and we're going to configure logging. And I'm going to for the sake of time, just copy and paste the configuration because we've done this before. There's no sense in repeating it. So we're saying logging basic config and we're going to have a format of level time message. We want to specify the logging format is hours, minutes, second. And we want to get the logging level of debug because we want to capture everything from here. It becomes very, very simple. I want to say logging info and start. And then I want to know when we're finished. So in here is where we need to actually do our threaded code. I'm going to say workers equals five. And this is the number of actual worker threads we're going to do. Now we want items we want to work with. So I want to say 15 items. Notice how we have more items than we have workers. This is why a thread pool exists because we want to work with these items in well chunks of five because we only have five workers. So we're reusing the workers as each one of these becomes free, it'll grab another item and so on and so on. From here we're going to say with and we want the thread pool executor. We're going to set the maximum. And actually want the max workers. There we go. The our workers number. This tells it, hey, use a maximum number of threads and we're setting that to five from our workers variable here. Now we're going to say as X the Qtor. There we go. I like that word executor. So now that we have this, we need to work with the map function. I'm going to say executor dot map and we've covered map before. There's a reason why I covered it earlier. It's because we're basically mapping a function to a list of values. So we're going to say take our test function and we just want to just simply say range items. We're going to let range actually build up that list for us. And that's really it. We don't have to do any fancy joining. We don't have to monitor the threads or anything like that. So really all we're doing is say thread pool, make a pool of threads, give me a variable that represents it and then do something with it. Go ahead and run this and see this in action. Oh boy, this looks kind of crazy. So we're just going to give this a minute, let it run and you see at the very end, finished, finished and maybe, maybe, maybe. There we go. Application finished. So let's actually flush this out a little bit better. Say app start and app finish. That way we know exactly what's going on here. Let's clear this and let's actually get some real estate here. Let's see what's going on under the hood and thread land. You can see the app start. Now we have all these threads that are spinning up. You notice the only going chunks of five started at zero went to four and now it's just kind of random numbers. There really is no guarantee on how these threads are going to be run. That's the operating systems job to figure that out. You can see we're getting thread IDs along with names and they're named thread pool executor zero underscore something. The naming convention is internal to the thread pool executor. Don't really rely on that. You can change the names if you want, but I would not recommend it. The IDs are given to us from the operating system. And then sure enough, you can see finished, finished, finished. And then finally, all the threads from the thread pool executor are joined in the application itself finishes scrolling all the way up to the top. You can kind of see the order app started. Then we have thread zero thread one, two, three, four. Those are our five workers right there. And then something magical happens that work gets put into a queue. And the thread pool executor waits until it has a free worker. In this case, it was thread to pop up and said, Hey, I'm available. So the thread pool executor said, Hey, thread to go perform this work. It's actually a very delicate dance and it's highly efficient in terms of memory usage. So if you have a large body of work and you want to make it multi-threaded, do not create a thread for each item instead use the thread pool executor. I hope you enjoyed this video. You can find the source code out on github.com. If you need additional help, myself and thousands of other developers are hanging out in the Voidromes Facebook group. This is a large group with lots of developers and we talk about everything technology related, not just the technology that you just watched. And if you want official training, I do develop courses out on udemy.com. This is official classroom style training. If you go out there and the course you're looking for is just simply not there, drop me a note. I'm either working on it or I will actually develop it. I will put a link down below for all three of those. And as always, help me help you smash that like and subscribe button. The more popular these videos become, the more I'll create and publish out on YouTube. Thank you for watching.