 Welcome back everyone. My name is Brian. We're going to continue our journey into Python. All right So we're gonna talk about a few different things with threading and we're really leaving beginner land and just jumping head first into advanced territory here So we want to avoid a few things. For example, the dreaded race condition and deadlocks. You don't know what these are a Race condition is the same resource notice. I didn't say variable the same resource Modified by multiple threads pretty much at the same time or close to it We have to have something called locking and that's what we're gonna cover in this video Now the problem with locking is you can create another issue called a deadlock, which is multiple threads Waiting on the same resource So think of it this way think of your computer as a giant cookie jar Now the problem is the mouth of this cookie jar is only big enough for one person to reach in at a time So if you have multiple people trying to reach in at the same time Either they're going to reach a deadlock meaning neither of them can get in or they're both gonna magically get in but then neither of them can pull their hand out That is confusing We're also gonna touch on another subject and I'm going to just copy and paste this giant text wall here bang It's gonna be out on get out in case you want to read this whole thing But I will summarize we're gonna talk about the gill or the global interpreter lock and python gets a lot of flak For threading because it handles threading differently Then other languages and to just dispel with submits and rumors. Yes python is Truly multiple threaded what I mean by that is when you create a thread You can actually see on the operating system a thread is created The problem is all of this bottlenecks into python in the gill and what this is is basically One thread one CPU at any given time. It just does it blistering fast So from our programs perspective, we can have multiple threads and they appear to be all working in unison But really under the hood See python supports multiple threads within a single Interpreter Meaning so far we've been working with just one instance of python, which means these threads all go through one gill There's one door. They're all trying to run through at the same time to kind of circumvent all that after reading all of this You have to use something called Multi-processing which is using multiple python interpreters running in tandem Highly advanced topic we're gonna cover later, but I want you to understand as We talk about threading someone's gonna say oh python's not truly multi-threaded. Well, it is It just has this global interpreter lock and there are ways around it. You have to use Multi-processing which is blistering fast, but uses more RAM. So let's dive in and let's take a look First things first we need our imports and we're gonna make a global here So I'm going to just for the sake of time. So you're not watching me type all this crap out There we go. We're gonna import logging threading and the concurrent features. We're going to import the thread pool executor We've covered all these in previous videos Which is why I'm not taking the time to really dive in each one of these We are going to stop the current thread and we're going to use a random number generator But additionally we want to use a global variable and that's why I wanted to slow way down at this point and say this Is where we're going to have problems counter equals zero This looks very innocent and we've done this before but now we're talking about multiple threads So take a moment and just really absorb what we're doing. We're making a global variable called counter We're going to spin up multiple threads and increment this counter. What could possibly go wrong? Remember what we're talking about is thread locking a Race condition is multiple threads accessing the same resource. What are we doing here? We're making a single resource that multiple threads are going to access Let's take a look at how python actually works with threads Let's go ahead and set up our test function. This is what we're going to call the threads on I should say the function is going to be called on the threads. I'm say depth test and We want some sort of count that we're going to increment our counter with We're going to use that global variable counter And we're gonna go ahead and get the thread name And this is going to be the threading and we've done this before So I don't want to spend a whole ton of time on this if you miss the videos definitely go back and rewatch There are 45 other videos in this series so far I'm going to make a lot more But we've talked about what the current thread is and how to get into it Also, we're going to make a wild assumption that logging is actually set up at this point We've covered that in a previous video where you have to configure logging before you work with it We're doing it a little bit backwards here. We want to start using it before it's really configured We'll catch up with that in a moment. So we're gonna say starting and then whatever our thread name is And we're gonna mess around with this function a little bit over the next few segments But I just want to flesh this out really quick so you get an idea of what we're going to be doing here So we're gonna say 4x in range very simple and we want you range on the count Which is the argument that we're given and then we're just going to say Logging and I want to do count thread name plus equals And we want the out just so we can see what's actually going on here We're gonna play around with this and I'm gonna show you locking and unlocking and acquiring and releasing and all that But I really just want to show that this is what's going on so Now once we're done, we just want to say completed Whatever our thread name actually is so this is gonna be our test function now right in here is Where we're going to do some work Put them out work here Okay, now that we've got a test function. We're gonna go ahead and do our main function. So Done all of this before so I'm not gonna spend a ton of time Explaining what all this is highly recommend you go out and watch the other videos Just so you understand what's going on in case you skipped any of them So we're just going to make sure that python knows if we're running as main to run the main function and then inside the main function I'm going to Copy and paste some config code here So we're going to configure our log Using basic config and we're going to say level sync time now one little bit you should notice is instead of just string I have dot and then the milliseconds three decimal places So I want to know the actual milliseconds and we're going to log that and then in the rest of it's just the same thing We've been doing with the hours minutes seconds It'll have the milliseconds behind it and we're logging debug. So we're collecting everything now from here I'm going to The old copy and paste by mouse wants to work We're going to say app finished and then in between this is where we're going to actually kick off our Red pull executor which we've talked about in the previous video some say workers Equals and we just want two threads that we're going to use in the pool With And I want thread pull executor Go ahead and set our max workers equals workers That way we're telling the thread pull executor use only two workers as And I'm just going to say as ex and then four x in range And I want to do the workers Times two so basically I want to do more work than workers That way we can actually see the thread pull in action and it does well exactly what it's supposed to be doing And missed a little there we go Now I want to get a value and we're going to say just a random number And I want a ran range. We've done this dozens and dozens of times But basically we're making a random number between one and five Now I want ex.submit and we haven't talked about this before we've talked about map which does the map function Submit is a little bit different similar syntax, but different So we're saying a function and A parameter so we're directly calling a function here See some sort of callable function with arguments. So really what we're doing is saying thread pull submit to this function a value And it's going to call this little guy right here. Beautiful how that works. I absolutely love that. It's so Simple to use. It's just ridiculous So this is to a function with an argument And then from there, it's very very simple. Let's go ahead and just run this just to see what happens And you can see we have no errors and it actually does kick up some threads in the thread pool Okay, if you've been paying attention so far, we have a multi-threaded application that's doing well basically nothing We're starting a thread pool. We're submitting to a function and then this is being called up But we have this work here. So we haven't actually done anything yet That's going to change. We're going to look at the global interpreter lock or the gill What is it and why does it exist? Think about what we're doing here. We're taking a global variable across multiple threads and we're going to modify it This is very very bad. So I'm going to say counter plus equals And let's go ahead and increment this by uh, What do I want to do here? Let's go ahead and just add one just to see it work Now if you've come from another programming language and you know threading, you know, this is very very bad Because what have we just done here? Well, we've created a race condition Because multiple threads are hitting this simultaneously You can see down by our hours minutes seconds milliseconds. This is firing off at the same millisecond so Race condition is notorious for crashing programs because what happens is The program doesn't know the value Let's say the program's in the process of modifying this and another thread comes in and tries to modify it What's the actual value? That's When the program gets confused and ultimately just crashes. Let's see it in action No crash notice that No crash at all application finished So what's going on here is well You may have guessed The global interpreter lock is coming in and saying nope only one thread at a time through the door and we can even test this By getting that actual count just to make sure this is an actual integer and nothing has exploded here So let's just clear this out And run again So we have a count of seven Now you can play around and this is Admittedly a bit watered down or simplistic or almost silly example because we're doing basically no work But I want to dumb it down so we understand the concept Of the race condition and what we've just done This is why the global interpreter lock exists. It makes threading in python ridiculously simple We don't even have to worry about locking this even though we should So we're going to cover locking in the next segment, but just understand what we've done here Multiple threads at the same time are hitting this resource and modifying it So far we've been trusting python to take care of all the murky details for us But I'm not a trusting individual so we're going to do this the right way We're going to use what's called locking And this is kind of intimidating But we now control that resource. So I'm going to say lock equals threading Dot lock and when you see the word lock in the context of threading Think of it literally like a lock on a door We are now shutting that door and locking it so no other thread can go in and mess with it But to do that we have to actually acquire the key Now because we've acquired the key and locked the door we have to unlock the door now before anything else can be done So let's do this. We're going to say try finally And we're going to literally just copy this code here And then we're going to say lock Release and release is well literally unlocking the door so other threads can now go in and do it Notice the structure here Only the resources we're going to modify are between lock and release or I should say acquire and release Because we want this window of opportunity to be very very small That's what's going to keep our applications nice and nice and fast Now Doing this is great and all but there's another little issue. Let's go ahead and run this just to show you that it does actually work Ta-da Now let's go ahead and we're going to Actually create a deadlock So let's say we accidentally did something like this So this is a deadlock what we're doing now is we're saying we are now waiting on Because this is locked the door is locked. We can't get in But now someone else is saying hey Try and open the door the door is locked. Oh That is super frustrating. So what's going to happen here Is you notice how it just stops So whenever you see a multi-threaded application just stop like this how it's just sitting here forever and ever and it's not Going to crash is just going to sit here You have just ran into a deadlock So if you're ever sitting around at work and somebody says hey, why is my thread stopping? It's a deadlock. That's really what's going on Now so far all of this has been great and all but it's also very ugly. So we're going to do something called locking Simplified and if you played around with this code a little bit you'll notice that well Python takes care a lot of the details for us. So we don't even have to worry about it So let's go ahead and comment all this out It's going to keep it here that way if anybody grabs the source code. They can see what I did So locking simplified We're going to stay away from this try finally and we want to lock and immediately unlock So i'm going to say lock threading dot lock And now I want to say with lock And we could have said with threading dot lock, but i'm just showing you that we have some sort of variable here So with lock and what's going to instantly happen is we're going to acquire that resource and then release the resource When it's no longer needed. This is incredibly cool So i'm going to Just grab this And we're going to pay some code in there. So i'm going to say so let's change that to plus one Time sleep two so each time we do this we're going to sleep. I'm slowing this way down So just to scroll up. I know we got a lot of comments here I'm going to inter test we're going to grab that counter We're going to grab the thread name do some logging and then 4x in range count We're going to go ahead and say get a lock lock and acquire Do something go to sleep and release. So i'm intentionally slowing this way down so you can see it in action Save and run and you can see it's going going going it almost looks like it's locking it because we're putting all the threads to sleep And eventually this will finish out Ta-da everything's working as expected I hope you enjoyed this video you can find the source code out on github.com If you need additional help myself and thousands of other developers are hanging out in the void realms facebook group This is a large group with lots of developers and we talk about everything technology related Not just the technology that you just watched and if you want official training I do develop courses out on udemy.com. This is official classroom style training If you go out there and the course you're looking for is just simply not there drop me a note I'm either working on it or I will actually develop it I will put a link down below for all three of those and as always Help me help you smash that like and subscribe button The more popular these videos become the more i'll create and publish out on youtube. Thank you for watching