 So I'm Ruben, and I have a fantastic job. I spend just about every day helping people become more fluent in Python. What could be better? I do this in a few different ways. I have a free weekly newsletter called Better Developers, read by about 12,000 people per week. I do corporate Python training. So every day I'm in a different city, country, company teaching people Python. It's good we're in the Shanghai room, because next month I'll be in Shanghai. I also do a family of courses called Weekly Python Exercise. You will be shocked to hear that involves a weekly Python exercise to help people improve the fluency in Python. I also have a bunch of online video courses. About a month or two ago, my new book for Manning was released. This is this called the Python Workout. It's 50 short exercises in Python to improve your fluency. This is the first time Manning has released a book with the author's picture on the cover. About two, three weeks ago, I started a new thing on YouTube. I'm doing the Python standard library video explainer series, where I'm walking slowly through the Python standard library, trying to explain it. And by the time I'm done, I will have great, great grandchildren. So this is a bit of an advanced talk. So if you don't understand some of it, that's OK. I promise within 20 years or so, you will. So let's decorate a function. What does it mean to decorate a function? Well, it's going to look sort of like this. At my deco, def add a and b, return a plus b. Not so exciting, certainly a bit of a boring function. But what's important is to understand what's going on here when we decorate this function. It's important to understand the whole process that's happening in Python behind the scenes. So what's going on? Well, first of all, our function is getting defined. There we go. And it's important to remember what happens in Python when you define a function. When you say def add of a and b, return a plus b, two things are happening. First of all, we're creating a new function object. Secondly, we are assigning that object to the variable, to the identifier add. Both things are happening. So when we decorate this function, what's actually happening, another step. After we define our function, this is happening. We are calling my deco with an argument of add. And the result of that is being assigned back to add. So this means that we've got a few different functions running around here. And functions or classes, for that matter, are known in the Python world as callables. Now, here we have add, our original function that we're defining. Here we have add the same name, but we're doing a little switcheroo. We're changing what add is referring to. It's no longer after that third line they're going to be referring to the original function. Rather, it's going to be referring to the result from calling my deco on add. We have three callables here. First callable, our original function. Second callable, my deco. Third callable, the mystery quiet. Hidden callable, and that's what we get back from calling our decorator. So if we're going to implement a decorator, we're going to need to keep track of these three callables. And we're going to have to write a callable that takes a callable as an argument and returns a callable as output. Well, that sounds exciting and not head-splitting at all. So how am I going to do this? Well, I'm going to write a function inside of a function. Now, functions in Python are objects, just like everything else. When I define a function, as I just said, when I define a function, I'm creating new object and I'm assigning it to a variable. When I define my deco here, what am I doing? I'm defining a function, assigning it to the variable, the global variable my deco. Why global? We're in the global scope there. But when I define wrapper, wrapper is being assigned to a local variable inside of my deco. And each time that I run my deco, I'm defining a new local variable wrapper inside of that particular function's local scope. Well, I want to be able to decorate lots of different functions with any sort of function signature. And so the arguments, well, they can be quite varied. So I'm going to write my function so that it takes splat args, double splat kw args, any positional argument, any keyword argument, any combination of them. Okay, so it's great that we can get these different inputs. But when I want to actually run the function, how am I going to do that? Well, I'm just going to use funk. Funk is the function that was originally decorated, what was passed to my deco. But wait a second, how does our inner function have access to this? L-E-G-B, local enclosing global built-ins. That's the search path that Python uses for variables. So inside of wrapper, it says, I want funk. Is funk a local variable? No, it is not. Is funk an enclosing variable? Is it a local variable in our closing function? The answer is yes, we grab it from there and we're going to execute it. But wait, before we can execute it, we need to get its arguments. What are its arguments going to be? We're going to take args and kw args and sort of unroll them. We're going to take that tuple args and turn it just into a bunch of positional arguments. And we're going to take kw args and turn it into a bunch of keyword arguments. And so we're able to, in this case, do nothing basically. That's okay, there are lots of people who do nothing also. In this case, we're going to have our function basically do nothing, take the arguments, call the function and then return whatever it gave to us. This is how decorators can be created. This is how they work. We have our decorated function, we have our decorator, and we have the returned callable as well, which is returned and that's what's replacing our original function. We have another perspective we can use on this. I can say that that outer function is executing once and only once when we decorate our function. This is our opportunity to sort of hijack the definition of our original function. But I also have the inner function, which is executed once per time we want to execute our inner function. So decorators are pretty cool, yeah. But you know, sometimes you see something and you're like, wow, that is super cool, but where would I use it? Is it sort of a solution looking for a problem? And no, actually decorators turn out to be useful because whenever you have code that might repeat itself inside of functions or inside of classes, you can extract it, you can refactor it, turn it into a decorator, and then have something that's sort of generalizable and usable all over the place. So what I want to show you in the rest of this talk is five different examples of where we can use them and get you thinking in sort of the mode of decorators. And I hope you'll understand not only how they work, but why you should use them and how you can use them. So first example, timing. This is like a classic example of where to use decorators. Well, you have classic, it's not like people have been doing decorators for 100 years, but let's call it that. So how long does it take a function to run? So I wanna know, let's assume I have a large code base and I know that I have a few functions that might be sometimes taking a little while to run. It would be nice to find out how long they're actually taking, but for me to actually go into each of those functions and modify it so it's gonna write out the timing, that's super painful. Plus my QA people might not be so happy with me monkeying with my functions like just to check timing. So what am I gonna do? I am going to write a decorator. My inner function wrapper will run the original function. That's not gonna change, but I'm gonna keep track of the time before and afterwards and then I'll write it to a log file. So how's this gonna look? Well, I'm gonna have log time, that's my decorator, the outer function here. And I'm gonna have then wrapper, the inner function. What's wrapper gonna do? Well, it's gonna check the time with time dot time. It's gonna then run the function, get its result and then it's gonna find out how much time it took to actually run things. And then we're gonna write to a file. By the way, if you have a function that's really like sort of time sensitive, writing to a file might just happen to slow it down a little bit, but let's ignore reality for a moment, all right? So basically we're gonna open the file with A, A for a pen because we don't want to obliterate our log file every time we write to it, that like, not so useful. And we're gonna write to it the name of the function, how long the function took to run and the timing for it. Fantastic. And let's apply it then. So I'm gonna have at log time on my slow add A and B, adds things, timesleep2, return A plus B and something similar for multiplication. And I'm putting in the timesleep because modern computers actually add things pretty quickly and it's not so useful to do this with them otherwise. And this is the sort of log file I'll get back. We have on the left column there, the timestamp of when I ran the function. In the middle column I have the name of the function. Notice each function is decorated separately but because we define the log file to be common, they're all gonna write to the same place. We don't have to, of course we can. And then the right side is how long each of them took to run, fantastic. And so if we now look at our function definition, our decorate definition, we have our decorated function coming in as func. We have the decorator here, log time, and we have the result of invoking it, or we have basically the result of invoking log time on our function, we get back a wrapper which is replacing our original function. Terrific. Wait a second though. Have you ever done this, help on a function? I hope so, right? And you can get it also into different IDs and so forth coming up as help. What if I now do help on slow add? It's super helpful, it tells me it's called wrapper and takes splat args, double splat kw args. Well we all want a little mystery in our lives, right? And here I gain the same thing for slow mo. Okay, this is a little too much mystery even for me. So like this is Hitchcock industry here. So basically what do we wanna do? We wanna somehow swap things out in it. Now we can do this, we can do this manually but why should we work hard? Can we solve this? Of course we can solve this how? With a decorator, come on. So that's right, if we want to we can go to func tools and import wraps. And then we run at wraps on our function where just above our definition of wrapper. And what that's gonna do then is it's gonna say but wait, let's do help on slow add. Aha, we have our doc string and our original function signature. And if I do that to mo, same thing. So it turns out that func tools wraps helps us with this sort of integration with other systems that allows our decorator function to pretend that it wasn't decorated after all. Okay, second example, let's do some rate limiting. I was just talking to someone a day or two ago, I can't remember who it was, was saying that they had used a decorator to do rate limiting on Django requests. This is exactly the same thing minus the Django part. You know, small piece of that I'm sure. So what we wanna do now is run a function but we wanna limit how often it could be run to once every 60 seconds. And if it's run more than once every 60 seconds we're gonna get an exception. How are we gonna do this? Well, I'm gonna have my once per minute decorator. It's gonna take func as an argument. So here is my decorated function as an argument. Here is my decorator and here's what we return. Fantastic, just one little thing. What in the world are we putting in the middle there? Well, I wanna make sure it's not run more than once every 60 seconds. But if I use a local variable inside of wrapper that's going to get zeroed out or erased each and every time. So we can't do it there. I could use a global variable but then I won't be able to look at myself in the mirror every morning when I get up. Therapy, you know, the whole thing. So how can I do it instead? I'm gonna stick it as a local variable in the external function, in the enclosing function once per minute. I can do that because we have non-local. And non-local basically says if I assign to a variable I'm not making a local variable. It's non-local, right? I'm not making global variable. I'm affecting this variable that's otherwise unreachable in the outer scope. So I'm gonna set here last invoked. Last invoked is gonna be a number. It's gonna be a timestamp. And that is gonna stick around across my various invocations. Then I say non-local last invoked. And that non-local statement means whenever I say last invoked equals I'm affecting that variable in the outer scope. And then I can just assign to it. Last invoked equals time dot time and then it'll check. And if we now run it, well we're gonna run our decorator once on the outside but many times on the inside. This is the division of labor that we have. And so when I actually run it, let's say I run on add of two and two and add of three and three. So the first time it's four, big surprise I know. Second time add of three and three and we get this error called too often error. Called, yeah. All right, so we see that this actually does work. But let's try to make this a little more generalizable. I wanna say once per end. I wanna make it so that I can decorate a function and say this one can run every 10 seconds but this one can run every 100 seconds. Well, how do we do this? Let's remember now how our decorators work. When I see at once per minute over a function, def add. What's really happening? We are defining the function and then we're doing that little switcheroo at the bottom. Add equals once per minute of add, fantastic. What do we do now though? I wanna say at once per end of five. So it's gonna have to look like this. I'm gonna have my function definition and then at the bottom it's gonna do this sort of switcheroo. I'm gonna invoke once per end on five. The result of that is a callable that's gonna be invoked on add. And the result of that is going to be return back to add and assign there. That's right. We have four callables here. Sorry, I don't have any headache medicine. So basically I've got decorated function up there. I've got the decorator here once per end which we're applying to five. We've got the result from that which we're invoking on add and then the result from that which is assigned back to add. Now if for three callables I needed two functions for four callables I'll need three functions. I have a function in a function in a function here. The outer function, and by the way it doesn't go deeper than that, right, okay. Just like to say you know, I'm not heading like to infinite functions. That might also be a longer talk. So basically if I say once per end that's now the outermost function and that's what's gonna get our argument. That outermost function is gonna then return a callable. What callable? Our middle function. That middle function now is gonna get our function, our original decorated function as an argument. And it's then gonna return wrapper which is then gonna be invoked each time. So what we have here is last invoked is still there but now it's in the middle function rather than the external one. We're still saying it's non-local because non-local works at all these different levels and we're still assigning to it. The difference is that the decorated function is now an argument here to middle. Our original decorator, the outermost thing is the decorator that we recognize. We are returning this when we invoke once per end and we're returning this from middle of fun. This is executed once when we get the argument. This is executed once when we decorate the function and this is executed lots of times each time we invoke the function we wanna check has it been run too often? Well, does it work? It sure hopes so because I've prepared these slides. Basically if I say here slow out of two and two, yeah we get four and we do slow out of three and three. There we go, you know it's working great. So we can actually do this and pull it off. Okay, example four, memoization. So memoization, it's not memorization. Memoization is a very old caching technique from more than 50 years ago and the idea is that if you have a function that forgiven arguments will always return the same value. So there's no state, there's no disk, there's no network, none of that stuff. Then what we can do is we can call it once, cache the solution and then the next time we call it check in the cache. Now doing this for adding and multiplying is kind of stupid, right? But doing it for harder to calculate things, really complex things even so like SHA-1 or MD5 that might be worth it. So how does that look? Well, I'm gonna define my outer function and my inner function. My outer function here is gonna cache things. How do we cache things in Python? We use a dictionary of course. All right, and it works perfectly for that. So what am I gonna do? My wrapper is gonna take once again splat args and double splat kw args and I'm gonna use args, my position arguments which are all in a tuple and a tuple is of course hashable so I can use it as a dictionary key. I'm gonna say, hey, have I ever seen this before? Is args not in the cache? Oh, we've never seen this before. So what I'll do is I'll run the function and stick its result into the cache and then I can just return what's in the cache because I know by the time I get to that line, yeah, I've got it, it works great. By the way, why don't I need non-local here? I'm using a variable in the external scope. Aha, but I'm not assigned to it. I'm not saying cache equals. I'm saying cache square brackets equals and that's a world of difference. I can do that because I'm retrieving cache as an LEGB but then I'm assigned to it using square brackets so totally, totally different thing even though it might not seem that obvious. So I have my decorated function out there. I have my decorator there. I have my return thing there. Terrific. I also have this part which executes once so I'm setting up my cache once only but I'm gonna be using and applying the cache and retrieving from it again and again and again. Does it work? Well, let's find out. I'm gonna do here add of three seven, the mall three seven, each of those twice. So when I say add of three seven, what does it say? Oh, look, I've never seen this before. I should really run that. Running add, we get 10. I run mall, same thing. Notice, different functions are gonna be decorated separately. They're each gonna have their own little private cache. And then we say add of three and seven again. It says, oh wait, I've seen this before. So I'll just give you back the result right away. No reason to waste your precious time adding integers. And here I'm gonna do the same thing for multiplying. I'm gonna pull it out of the cache. So this actually works really well. Wait a second though. What if args has a non-hatchable value? And what about kw args? Yeah, okay. So pickle, pickle to the rescue. What I can do is take args and kw args and pickle them. Pickle is Python serialization system. It works on just about everything. And I get back a string or nowadays a byte string. Byte strings are of course hashable. So what I can do is I can take args and kw args, pickle each of them into a byte string, and then use a tuple of byte strings as my key. And indeed, that's what I'm gonna do here. I'm gonna take args and kw args, run pickle.dump s on each of them, get back a byte string for each of them. I use that tuple and then I check, have I seen this before? Oh, I haven't seen this before, let's run the function. I have seen it before, just return it. Now, it's true that Python already comes with a version of this. It's called LRU cache. LRU is a super intuitive term for last recently used. And basically what it means is, let's keep the most recently used stuff around and it's actually smart enough to sort of get rid of the older stuff. But so far as I know, it doesn't handle unhatchable arguments. I don't think it looks at kw args at all. Okay, another example here, attributes. Wouldn't it be nice to give a whole bunch of different objects the same attributes? Or the same methods. There's a certain method that I want a whole lot of different objects to have. You could say, oh, that's for inheritance, right? Yes, but inheritance is fine when I have a whole hierarchy of objects of classes that are similar to one another. What if I wanna provide the same functionality of many objects that are not similar to one another, not related to one another? To which you might say, oh, we could use multiple inheritance. Well, you might not know this, but there is a deep division in the computer world on multiple inheritance. Some people say it's a terrible thing and the rest that people say it's the worst thing ever created on the face of the planet. No matter your perspective, and you can see I'm very pluralistic on this issue here, we might not wanna use multiple inheritance. We might wanna consider a slightly different thing. So what am I gonna do? I wanna have a bunch of attributes consistently set across classes, not use multiple inheritance. So what am I gonna do? Here's my example, I'm gonna improve wrapper. Rapper anyway, I mean, let's be serious. It's really not that useful as it is. So we are gonna improve it. We're gonna have fancy wrapper. Fancy, gotta be good, right? And it's gonna say what the object is, what type it is, and what its vars are. So I wanna apply this to across a whole bunch of different classes. How are we gonna do that? Well, I can say here, death better wrapper of C. C is now gonna be my class. I'm gonna say C.dunderrepper equals fancy wrapper. And you can do that. I'm gonna assign a method to a class in exactly this way, and it takes care of all the magic it needs. And then I'll define wrapper, because we need a wrapper there, right? And it takes args and kwargs, and we'll say C equals, we're gonna create our new object, and then we'll return the object, and then we return wrapper. And then we'll have the decorated class, and then we'll have our decorator, and we return the callable. Or we can just do it like this. Because basically what we wanna do is have, what we wanna do is get C, our class, as the argument. We wanna get our decorator, and we wanna return a callable, but a callable can be a class also. You don't have to return wrapper just to sort of be this pass-through kind of entity. Okay, does this work? Yeah, I can define foo here, where foo gets x and y, and we're gonna say, f equals foo of 10 and 10, 20, 30. And then if I print f, what's it gonna do? It's gonna print my fancy wrapper, and I can then apply this with my add better wrapper to whatever class I want. Okay, so we set a class attribute. Can we also change object attributes? Of course we can, it's Python, come on. So what do I wanna do here? I wanna give every object its own birthday. I mean, come on, they work so hard for us. They deserve a little bit of a celebration, right? So basically I'm gonna create this add object birthday decorator, and when I apply it to a class, every object would automatically get this underscore created at attribute of when it was created. How are we gonna do that? Well, once again I'm gonna need to create a function. The function's gonna get a class. The function is then going to need to use wrapper. Rapper is going to invoke C. It's gonna invoke the class with args and kw args to get back a new object. And then, sort of after init runs, but before we return that object, we're gonna stick that new attribute on there. O dot created at equals time dot time. And then we're gonna return the object. So now we have our decorator class and our decorator and the return wrapper as usual. And this works really well. So now when I print the object, print f, I get the old wrapper. We'll talk about that in a moment. And when I check what is created at, it works just fine. So I can indeed add things to the different instances. Let's do both. So I'm going to, at the outer function level, set this attribute done to wrapper on the class. But on the inner function, I'm gonna work on each and every instance. So here I add the method and here I add the instance and it works just great. So decorators let you dry up your callables, whether they are functions or classes. I mean, if you've ever been tempted to write, okay, we've all been tempted to do this. We say, oh, I'll never need to modify or maintain this code. What are the chances? I'll just copy and paste it. I will tell you what the chances are, 100%. You do this, you will be in a world of pain because somewhere down the line, you're gonna be like, oh my God, I can't believe. Like my past self was so stupid, it should have asked my future self. So basically what you can do is sort of cut that off and say, wait, I see that I have similar code in multiple callables. Let's extract that into a decorator and then call the decorator and then I can test that separately and test my function separately. Now, all of this depends on the fact that in Python, everything is an object. Functions are objects, classes are objects. Objects are objects, right? That's sort of a tautology there. And so basically, the fact that everything's an object means that we can mess with them in this way. We can look into them, we can change their attributes, we can pass them around. And this gives us tremendous, tremendous power. Okay, if you have any questions, I think we might have a little bit of time for them. And if not, okay, we've break in five minutes. I'm happy to take questions here or afterwards. If you want, I have a few weekly Python exercise t-shirts and stickers. So programmers need nor t-shirts always. Or so I tell my wife. And basically I'd be happy to hear from any of you. If you wanna download the code and the slides from the talk, go to practicaldecorators.com and you can get it from there or search the, scan this QR code and I'm very happy to be here and answer any questions. Thanks very much, everyone. Okay, thanks very much, Ruben. Are there any questions in the room? Yes, there is one. Can you please get up? Thank you for the presentation. Yes, it's on. It's on. Thank you for the presentation. Thank you for the presentation. I have a question. Often when I need to write a complicated decorator, I use not a function, but a class and a call method. What do you think about such approach? So for a long, long time actually when I would teach decorators in my courses, I would start off doing it with classes because I find it to actually be easier to understand in many ways. Nested functions tend to give people like, I don't know, seasickness. But basically in a class you can divide it up between in it, like dunder in it and dunder call and the separation is much clearer. The problem is that you can't easily use a class-based decorator to decorate certain methods, for example. I think it might have been cleaned up recently. I might be wrong on that, but I'm not sure about that. So I basically over the last two years or so have switched to just sort of starting off talking about functions. If it works for you, first of all, stick with it, right? Second of all, again, I think it's a clearer separation of powers there, but I think that it's like, I've seen more people are writing them as nested functions and you can't so far as I know then do the whole like take an argument thing. I could be wrong there, but I don't think it's at least not as easy. Okay. Can you try that? Hello, amazing talk. I have a question about where, if you have this type of decorator, where are you like putting the memory that you are using in between? Because if you call a decorator a hundred times, you are allocating memory, where does it lay and how can you free that? Oh. Wait, we have to worry about memory? We're in a high level language, it's all magic. Look, yes, it does take memory because what's happening is that the stack frame from the outer function is still around. Now, it's not gonna be like that big, but it does take something. Now, when does it get released? Basically, so long as your decorator exists, so long as your decorated function still exists, and let me put this way, so long as the result of decorating the function still exists, it's then pointing to that stack frame. So as soon as the function goes away, then that should be released because that's the only thing holding onto it, but that might never be. If you're defining a global function, then global stick around basically until the end of your Python session. That said, that said, I would say, I mean, you're right that you do have to think about it, but how many functions are you having decorated there, right, compared with all the other, I mean, if you're doing thousands of decorated objects, then maybe you have to think about it more, but I think I'm guessing here, and this is just a guess, the overhead of all the other objects in Python is so much greater relative to that that I wouldn't worry about it too much. Okay, we have time for one more question if there's one, and if not, let's use the time for another round of applause for Ruben. Thank you very much. Thanks very much.