 Thank you So taxonomy of decorators first. Let me introduce myself. I'm Andy funding her. I've been a Python developer since Python version 2.4 at the time all the books came out they said Python 2.2, and then they had a little mark said newly revised for Python 2.3 And there were a couple pages in the back that said once Python 2.4 comes out Here's the new stuff. You'll get to use so that's where I started with Python Low those many years ago You'll see some notes on older things in these notes Everything we're talking about today Works in Python 3 or such older versions as you might happen to still be running What is Bloomberg Bloomberg is? First and foremost a technology company. We sell financial data and Analysis we're also a media company We have major locations in New York London, San Francisco and all over the world Now what is this talk? This talk is about creating a taxonomy of decorators we use decorators, but Describing them gets a little bit fuzzy It's we don't really have established categories, so I thought I'd try my hand at throwing out some categories and saying maybe These are good categories to talk about decorators in I'm hoping with a common set of Terminology we can discuss these more easily sort of like with design patterns We can say that's sort of like a singleton pattern even if we're not implementing a singleton pattern We don't want to implement a singleton pattern. We're not using it, but it at least is a data point This talk is intended for intermediate developers who can write decorators, but maybe aren't sure when or why and Architects who may need to work across teams to implement and manage their design We will be going over the syntax of decorators But I'm not attempting to teach how to write decorators in this talk I'm going over it just so that if you need a quick refresher. You've had a quick refresher That said we are going to go over without the at sign function decorators class decorators decorators with arguments and decorators with it written as classes Relatively quickly if you've taken other talks on decorators, you'll know that could be a talk right there Remembering of course anything in a decorator you could just replace it with the code that's in the decorator and put that code in the function There's no special magic for decorators What they're generally doing is inserting a section of code of arbitrary complexity using a single line and and Generally causing both the debugger and our eyes when we read the code to skip over that line For better or for worse Which is a lot of what I'll list as the problem with any particular kind of decorator So basics and tax going back to our first decorators Static method and class method these came back before decorators in Python 2 2 we got static method and we wrote them like this When we wanted a static method we took a method we already had we ran the static method function on it We assigned it back to its own name And it worked in Python 2 4 we got some special syntax to do this and that worked too We could also write our own So we could write our own say something like this And it's just creates a nonsense function that does nothing and returns that and of course by the time We got to Python 2 6 we could decorate our classes We can go ahead and run our nullet decorator on a class If we do that we run it This will actually work if you run it now your class decor your class constructor returns true That's not what you want it to do, but it does work Syntactically Then we can look at decorators with arguments now the trick for a decorator with argument is that it's not doing anything special They're simply calling the function that you have written with the arguments that you have given That function is returning a decorator That decorator is then being used to decorate The function that follows it so here we have multi arg Which is a quote-unquote decorator with arguments it takes an argument malt It returns an actual decorator deco that takes a function which returns an actual wrapper Which is the wrapped function If we go ahead and see that in use at malt arg 3 We decorate print X Print X does nothing but print X If we print X of 1 well, then it kind of makes sense because it prints 3 which is 3 times 1 and all that math makes sense if We do print X of hello. Well 3 times hello is hello. Hello. Hello Because we can multiply strings Whether we want to or not we can And off we go The thing to notice here is that we actually have two closures going on one is For the argument malt and another one is the function has its own closure funk Okay, now for reasons that I've never been able to understand really We normally write our decorators as closures It may be a couple less lines of code and a couple more lines of Unclarity There's definitely no particular benefit to this You can definitely write your decorator as a class and if you're writing a more complex decorator I will suggest you do so They look like this In it takes that function and and really ought to do something with it and then you'll need a callable class Because you have to return a callable So that should do something with that function so In this case we store it away when it's called. We're just going to print whatever arguments were passed. It's a nice little tracing thing and This does what it says on the tin We decorated rand with trace it When we run it we print the arguments we run the result and life moves along Right along to our actual taxonomy So the first time in our taxonomy is our argument changing Decorators the argument changing decorators are one of the most common Types of decorators and probably the most common decorator that you will use when you go to write an example of what a decorator does I think half the examples I just used were argument changing decorators argument changing decorators add or remove or change an argument when the function is called They're messing with the arguments They're changing the type as the arguments go through or they're changing the return value because that's really functionally the same as an argument What's the problem here well the problem is if you actually just look at the function put together the arguments call the function it immediately fails or Subsequently fails or does not do exactly what the plain reading of the code says it should do So if you're trying to test this function You're going to have to do whatever it takes to set up the decorator In order to make it actually drive the function in the proper way An example of this actually comes right from pie test pie test tests always have no arguments except pie test mark parameterize We'll go ahead and create some arguments and fill those in for you So these arguments aren't there the decorator takes them out Of course writing these is pretty simple We often write these as examples, but here's one just so we see what we're talking about Here's a funk with name decorator that tells the function what you called it so In this case the first argument to the function becomes the name of the function So function self-aware gets its name and another argument when it's called And does what it says on the tin So moving on to be the binding decorators Binding decorators implement the descriptor protocol to change how functions behave The standard library has a few of these we saw static method earlier class method and property or two more The problem here is they create an alternative to instance methods and attributes and things that your teammates are familiar with They create new or they enable new language patterns that maybe are better for other languages where you got them from Maybe you should leave them there Or they might actually cause an error to occur sometime other than when it should have occurred You can use these for lazy execution Lazy execution will cause your errors to occur sometime other than when they would have otherwise occurred That's why it's lazy SQL alchemy has one of these in the hybrid properties Which if you access it in one context, it'll give you a value if you access it in another context It'll give you a query It's kind of cool, but it's definitely not doing exactly what you would expect Here's a particular implementation of it just to show playing around with it normally our normal methods in Python if You call it if you access it from a class you can get it and you get an unbound method Fine it's not necessarily useful, but you can get it and you can bind it later Maybe you don't want people to bind your methods later. Maybe that would ruin your day I'm not sure how but maybe you have a very complicated inheritance thing and you don't want people to pick up a method and rebind it somewhere else So what this does is this is an instance method. It cannot be accessed from the class If I decorate something with this it cannot be accessed from the class because it implements the descriptor protocol And if you access it from the class it raises a type error So simple method and normal method are exactly the same except Simple method is decorated with the instance method Decorator if you use them normally no one can tell if you pull the normal method and access it from the class you get The unbound method in Python 3. That's a function object Python 2 it's actually an unbound method object if you try to do the same thing with simple method You will get an error The tricks you can play here can be much deeper. I Was seeing one at an open space where they were actually using this to trigger events to a UI on access When you change the value it was giving a chance for an arbitrary collection of functions to Mutate the value that was being set before it was being set and Callback arbitrary numbers of things to do arbitrary things based on the fact that a value was being changed Because there was a decorator So that's binding see control flow decorators control flow decorators are In my experience kind of the other most common Decorator these are the ones that change whether a function will be called and how many times The problem Well, you don't know whether it's going to be called So if you're looking at a debugger or if you're trying to figure out When did my error happen or why will it happen? Well, now you don't know It might be Zero one or many executions of a function Our common example here is the retry decorator from retry We probably use this a lot and we probably just use it a lot more And hope that it makes things work We can write our own that's not so bad Infinite retry just says as long as we're getting a runtime error Go ahead and try it again and again and again and again It looks like that descriptive decorators are ones are kind of one of these ones that People may not even think about existing or existing as a thing This is one that that someone told me didn't exist until I vote one out for them What these do is they add the decorated object to some sort of collection That collection will serve some other purpose like documentation Dispatching or plugins The problem is once you've done that you don't know why you did it or where that collection is or How that's maintained or maybe how you'd get it out of there or any of that stuff We'll go back to pie test for another example because pie test does a lot of decorators Pie test mark does this one. Oh Look, I just marked it as a web test. What does that do? Look It's an app route What does that do? Where do I find that? Look here. I'll write my own. Here's a qa decorator It's gonna maintain a global list of all the things that I should really qa We actually do have quite a few of these most of your Frameworks if they're following any sort of the flask model will use something like this and register something It's not an uncommon pattern, but it is an under recognized pattern. Oh Yeah, there's my qa list now One that is uncommon and fortunately uncommon. It's the execution decorator These are the ones that read the code that's been decorated and They may totally reinterpret it to be something else Under problems, I'm not even gonna try to list these the code you wrote is not what will run It might be just analyzed for dependencies It might have the AST swapped out It might be recompiled under different rules that aren't really Python anymore Which is exactly what's happening here Scython has a mixed mode in the mixed mode Scython you can decorate things to be Scython or not Perfectly reasonable. How else would you mark it? But that's exactly what's happening. It's no longer Python inside this decorator Here's an implementation in case you ever want to write one This one goes ahead and gets the source off the function that you got that you decorated splits off the lines Throws away the zeroth line because that's at replacer. I know that Takes the first line because that's a function name takes everything else runs it through the search and replace execs the whole thing in global name space Takes out of the global's name space and returns that as a new function Not even the best way to do it. Don't worry So here it is without the decorator Here it is decorated for B Down the bottom that's 36 and It gets even more fun if you decorate it for replacing a with a times three Because now the value returned becomes 14, but it also found that other a in sample There are some more practical tools for this So you can do these things. I'm not gonna say safely but with controlled danger You can manipulate this at the bytecode or AST level They just don't fit quickly on slides is the main reason. I'm not showing those today And that's how some of these do it So Here's our taxonomy in conclusion We have the argument changing decorators The binding decorators the control flow decorators the descriptive decorators and the execution decorators. I Do believe there's more I think there's probably a hybrid or a mixed mode or a messed up decorator Not sure that that needs to be enumerated I have found some other decorators and listed them here Just to put them out of the way And I think we have time for questions Yeah, so let's thanks Have some time for questions What is your opinion on decorators? I Actually think that decorators are a Member of a category of Python features that I've started to call shovels They are Python features that help you dig out of holes But shouldn't be used prematurely Because if you've dug yourself into the hole using the shovel You'll be at the bottom of the hole and you might already have broken your shovel So they need to be used sparingly carefully they should not be used prematurely So they should either be used as part of your initial design or After it's become very evident that this is exactly the right thing to do Okay. Thank you. Here's another question. Hi. Thanks for a nice talk quick question How do you debug and test decorators when you write them? So I tend to actually use a full-up debugger from pie charm So if I believe that I have an issue in a decorator, I'll put the breakpoint in the decorator Also, I will often write unit tests for the decorator and need my decorators to be pretty highly reliable code so that You know issues in the decorator Appearing in other code is rare Same thing needs to be done with context managers again, because they Are sort of out of band code Beyond that it's not unreasonable to have them log a debug level statement that still indicates what's happened And if you're looking at a control flow decorator Then you need to be aware and this is one of the places where I hope that Either this taxonomy or this talk alone has helped you if you have a control flow decorator Realize when you're debugging your code may not even be running and there was one big one for that right here LRU cash in the standard library An American questions, thank you for the talk How can we build on this foundation to Think of better ways of writing decorators or things to avoid because it seems that There is indeed a lot of problems coming from each tips each types of decorators Is there some you would definitely say avoid them at all costs some that we could find more than unit tests some ways to make them Better or what do you suggest so my suggestion would be when you're writing a decorator Especially when you're well anytime you're writing a decorator for any degree of publication Document the decorator. I would love to have you documented in these terms, you know, this is a descriptive decorator it registers To here it this is what it does like if you look at descriptive decorator, I noted the problem with it is Primarily that you don't have a good idea where Where it's registering where it's putting its information So this is a descriptive decorator the net effect is that's maintaining a list in the module namespace of this module Okay, and Then you know anyone Can look at that and go oh this is a descriptive decorator like this other descriptive decorator that I'm passingly familiar with and I understand that You know, this is a binding decorator or this decorator implements the descriptor protocol Which you know the descriptor protocols right in the Python docs so we can use that terminology too You know this this decorator does exactly this That's that's what I think would really help is we Sometimes just put our decorator in and we're like it's a decorator It does it does this and we describe it in purely functional terms that don't establish or maintain the commonality with other things Yeah Let me see if I can even get it out as like an official Bloomberg white paper that would make it easier to find than buried on my own blog Yeah, I would be a tech at Bloomberg. I'll try to get it out I Mean I have I have my own blog and Twitter and Twitter's linked from the conference stuff, but And I guess I think we can get this up through the conference slides, but that's not very referable now is it? We see what the most referable thing is Okay, more questions Okay So decorators have this funny situation where you can pass some arguments to the decorator or you can just leave off those brackets all together This means when you implement a decorator which supports both You have to have a multifaceted approach because the first time you're called it may be with arguments Or it may be when you're given the function when the decorator is applied. Do you design the implement decorators that? Do both or do you agree? This is a water python that we have to as Decorator implementers cope with this No, I I probably would not bother in most cases Implementing the ability to do both In some library cases, you know the people that are worried about that I think they have legitimate concerns If I ran into that, I think I would probably look at I probably would adopt exactly the sorts of things They're doing without too much hesitation But I don't personally generally worry about that. I would simply say like you always need parentheses You know move along And there's a there's a decorator module that I think helps with this is a yeah, that looks That looks good, and I'd probably just use it and move along with my life But like like I said the the whole key to understanding Decorators with arguments is that it's not doing anything It's absolutely not a syntax. It is simply calling that function and doing using whatever it returns from that function and as as a decorator So I think if I actually went to make a kind of comprehensive solution for that. I'd probably do a class But that's some sort of class-based solution and maybe Pre-initialize it on module load as a singleton. I'm not sure quite how I'd go about it But if I had to resolve the problem, I'd start hunting down that path Okay, unfortunately, we don't have any more time for questions. So let's thank Andy once again