 Welcome back Python Stars. We are pleased to welcome Mr. Andrew Knight, a very special guest from Automation Panda. In his own words, Andy Knight is the Automation Panda, an engineer, consultant and international speaker who loves all things about the software. And today he will be sharing with us his expert insights on how decorators function in Python. With that, I ask you that you give your full attention to Mr. Andrew Knight and help me in welcoming him to the stage. Over to you, Andy. Thank you. Good evening, good afternoon and good morning around the world. Can everybody hear me okay? Cool. So, I'm glad to be here streaming to you live from the United States. As he said, my name is Andy Knight. I also go by Pandy for short in the Python community. I am the Automation Panda and I'm a big Python fan just like y'all. So, have you ever seen those tags on top of Python functions? Maybe you've seen them on top of methods in classes too. Those are decorators, one of Python's niftiest language features. Decorators are essentially wrappers. They wrap additional code around existing definitions. When used right, they clean up your code better than OxyClean. Let's learn how to use them today. So, best way to learn is by seeing actual code. Here is a regular old hello world function. When we run it, it prints hello world. Nothing fancy here. Now, let's take that same function and bam, let's add a decorator. Using this at sign, we just added a decorator named tracer to this hello world function. So, what exactly is this decorator? Tracer is just another function. But it's special because it takes in a function as an argument. Since tracer decorates hello world, the hello world function is passed into tracer as an argument. Wow! So, what's inside tracer? This decorator has an inner function named wrapper. Can you even do that? With Python, yes, you can. The wrapper function prints entering, calls the function originally passed into the decorator, and then prints exiting. When tracer decorates hello world, that means hello world will be wrapped by entering and exiting print statements. Finally, the decorator returns the new wrapper function. Anytime the decorator function is called, it will effectively be replaced by this new wrapper function. So, when we call hello world, the trace statements are now printed too. Wow! That's amazing. That is how decorators work. Decorators wrap functions around functions. Think about them like candy bars. The decorator is like the foil wrapper, and the decorated function is like the chocolate inside. But how is this even possible? That decorator code looks confusing. Decorators are possible because, in Python, functions are objects. In fancy language, we say functions are first-order values. Since functions are just objects, we can pass them into other functions as arguments, define new functions inside existing functions, and return a function from a function. This is all part of a paradigm called functional programming. Python supports functional programming because functions can be treated like objects. And that is awesome. So, using functions as objects, decorators change how functions are called. Decorators create an outer decorator function around an inner decorated function. Remember, the outer function is like the foil wrapper, and the inner function is like the chocolate. Creating an outer function lets you add new code around your inner function. Some people call this type of code advice. You can add advice before or after the inner function. You could even skip the inner function altogether. The best part is, decorators can be applied to any function. They make sharing code easy so you don't repeat yourself. You can follow the DRY principle. Decorators are reminiscent of a paradigm called aspect-oriented programming, in which code can be cleverly inserted before and after points of execution. That's neat. I recommend you check that out after this talk. So remember, decorators wrap functions around functions, like candy bars. Hold on now. We have a problem in that Python code. If the wrapper function effectively replaces Hello World, then what identity does Hello World report? Unfortunately, with the given code, its name is wrapper, and its help also returns wrapper. That's not right. Never fear, there's an easy solution to this. The functools module provides a decorator named wraps. Put functools.wraps on the wrapper function and pass it in the inner function object. And decorated functions once again show the right identity. That's awesome. But wait, there's another problem. How do decorators work with inputs and outputs? What if we decorate a function with parameters and its return value? If we try to use the current tracer, we get an error. Arguments broke it. But we can fix it. First, add star-args and star-star-kw-args to the wrapper function's parameters, and then pass them through to the inner function. They will make sure all arguments go through to the decorator. Through the decorator into the decorated function. Then capture the inner function's return value and return it from the wrapper function. This makes sure return values also pass through. If the inner function has no return value, then don't worry. The decorator will pass through a none value. Thanks, Python. When we call the function with the updated tracer decorator, we see tracing is now successful again. When we check the new return value, it's exactly what we expected. Hello, Andy. It works. Wow, that's awesome. But wait, there's more. You can write a decorator to call a function twice. Start with the decorator template, and simply call the inner function twice. Return the final value for continuity. Bam, it works. But wait, there's more. You can write a timer decorator. Start with the template, call the inner function, and surround it with timestamps using the time module. Here, we'll be printing the elapsed time as the difference between start and end. Now you can time any function. But wait, there's more. You can also add more than one decorator to a function. This is called decorator nesting. Order matters. Decorators are executed in the order of closeness to the inner function. So in this case, with these two decorators, call twice is applied first, and then timer is applied. If these decorators are reversed in order, then each inner function call is timed. Cool. But wait, there's more. You can scrub and validate function arguments. Check out these two simple math functions. Add and subtract. Nothing too fancy here. Create a decorator named int inputs to scrub and validate inputs as integers. Add the wrapper function, and make sure it has positional arguments. That's the star args in the wrapper parameters. Then cast all args as ints before passing them into the inner function. Here, I've done that using a list comprehension. That's that. Square brackets in A for A in args. If you're not familiar with list comprehensions, think of it as a really cool way to do processing on lists or collections without using a loop. Definitely recommend looking that up afterwards. Now, when calling those math functions, all numbers are integers. Using non-numeric inputs also raises a value error. Cool. But wait, you guessed it, there's even more. You can create decorators with parameters. Here's a decorator that will repeat a function five times. The repeat function is a little different than the decorators we've seen before. Instead of taking the inner function object, it takes in the parameter, which in this case is the number of times to repeat the inner function. Inside, there's a repeat decorator function that has a parameter for the inner function. The repeat function returns the repeat decorator function. Inside, repeat decorator is the wrapper function. It uses functools.wraps and passes through all arguments. That's star args and star star kw args. Repeat decorator returns wrapper. Finally, wrapper contains the logic for calling the inner function multiple times, according to the repeat decorator's parameter value. That's what the for loop does here. Now, hello world runs five times. I could change that to be four times, eight times, 99 times. Nifty. But wait, there's even more. Decorators can be used to save states. Here's a decorator that will count the number of times a function is called. Count calls has the standard decorator structure. Outside the wrapper, a count attribute is initialized to zero. This attribute is added to the wrapper function object itself. Remember, in Python, we can dynamically add attributes to any object. Inside the wrapper, the count is incremented before calling the inner function. The count value will persist across multiple calls because it's associated with the wrapper function object itself. When we run this, initially the hello world count value is zero, because it hasn't been called yet. After making two calls, the count value goes up as expected, and now the count attribute returns two. Awesome. But wait, there's more. Decorators can be used in classes as well as for functions. Here, our timer decorator is applied to this hello method. As long as parameters and return values are set up correctly, decorators can be applied equally to functions and methods. Decorators can even be applied directly to classes. This might blow your mind a little bit. When a decorator is applied to a class, it wraps the constructor. Note that it does not wrap each method in the class. Since decorators can wrap classes and methods in addition to functions, it would technically be more correct to say that decorators wrap callables around callables, instead of saying the wrap functions around functions. Interesting. Python is cool like that. So that's all great. But can decorators be tested? Good code must arguably be testable code. Well, today's your lucky day, because yes, you can test decorators. Testing decorators can be a challenge. We should always try to test the code we write. But decorators can be tricky. Here's some advice. First, separate tests for decorator functions from decorated functions. For decorator functions, focus on intended outcomes. Try to focus on the wrapper instead of the inner function. Remember, decorators can be applied to any callable. So cover the parts that make decorators unique. Decorated functions should have their own separate unit tests. Second, apply decorators to fake functions used only for testing. These functions can be simple or they can be mocked. That way, unit tests won't have dependencies on existing functions that could change. Tests will also be simpler if they use slim down decorated functions. Third, make sure decorators have test coverage for every possible way it could be used. Cover decorator parameters, decorated function arguments, and return values. Make sure the name and help are correct. Check any side effects, like saved state. Try it on methods and classes as well as functions. With decorators, most failures happen due to overlooked edge cases. Let's look at a few short decorator tests. We'll use the count calls decorator we wrote earlier. There are two decorated functions to use for testing here. The first one is a no operation function named no op that does nothing. It has no parameters or returns. The second one is a function that takes in one argument and returns it. Both are very simple, but they are two equivalent classes of decoratable functions. The test cases will show, will verify outcomes of using the count calls decorator. For count calls, that means tests will focus on the count attribute added to the decorator functions because that's the side effect. That's the outcome. This first test case here verifies that the initial count value for any function is zero. The second test case calls a function three times and verifies the count is three. This third test case exercises the same function over to the left to make sure arguments and return values work correctly. It calls the quote unquote same function, asserts the return value and asserts the count value. This collection of tests is by no means complete. It simply shows how to start writing tests for decorators. It also shows that you don't need to overthink unit tests for decorators. Simple is better than complex. And to be honest, I wouldn't be the automation panda if I didn't cover at least a little bit of testing. So up to this point, we've covered how to write your own decorators. However, Python has several decorators available in the language and in various modules that you can use. Absolutely free. Decorators like class method, static method, and property can apply to methods in a class. Frameworks like Flask and PyTest have even more decorators. So let's take a closer look. Let's start by comparing the class method and static method decorators. We'll revisit the Greeter class we saw before. The class method decorator will turn any method into a class method instead of an instance method. That means hello, or that means this hello method here can be called directly from the class instead of from an object of the class. This decorator will pass a reference to the class, or it will pass a reference of the class into the method. So the method has some context of the class. Excuse me. Here, the reference is named CLS to represent the class. And the method uses it to get the class's name. The method can be called using greeter.hello, as shown on the right. Wow. The static method decorator works almost the same as the class method decorator, except that it does not pass a reference to the class into the method. Notice how its parameters are empty. Methods are still called from the class, but here with greeter.goodbye. You could say that static method is just a simpler version of class method. Next, let's take a look at the property decorator. To show how to use it, we'll create a class called accumulator to keep a count of a tally. Accumulator's init method initializes a count attribute to zero. An add method adds an amount to the count. So far, nothing unusual. Now, let's add a property. This count method has the property decorator on it. This means that count will be callable as an attribute instead of a method, meaning that it won't need parentheses. It is effectively a getter. The calls to count in the init and add methods will actually call this property instead of a raw variable. Inside the count property, the method returns an attribute named underscoreCount. The underscore means that this variable should be treated as private. However, this class hasn't set that variable yet. That variable is set in the setter method. Setters are optional for properties. Here, the setter validates that the value is set to a non-negative value. If the value is good, then it sets underscoreCount. However, if the value is negative, then it raises a value error. UnderscoreCount is handled internally, while count is handled publicly as the property. The getter and setter controls added by the property decorator let you control how the property is handled. In this case, the setter protects the property against bad values. So let's use this class. When an accumulator object is constructed, its initial count is zero. After adding an amount to the object, its count goes up. Its count can be directly set to non-negative values. Attempting to set the count directly to a negative value raises an exception as expected. Protection like that is great. Python packages also frequently contain decorators. For example, Flask is a very popular web micro framework that enables you to write web APIs with very little Python code. Here's an example, Hello World Flask app, taken directly from the Flask docs online. It imports the Flask module, creates the app, and defines a single endpoint at the root path that returns the string. Hello World, it says. Flask's app.root decorator can turn any function into a web API endpoint. That is really awesome. And it's one of the reasons why Flask is so popular and so loved. It's so easy to make a web service. Another popular Python package with decorators is PyTest, Python's most popular test framework. And it's probably also my personal favorite Python package. One of PyTest's best features is the ability to parameterize test functions to run for multiple input combinations. Test parameters empower data-driven testing for a wider test coverage. To show how this works, we'll use a simple test for basic arithmetic called test addition. It asserts that a plus b equals c. The values for a, b, and c must come from a list of tuples. For example, as shown here, one plus two equals three, zero plus one equals one, and so forth. The PyTestMark parameterized decorator connects the list of test values to the test function. It runs the test once for each tuple in the list, and it injects the tuple values into the test case as function arguments. The test case would run four times. Test parameters are a great way to rerun test logic without repeating test code. So act now before it's too late. When should you use decorators in your Python code? You may be wondering. My advice is to use decorators for aspects. An aspect is a special cross-cutting concern. There are things that happen in many parts of the code, and they frequently require repetitive calls. Think about something like logging. If you want to add logging statements to different parts of the code, then you need to write multiple logging calls in all those places. Logging itself is one concern, but it cross-cuts the whole code base. One solution for logging could be to use decorators, much like we saw earlier with the tracer decorator. Good use cases for decorators include things like logging, profiling, input validation, retries, and registries. These are all things that typically require lots of extra calls inserted in duplicative ways. Ask yourself this. Should the code wrap something else? If yes, then you have a great candidate for a decorator. However, decorators aren't good for all circumstances. You should avoid decorators for what we would consider main behaviors, because those should probably be put directly in the body of the decorated function. Avoid logic that's complicated or has heavy conditionals too, because simple is better than complex. You should also try to avoid completely sidestepping the decorated function. That could confuse people. Ask yourself this. Is the code you want to write the wrapper or the candy bar itself? Is the code you want to write the wrapper or the candy bar itself? Rappers make good decorators, but candy bars do not. I hope you found this infomercial about decorators useful. If you want to learn more, check out this real Python tutorial by Gar Arn Hell, named Primer on Python Decorators. It covers everything I showed here and a little bit more. So thank you so very much for listening to me. Again, my name is Andy Knight or Pandy. I'm the Automation Panda and a bona fide Pythonista. Be sure to read my blog at AutomationPanda.com and follow me on Twitter at AutomationPanda. I'll be taking questions offline, either in Zulu or you can also tweet at me at AutomationPanda. So thanks so much. Stay safe out there and enjoy the rest of PiconIndia 2020. All right. Awesome talk, Andy. On behalf of PiconIndia, I thank you for the talk for the questions posted over Zulip already. They have been escorted over to SpeakerLounge as well. For any late questions, please feel free to catch Andy at the SpeakerLounge over Zulip and we'll be back with the final keynote for the day in about five minutes at the Bangalore stage. Stay tuned.