 Hello everybody. Thank you for being here. Let's talk about this unique feature of Python First let me start by saying that I'm not the only one who is very fond of the scriptors You will find like many Experts developers in Python who will also tell you how good the scriptors are There is some being that not only they're a powerful feature, but they're a fundamental part of the language itself So I understand the scriptors you will go more proficient in Python because it's an important tool to have in our tool set And you will understand the language better and as a result of that you will become More expert So let's start with the basics something that applies to all mainstream programming languages that have support for object orientation We have objects in the form of instances of some class and in there we hold data based on Attributes so when we can set that values to them and when we try to request The value we obtain what what we expect which is pretty much what we said previously But what if I told you that when you run something like this when you request the value of an attribute We can actually run arbitrary code controlled by us and that going on is going to be managed by another object of a completely different class Well, basically that's the idea behind the scriptors the scriptors are a powerful feature of Python by which we can control What happens at the very low level over the basic Corporations of the language and means that we can control what happens when we try to get set or delete attributes on given objects that had these Protocols of the scriptors so the scriptors are achieved by this protocol, which in Python is Intel by these magic methods as always in Python Special behavior is achieved my by magic methods in this case these four methods comply with the script or protocol basically under get set and delete and the last one was added in Python 3.6, which was an interesting enhancement to the language So the rest of the presentation will follow Python 3.6 as a reference because there has been some improvements So the talk is not only discovering the scriptors are perhaps also really scoring the scriptors So even if you are familiar with them, it's also interesting to see that there are new stuff in the current version of Python So let's go by parts first. Let's classify the scriptors There are two types basically what we have to know is that there are subtle subtle differences between them We can classify them in data descriptors or non data descriptors In terms of implementation the difference is where or not they implement and under set if they implement and they're set They're also called overriding the scriptors and they will take precedence always Instead of the dictionary of the object. Whereas if they're non data descriptors We can actually override them because the dictionary of the object will take precedence and will be run first So let's start with something simple with a non data descriptor For this the problem that I have is that I have an object that has some attributes and I want to automatically run some computations on those attributes and Kind of like a property, but it turns out there are many attributes So if I want to use a property, I will have to repeat the goal many times So I will use a descriptors and this means that I will have to create at least two classes one for the descriptor itself And the other one which is the so-called managed class Which is a class that I originally intended to have in the first place But now it's going to be managed by a descriptor object. So First let's inspect the descriptor. I'm going to create an object like this, which I'm going to go date formatter What it's going to do is it's going to format a daytime into a stream So this has a name. Well, we'll see that why the descriptors need a name shortly But important thing here is that they have it has a standard method because this is what defines this class to be a Scriptor because it implements one of the methods of the descriptor protocol at under great text takes two parameters The instance which is the object being accessed at that time and the owner which is a reference to the class And what I do in that method is I will try to get the attribute with Holding for that object and I will return a string representation of that daytime So now we can take a look at the managed class how this is used for the original class I intended to have and for this if my logic I will have this which represent a file stat which basically models a file like object in a unix like system and This object is defined by a file name. I'm I'm interested in tracking also the date it was created updated and removed that and these three Attributes are going to be instances of daytime, but I have some other attributes which are going to be called class attributes Which are basically the same but with the prefix str str created at and they're going to be instances of the descriptor And this is important. This is an important detail for the descriptor protocol to work They have to be defined at the class level so they have to be class attributes or the work otherwise the descriptor will not work So basically the logic is like I created a very simple naming convention by which str created at will be managing the attribute name created at So let's see now we have two things Individually, let's see this How they work all together So basically I created me my object my file stat with a file name and I'm going to pass a date for the creation and updation and Once again, these are instances of daytime from the start of every and now I request str created at for that object and I get the string representation of created at So in order to see how this happened is important to understand in which order and Python resolve things So let's go by parts first I created the object and I passed all the parameters today initializer and then I requested this attribute Which is str created at so the first thing it will happen is Python will look this into a dictionary of the object and it's going to try To fetch a key with the name str created at and as it happens It's not there because the object has created that not str created at so that search fails, but okay. It's not a problem Then it's going to continue and the next search is going to be in the dictionary of the class And this time the key is there because remember that in the managed class I define is created at to be an instance of the dictionary So the class has str created at and it's a mapping a key is mapping a key with a value So you think you might think okay It's going to return the value for that object as it happens with all properties Well actually no because this is a descriptor so Python is smart enough to tell that this object has a standard method So what it's going to happen is that instead is going to call the underget method of that object And that's the result I am obtaining So in the end what happened is that when I created this like str created at Python will call the class Dot the descriptor which in this case is str created at Dot underget And it will pass the object as a first parameter and the class as a second parameter and inside that method again We are in full control of what we are doing so we can Run any logic that we want in this case the logic for formatting the the daytime according to our requirements There's another case when you call the underget when you're trying to delete something there is a descriptor But you call it from the class not from an object in that case Instance which is the first parameter of underget is going to be none and the most common pattern or idiom to Have in the situation is to just return self which is a descriptor So basically the idea is that if you have a descriptor and you're accessing through an object you return the computation that Was defined in underget whereas if you try it to get it from the class you get the entire descriptor object Pretty much like a property like when you define a property in an object if you call it from an instance You get the result of the computation being decorated by the property But if you call the property from the class you get a property object. This is kind of similar so Let's see how the descriptor got its name because so far we assume the descriptor was able to retrieve the attribute and just Transform it in the way we wanted to but actually I had to explicitly pass the name to the descriptor And was they in the need I made the descriptor to able to store the name internally So basically for example str updated at I had to pass a string updated at to tell the descriptor You're going to be managing the property of the attribute under the name updated at But this is somehow repetitive and I don't want to have this logic I want something like in the last example She has created a descriptor and to make it able to automatically configure itself So if you wanted to achieve this in pre-version of Python had to resort to some complicated stuff For example one way of doing so was like having a class decorator and the class decorator you weren't defining the mapping of each name to the descriptor and Inside a decorator you were setting the name when you created the descriptor or another implementations I have seen are with meta classes that again when created the object You try to set at creation time by overriding than they're new But those things in my opinion are way too complicated for something that should be like plain simple So luckily there were some enhancements in Python 3.6 and this method was added to the class creations Which is set name and it receives the class when the owner which is a class as a first parameter And the name which is the name as the attribute appears defined in the class as a second parameter And this is going to be called automatically once the class is being created So taking advantage of this we can just use that method to automatically store itself that name and make the descriptor know the name of the attribute that is going to be managing which Makes things more simpler at least from the user perspective because the user doesn't have to know or remember to configure the descriptor So okay now we have the basic the data descriptor and we know how the descriptor gets their name of the property that are going to be Modifying let's move to something is slightly More complex with a data descriptor this time we're going to implement than they're set And the problem to illustrate this is just an example by which I have an object can be any object That has some attributes And what I want is to automatically count how many times those attributes change their value over time And if I try to think strategies to achieve this again Python as always offer offers multiple solutions So I couldn't do that with a property for maybe with Setter and in the set there I can have an internal counter that gets updated all the time But if I have many properties that will be repetitive I could also do that with set attribute but over writing this magic method or we can use descriptors Which I think is much more simple and convenient as we'll see in the next slide so For a script or first I need the name and we know we now know how the descriptor gets his name by the under set method The under set name So in this case is going to store the name as always and in the second line is going to store count name Which what it means is it counts how many times a property under the name Change their value over time Just for simplicity and for the sake of example I'm going to assume that in this case everything that it starts with count underscore corresponds to something That is the count of another property and not a property itself So now we can move to the under set method and see the interesting part So basically under set receives the object being modified at that time and the value that we're trying to set to that descriptor in that time So basically what I'm going to do here is I'm going to try to request The property from the object by inspecting the dictionary with the name that we know that the descriptor has If it's not there I'm going to set the counter to zero because it means that it never changes value is the first time we're assigning something if it is there In the near the end We're seeing that if if the value is different than the one it has it means there is a change So I'm going to increment the counter by one and in any case no matter what I want to actually Set the the new value that I'm calling that user is currently assigning So in order to exercise this this imagine that I have this object Which is a traveler but again it can be any object. I'm interested in seeing how many different cities and country that person has Visited so I'm going to find city and country to be both instances of class attributes Instances of the descriptor I just created so it's a trace property and basically what happens here is that first I set something to the object and we see that both counters are set to zero Then when the tourists moves from one city to another we see that the counters are Incremented and so on and so forth you can get the rest of the examples But the important thing is that if you look at this this is from the user perspective This is completely transparent. You the user is only Modifying values or attributes in the object and what's happening is that behind the scenes that the script or which is So is taking place and updating the internal representation of the object So this is completely transparent to the user and enable us full control over what's happening when we try to set something to the value So again similar to in the first example This is also in texture because when we try to set something to a descriptor that has than they're set defined What's going to happen is that it's going to call the class dot the descriptor dot set and it's going to pass The object as a first parameter and the value we're trying to assign as a second parameter And again in that method we're in full control of the logic We can use this to create validators default values inputs and utilization, etc At this point, maybe it's becoming clear or you might already guess what under the lead it might be doing Basically is the method. Yes, as you might have guess what it's going to be called when you try to delete something That is a descriptor and has this method defined so For example imagine that I want to protect some attributes against deletion where if delivered or accidental one way of doing so will be to create a Script or that implements the under the lead and raise an exception saying that you can't delete this Attribute for this object. So for example in my application if I want to protect a username Against deletion one possible way of doing so would be to actually create it as a class attribute. I used this descriptor And now we see that if I create this user and I try to delete it You will raise an exception selling me exactly what I want whereas if I try to delete anything else It will let me because there are just regular attributes nothing particular about it So that's the difference between something that has a descriptor and something that it doesn't have a descriptor So okay, now we know the basics about descriptors. We have seen the basic methods and Roughly how to implement it and basically basically hopefully with some examples You can actually take that advantage But how do we know that we're making sense out of the script or like we're actually using it properly Basically a descriptor is any is like any other good Python object So probably a good descriptor is one that makes sense with Python itself that is consistent with the language So it's Pythonic as we call it So in order to say that probably a good idea is to take a look at how the scriptors are used internally by CPython Because as it happens they are deployed in the language infrastructure There are a fundamental part of the language itself as I announced at the beginning of the talk For example properties are descriptors if you recall the first example I gave I mentioned that there were some suspicious similarity between the scriptors and properties And I said that when you get the property from a class you get the entire property object And when you get the descriptor from the class is expected to have the entire descriptor object That is because in the end properties are a particular case of descriptors They actually implement the full descriptor protocol and what happens is that when you decorate something with the property Python is using that decorator to store the method internally and place it under Set get or delete respectively. Also class methods are descriptors that what they do is they allow you to Have the class as a first parameter already provide That's why you can call the class method and you will have to explicitly pass the class because there's a descriptor going on Applying the decorator that automatically does that and the same happens with static methods that make sure that you can call that function Both from an object or from a class or from an instance and it's going to work I'm probably one of the most interesting Applications of descriptors are methods because this is the way methods work in Python because in the end methods are just functions so let's take a look at that and Also functions are descriptors. They have a get method What they do in that in get method is they return the function applied to an object Which is basically saying that the function is converted to a method when it's applied to an object so Basically, let's see this by part first. Let's try to prove that methods are just functions So I define a class like this and I create what we call a method And if I inspect the dictionary of the class we see that internally method is a class attribute, which is an instance of a function, right and Remember from the first example, what happens when you have something defined at the class level that has a get method Python Python doesn't return that object just yet instead. It calls the get method and returns that result So now With this information in mind we can know that when you when you call a method Imagine that I create an object and I call the method We're not actually calling that directly because that's a descriptor. So what's happening is that it's calling the class The descriptor then get method passing the instance Which is the object and the class and that parallel up to the first cluster of parentheses that will return a method Which is basically the construction of that function for the object and then it's a callable that accesses We can access with the rest of the parameters So in the end it happens that the script are having there all along there are fundamental part of Python and maybe they are invisible that we were using them probably without even noticing and This is an elegant solution by which Python is able to actually transform the same object Which is a function to a method in a class and they work the same way in a completely and the user doesn't even know about that So okay now we have seen How Python uses descriptors to their advantage, but how come we use descriptors to our case to our advantage? I'm probably the best thing that the scriptors have is not only that they're so powerful and they are They allow us to have total control and do a lot of things but probably that they can fix or Solutions in in the code for example, we can write better decorators by using descriptors In particular for those decorators that change the signature of the function So for example imagine that now I have a decorator that I was going to transform functions And it's going to change their signature in order to make it more convenient for the user and avoid Repeated code so for example, I have this Resolver that takes some arguments and with those arguments I construct an object of my domain and the rest of logic only uses my domain This is so I made up example But something like this happens in many web frameworks when you actually define your views to receive a request But you apply a decorator to it so basically what happened is that the decorator is changing the signature of the function and it provides you with the With a descriptor etc already But that changes the original signature because I have a decorator like this one That basically exposes the same all four parameters as before It creates the object that I need to and then it calls the wrap function with that object So now I can change my function to actually only receive the the object that I need to work with and nothing else But we have to keep in mind that the original function the one that is Ended being created after the decorator is applied. It still takes the original for arguments But this example works and if I have many functions like this I can reuse this this logic and apply that this will work and I think that this is fine, right? but I keep Looking at the code and I find that now I have the same situation But this time in a method of a class and I try to apply the decorator to that method and hoping that it will work But actually the way is defined so far It will not work because remember that methods take one extra parameter, which is self Which is the instance of the object being created So basically what's happening here is that there's a parameter mismatch and everything's like shift one place to the left So if you want to try to fix this decorator might we tend to find or resort to like complicated solutions like inspecting the code or inspecting the trace or the frames, etc But actually after seeing how Python makes functions be able to work as methods We can get inspiration from that idea and use the same technique or the same trick to our advantage So basically with our idea in mind, the only thing we have to do is just implement under yet And make that the decorator also a Scriptor and what I do here is I'm only I'm going to delegate the problem to Python because get method is going to call The get method of the function being wrapped which automatically does that and converts that to a method And then I'm going to re-trigger the scriptor But this time with the method instead of the function and now with this simple change with these two lines of code I fix the problem or something that would have been otherwise Quite difficult. So this is another cool application of the scriptors And how they are used So Basically to try to to wrap up We have seen first the methods of the scriptors how they work Basically each method what it does We have seen that there is like a composition that the one object goes to our object Automatically and we can apply all logic then we have seen How Python uses the scriptors? internally In particular to use to call functions as methods and then based on that idea How we can use the scriptors in a similar way as Python does for their methods. So With these I like to just share some final vices Which are basically that in general is a good idea to implement as few methods as possible So from the entire descriptor protocol we have seen has a four Maching methods partial implementation is not only possible is desirable because the object will be Much more generic and it will run faster because it has to do less checks in in runtime, etc and another important point is that we should like reserve these for general purpose solutions like to using the scriptors when you actually know That you're creating a solution that is abstract enough because this is a feature Mostly using frameworks on libraries. For example, SQL alchemy uses the scriptors um For example the models as the scriptors and all the logic is there. So this is something more targeted to towards libraries or frameworks or Yeah, something where you will have an API So if you can get away with just a property or something that is it will work once I encourage you to do so and reserve the scriptors when you actually need to to have something That you know many people will apply in different ways. So that's all. Thank you very much for listening And I would like to to hear your question So do we have any questions? Okay, that was everything was clear. Thank you again. Oh, I'm sorry Thank you. That was a great talk and thank you Can you recommend or suggest any patterns where you would actually use especially like the 3.6 feature the set name The set name want to use set name Any like patterns like what would be a typical example when you want to use it? No, I think in general Like always because the scriptor always knows Always has to know the name of the property that is going to be modified So set name is like I would say like in python 3.6 should always be there It's not like you will have it once or not So if available I would recommend to always use it because it's more convenient at your solutions So so my answer if I understood your question correctly will be like always implement another set name Thank you Any more questions? Yeah Hi, thanks for the talk. Um, you had a slide and you've been talking I guess about c python this whole time You said this is how descriptors work in c python. Does that imply that they work differently in other python versions? uh, actually I don't I would expect they work the same way, but for the talk I use C python as reference so I I took a look at the code as as it's implemented in c python and the documentation So the reason why I mentioned c python specifically is because it's the reference I use I don't know if it or I would guess the yeah because our implementations like pi pi would do the same So I guess would be that the same Thank you Any more questions? Okay, thank you again