 OK, there we go. Hello. So we're going to talk about eight things that happen at the dot. This talk came out of a couple things. One was a discussion I had with Guido, where I said, hey, what in Python does still kind of need to you? That you wrote. And he said, well, descriptors are kind of neat. And the other one is this is something I've occasionally asked on interviews and that people are not real familiar with what happens here. Usually, if it's going very well, we'll see how it goes. So first, who am I? I'm a new father. I have a daughter who's about a year and a half now. I'm bringing her to London for the first time next week. So if anyone has suggestions, let me know after. I've been a Python developer since Python version 2.4. When I first started, the book said, based on Python 2.2, newly updated for Python 2.3. That said, this talk is all based on Python 3 and 3. whatever. And it's all stuff that hasn't actually changed pretty much the whole time. This is all stuff that has not changed since Python 2.2 with one exception. I've previously approached the topic of descriptors for type control in a talk at PyCaribbean in 2016. And I've previously given essentially this talk at PyGotham in 2017. I'm at Bloomberg now. Before I came to Bloomberg, I worked in industrial automation. I worked in metaverse development. I worked in some investment banks. Currently, I'm at Bloomberg in the data license group, which is the group that packs up bunches of Bloomberg data and sends it to clients. So for clients that want, say, all the data about the equities in North America. Our group of about seven people uses a wide array of technologies. This is fairly typical of Bloomberg. We use Flask, Celery, PyCSVW, Jenna, Riot. I think we got rid of the PHP, but it's probably still hiding somewhere. Perl, JavaScript, C++, and they keep telling me there's Fortran hidden away somewhere that our group's responsible for. If we ever find it, we'll probably replace it with Python. That's typical of Bloomberg as a whole. We are enterprise scale, but we are extremely agile. And there's some reasons for that. Marketing gave me this slide, but it actually is kind of interesting. So 5,000 developers, 150 of them, for example, in machine learning, one of the largest private networks in the world. Our scale is kind of big. There's about 120 billion pieces of data going through our system, peaking at about 10 million per second, and they aren't all simple. Two million of them, as much as 500 per second, are news stories that are being digested, analyzed, and distributed from 125,000 sources. A billion of them are instant messaging chats, which is really kind of big. But moving on to our friend, the humble dot. The dot usually kind of sits there between our object and our attribute. And we go, oh, that's syntactic sugar for get adder or attribute name. Usually that's just going to go to dunderdict and index it and return what it finds, unless of course you went to Gordon's talk earlier when you know it doesn't quite do that. But we'll go a little deeper into some of the things it does and how. So we pretend it does this. We pretend it goes to the instant stick, but it could also go to the class stick. So we know it'll go ahead. It didn't find it in the instant stick. It'll go find it in the class stick. Here I put a value of 4 on the class attribute for y, and I can find that from any instance. Of course, it might not do that. It might go to the parent class. So here we have a base class for my object. I put t on the base class. I go ahead and get that from any instance, and it works just as well. But this is Python. And in Python, we have some more powerful tools. This is actually even older than Python 2.2. This goes back to the original object model in Python 2.1 with get adder. You call it. It doesn't find it in the instant stick. It goes to see if there's a get adder. It calls the get adder, says I was looking for this. Can you give me one? And whatever get adder returns, it returns. Now with get adder, if a value is set, it will no longer call get adder. So we can see that here. Once we put our into the instant stick, we no longer see get adder being called. On the other side, we have set adder. If we're on the left side, that is actually pretty simple behavior. If you call it, it does what it says. Now you will notice it doesn't update the dict, so you can disable anything being set onto your object by simply implementing a set adder that doesn't update dunder dict, which is what we used to have to do until they gave us get attribute. Get attribute is always called. That's great. When get attribute is called, you will be called whether they put something in the instant stick or not. So here it is. We access it when the instance attribute is not there. And we just get attribute is called. It prints it out. And off it goes. It returns any var. We put it on there with a p.s equals hello. And we call it. And it still gets called. But actually, get attribute is doing a lot more than that. Get attribute from object, which is where that code actually lives, is actually looking at what it found in the class stick. So we create this probe nd object. And we'll go ahead and make that a class attribute on my object. Now, it's z. So of course, we expect we're going to call o dot z. It's going to go up to the class. It's going to find z. It's going to return z. But it actually doesn't. What it actually does is it calls dunder get on the z that it found once it found that there is a dunder get there. Similar to get adder, if we put something into the class stick and here I've gone to the instance stick, and here I've gone ahead and done it directly just for clarity, that will be accessed and get attribute will not do the whole access on the class stick or will not run it. However, if you do as little as add a dunder set to that object in the class stick, the behavior of object get attribute changes. So here, probe dd inherits from probe nd and adds a dunder set. Now, if we take a look here, I'm putting a value for z into the instance stick. And then I'm calling o dot z. Now, we would expect from the previous behavior that it would not actually call dunder get because why would it? Why would behavior change? But the actual behavior is if you have a dunder get and a dunder set, dunder get will be called even if that attribute is in the instance dictionary. And that's what we're seeing here. It's there, but we still call dunder get. For completeness sake, yes, dunder set is called. It is called about how we'd expect. If we set a value, it gets passed into that function, and that function does whatever it darn well pleases. It does not magically put it into dunder dict or anything. I don't do that, so it doesn't get done. OK, so let's take a look at the eight things. We've got the first three things. We'll call them the simple lookups. We look in the instance stick, the classic, the base classics. Go ahead and see if we find anything wonderful. Then we've got the next three things, four, five, and six. We'll call those the old hooks or the simple hooks. Get adder, set adder, and get attribute. Get adder will be called if it doesn't find it in the instance stick. Set adder is just always called if you're setting a value, and get attribute is just always called. And then we have seven and eight, dunder get and dunder set, where we know the behavior of dunder get is changed based on whether dunder set is there or not. This is what's called the descriptor protocol. And the descriptor protocol is new in Python 2.2. OK, I don't know if we can still call it new, but they are officially called new style classes throughout the run of Python 2. In Python 3, we call them classes. The descriptor protocol in totality is these four functions, dunder get, dunder set, and dunder delete. Dunder get is called whenever you try to access that attribute. Dunder set is called when you set a value. Dunder delete is called when you delete a attribute. Does anyone actually delete attributes here today? OK, if anyone's not willing to raise their hands but still would tell me about it, I do want to know. Of course, we saw the subtlety with dunder get being called based on the presence of dunder set or not. And if you call a attribute at the class level, you'll get an object of none instead of an instance. There is one addition in Python 3.6, set name. This is very cool because now a descriptor can find out the name of the attribute that it was assigned to, which is terribly handy and solves many cases, none of which I'm showing today because I promised I'd stick to Python 3 simple stuff and not too cutting edge. Two names here. If you just have dunder get, it's called a non-data descriptor. If you have dunder get and dunder set, that's called a data descriptor. You can flip back and realize I called it probe ND and probe DD for a reason. Technically, you can write a data descriptor with just a get and a delete, but none of you use delete. And as I said, if you do, please tell me. So what is this used for? If you take nothing else away from this talk, what I'd like you to know is that this is not an obscure set of hooks. This is what makes method binding work. Every time you bind a method, every time you access a method from an object, the descriptor protocol steps in and does the binding. So here is a totally trivial class with a totally trivial method. And if we go and look at the class of the function, if we look at the class of that method, it is function. If we look at function, it has a dunder get implemented. If we access that attribute, functions dunder get runs returns a different object of a different class. Now the class is method that is what your bound method is. This happens every time you access a function from an instance. And this is how it works. So what else? Well, every variant method binding is using this. Static methods, class methods, and properties are all just let's bind functions differently, exactly where we would otherwise. So they just create a different class with a different dunder get that does what it needs to do. Frameworks love to use this. SQL Alchemy does it. It's hidden in the Flask config module. Any framework that you're sitting there going, I'm not quite sure how I would do that, or I'm not quite sure how I would do that without descriptors or metaclasses, is probably using descriptors. Maybe metaclass is two different talk. It's also useful for lazy execution, proxying, monitoring, runtime type checking is great to do with this. And of course, many other kinds of behavior things. Let's look at another couple of examples if we still have time, and I see that we do. This is one I wrote it up for this talk, and then a couple of days later at work I realized I needed it and put it into some code. So this is an alias descriptor. This is, hey, I have two names for one thing because I'm transitioning some code between the names. So I want to call them both the old name and the new name. So we construct it. We just give it the name. We just give it in its initializer the other name. And then we have a dunder get and a dunder set. If you do the get, we're going to go get the other attribute and return that. If you do the dunder set, we're going to go to the other attribute and set that. And that's all we have to do. There's one subtlety here, my choice. If you call it at the class level, I'm going to give the alias descriptor itself in case you're, I don't know, switching an alias around. That's a very common choice in descriptors. So usage code here, we have legacy x, y, and z. Store data in it, whatever. But then we're going to start calling them data x, data y, and data z at some point in the future. So we alias those up. And the behavior here, we construct one. We call it high pylendinium 2018. We can see data x, legacy x, both got high. You can't really tell from anywhere outside the class. We can see the value of legacy y, which is the actual value is pylendinium. And dc, data z, and dc, legacy z are the same thing. If we change the value of data x, it also changes legacy x. And it's an alias. It's working. Take another one, defaulted method. Hey, guys, let's implement class methods differently. Instead of class methods working on class objects, how about if we just create an object, if you call it at the class method, at the class level. We'll just run the initializer and make one for you since you didn't give us one. That's probably a horrible idea, but I implemented it. Just to give us an example of a non-data descriptor that's playing with binding. So in this case, for the initializer, I just need the function because this is both a decorator and a descriptor. We grab the function, we store it away. Fine. When Dundergit gets called on us because we're sitting on the class, if we were not given an object, then we're going to go ahead and create an object. And just to give you a different example that is totally equivalent, I bound this up with a partial. Do not use that in a real code. That's just because I thought some people in the audience might be more comfortable seeing a partial do the binding. The proper alternative would be to call Dundergit on the function. And then if I was given an object, I just called Dundergit from my own function and returned that. So we implement it. We have name printer. It stores the name that it was given. It has two functions, name print and print name. They do the same thing. They print the name. If I construct one and I give it my name and I call both functions, they both do the same thing. Code works as advertised. However, if I try to call them both without giving them anything, then the first one will work because it will hit defaulted method. It will go to get. It'll construct one and then it'll call print name and it'll have an object that's fine. The second one will not work because that's a normal method being called at the class level when it's an instance method. What do we have? Oh, perfect. Wonderful. Because this is the point where I really like to have questions. Do we have questions? You got questions? Who has a question? So really, eventually, what get attributes was needed for? This is the big question that was left from the previous talk. So why there was the need to get attributes? So actually, in order to implement descriptors. So the bindings in the old style classes did a number of special, this is only a function thing. This is only a class thing. This is only an instance thing in order to bind up functions. When they said, hey, let's clean this up and make this a normal, regular thing that can be done in other ways, like class method, static methods, et cetera, then they had to kick it up to something that could do that kind of change. They implemented object to get attribute, and then the rest flows more or less out of there, as well as I understand it. Thank you very much. It's very insightful talk. So when you mentioned that when we call a method within an instance or a class, it always says, one of your slides, you said, when we call a method or a function, it always binds it, or there's a slide mentioned that the class always does that before we actually execute the method. So can you find a little about what it actually does? Sure, sure. Let me see if I can flip back to that slide quickly enough to make it matter. So that should be this one. So method binding, this slide. So method binding, in all cases, it binds the method through the descriptor protocol in Dunderget, and then it evaluates what it got. So I've broken it up across two separate cells here. So I bind it here, and I execute it here. But even if I move the parentheses up here, it still does this operation, gets this object, and then executes on it every time. Does that clear it up? OK. Hi, thanks again for the talk. Could I just quickly label that point of the getatter and getattribute? Am I right in thinking that you're saying that the getattribute allows you then to include descriptors in your calling? Or is that something else? Getattribute is what actually calls and implements the whole descriptor protocol. So getattribute actually does it? Does the actual descriptor call? You can check the source code for subtleties on that, but at that level of abstraction, can we get a mic over here? I recently had a use case for accessing my snake case attributes using camel case. Can I abuse the descriptor protocol to do that? Well, you could use my alias descriptor if you wanted to alias them individually. That you could absolutely do. If you wanted to, say, make something that generically anytime there's a snake case that you can't find or anytime there's a camel case you can't find, then do the search. That you would do in getattribute override. We have another question over here. Yeah? You said that getattribute looks in class.dict. Does it actually use class.dict? Or is it calling getattribute on the class? It's calling. That's a good question. Getattribute, because if it's calling getattribute on the class, then you could do tricks into the meta class on it. But a lot of these accessors do bypass that. I'm honestly not sure. I have not tried mucking around with that level of the search because did you have a use case for that? Oh, I was hoping you did. OK, I may have to play with that. That sounds like fun. All right, well, that's all the time we have for questions. I think Andy will be here for the rest of the day if you want to catch up with him, or maybe Andy and Gordon together. And we have lunch right now on seven. And I've been told there's two cues today, not just one.