 The next talk is going to be by Scott Irwin who is a senior developer in Bloomberg So today we're going to talk about Dunder methods and why they are special So we're going to cover pretty quickly what special methods are Talk a little bit about what the Python documentation calls basic customization Then we're going to dive in and we'll cover some of the Dunder methods some of the special methods there And then we're going to go more into some special cases like callable Look at numerical operators and then finally we'll finish off with protocols or at least a couple of the protocols One thing to keep in mind is there's 40 plus special methods So there's no way I can cover all of those in this talk. So I'm cherry picking a few So special methods, what are they? So special now methods are the name used in Python documentation You might also have heard these referred to as magic methods or Dunder methods And for those of you that haven't heard the Python slang Dunder stands for double underscore the official definition from the Python docs is Special methods are Python's approach to operator overloading allowing classes to define their own behavior with respect to the language operators So one example is Dunder add method, which is called by the plus operator and is defined by various built-in types Such as int float and stir so why are special methods useful to you as a developer they enable you to Give your custom types your classes to have the same features and expressiveness as the Python standard types So one general note is when you're emulating a built-in type You should generally only implement the special methods that you need which is to say if you are in your own application You shouldn't necessarily worry about completeness You should just care about what you need to get your application running now if you're a library writer and you are Accepting things from other people then you might want to pay a little more attention to completeness if you want to block Specific functionality you can essentially set the special method in your class to be none And then that will prevent that type of operation one example would be there's a special method Dunder'd reversed Which allows an optimization for reverse iteration across sequences if your sequence doesn't support Efficient reverse iteration then you could block that by setting Dunder reverse to none so Basic customization so what is the what is in the basic customization? So one of the ones is probably if you've written any Python classes at all you have seen this special method and this basically takes an already obstantiated object and fills it in so this Should not have a return statement because within the language It's a type error to return anything other than none and since in the Python World if you don't have a return statement the default return type of your function is none Best thing to do is to leave the return out One little note is if you have a derived class and you want to make sure that you are Using the base class and initializing there as well you need to explicitly call the base class and it Dunder and it function So that's what this little bit of code down here is showing the the base class and then the derived class calling the Parents class Dunder and it Also in the basic customization is Dunder repper. This is the official string representation of an object If possible, and this is from the documentation if possible, it should look like a valid Python expression This is typically this is the string representation that is typically used for debugging So it should be information rich and unambiguous. So if you have spent any time at the Python Ripple and you've typed a variable name and then watched what the values get dumped out You've invoked Dunder repper a closely related special method is Dunder stir This returns an informal or you could think of it as a pretty print string representation of your object But this is the one that is called by the stir constructor also by format and by print But you don't necessarily need to implement this on your class because the default Implementation that's inherited from the Python base object will call Dunder repper as its default implementation One of the other things that you might want to think about is implementing Dunder bool. This defines the truthiness of the object Generally, this should return true or false But if you're if it if you don't implement Dunder bool The Dunder len method is called so for containers. This is why an empty list an empty dictionary an empty set Return false is because they call the Dunder len method for for those classes And the truthiness is is if len returns zero, it's false if it returns any other value. It's true now if you don't implement either of these methods then all Instances of your class will be considered true and so here in the We have a class swallow and We're keeping track of the state of this swallow and we are only true if we have an unladen swallow There are some additional customizations way too many to go into there's Some more around representations such as Dunder format and Dunder bytes There's what the documentation calls rich comparison methods, which is the less than less than or equal to someone and so forth And then there are some advanced Dunder methods such as Dunder new Dunder Dell and Dunder hash Which you should only be implementing when you really know what you're doing you can get yourself into serious trouble with those three So with that let's move on to callable So the definition of a callable is any object that can be called So it's a little bit of a recursive definition there, but basically it's any object that supports the The paren operator, which is to say it can be called like a function Python callable objects that you might be familiar with are any of the built-in functions Any function that you write yourself is a callable the class objects that you define are callables methods on built-in objects and Methods of your classes that you you define all of those are callable one additional thing and this is where the special method comes in is if you've defined a Dunder call method then Objects of your class are also callable as a function. So what does that look like so informally? What you end up with is being able here We're creating an instance of this multiply by and then we're invoking that instance with arguments just like we would any other function and Where you might want to do this is if you have you want to create a function with state So here we've given it a state of three. So this function always multiplies by three. I Could create other instances of the same class that multiply by something else And I can have all of those active in my system at the same time Next up are the numerical operators This allows you to emulate numerical types by defining these methods So they're the operators the plus minus star single slash and double slash. We're talking strictly Python 3 here And those dispatch to Dunder add Dunder sub Dunder mole Dunder true div Which is true division versus Dunder floor div Which is the basically the integer style division there's a bunch more methods that are available if you really want to go and Implement a fully functional numerical type. There's matrix multiply mod div mod pow left shift right shift and The bitwise and Xor and or if you're not going to Implement it or I'm sorry if you are implementing it and you're not going to support a particular style of argument You should return not implement it. So if you have say You're a numerical class and somebody gives you a string object And you don't know how to add a string object to your object. You should return and not implement it There's also this concept of reflected operators All of the numerical operators have a reflected version. So where we had Dunder add We have a Dunder R add and a Dunder R sub and etc the right hand Operand of an operator is its reflective operation is called if the left hand Operand does not have the supported operation Or returns not implemented. So one of the ways to look at this is if we look at an example So here we have a star B First we'll try to call the mole method on the a object Giving B is the argument if that returns not implemented and a and B are different types Then it will call the R mole method on the B object passing it a as an argument and where you can see this in regular Python is that's how This duplication of string methods work The int doesn't have a mole method that takes a string But the string has a mole method that takes an int. There's also augment augmented assignments Nearly all of the numerical operators have an augmented assignment Which is to say an in-place version, which is where the the I comes in here So we have I add I sub so on the only one that does not have an in-place version is div mod and These are the things that implement plus equals minus equals so on and so forth Generally these methods should do the operation in place, which is to say modifying self and return the result That said well the result could be self it doesn't necessarily have to be self and one of the places where you would see this in One of the basic types is integers the integer instances of integers are immutable So if I do a plus equal operation, it's going to Do the operation and return a new object so if I do 42 plus equals one I'm going to get Not self of 42. I'm going to get a new object 43 There's also fallback behavior if there's no in-place Version available the the operator falls back to the normal method So if your class does not support plus equals essentially it would break down into one of these two where it would do the assignment and Do the add or our ad depending on what the types are here? So an example of that is the stirred class does not implement an I'm all method But it does support the star equals Because of this fallback behavior here There's lots more operators Some of these are These are the unary negative unary positive abs invert so on and so forth If you are interested in these these are well documented in the Python documentation so on to some more Complicated things let's talk about protocols so protocol or Let me back up so Protocols everything that we've seen up till now we could get away with implementing one and only one special method on our class If we wanted our class to only support ad we could do that For protocols these are sets of special methods that we would need to implement in order to fully implement the protocol Python has multiple protocols that you can use to tap into some of the most common protocols are things like collection and iterator They're so common that the Python has a module collections dot ABC which is to say abstract base class that will help you implement these it will guide you towards implementing the the necessary methods to The to implement the protocol So let's take a quick look at those two things. We're going to look at collection first So a collection is a sized iterable container class Special methods for the collection include Dunder iter which returns an iterator for that collection It also needs to support the Dunder contains and the Dunder lend method because remember we have we're supporting sized here So we need to be able to know how big we are So These slides are hosted on github. There is also a Implementation of this linked list class in that same repository Where the where these examples are coming from So here I'm pulling in the linked list I'm instantiating a linked list and then I'm just quickly adding some things here and then I can do things like some on the linked list and This works on my linked list because it's just using the underlying iterator And I can do testing of contains that's what in is calling the Dunder contains and Then lastly lend calls Dunder lend. So what does this implementation look like? so that the Dunder iter just returns an instance of the linked list iterator class with is instantiated with The particular container we're working on right now the basically self of the linked list Contains is just it's just going through the list again. It's using the iteration it's doing the or This is essentially causing an iteration and I'm just searching through And then lastly The linked list class happens to keep track of how many nodes so the lend operation is very efficient in this particular case so a sequence methods for sequence containers They are everything that a collection is plus get item Which is the square brackets It could also include optionally reversed index and count Some additional methods if you're going to have a mutable sequence. So this is like tuple versus list Is set item and Dell item and then there are all there are several more optional Methods that you might need to implement all of these are documented in the collections dot ABC documentation But again, I'm going to Remind you that you should only implement the minimum necessary functionality to suit your application within or your use case within your application for mapping Mappings are also containers. So they start with a collection, but then they also support get item iter and lend But they also Include things like contains keys items value. So basically it should Work similar to a Python dict for additional methods if you want to have a mutable mapping You would need to have set out set Dunder set item and Dunder Dell item And there are several other methods that you could include such as clear an iterator is an object that iterates over an iterable An iterable needs to implement the Dunder iter, which we saw before in the container The iterator needs to implement Dunder iter. It's as well, but it just needs to return itself So if you call iter on Something that is an iterator or it just returns you a reference to itself But an iterator also needs to implement Dunder next and the Dunder next is basically how you step through the iteration and It should return the next item from the container and when you reach the end of the container It should return a stop iteration. It should raise a stop it raise stop iteration and This is the internal workings of Python are very much reliant on this So let's take a quick look at the implementation for the link list iterator, so Dunder iter is very easy to understand the Dunder next is Basically if we're an empty list, there's nothing to do or if we're already at the end of the list Otherwise we move on to the next position and then we return the data of whatever position we're at and This is just conforming to the definition So some examples of iterator behavior So again here, we're just quickly building up a link list and Then we're getting the iterator on that link list and then we're getting the next item and Then we're getting the next item. So next here is calling our Dunder next to make this work When I get to the end of the list I get the stop implementation or stop iteration exception and That is used implicitly by The the for loop if you were to iterate or if you were to do a while x in blah blah you would see that so inclusion We have special methods are pythons approach to operator overloading and They enable your custom types to have the same features and expressiveness as the Python standard types so Essentially what is the Python language has done is exposed the inner workings and Hooks to allow you to write very fully featured expressive code with your own classes References this is a direct link to the language reference and then this is a this is the link to the slides The github repo is basically SJ Irwin or a github.com slash SJ Irwin Which I should have put on the slide, but I forgot And that's it. Thanks I So we went a little fast. So I have time for a few questions for one question if someone would like to Hello, and thank you. Is it working? Okay Just carry on. All right. Thank you for this amazing talk I would like to just ask you one question regards to it You have given us examples of what's what actually caused this next dunder method in under the hood like a for loop Like a while loop at the truck, but I was thinking I think generators They also call when when you when you call a yield for example, it actually caused the next under the hood as well It's alright. Yes That's all. That's all. Thank you Any other question? Okay, everyone and there's oh, yeah So I think operator overloading and things like that has a lot of scope for horrible horrible things to be done Do you have a story of the most awful overloading that you have seen? um, I Think I'll keep that private. I Don't want to embarrass myself or anybody else. I think this gentleman on the end had a question. Oh Same question All right. So If there are no more questions, there is a coffee break for 20 minutes on the seventh floor And there is also going to be a book signing by Ian Oswald So, thank you