 Next talk up, Anders Hemekist is going to be telling you about using metaclasses to make a declarative GUI, and then it's time for lunch. There will be questions at the end. All right. Hello, everyone. As you said, I will be talking about using metaclasses for making a declarative, it's been sitting here for too long, there we have, making a declarative GUI implementation. I will also be telling you about metaclasses, it's sort of an inspirational talk for you to use metaclasses on your own. The work on the background that inspired me to do this talk, I did together with my colleagues Henrik Bore and Mikkel Schoenenweig, but first a bit about me, I motivate my five-star Python rating by having been at it since Python 1.4. I have been working at OpenEnd since 2001, and I have been consulting at AutoLabel where we did this work since 2014, and I do all kinds of stuff because we are small companies so I get to do everything, but to start with, does everyone know what a metaclass is? Okay. About half of you. I can tell you that you have all used them. If you have used the class keyword, you have used the built-in metaclass type. The other things we will be talking about is a bit, we will be seeing a bit of GTK, which is the GUI library we used, and of course I need to tell you why we all did this, and this is AutoLabel makes these cool labelling printers. You have the printer printing out the label and it has an arm that sticks it onto the box, and it's time to revise the software, and the software running, you see the display here, that's all Python in GTK interface. The one that's in this picture is running Python 2.5 and GTK 2, and we're upgrading it to 2.7 in GTK 3, and this is what the new user interface will look like once it's done. So we wanted a nice way of expressing the layout. As I said, we started, the legacy application is running Python 2, well, both of them are running Python 2, and we had GTK 2, we decided in this process to upgrade to GTK 3, mostly using the compatibility layer, but trying to move on to the proper GTK 3. But this is sort of the syntax that I dreamed up. This is where we wanted to go with the layout, if I quickly go back to the, you see the hierarchical, you have the sort of major groups on the left, and as you go across, you go deeper and deeper in the type hierarchy, which is, I mean, that's how GUIs are built. You start with the window, and then you have the dividing up the window in different parts and you have different widgets in there. So this, I have some code that you can download and play with, and the code on the left will actually produce the little window you see up on the right. And just for comparison, this is what it would look like in the standard GTK 3 syntax, and I had to take out all the whitespace to make the code fit on the slide. But for the first part, I will teach you about metaclasses, and as I'm sure you've heard many times, metaclasses is all wizard level stuff. So once the talk is done, you'll all be wizards. So the metaclass, that's the class of the class. As I said, the built-in metaclass that every class in Python uses is type. And if you want to use your own metaclass, in Python 2, you use dunder metaclass. In Python 3, you add metaclass as a keyword argument on bar. The code examples are Python 2. I will try to mention how to do it in Python 3 when there's a difference. The example code that you can download, if you run it through Python 2.2.3, it will run fine on Python 3. There's a bit clearer illustration of what the metaclass is. Everyone knows what an instance is and what a class is, I hope. So the relationship between the metaclass and the class is exactly the same as between the class and the instance. And that, of course, leads to questions, can you have metametaclasses? And yes, you can. I haven't figured out a reason why you would want to use a metametaclass on the metaclass, but you can go on forever. It's just dunder metaclass, well, if you specify a metaclass in your metaclass definition, it will work just as well. But I said I haven't been able to figure out a use case for it. But if you do, please tell me. So the methods that you will actually be using if you write your own metaclass is the dunder new, which is the constructor. It returns the object that you're instantiating. You don't actually have to return an instance of the class that you're in. You can return whatever you want. If you return 42, instantiating the class will give you the integer 42. It will try not to do that. It can be very confusing. Dunder init, of course, once you have your metaclass instantiated, that is, once you have your actual class object, your dunder init will be called and you can do stuff there. In Python 3, there is a method that would have been useful for us in implementing this, this dunder prepare, which is called before the class declaration is parsed to return the dict-like object that will be used to populate the namespace. So if you don't want to use a standard Python dict in Python 3, you can. You can dream up any dict-like object. For instance, if you need ordering, which we need, you could return an ordered dict from the dunder prepare. Then also, a difference here with Python 3 is that you see the KWR on the dunder prepare. You have the keyword arguments for new and into also. They come from the class inheritance list. If you add keyword arguments there, they will get passed into the constructor methods, which can be useful in some cases if you need to customize your metaclass. Right. So if you add other methods on your metaclass, which you may want to do because you don't want to have 300 lines of new, those methods will be visible on the class, but not on the instances of the class, which it's good to know because you'll be less confused about why this, why can I do call the foo method on the class, but not on the instance, how, where did it go? Ah, it was on the metaclass. So one thing I wanted to sort of take back from this about metaclasses, the class declaration syntax in Python you see on top, it's really only syntactic sugar for instantiating the metaclass. So the top code and the bottom code, the result, the foo class, it's exactly the same. Once you've run the code, you cannot tell the difference between these two. So try to remember that, that you can use the class declaration syntax to instantiate the class essentially, the metaclass that is. That gives you lots of power to use the class syntax for things that, well, there's classes, but not really the standard Python classes, which is what I've been, which is sort of the idea that I used to dream up this gooey declaration syntax. Just for comparison again, just not all of it, but the top is the iterative syntax that we look at how we did in the bottom is what it would look like if we hadn't. And I just have a little pause here to see if you're following along. Are there any questions on the automatic classes? Or should we continue to look at the code? So right. As I said, the first problem we ran into was that in a standard class, you just get the dict with all the attributes in some random order. And when you're declaring a gooey, obviously you want to be sure that whatever you put on top, the first spot in your window, it shouldn't randomly appear that somewhere in the middle. You want the order well-defined. And since we were using Python 2, we couldn't use the dunderprepare. So this was really the, I'd say it was the hardest bit of making this work, figuring out how to do this. Not that it was very hard, but then I've been at this for a bit. I might not be the one who should be judging how hard it was. What I did was that as long as the contained or the attributes in the class were also instances of our meta class, I would be able to sort of detect when they were created. And this, for our application, since everything was going to be a GTK widget, this was no problem. Everything was going to be instances of our meta class. So whenever you had the class on the inside, I would, I should actually back up a bit to this slide. The way, if you look at the top half, the way that the meta classes get instantiated is that it will instantiate from the inside out because label is going to be an attribute in the group class. So the label will have to be created first in order to be able to be placed in the namespace of group and so on up to the top. So therefore when, I'll go back to that slide, it's easy. When label is created, I will remember that, ah, now label was created. And then when we get to group, okay, let's see, what has been created recently? And I'll go through the list of the recently created classes and the order they were created in, and I'll say, ah, label was created recently, is label in my namespace? Yes it is, okay, so that one must have been first. And then I, of course, once I found them, I will remove them from this list. And it will waste a little space, well, not really because you will be keeping the classes around anyway, but you'll have the list of a few of the top level classes still remaining because there is no meta class for the module basically to clean up this list. But that's small enough that doesn't matter. And I'll take this opportunity to have my first GTK quirk that I ran into also, and that was that the G object also has a meta class. And it took, I didn't realize this at first, and I was really confused, and why can I not inherit the GTK widgets in my widgets? And it turns out it was because the G object also had a meta class, and they were colliding. But once I inherited it, the G object meta class, as in, as you know, type is both, it's a bit of a magic built in because if you just use type as is, it's the base meta class, but if you call it with an object, it will return its class. And since G object is already a class, it will be the meta class. So yes, this is our meta class for our basic widgets. And you see here when, actually, I think I missed the code where I actually add, or maybe that's on the next slide, we'll see. So here, anyway, these attributes, class attributes, or meta class attribute, it would be in this case, that will be populated with the list of classes as they're instantiated. And then I will go through it and look for whatever is in my namespace, append it to my local list of ordered attributes, and then remove them. And as I'm sure most of you know, you cannot remove stuff from a list while you're iterating over it in a for loop because then you will miss stuff. Namespace is the dict that is created from, let's see. For instance, if you look at the top, the namespace is the dict that is created from the attributes in the class declaration. So in the case of top here, namespace will have title world, and it will have group, which will be the instance of the group class. So I look through there for instance of, what, classes really, instances of the meta class. It's a bit confusing with the meta class because you have meta class instances, which are classes. So the terminology gets a bit confusing. And then at the end, I will, of course, instantiate calling the super class Dunder new, which will instantiate the actual class. I realize here I forgot, this code is actually wrong. On the slide, the code that is downloadable is slightly different, so that one is right. This should be a, the return from the Dunder new, I should, of course, have saved in some place because that's the actual class that needs to be passed on up. Yeah, so, you've seen this a couple of times, but using that meta class you just saw, assuming that you actually returned the result from the super Dunder new, this code will get you a attributes ordered attribute on the top class, which will be the ordered list of the inside classes. So that's our, now we solved our first problem. Now we can actually order stuff in our classes. The next one is, we wanted to do, of course, we wanted to be able to set properties on the GTK widgets. And also on our own complex widgets, we have special widgets that deal with settings, and they know all kinds of magic stuff. So, in the case of window, title there should obviously set the title of the window. And in GTK, the title of the window is property, which is not a standard Python property, but it's a little bit different than, so I need to add some code to set it in the proper way. And also I wanted to be able to, if our code has a standard Python property, I wanted to be able to set that property in the declaration. So, for instance, normally, if Foo widget has a property, Foo, setting Foo in a subclass like this would overwrite the property, and you would lose whatever setter and get the methods you had and just have a standard 42 in the class dictionary. But with some magic, you can make this syntax instead call the setter. One way I thought about maybe doing this was to use a setter on the meta class, because then maybe it would be called once you set the 42 and you could use stuff in the meta class, in the property on the meta class. But it turns out that the name space, the initial name space doesn't bother about calling, doesn't bother about looking for properties. So that didn't work. So basically what I do is I put an attribute on the class under defaults, which I will populate with any, first in the meta class, when you write your, this is when you have your base class in this example, when you create your Foo widget class, it will look through your attributes and see are any of these attributes properties? Either in the case of the Python property, it will look for a dunder set on the attribute. So if Foo in that case had been a property, the property will have a dunder set that would get called whenever you change the attribute. Properties are subjected for an entire other talk, because they're really useful too, but I shouldn't get into them too much here. And in the case of the GTK properties, there's a props attribute on the GTK class. So I look for if the attribute you're trying to set is in that one. And if it is, I will take the attribute you're trying to set and move it to this dunder defaults dictionary. And then the corporation from the base widget. This is a normal class. This is not the meta class. When you instantiate that, I look through this dunder defaults, which is in the class. And if there's anything in there, I will set that or that. And since I remove the attribute from the class, it will now call the property setter method. It'll be roundabout way, but it works quite nicely. And of course, if it was a GTK property, I will call the setter on the props attribute instead. Right. I should mention the dunder MRO. How many of you know what dunder MRO is? OK, it's about half. I want to mention it because I used it, obviously. It's the order that you should look through your base classes when you're looking for an attribute or a method. So for instance, here, I put up the dunder MRO of strings, just as an example. So you're supposed to start with the first one and go through them. And once you come to the class that has the method you're looking for, you should call that. And normally, you do that using super if you are involved at all. Usually, if you just say foo.bar, Python will look for bar for you. But in this case, I needed to look through them myself. And it turned out in all my use cases, it was much easier to just overwrite what I found than it was to look and see, is it here? No, is it here? No, is it here? So I walked in in the reverse and just made sure the top one won in the end. There is a really, I wouldn't say complicated, but it's a really long description of how the MRO is calculated at this URL at the bottom if you're interested. Right. And then, of course, the next order of business is instantiating the widgets so you get your actual window and all the actual widgets inside your window. If you just do nothing special, instantiating top will get you a top. And it will have attributes in it, say top.group, but that will still be the class group, which is not the actual Vbox that you want. The simple way to do this is, of course, in the base class when you instantiate top, have it, in turn, go through its attributes and instantiate whatever is there. And then, of course, it needs to, since it's widgets, it obviously needs to put them somewhere also, but the base class can't know where to put them so that each individual widget will have to figure out where to put them. And so that code looks like this. And again, I'm working backwards through the MRO to find all the widgets, because I don't actually know if I needed to do this, but I wanted to instantiate them in the same order that they were listed. And then I replaced the, or I set the attributes ordered list on the class with the instantiated attributes instead of the classes, because I felt that, well, if I needed the classes, I could go through the class and made stuff look a little bit nicer, and I didn't have to invent new names. And then for the last order of business, we have mixins. We wanted to do mixins. GTK doesn't allow multiple inheritance. I hope you can see the problem here. Python doesn't know that GTK doesn't allow mixins, so it will happily let you try. Things break in strange and mysterious ways sometimes. But it turns out that if you do nasty things with the GTK meta class, you can sort of work around the fact that you can't subclass GTK classes. And then we'll just show you the code. So basically, in the meta class, if we find that we have more than one base class, ah, this is multiple inheritance. We may have to do something. The DunderG type is set on all the object classes. So if you have a DunderG type, this is a GTK class. OK, I'm getting to dangerous territories here. And if I do, it turns out that I want to put an extra layer in between. I don't know why this works, but it helps for certain bugs or features, I guess, in GTK. So I quickly invent one more level of class hierarchy to bring in all my base classes into one. And then I can add my stuff to look through here and find stuff and put it all into my namespace and eventually instantiate mine. I don't know why it works, but it does. Yes. Finally, we're at the end. So I hope you've learned a bit more about what meta classes are and what they can be used for. It is obviously how your class object isn't associated. You can go and customize it and do weird shit. And I also hope you've been inspired a bit to try and abuse Python into doing what you want and giving you a nice syntax for your problems. Not everything that you have is beautiful code, but hopefully you can make it a little bit more beautiful. I have code for you to play with. It's a little bit bigger than what I've shown here. It will run on Python 2 as it is. If you pass it through Python 2 to 3, it will run on Python 3. I have tested both versions, so it should work. Famous last words, you know? But there you go. You're hopefully now all wizards. And now I should just put up the URL for where you find the code. Hi. So first of all, thank you very much for the talk. I think it's the first talk about meta classes, so kudos to that. It's actually my second talk about meta classes, but not this year. No, I mean at the conference. OK, so I think we all agree that meta classes are one of the best features in Python. And so I had a bit of a problem with meta classes because in some design patterns, you would want to use multiple meta classes for one class. And the magic method underscore, underscore meta class takes only one argument. And the solution for this is a bit cumbersome. Namely, you have to create an abstract class from that takes mixings as meta classes and throw then that into your class. And I was wondering if you have a better solution. No, not really. I suppose that for some things, in Python 3, you could use the keyword arguments. But I'm not sure. It depends a bit. But for Python 2, no. Next question. Anyone else? Hi. Aren't you afraid if a new developer comes to your team and he has like two stars or three stars on his badge, he won't be able to follow what you've been written? Actually not. Because once you look at it, you have to grasp that you have one more level of class instantiation. But once you pass that, it's not really that difficult. I mean, have you ever written a meta class? Well, the only place I actually actively used them is in Django. But that's a bit different story, I guess. I hope you take a look at the example code I have here. So you can hopefully see that it's not so much magic. I think mostly it's lacking is telling people and documenting how to use them. Not so much the difficulty. But as always, good documentation makes hard problems simple. Hi. Thanks for this presentation. I was not aware of meta classes. And I tried to solve the same problem with GUIs and Python that you did without meta classes. And it looks quite ugly. I was just wondering, when you linch your code, do you get any problems with having so many nested classes? No. I mean, there is one small problem with having the nested classes. And that is that it's hard to find, when you're in one of the nested classes, you cannot easily get if you're in the dunder new or dunder init method of one of the nested classes. It's hard to find its actual class instance because the class outside hasn't been completely initialized yet. But that's the only problem with the nested classes. I can do this just to make it a bit clearer what I'm talking about here. When you're in the dunder title, sorry, the dunder init of title, group and top don't exist yet. So you can't say top.group.title if you need to call, for instance, super. But other than that, it's no problem. Thank you very much. I just wanted to say that I think you should retitle your talk to, I never met a class I didn't like. OK, thank you. So hey, it looks like what you've done has taken GTK objects, which are pretty horrible to use and initialize. And you've made a much nicer API declarative syntax for writing it. So I guess if I was starting to write GTK GUI code, I would rather use your system than the default one. So have you thought about open sourcing your package and making a sort of GTK zero and contributing that back? Making the world better. I have. What we're using right now is a bit too deep in our system with all kinds of strange stuff that doesn't really apply to a generic system. I hope that the code that I've put up here can grow into something like that. I will be happy to try and sort of help that effort along. And the code here is, of course, open source. One more question. Is it hard to unit test the code you write like this? No. We have unit tests for pretty much everything, so it's not. And there is unit tests with the example for the implementation, too. All right, is that us done? Brilliant. Thank you very much.