 organizing teams, and privately I'm a cyclist, I make bikes, I repair bikes, I write bikes, I make photography, and also I'm an Ultimate Frisbee player. If you don't know that game, this is something like Python among the programming languages. The same thing is about Ultimate Frisbee in team sports. So check it out. It's a great thing to do. So the talk is going to be about how I already discovered the scriptors, and I did it when I was working on tree structures. The tree structure, that's basically where you have the parent notes, you have some fields inside, so maybe other way. So you have many notes. One can be put inside another, then we call this node a field of the parent note, and you can also put value inside a field. So the basic tree structure is like in Django, you know the models in Django, you know Django. So we have fields in there, this is one level tree. And what I discovered when I was watching the presentation by my friend is that I think many people have seen the scriptors have even heard about where they are used, but they didn't see the obvious way to use it in other places or to override the default behavior. So have you heard anything about the scriptors? Can you raise your hand? Yeah, and have you ever overridden the default behavior? That's what I thought. And so I'm going to start with introducing the project to give you some context. I'm going to tell you about fighting the legacy code and how I, when I got the opportunity, I have overridden the tree structure that was used over there and what I learned from that. So there's going to be a legacy code ahead. First the project. The project was a positioning system. It means we have a mobile phone which sends the GPS position along with what it sees in the radio. So what base stations of the cell network it sees. And we process that, we calculate the positions of the radio stations, we gather this data and then when the another mobile phone, the one down, sees only the network stations, we can give it an approximate position. And this is used in assisted GPS when you, because otherwise the phone would take like 10 minutes to lock onto the satellite because it doesn't know anything about the satellites and the receivers are cheap inside mobile phones. And that was the C++ project. And we had a test project alongside that was written by a C++ programmer. And this project basically replaced the mobile phones and also had access to database. So we were sending binary data, we were comparing the responses against what we want to see. And we were also checking within the database that the positions are updated in the right way. So that's basically it and after the code. So there were a few problems with the code. We had start imports because the C++ programmer didn't know another way to deal with the modules. He wanted everything basically in the same namespace. So every package was a namespace of itself and it was importing everything from the submodules themselves. That's the thing we didn't deal with because the project was used in other areas also. And we had a few things that I think stemmed out of not having introspection in C++ in such an extent as in Python. So we have repeated names. So as you can see here, there are slots that protect us from assigning to the wrong name. We have an initialization where we call the base element and we tell, okay, so under these names we want such fields. And we also give those names of the fields again to the fields themselves. So that's a bit of a hassle. Then again in the top level structure, we have all the functions defined in such way that they are manually looking at the tree below and they are doing the printing, the serialization, the comparing to other objects and that doesn't have to be like this. It can be automated because we can know everything about this data structure. But there is also something very peculiar and which I would like to direct your attention to. This is the Mortal Kombat guy over here to show you. This is the call function. And the call function was used to store the value inside a field. So instead of assigning where you would lose the field and instead had a different, you would put this simple object inside this attribute, we are calling the field and this way we put the value inside the field and the field is still usable with all these methods. So what happens when you have a structure like this? So you have to be very careful with references and so having variables to collect some parts of data and then constructing it, it's rather risky. So all the test definitions were done in long chains of attribute access where the lines were copied by something that's not very usable. Also we had another ingesting from C++ where we used the static methods within the class. So we are basically making a singleton out of a module which already is a singleton. So this is another thing that's funny here. We have some special functions that deal with some of the data like the set version. We have those calls as you can see where the cut is, the second cut that's actually set the data and that's about it. So when you think about such structure and what is your goal when you are doing such test project, I thought my goal was actually to provide for the testers. So they are the guys that are needing my work and what I wanted to do is give them easy to use trees so that they can easily define the data and that's how I imagined it. So I would not have this repetition, I would have a clear structure, I would know which element goes where and when you look at it, because those three structures were used everywhere, they were used in server configuration, they were used in difference reporting, they were used in fitting data, they were using fitting database. So when you look at it, this definition looks very much like an assignment. And so I thought this is the right way to go and I thought, well, I'd like to have something like Django where you have those fields where you can assign values but you can still use them and call method on them. So I tried to make it like that and the first step was to use the keyword arguments. And so I added those keyword arguments into the base element and I used them everywhere in the hierarchy below. So we have this ABC element which inherits from base element and I have keywords there and the second thing I did is I actually used the printing to give the differences. So I used the deflibrary to give the reporting to the tester and this way you didn't have to walk the tree for difference reporting. But again this Mortal Kombat guy shows you the place where there is something interesting happening, so in the init we call the get at on the item and then we call it with the value to set up the value. And so we have this cooperation between the call in the parent which is the init one and the call in the child which is the call one which intercepts the value and puts it inside. And so that was the first thing and pretty soon another protocol came and I didn't have to deal with the rest especially because other testers were also using it, not only my team. So what I left there was the start imports, I left the repetition when defining the data, I left the serialization because it was using some C plug-in and because the other projects were using it. So next time I had a chance I took the opportunity to actually implement those data structures because we had a new protocol to implement and again I was looking at Django and I thought maybe not only the initialization can be dealt with but also like assigning the values to the attributes. So when I assign I want the value to get into the field and I want the field to be usable still. And so the first straightforward way as I've seen it was to use the set add which I have overridden the default behavior and I redirected the assignment to the assign method and so in this place I use, so the set add is used in the parent when you access the attribute and then this assignment is redirected to the child and this child is dealing with putting this attribute in the right place in the field and also putting the other fields that can be in the subtree. So I actually had this in two separate classes but I couldn't fit that in here so this class is basically both the parent and the node. So each node has this ability to intercept the assignment and each node has this ability to take care of it when it's a child and so we are taking care of the value and we are taking care of the subfields that are in the child. And another thing I took from Django I have stolen it was the creation counter which let me get rid of the repeating the names in the slots so we have a creation counter we copy it from the class into the instance and this way we can easily sort the stuff because as you go when you define, when you call those classes instantiate those fields they each get a creation counter in the sequence like the lines go in the file and so it was almost there. It happened to be that actually I invented something like a data descriptors but data descriptors just go in a different way so I have overridden the set magic method and I called a sign and what the descriptors, data descriptors do they actually have this mechanism within the get attribute and they use set for setting the value inside the child. So those two being set and assign change the functionality and I put them instead there is a get attribute which would call our set and the set is in place of our assign so this is the transformation that happened and basically I'm using the data descriptors. And what I think this doesn't seem scary I think people get scared because they see this cold tree and think oh my god this must be complicated I don't know why is it here and this is complicated because this is actually few things mixed up together there is inheritance here there is descriptors and there is data descriptors so the ones in red and again I have those two matching pairs marked pink so there is a parent call and there is a child call and the red ones are the data descriptors so we are basically checking within the class because we have to do it before we access the field of the instance and if there is a get and set we call set and the set is doing our work and now the green part is basically an inheritance change so we take care that we take stuff from the instance and in case that it's not in the instance we take care to take it out of the class and if we take it from there we check again if there is a get method so that we can override the behavior and at the end the white one is a fallback which you can use and which is also quite fine to define something like a default dictionary with attributes when there is an attribute missing you just you can override the behavior when what happens when you get no such attribute and in the default that's attribute error that you get and so after you have done all of this and then you make a structure all inheriting from this base element you can do things like put some fields in there to give you the ranges for the validation put some fields to give you default values and all this is safe from writing when you are defining data so this is a type definition I was I had actually two kinds of the first split was between the structure and the atom so the structure was dealing with copying the fields the atom was dealing with copying the value within the field so so the structure was basically like a subtree the the atom was like like a single element within the structure also at least and then you simply define the data say something about the max value and when you have a you win then for example you inherit in from in and at the minimum value of zero and so on and so on and this becomes more readable but the most important thing is that you actually get this kind of use that let's look at the down the lower part and you have this ABC element which has a version and this has a minor field so we want to change this minor field and we don't have to call it so it doesn't look strange but you are when you assign to it you still can validate it you can serialize it and this way you move all the logic down with the structure so you'd only care about what this particular field is doing with in respect to serializing and then the field above is doing is calling this field to get the serialized data and then construct the bigger parts so this way you can spread the functionality and actually in most cases you don't have to write it again in in the fields in those small items over there and so the descriptors are basically that they are overriding the access to the attributes and this gives us the possibility to to put it somewhere else put the thing that we are calling it with like the assignments somewhere else and then still use the magic field that has some properties and and methods to be called and as a bonus in this project it gave it the structures that looking so easy that I was actually able to redefine the printing in such way that I was able to print those structures onto the screen so from the previous situation on the right where we had a different way of printing and different way of defining that we came to the situation that the printing and defining was basically the same and so we could do regression testings within the project and the testers got a really big performance boost on that because they were able to copy the structure from the screen and just put it in place modify some things and and test regression regressively so and that's about it so I think the moral of the story is that you can prove anything with a country's example and I like how it's how Joe has Polsky put it and I also seen it in the morning on some other talk that there is this Pareto principle and I wanted to say that too that you should actually when you are doing something you should choose the most bang for the buck and try to not get into rewriting stuff until you get to new features because it doesn't bring business value and and as I did the I did the new solutions on new features so if it's broken don't fix it do you have any questions I think I rushed through it yeah I think I still have time so so the story is first I I took care of the low hanging fruit and I put the keyword arguments in there to initialize the value and then I realized that when you look at the data it looks pretty much like an assignment and I wanted this assignment to not only be used in in the initialization but also to be able to assign again to that data structure so when constructing data structure I could have some definition but then I could copy the structure and then assign again and modify it for some other test so have some basic template and modified it modified later and this led me to try to override the assignment which is basically attribute access plus plus the assignment and I try to do that with the setup and assign and this works but then I realized that actually those things are already there so the how I think about it I think the strict the script or API data descriptor API is mostly about this kind of thing where you try to take over at the moment where somebody is assigning to something then take this data put it inside the field or a note and then still have this magic note that does all the work for you but you have the ability to write the code like like that where you assign to something and then call the the stuff that is needed for you and I think it led me to thinking that the scriptors are not very much the big call graph over here but this is only the the red part which checks if the methods are there and calls them if they are there so this is the default behavior of the get attribute stuff get attribute methods in Python and I thought I'd like to share it no I just called those sub trees which contained another other fields which could be also sub trees struct so I built a class hierarchy so I didn't have to re-implements the stuff that would be common for example to integers to numbers to build Booleans and so this is the items and for the structures I made by that I mean least trees and so here I'm actually showing something that is a there's a mix of the two because I'm copying over here I'm copying value that's what the item does and I'm copying fields because I didn't have space and I didn't want to complicate it so I wanted to say that there is a note you can put a value in it but you can also have sub fields and those sub fields can all again be a note you can put a value there you can put sub fields and this way you build a tree from that as I said I think it had a big impact on the performance of the team by by means of this one where they could actually copy paste the data from the screen and I think that was the biggest thing from the other stuff I also changed the comparator so we had two structures normally it was walking down the tree and trying to compare each of them and then said okay this is different from that one but maybe even didn't give the value so I changed it to such way that I actually use the printer I printed the both structures and then put a deep on that and removed all the non differing lines from the deep which I don't show there actually and this is this is a deep I copied from the from the code so so these were I think the two biggest improvements the other improvement was about the configuration but this was a bit a bit different tree so so I think it's not related well I no longer work at the company so right now I'm doing web development other stakes and this is a much better line of work and I think we have refined process with a lot of we have almost 100% coverage we have the ends around the stuff we have a working scrum and I think I'm not going back over there to fix that stuff anything more so I guess thank you for your attention