 So I have here with me today Smith Packard and we're talking about writing clean abstractions, right? Yep. Perfect, so take it away. All right, people, let's get started. So, yeah, as she introduced me, I'm Smith and I currently live in Dubai. And I work for a biggest classifieds company in Dubai called to be so. And today I'm here to talk about, as you can see, writing clean abstractions. So abstraction is somewhat abstract topics. So I will try to make it more concrete for you and give you tips on how to write clean abstractions. So, at Dubizal, I take care of building highly scalable and resilient infrastructure with infrastructure as code as a platform engineer and also focus on building tooling for developer productivity. You can find more about me on my LinkedIn or GitHub. So, yeah, let's get started. So to start with, if you were to explain someone what is an abstraction just to just so that we all of us are on the same page. I would say in layman terms abstraction is a way to say to your clients, give me the gist of what you want to do, and let me worry about the details of how if you are a client of abstraction you trust the provider to handle the details correctly. So, why to abstract, because abstractions give you leverage leverage is a term that comes from a physical liver, which let me move a huge object which I couldn't possibly move without a liver to amplify my force. Similarly abstractions hide the low level details this is called encapsulation and allow us to develop more and more complex systems. One of the classic example is this abstraction tower, the abstraction towers shown in the example shows us how building abstractions on top of each other have enabled us to do really complex stuff. So for instance, here is an here you can see assembly language is an abstraction of machine learning code which is in hexadecimal which is an abstraction of signals controlling the complex integrated circuit, which abstracts the components built out of gates where gate is an abstraction of certain arrangement of transistor, and we can go on and on about it, someone who studied transistors in college may be able to tell us what transistors are abstracting on, and so on. So, before we talk about good or bad abstractions. Let's understand the characteristics of abstractions. So I like to think about things as strong and weak abstractions, strong abstraction encapsulates a lot of complexity, and this is not necessarily a good thing. There are many times when you don't want to hide a ton of complexity, conversely a weak abstraction encapsulates very little complexity. And here Tom week doesn't necessarily mean a bad thing sometimes you actually need people to get down to the nuts and bolts of what you're providing them, because that's what they need in order to build something on top of your API or a plan or your platform. Another characteristic of abstractions is that they all leak and we have to live with that fact. By leak, I mean, you need to know sometimes the underlying detail to build on top of it. Some of the common example can be a two dimensional array or SQL or NFS something as simple as iterating the large two dimensional array can have a radically different performance if you do it horizontally rather than vertically, depending on the grain of the load. One direction may result in vastly more page faults than the other direction and page faults are slow. The SQL line. The second example is SQL. So the SQL language is meant to abstract the way the procedural steps that are needed to create the database instead of allowing you to define, define merely what you want to do and let, let database figure out how out the procedural steps to create, but in some cases, certain SQL queries are 1000 times lower than another logically equivalent queries. You're not supposed to have to care about the procedure only the specification but sometimes the abstraction leaks and causes horrible performance and you have to break down the query plan analyzer and study what you did wrong and figure out how to make the query run faster. This is another example of a leaky abstraction can be NFS. Even though network libraries like NFS and SMB let you treat files on remote machines as if they were local. Sometimes the connection becomes very slow and goes down and the file stops acting like it was local. And as a programmer, you have to write code to deal with the, with this, the abstraction of remote file is same as local file and it leaks. That being said, things can come crashing down if you do do abstractions wrong. If abstractions are not done. Write the side effect of abstractions can be you introducing a technical depth to support backwards compatibility, or you might end up rewriting abstraction in which case you will break the backwards compatibility. You have to think carefully when we design the abstractions or you will end up spending time to create spending time to clean up the mess in your code base. So what do we do. We should think and analyze before we go ahead and make changes in the code and write clean abstractions. So before we, before I talk more about, you know, how to write clean abstractions, let's understand what are bad abstractions and how they are formed. I know it's not patriotic to use Java snippet in Python conference. It's just the static typing help to deliver better example rest assured I have five examples in the future slides. So here. So I wanted to have this interactive but unfortunately, I think the audio is one way only. So the question was that which which side left or right looks more cleaner and you think it's a good abstraction. In my opinion, it's the left. Why I don't think there is a question because because the second one is a complete mess. If I look at the second one there are questions that pop up in my mind like connect to what what is what in the world is all why do some mutators return nothing. Why does close have a bullion to tell it whether or not you want to close, of course you do, or you wouldn't call close. Why are there two deletes that require substantially different information is one better somehow. What does that thing piles do why does only add one only some things. Notice the core of the core of the objection has to do something with the abstraction, respectively. For example, there is open and close but no method for connect so it's a complete mystery what this does and if you should use it. The second overload of all adds a mysterious parameter that seems to indicate this overload is some kind of consolation price method or something meaning a possible temporal dependency. There appears to be some ad hoc mixture of exception or ever code handling close wants a state flag you need to keep track of the things internal state for it. It's inappropriate intimacy inappropriate by the way inappropriate intimacy is a code smell that describes the method has too much intimate knowledge of another class or methods in our working in our data, etc. Then does this interface want ad hoc primitives or first class objects, it can't seem to make up its mind what defines a customer. The file stuff makes it seem like a class is a database access class that profited awkwardly for a corner corner case involving files, which is completely different ballgame. The rest of the operations have at least one overload that deals with customer but add doesn't indicating add is somehow different than other code operations. So, how are bad abstractions formed like we developers are not evil we don't want to write code that could possibly, you know, provide bad interface to our clients right. So, how, how are they formed. As you can see in this example. The existing code has powerful influence. It's very presence argues that it's both correct and necessary. We know that code represents effort extended and we are very motivated to preserve the value of this effort. And unfortunately the sad truth is that more complicated and in comprehensible code deeper the investment creating it the more we feel pressure to retain it. Like saying some cost policy, which refers to the fallacy of honoring the sun cost, which decision theoretically should have just been ignored. It's as if our unconscious tells us goodness that's so confusing it must have taken ages to get it right. Surely it's really important. It would be a sin to let all that effort go to waste. When you appear in this story in step eight above in above this pressure may compel you to proceed forward is that to just implement a new requirement by changing the existing code attempting to do so is however brutal. The code no longer represents single common abstraction but has instead become a condition laden procedure with which interleaves a number of vaguely associated ideas. It's hard to understand it and easy to break. If you find yourself in such situation. Please resist being driven by the sun cost when dealing with wrong abstraction, the fastest way is back, the fastest way forward is back to following reintroduce duplication in the abstracted code back into every color within each color use the use the parameters being passed to determine the subset of inline code that this specific color executes the little bits that are needed for this particular code and yeah, we'll we're in the formation of a bad abstraction. So, how can I go around building a good abstraction. Well, there are certain points you can keep in mind or you know, I would call it a checklist maybe. So the first thing you should do is identify your data flow and primitives, then define the methods that are essential to do all required then another characteristic of a good abstraction is to have safe defaults every good abstraction have safe defaults. So, for the ease of use, take anything from db's to web framework like Django and flask and text editors which are the best example they are simple enough to be easily adapted by these, but are customizable enough by advanced user. So yeah, have safe defaults, and then build layers on top of the abstraction for further common operations. And then last but not least, follow the sid principle from solid which comes in handy while designing the software. So I'm going to emphasize a bit on sid, which is a single responsibility interface segregation and dependency inversion. So, by sid, I mean, you know, the sid principle says that a class should have one and only one reason to change one class should only serve one purpose this does not imply that each class should have only one method but they should all deal directly to the responsibility of the class. A client should not be forced to implement an interface that it doesn't use this means that we should break our interfaces into smaller ones. So they better satisfy the exact needs of our clients. Similar to single responsibility principle, the goal of interest phase segregation principle is to minimize the site consequences and reputation by dividing the software into multiple independent parts. Depending on the abstractions not on conversions by applying the dependency inversion, the models can be easily changed by other modules, just changing the dependency model and high level model will not be affected by the changes to the low level Well, that being said, let's take a small example of sid. So I try to come up with a more real world and a pythonic example that we can all relate to. Well, I stole some snippets and also just mapped how the redis pi is written because I found it quite beautiful and well abstracted. So, as you can see here in the init method it has safe defaults with ability to have fine gain control for the configuration of redis. You can see here that we are using a lot of inheritance where extension is required. Another thing I would like to point is, please always differentiate between abstract and non abstract class. I've seen this a lot people instantiate the base class which is overridden a million times in the codebase and then you know that damn method that is instantiating the base class always breaks for no reason. So it's better to prefix the abstract classes with something like base for instance here we have something called base parser Yeah, and because it clearly indicates that it is only meant for inheritance and it must not be instantiated. Another thing we can always see things are abstracted out with sid principles in mind all these classes we follow we see they follow single responsibility principle and they use composition or dependency injection to work with each other. Okay, so what is composition in another it is another object oriented programming approach. We use it when we want to use some aspect of another class without promising all the features of that other class. In my opinion dependency injection is quite similar to composition. The only difference is when class B is composed by class A class A instance owns the creation and control or controls the lifetime of class B whereas in dependency injection we inject the instance of class B from outside you can see we pass the instance of connection pool into the Redis client. Example of composition would be based on the availability of high Redis we are swapping the parsers as you can see here and another example would be the connection class based on the URI we are setting the connection class in both cases we are creating the instance inside the class well that covers up the sid and I think I went too fast but that's all I had to present any questions. Hello hello yeah you still have 15 minutes left let's see then so well thank you very much and we have our first question very good so Thomas is asking how to balance abstraction which is complexity and flexibility which which introduces complexity can you repeat the question again yes so how to balance abstraction which sorry which is complexity and flexibility which introduces complexity so how to balance between how to balance abstraction and flexibility so what I would do is I would clear a layer of abstraction so I'll create one class with just the primitives and then for the common operations for the ease of use I would create another layer on top that way if someone wants to touch the low level details he can use the primitives and if someone wants to perform the common operation for the ease of use I would use the layer above the primitive class and also yeah safe defaults can also give you ease of use so yeah I hope that answers your question I think it does very good and then we have another question from Jill so Jill is asking do you know it sorry do you know any good examples of code following this following Python projects yeah you showed JavaScript code in the middle of a Python conference by goodness no I also took yeah I'm sorry about that guys I'm really sorry the redis client of Python is is really good example another thing which I found pretty beautiful was the requests library I think they are just reading the code there would give you really good idea and since you are a Pythonist you have the context of you know how these things work so you will be able to relate stuff also and understand the code easily perfect thank you yes very good so we have another question can you say something more about refactoring incorrectly created abstractions towards something more flexible refactoring incorrectly yes so how incorrect refactoring create abstractions towards something more flexible I'm sorry I didn't get the question like incorrect refactoring would create abstractions towards more flexible abstractions I think that's what he's asking so the literal question is can you say something more about refactoring incorrectly created abstractions towards something more flexible yeah sometimes what I've noticed that maybe you are too generic and it becomes too complex either for the users or trying to use that abstraction but you thought with genericness in mind so I don't know like you should have a balance between genericness and what you are trying to watch you maybe just limit your scope to certain use cases that you are trying to solve don't try to solve all the possible things that come to your mind sometimes I see that you know we don't keep business context in mind we just try to you know solve every possible scenario and I think that that's what leads to way too flexible and way to generic abstraction which brings in a lot of complexity while refactoring does that answer your question cool so we're waiting to see if he replies on this course or to see if that answers the question because we're all confused a little bit by the question don't worry well perfect for now the questions are over thank you very very much and there's still people typing so I'll give you it this is the last call for questions Okay, so Jill is trying to understand exactly what Paolo asked beforehand and so restructuring the question and we think could be something like how bad refactoring could lead to a worse design so a less flexible design but you answer that question pretty well so it's all right cool so any more questions at all nope it doesn't seem like it well thank you very very much