 We have Praveen with us. Hey Praveen, welcome to the stream. Hi, hello everyone. Hey, nice we can hear you clearly. So Praveen will be taking us through the Pyconic Interfaces presentation and stay to solve yours Praveen. Thank you. Hello everyone. I hope you're all having a great time at Pycon India. And I hope your day one was very interesting and enjoyable, welcome to day two. So my name is Praveen Shirali and I'll be talking today on a Pyconic approach to interfaces and interface enforcement. If this is the first time you're exploring the subject then don't worry. We'll be discussing concepts from scratch. At the end of this talk, there is a slide which has references and links for further reading and that should help you dig deeper into the subject. So you can check that out later. So I've been told by the coordinators that there could be a lag between me presenting and you listening. So if we pause for questions in between that may delay things a little bit. So in the interest of time, I'll take questions at the end of this talk. At the bottom right of the screen, you'll find slide numbers. And if you have questions which are specific to a slide then do make note of the slide number. I'm happy to run back through the slides to refer to that. The slides are also available on the URL listed here at the bottom. So you can refer to the slides independently during Q&A. If you don't find sufficient time for Q&A then please feel free to reach out to me directly at some later point. The last slide has information on how to reach me. So with that, let's get started. So yeah. So in this talk, we'll try to understand what really is an interface and why is it important? We'll see what benefits we derive from enforcing interfaces. We'll also explore what options we have in Python 3 and third-party libraries to help us with this. Finally, we'll look at a couple of factors to keep in mind when choosing a solution and what are the pros and cons and stuff. Okay, so let's get started. So what is an interface? An interface is a shared boundary across which two or more components share information. So do note the terms highlighted. So we'll try to explore this with an example. Okay, so consider an application and that's sort of represented by the box over here. So this application has a class called a data processor which is responsible for processing some data. Now, this data processor is responsible only for processing the data and it relies on another component somewhere inside this application to help fetch that data. And somewhere in this application is a class called a file IO class and this class is capable of reading data from a file and writing data to a file and so on. It does this through the read and write methods typically found on file-like objects. And the data processor depends on the file IO instance and calls its read and write methods somewhere during its execution. Now, let's say that the application needs to support more such data sources in the future. So consider that there is a class called memio and this is very similar to the file IO class. It defines exactly the same methods as file IO. They behave in the same manner as well. But the memio class operates entirely in memory. It doesn't read or write from a file on disk. So this gets interesting for the data processor because it has to work with classes which are similar. So you could say that the data processor now requires a class which implements a read and write method. So to put that in context, think of data processor as a first person. So the data processor is advertising to everybody else that I can perform read and write operations and I can work with any class which helps me do that. And what I'm doing is I'm gonna call your read and your write method. And for me to do that, I expect that your read and your write methods follow a particular format in the form of the arguments that have to be passed. So if we were to express this relationship in the form of code, we could define a completely new class. So let's call it a read writer because it does read and write. And let's define the read and write methods inside it. But you got nothing else. There's no implementation whatsoever. It is just empty methods. Now this represents an interface. It's an empty class which only defines what methods need to exist. And this serves the blueprint for other classes which actually implement these methods like fileio and memio implement these methods. Now does fileio have methods with the same definition as read writer? Yeah, it does, right? So we expect read to read and we expect write to perform a write operation and so on. So even the behavior is as per the spec. So you could say that fileio class satisfies the read writer interface. Mind you, we are like doing this just implicit assumption because we are comparing it by just visual inspection. But if we could programmatically verify this, then we would say that the fileio class implements the read writer interface. Similarly, you can observe that the memio class also satisfies the read writer interface. Now, if there are future needs for similar io classes, could a completely new class implement the interface? Yeah, it could, right? So all it needs to do is it needs to ensure that the read and write methods are exactly the same. Or they're implemented in exactly the same form. But is it really necessary to have interfaces? Like why do we have to be so strict? Could we choose not to be strict? Like for example, this class here is a less strict version of the io class, right? It defines read and write methods which could work with the data processor, but you don't see it matching the interface, right? But technically it should work. But the question really is, do you want to take a chance with this? Because if there are issues, you will discover them during execution when those read and write methods actually get called. So sure, you may have unit tests and stuff, but it's like pretty late. You're gonna discover them when you run those tests. And everything depends on the fact that you've written your tests well and so on. But ideally it would be awesome for you to know about these when you're writing the code itself, right? That's a good early stage to know when things are deviating. So this leads us to an interesting thought. Could we detect and verify interface deviation automatically so that we don't leave this to chance? Rather, can we write code to enforce the fact that the implementations like file and MIO must implement methods as per spec, in this case, as defined by read writer interface? Because if you're able to do that, then we can detect missing methods, wrong arguments, misspelled method names, bad declarations, and absolutely any deviation that might arise. We don't have to test for such deviation specifically, but we just get some code in our application to do that for us. So what does interface enforcement mean? Exactly this. We basically find some code which ensures that a class adheres to an interface. And if it doesn't, then the application raises errors. So rather than testing for it, we make it a requirement for the application for it to start or uncorrecting. And what do we really want to enforce? We want the method names to be correct. We don't want to miss any methods. We want method signatures to match, that is the arguments, the order of arguments, type annotations, et cetera. We want it to respect descriptors and decorators like static method, property, et cetera. If some methods are generators or async methods, we should be able to direct those as well. And really anything along those lines, if you feel that there's some sort of structure that has to be followed by all of the implementations, you could try to enforce it. So in summary, the expectation is, if you have an interface and this interface defines certain methods in a certain format, and there are specific arguments, specific decorators which define it and so on, you want all implementations to implement it in exactly the same way as the interface defines it. Why is this really important? Well, clearly there are some benefits, but what problem does it solve in the real world? So let's again try to understand this with an example. Consider an application again, again represented by this box and assume that this application consists of a lot of components. And these components could be modules, classes, functions, and so on. Just code distributed across the application. And all of these components or classes, they talk to each other. The instances talk to each other. And over time, the application is constantly evolving. Some components are getting added. Some components are getting removed. You may improve some of the components as well. You'll have better versions of the same components working in its place. And over time, the application begins to take shape. At some point, when you introduce a change to a component, we end up breaking the application. Now, one of the reasons could be that the interface between these components are not clearly defined. So one of these components, which could be these empty boxes, makes a call to the component in the green box expecting that a certain method is available and expecting that it will behave in a certain manner which it doesn't anymore. So what we ideally want is a scenario where we can plug and play such components or classes without worrying about breaking method calls. At least if we're able to ensure that, then that the interfaces are honored, then that's one less problem to worry about. So what really are the benefits? For a start, we get clear boundaries of separation and responsibility for each of the classes. And that makes it easier to maintain applications because they are less prone to errors. If we're able to enforce interfaces, then we could prevent some errors as well. And knowing them very early is a huge win. It saves time, a lot of engineering effort, and also improves quality. So how do we enforce interfaces? So Python has a library called abstract-based classes, ABC in short. And ABC provides mechanism to achieve what we discussed to some degree. So interfaces are enforced using meta classes and interface class sets ABC meta as a meta class. ABC meta is a class inside ABC. And ABC meta actually has the code that does the enforcement. Additionally, we need to decorate each of the methods with this abstract method decorator. Indicating that this must be present in the implementation. So an example of what this would look like. In this case, let's look at memio. So memio inherits from readwriter, which is the interface. And then we have the read and the write methods actually implemented in this class. And when you try to create an instance of memio, that is when the checks are performed you see whether read and write are implemented or not. So just note this once again that the interface marks the abstract method. Interface marks those methods which are meant to be mandatory. And the implementation actually implements it. So how does the enforcement kick in? So let's say you have a version of memio which does not have the right method. Now, when you try to create an instance of memio, then the compliance code will kick in and say your memio class does not implement a right method. So that's the type error that's raised over here. And this error is raised when the instance is created, when you're instantiating the class. Now, there are some limitations to what ABC can do. You can see an implementation here of memio where the read and write methods have no argument at all. However, no exceptions are raised when the instance is created. So while abstract-based classes can enforce the presence of methods, they can't do anything beyond that. On the positive side, as the entire mechanism is defined using meta classes and inheritance, the subclasses tend to automatically benefit from the advantage of interfaces which are already defined in the interface. The implementation tends to benefit from it. But ABC has actually been around since Python 2 when it was designed. And perhaps at that time, the goals were different. Now, there's an interesting talk by Raymond Hettinger on this, which shared some more light on the ABC's history and so on. So the link for that is there in the reference slide and you can have a look at it later. But Python 3 has evolved a lot. And let's see what we can do with that. So let's explore Python 3, some of the libraries and what it can offer. So Python 3 has support for type annotations, async, better typing. There is an inspect library, which is part of the standard library and the inspect library is actually kept up with the times. Besides, Python offers expressive ways of defining APIs. Methods can be defined in a lot of ways. We're gonna look at a few of them now. But obviously there is more that Python can do than what we're covering in this talk. So let's look at some ways to define a method. In this case, foo. And while we do so, let's also observe some factors that matter. It's likely that any Python 3 application you are writing today will likely have methods defined in all of these forms or at least a majority of these forms. So this stuff is very relevant to Python 3 applications of today. So let's start with the first one. Obviously the name of the method is very important. It is the identifier with which the methods can be retrieved and everything else is verified using that as the first step. Next, the method signature is important because they define the arguments, type annotations, et cetera. Then we have data descriptors like the property decorator, setter, et cetera. These are interesting because though the members look like methods, they're not really callable and they behave like attributes. Then we have class method and static method decorators. We also have generator functions, coroutine functions which lend themselves to different behavior as compared to regular methods. So what can we do with inspect? So inspect has a function called signature and as the name suggests, this function returns a signature object and the signature object represents the arguments, their order, type annotations, basically everything that follows the method name and signature objects can be compared against each other. So you can use this easily to verify signatures across an interface and an implementation so that you can know whether an implementation actually implements it correctly or not. Now again, data descriptors are special. So you have methods which look like they have the same name but they are decorated with property setters, getters, et cetera. These are not callable, so inspect.signature will fail on them but there's a handy function called is data descriptor which can tell you whether a method is a descriptor or not. And you can use that as a mechanism to further fetch the actual getters and setters. There are also functions which can detect generator functions and coroutine functions as well. So there's is generator function, is coroutine function, it's pretty straightforward. This is a little interesting. So the class methods and static methods can be detected first by fetching the method from the class that actually implements it and we can do this by navigating the classes, MRO, the MRO stands for method resolution order and the Dunder MRO attribute holds the entire class hierarchy. So once we fetch, we can then use ease instance to check whether a method is a class method or static method. And the reason you need to do this is because the methods could also be inherited by an implementation. Though the inherited class does not define it, it's likely that the parent defines it and the decorator is actually used in the parent class. So that's why we need this method to fetch the actual class where that method is implemented and then check on it. So let's revisit our list. So if you look at the method definition list here, we have a way to identify everything that's on the list. So it's pretty easy to build enforcement capabilities. So if an implementation does not satisfy a check, I'll just raise an error and that's it. Yeah. Sorry to interrupt, but the attendees are saying that the slides are not getting changed. Are you on the same slide, slide number 49? Yeah, sorry, sorry. I could see that, yeah. Oh, I'm sorry, let me just run through the slides again. Sure, thank you. Yeah, thanks for pointing this out. Right, so I'll just take one minute to quickly run through the slides that I explained. Sorry for the miss. So we were looking at this slide where all the methods were defined and like various methods, ways in which you can define these methods and what we can do with InSpec. So the first function we saw is InSpec.signature, which can be used to retrieve a signature object. These signature objects are actually comparable. So you can use this to verify whether a defined signature from an interface matches that of an implementation. For property decorators and data descriptors, there are some special mechanisms in place. So basically the method names look the same, but they're actually decorated. So the InSpec.signature does not work because these methods are not callable. But there is a function called isDataDescriptor. So you can use that to detect whether a method is a data descriptor or not. And if it is, then you can start fetching specific elements from it. For generator and coroutine functions, there are again ready functions, which you can just call. For class method and static method, you would need to go through a route to actually fetch the method from the class which implements it and then check whether a class method or a static method decorator has been applied on it or not. So coming back, the types of method definitions that we originally listed, we feel that there is a way to actually verify all of these and build an enforcement mechanism to detect these and also raise errors. So in fact, there's some third party libraries which already do this pretty well. So there are more than these libraries. So I'm just listing three of them here. Implements, Python interfaces and Zope.interface. Implements is a very simple library which does a lot of enforcement. Basically it's just one decorator that you need to import and use on your implementation. Python interfaces has wider support for Python 2 and Python 3. It uses metaclasses and Zope.interfaces is a huge package. It's got very descriptive APIs and so on. So how do you choose a solution that matches your needs? So there are a bunch of factors which are of importance. So let's do some very quick comparison of just some of the main factors, there are a lot more. So one of the main ones is composition versus inheritance. And this is very evident from the way you bind the interface with the implementation. So in case of implements, you have your class which is your implementation. In this case, my class. And you just put a decorator on top of it that says implements and then pass it the class name which is actually the interface. So it's that simple. So the class, when you look at the class, you straight away know what interface it implements. So this is a very clean wrapper, it is very explicit. You can read it very easily. Now on the right side, both ABC and Python interface packages use metaclasses for enforcement. And the inheritance is used to propagate enforcement. This is not as explicit as the decorator-based approach because you need to know the entire class hierarchy. But on a positive side, because of inheritance, all these subclasses also enjoy the benefit of the interface enforcement. So if you already have a code base which uses inheritance heavily or you've already been using ABC to enforcement of interfaces, then Python interface is a great choice to work as a replacement. Another important factor is early versus late enforcement which is actually the main advantage that you can draw out of it. So implements and Python interface they enforce on import or class creation. So as soon as your application runs when all your packages are getting imported, that is when this kicks in. So it is really, really early. And it can stop your application right there in case some errors are found, right? And this is really useful. On the other hand with ABC, it is based on the philosophy that the right time to check for any sort of enforcement or adherence is just before the creation of the instance. Yeah, but however I feel knowing things early is a great dividend so you can make a trade off over here. Sorry, I didn't advance the slide. So yeah, so early versus late enforcement is like this. So I think we are at the end of my talk. So let's just do a recap. So we defined interfaces what those are and we've seen that interfaces help and interface enforce even better. We've seen an offer what Python 3 can provide and what some third party libraries can provide. And really in today's world with Python 3, there's really no need for using ABC anymore. There are much better alternatives. We can see how inspect can be used and how enforcement can help us find interface errors early. And with that we are at the end of this talk. So here are some references for future meeting. The links are, the slides are shared so you can actually go through these at a later point. So thanks a lot. These are ways in which you can reach me if you have any questions at a later time. All the slides are also available online on GitHub for you to refer to. So open to questions now. Thank you. Thanks a lot Praveen, that was great. Let me just quickly go through the attendees and what they are saying. And let's start from the beginning when you were almost midway through the slides. Shridha says we can have an adapter for data input for data processor, adapter design pattern. Right. Is that the question? No, he's just come in, he's trying to suggest that particular option. Abhinav says it looks similar to a Java interface. I think this is when you were before slide number 49. And we have a fairly big question. Okay, so Priyash says what's the link for slides on GitHub? So for all the attendees after the session is over if you want to have a conversation with the speaker feel free to visit Zulip and go to 2020 forward slash stage forward slash deli. And there you can find Praveen, you can ask any question you want offline as well. Praveen will also share all the resources that he wants to share the code base, the slides and everything there only. And we have a big question. Sairam says the advantage of developing in Python. Let me just quickly copy this and paste as a banner so that everybody else can see it. So how much time do we really have left for Q&A? So I guess we have a couple of minutes left and there is a five minute buffer. So we have, if you see the max amount of time we have seven minutes. So if someone else wants to ask a question feel free to write there. So the question goes, let me just- Can I take the first question? Can I take them in order? Yeah, sure. Yeah, so on the first question, yes, the adapter pattern helps. The adapter pattern is a way of bridging two different classes on what they offer and the adapter fits in somewhere in between and translates what one class requires and how a different class can offer it. Ideally, that comes at a fairly advanced stage where you want to, where you are at a situation where you don't have the flexibility of modifying a class because let's say it is being offered by a third party package and you can't really modify it. So you build an adapter so that you can bridge the connection between these two. However, the interface mechanism is there for you to establish a common language for two classes to talk to each other. So it is different from the adapter pattern in that it is trying to just formulate rules saying, I am a dependent class, say I am data processor and I need to depend on class number A, class A, class B, class C and in order for these classes to give me what I want I'm going to define this interface which they are supposed to obey. And if they are compliant, then I can work with them. So the purpose is slightly different though that you can use either of these to accomplish your need. So can we move to the next question? Yeah, I would just comment like it looks similar to Java interface. Yeah, it does look similar to the Java interfaces where you have it clearly defined and so on. So it's a bridge between Java interfaces and sort of what Go tries to accomplish, Go lang. But it's, yeah, we're not there yet. This is a fairly Pythonic way of trying to draw similar benefits from what other languages already implement. And it's actually part of their language as well. So what is it, it's something we have to build on. Another question is, I understand that Python interfaces and, sorry, that's a... Yeah, so implements is just uses a decorator called implement, right? And what implements does is when you give it an implement, when you give it an interface class, it goes through all the methods that it has. It fetches all of them and understands its signature, understands whether there's a decorator on it and so on. And then it tries to compare that with the implementation and see whether the implementation has it or not. So the moment, so all of that logic is built into that interface decorator, sorry, the implementation decorator. So the moment that decorator kicks in, it's gonna do that comparison for you on import. So the enforcement is done right there when you're like loading all of the modules. I hope that answers the question. I can dig to more details later. It's, there's another question. It's situational, but given that both methods are applicable. So yeah, if you're writing fresh code today and you have a complete choice of whether you want to do a lot of inheritance or not, then composition is a better way to go about it. But if you're dealing with a huge legacy code base where you're doing a lot of inheritance already or you're already using ABCs and you want to just transition that code to work with interfaces better, then maybe Python interfaces or Python interfaces also for one of these could be a better choice. It depends on what change you want, how early you would like to detect changes and so on. So it's just trade-offs and how much effort that goes into picking one of the methods. Okay, another question is too long. So it might cut off on the screen. I'm just gonna read it from the chat. Yeah, so the trade-off here is how clearly do you know the requirements for one class to depend on another class? So when you say fast development, the methodology matters here. Are you writing code to then understand how classes work or do you have a predefined understanding of how the classes are supposed to talk to each other? So if you already have some predefined understanding of how class A should talk to class B, then you will be able to define interfaces better. Sometimes these interfaces also come out of evolution of development as well. So I don't think it slows down the development process that much because the work involved in getting the interface in place is not much. You just have to define the methods clearly as a class and then all the classes we are supposed to implement it you just decorate it or insert it in some way. So the effort required to integrate is not much. But the thought that goes into exactly defining the interface that is much higher. So yeah, that's where the trade-off is. There is a second part of this question. What are the scenarios where interfaces must be enforced and where to relax? Interfaces give really good value if you have one class which is going to talk to multiple variants of other classes. Like we saw this data processor, right? And data processor talking to file IO, MIMIO. Let's say there are like four or five other such classes and you want to enforce rules on multiple such classes. So in such cases, interfaces work really well. Ideally you should be trying to use this everywhere. But there may be some places where the API itself is very experimental. You're not sure whether the method definitions you have defined today are the right ones or not. And for you to make changes, it may require you to make a lot of changes all over the place. So in those cases, you may want to relax so that you have solidified the interfaces to a certain degree where you feel that things are stable and then enforce it. Great, thanks a lot Pravee. And I hope you had to Zulip chat for more questions and to all the attendees feel free to reach out to Pravee if you have more questions. Thanks a lot. Thank you. See you all. Have a good day.