 Hi for the beginning. I want to say that this presentation will be about clean architecture with for me is really really Exciting and I think that Uncle Bob's book about architecture is also so exciting. It's like criminal book for me I'm a nerd. Yeah, so let's begin Okay My name is Gregor Scotland and I'm from Poland If you find my name difficult to pronounce, I'm from the city of Łódź have a luck But for simplification make life easier. You can call me Greg Eight years of programming in Python is a great advert for me And I'm really happy that I met Python as a first commercial programming language for me Currently I'm working at HDT and I share my free time in by creating local IT community MiGavky IT and dancing modern dress with my wife My goal is to keep commonplace stuff to a minimum While sharing as much as of my experience as possible That's a lot of slides. So I will try to be as painless as possible Now you're going to see this symbol from time to time I Use it to mark the parts. I think are most important that are most important for me So when you see it just wake up for a second For a little warm-up will begin with what's new in Python Anyone using type annotations? Yeah, I love those hands. Okay, great. So for the initiated Python annotations were introduced in Python 3.5 Initially, they only work in function declaration Here you can see that we have our input parameter integer and return type dictionary But let's take a closer look to that return type using dictionary type We can ensure that we won't call an incorrect function Incorrect method on the return value But what does that say us to us from the business point of view? Frankly, nothing. We have no knowledge what to accept from this dictionary and This is my pro tip number one With typing be precise not use to general type annotations how For example with aliases if you have been in previous talks about typings. Wait a second Here you can see order alias are just a dictionary of string key and string or integer values And we can use it as a return type of our function While you gain your doubly tea what is most important for me that we know what is inside this dictionary unordered from Python 3.6 onwards we gain possibility to Annotate variables So we can say that our order name for Europython is a string type Also, we can use our type annotations aliases To annotate order as you can see it is still a simple dictionary initialization nothing changed only the type is set and This is how fully typed function in Python 3.6 looks like However, this sole function definition tells us nothing about The fields inside this order We need to investigate the function body to find out that it has ID and name fields How do we buy past that? With Python 3.7 which places the cherry on the cake for typings. Do you know what it is? data classes Data classes as name implies our classes designed for storing data We only need to define what to store in them nothing more. We don't need to define unit functions We don't need to define string representations that's all you need and We can use instead our alias dictionary this data class as our return type and to create order variable inside this function And this is pro tip number two data classes work very well with type annotations Now we are super precise what we are doing There is no chance that by accident someone will forget a required field and This is terrible this now is fully automatically typed checked and verified Okay, that was that common play stuff that I want to keep to a minimum now. Let's talk architecture Let's begin with the big question how to design architecture With the idea of maintaining it for the next 10 years this is really important because Usually we change our job every few years and we forget that those applications will be still developed by other people and Keeping that in mind is important because maybe just maybe we'll get lucky and those karma came back up to us So how to achieve that? with clean architecture Clean architecture puts the application domain in the center of our application not the database It considers Database a part of the outside world. So what is application domain? It is where we implement our business logic and how we want to communicate with this outside world nothing more So let's see how to do that. It's time for coding We create a simple microservice with free responsibilities free use cases create order view order list and last night, but least our most important Fisher Fisher cash cow adding existing item to order I Like you to show the code that I will that I prepared for the demo. You will see later and this code is based on real production application and Those are steps that I take to prepare this code We'll begin with data definition or entities, which is data that we'll be working on First we implement base entity, which is not much as you can see But when we create our first real entity data class client it will inherit from our base entity But there is a small difference between the earlier data classes and this one Here we add frozen equals true parameter What does that do? For instance if we create me Gregor and someone will try to say hey Gregor now your name is Alice Python will protect me and say frozen instance error or no So this is called immutability an instance once created cannot be modified and Is it difficult? Oh, sorry And is it difficult to use frozen data classes? No, is it It will be useful in our demo also not so Why? Well Because immutability can be handy when our project gets bigger and bigger and we want to add Multi-processing or trading so concurrency Then immutability solves a lot of problems And for now it is important for you to remember that adding immutability to existing code base is really difficult It is far easier to start it from the beginning Okay, but let's go back to our entities We can add product definition also a frozen data class and the most complex one Order ID created Client as you can see we can use our order other Data classes as our type Total cost here. You can see how to define default value and items What can we say about our items without type? Not much Usually we need to dig through the code to find where and how it is initialized to find out It is a list or a dictionary or some other class instance and what methods we can run on it but By having well-defined types like this one We know for sure that this is a list of product class instance instances And if you want to say if you want to set an empty list as a default value We need to use field default factory like this one. Okay, that's all about entities Let's move on we'll implement some business logic called use cases Our order logic class Have three methods free use cases Create order for client ID and return order instance search for client orders and return them in a list and Finally our most important feature that earns us money add product order and return updated order instance But those are operations on a database. So should we start using database? It's a time to introduce Any guesses no an abstraction and This is pro tip number three Each time you have to communicate with outside world like database use an abstraction for that As you can see between our application and database We have an adapter and inside domain as I mentioned at the beginning We can define only how it should look like nothing more everything else is outside So in programming we create application Interfaces Sorry abstraction we create by creating interfaces like this one So we have a class I client repository where I stands for interface with two methods create and get and Both of them returns client data class To mark them as interfaces in Python views ABC package, which is a part of the standard library and Interface does nothing. So those function bodies are empty in exactly the same we implement prepare our iProduct repository and Of course at the end our order repository create Get and for the first time save we want to store some data and search With optional argument client as you can see we must strictly mark that our client's Argument is optional only by doing so we can pass non-values to it and We are returning of course list of orders that was found Okay, we define how we want to communicate with this outside world now We can go back to our logic or the logic class We need to pass all those interfaces in our init function So we can use them later But to mark to make our life easier. I use inject decorator Which is a part of injector package? that Does that for us? This is pro tip number four When we add an abstraction to our prior project It is good to be able to manage it somehow. This is the injector for This injector use dependency injector injection well known in type word and By using it we can configure how we want our instances interfaces to be created So we don't need to do that manually each time injector will do that for us So each time you want to create order logic instance injector will pass all those arguments and This is quite simple example with only three interfaces But imagine what will have what would happen if we will have over a dozen of those interfaces Creating all of the your instances managing them manually would be a terrible terrible Believe me, you don't want to be in that place Okay, so we have our interfaces we can now implement our first method search first we get our client from client repository and then We use it in orders repository to search and that's it search is ready create We've created we do the same get client object and search by client in and Create an order in order repository And as usual at the end our most important that product we get an order We get an order and product from these repositories and We want to add product to an order But what does strange construction fear as you remember we marked our data classes as frozen So we cannot modify them But we can replace them and we can do that with replace function first argument is our old instance And in quirks we pass only those arguments that we want to replace in our case we are adding new item and increasing total cost And of course at the end we want to use save to our new replaced order Okay, we did a lot of work wrote a lot of code added abstractions we use our dependency injections strange things in Python and All of that just to add single item to an order Why bother? Why do that things? Let's write some unit tests to find out We have our test name test at product increases total cost and This is pro tip number five use expressive names of tests Write them in a way That after reading all of them they will create a story a story Comprehensible for newbies and ignorance There are many benefits of doing so We are forced to create smaller test We can see where we made mistakes by only looking at failed test report and we create a living documentation of our application and To add product we need our interfaces. So using pie test. I create a fixture that returns static repositories And they are returning always the same object As you can see the return type of this couple are those repositories and I use alias for it and Passing this terrible Tappel to 20 or even more test would be a terrible ID You will start hating typing if you will do so. So aliases are made for such situations when we have such complicated tappels that we want to return or other complex objects and This is my pro tip number six. Aliases really works here. Well Okay, but let's go ahead and write this test already. I Personally hated writing test When I have to write three hundred lines of code just to test three or five real code Of our application of my application. It was a terrible nightmare. And if you heard someone says that he hates Units testing it is probably because he never see a test like this one Bump three lines of code That's all you need when you do clean architecture the right way First initialize logic Second call the logic and third validate the result We need a bunch of those to cover all corner cases of your application and other use cases This is really really easy and really nice to write Okay, our cases are written and well tested So now we need to implement our interfaces. We need to work on data So is it already the time for a database? No, no, no, no It's time for a repository and Wait for it memory Now we will literally prepare our own database implementation sounds easy We'll do that in a single class named RAM storage. It has internal storage type, which is a simple Dictionary Here you can see some strange type in construction when I first seen that my head went kaboom and So let's go step by step to figure out what it's going on First we declare T Which is a type variable so we can put some variable in place of T then we create storage type with integer keys and T values and Finally we inherit in our RAM storage from generic T What does that mean? When we create an instance of RAM storage like this one We'll replace all T in the first instance with client type and for the second instance with order So for the first insight inside this internal RAM storage We can only put variables of client type. We cannot mix them These guys this gives us a great control over what we are storing in our simple database and To be honest that was the scariest part of this talk, so I'm happy that you are still with me In our database, we want to implement such methods add Add an item to dictionary. You know how to do that, right? Get get an item from a dictionary for a given key search filter dictionary values Remove remove an item from the dictionary all return all values in form of a list and That's neat and that's all you need to create our own database implementation in Python basic dictionary operations So now using this simple storage we can implement our repositories reds. Let's start with product repository we have internal RAM storage of course, we mark them as product types and Inside get we just call RAM storage get get and Check if the result exists if it does return value if not rise product not found exception That's how we can implement all of our repositories But as you can see it is quite boring. So we will skip that part But I will provide you the link for the github repository so we can preview all of this later Okay interfaces check Now we'll expose our business logic to the world. We will prepare an API and And for this we can choose our framework and This is slide number 77 and protip number seven originally not now and The pro tip is don't choose your framework at very beginning now We know how our application looks like we know our use cases We know how we implement them and we can make an informed decision of what framework to choose My decision was flask and connection You probably familiar with flask, so let's go take a closer look at connection With connection you can use a young file to describe how your API will look like Here you have order search, which is a get method and it will be handled by operation ID our package and point search of course, we can have More description in here like input parameters response template and by doing all of that We can preview our API in a form of a nice UI Like this one in connection. We define how our API will look like Now I will show you the whole implementation of our endpoints in One slide. I only skip the imports nothing more Okay Search first argument logic second client ID. We want to return list in the order return Lodzy logic search client ID equals client ID. That's all Create first argument again logic here. We have second argument body This is how you received put and post body JSON data and body Logic create client ID equals body client ID Again, that's it and finally add product Again argument and return logic at product order ID product ID. This is the whole implementation of That connects API with our business logic Those more familiar with flask probably wonder where this logic Come from this order logic type because usually in endpoints. We received arguments that are passed from request Well, this argument is injected by injector To make our life easier. I use additional small package called flask injector It wraps all of our endpoint functions with inject decorator. So we don't need to do that manually To configure it. We need to pass our application and modules modules are a list of configurations of injections to configure such configuration of injections we need to inherit from modules from injector module and Overwrite configure method Inside it we use binder to bind order logic to order logic That's an easy case. So Let's take closer look to a more complex one Hey injector each time I Ask for a client repository place kindly give me an instance of client repository and Of course, this is our implementation in memory. So we want to share the state So we say The binder set the scope to singleton. So the instance will be created only once and That's it The second freaky freaky thing Is return type of each function in flask? We need to return data that is series a series Serializable dictionaries strings integers simple types order data classes Are not serializable. So how does it work? We do response serialization automatically Generally we can do that by inheriting from JSON encoder and Usually we are using that for saying that our date time instances or date instances Should be converted to nice ISO format So we can do the same with our data classes. For example client Each time We receive our client object will convert it to the nice dictionary Of course doing such thing for all of our entities in production application will be terrible So we can do that in a smart way only once Do you remember all of our entities inherits from base entity class that does nothing? but we can use it now and code code out our all of our entities in one place and convert them easily Okay, at the end we need to also remember to Call the default implementation because serialized serializers are called recursively Okay, that's all about API When we go back to this picture there is still One piece missing Borders Each color change is a border a border that's need to be protected We run tests to check if our code works we run black to check if our code code looks good and We also make do something to protect our borders and this is pro tip number 8 do it How? We can protect our borders by creating project structure that supports that Here our domain logic and interfaces are in the shop directory interface implementation and RAM storage is in RAM DB and finally flask in API But that's not enough. We can easily import from around DB or API in our shop in our application domain so we can go one step further and Create a package for each of those directories Now we have separate dot pi and what's most important separate requirements We can call them microlips so now we have 100% sure that no one will import the flask or RAM DB inside of our shop inside of our domain Of course, we can still add something to our requirements But adding Modifying requirements should be always considered as dangerous. So we usually keep Pay more attention to that But installing so many microlips in Dev environment can be painful. So that's why we can create Generic setup dot pi that can install them for us automatically and I will not show you that but you can also preview that in the github repo It is also there will be also a link for article where you can find how it was invented Okay, it is time for live demo. I hope it will go. Okay Small change Okay, this is our swagger UI that you seen before Here like a real backend developers. We can click and see how our response was defined and we can set client ID click try out and see that our URL was Executed and we have response body. We can see that Guido is one of our customers and he has one order With no items and total cost zero So now we can add some product to his order We have order ID one product ID one try out and Bum Guido has now a phone we can call to him Also total cost was updated as accepted and When we go back to our get and call it again, we see that this order is updated also here So we can try to add another product product ID to And we can see that we have also bought a nice new graphic card, which is quite expensive But what's most important that total cost has increased and simple math in our project works. That's great but what will happen when we try to break something like Real good testers Let's try something like this Hmm Response code 400 nice JSON message product not found element not found. How does that happen? As I promised, I was showing you all the code as you can see here are endpoints only Imports were missing but here is everything like in slides When we go to our ad product logic, it is also look familiar injector in it function ad product But when you go deeper In product to product interface one thing becomes apparent the return typed Thanks to spending some time on designing this architecture I design that all of those interfaces must always return an entity So inside the implementation If our product is not found we rise product not found exception. You see you saw that on the slides So how to convert this? Exception to nice json message. So we need to go to the our mine file where we define and Configure our flask API our flask API Here you can find the connection. We are adding this young file that you describes how our API look like Here we configure flask injector and the json encoder But one thing is new Here you can see error handler That code that checks if one of our endpoints doesn't end with element not found exception and if it does It convert it to nice connection problem object that renders this nice json message so That's it. The application is works. What are benefits? First of all business logic independence Hmm, we don't care probably about that but this implies very important thing to us is of technology update and change and This one give us the power to maintain our application over next 10 years We can easily update our framework. We can easily change our database as you can see We don't even have a database and it works So we don't need it. We can change it. You can update it. We can do everything we want with our technology Clear and secure borders and that's give us a power to maintain our ease of technology update Technology chosen based on knowledge as I mentioned We choose our framework at very end and we make conscious decision of what framework to choose Fast prototyping and proof of concept. I was creating this for like this second time this up This such kind of application and it took me like two hours only to prepare this demo and it works We can test it. We can prepare front-end for it and what is most important? We cannot send it to production Because if we run it in production and restart it with all begun because as you can see We have running the bugger here If we restart application If works and we refresh and it doesn't work client ID try out nothing the state was restarted So we cannot deploy this the production ease of testing you see three lines of code and We only installing the required Packages so this is important for example when we have workers we have API and We don't want our API to be installed on worker docker image or our worker Libraries in our API we can split them easily and to sum up for me architecture is a set of conscious decisions and Clean architecture is a way to import to Delay those important decisions further reading If you want to learn more about clean architecture, there's a great book by Robert C. Martin clean architecture and there is a Micro-lib article on medium that inspired us to create borders with If Python and I have one question for you, can I make some selfie with you? That's okay okay Thanks, thank you very much That's it for me One more thing if you have any suggestions won't ask me some question privately you can mail me you can tweet me I really will respond and please give me a feedback It is really helpful for the next presentation and here is this the link for the PDF with Slides and also the code Okay, thanks Thank you. Great Do you have question? So you say You start with with model and in the Python code and And then you choose your framework. Do you think Django will kind of go that way and integrate that Into its language and I mean after all one day Python 3.6 will be obsolete and then we can focus on 3.7 and do the same thing here And we have the choice. I mean we can go directly with Django and okay. It's nice to have flask in the end, but Yeah, what I mean could be the same solution with Django right away. Do you think that will happen? So two answers first of all, I don't start with model It is really important to remember that entities are Totally different thing that model For example, we had one entity that was saved in two different databases so we have one table and one Some kind of data table that were used to store only one single Entity so entities are not our models. It is different thing and for the Django part Well, it is difficult to use clean architecture in Django, but it is not impossible Well to prepare clean architecture in Django, you need to do some conversion between objects and Django or M models and you need to do the do it many times, but it is possible But it's not very easy Django is not Well prepared for that, but we have a bunch of other frameworks Not only flask, but also for example new framework fast API that looks really great and support sniping Hello, thank you for a good talk. I have a question about the order of your designing. I Would personally start from the API points and then Make any thoughts about that. Can you repeat because I didn't hear what middle part? Oh, yeah, so you started from designing Entity types and business logic. Yes, I would personally start from an API and points and if I'm not wrong this connection library can even define types of Input and output and validate them so maybe you can skip the Python typing part So that's why I mentioned the fast API framework because it do it nicer way You don't need to define your YAM file to define how your API will look like you will do that only by using type annotations So this is really cool feature that I like. I didn't try it yet but when I go back home, I will do that for sure and This will be much easier why I didn't start from creating an API is because it is easier to break rules of architecture when you start from API It is easy to break some borders So that's why we start from creating our business logic because we create it only using Python and nothing more We use injector, okay, but nothing more and we didn't use any extra requirements for that So there was no issue that for mistake. We import some global state from flask on etc The question is there a reason you don't put any methods on the data classes I mean you had this order class and then order logic but if you just like I don't know then you have product and product logic and As long as that does not get too big some small like like if it's only a couple of methods Do you usually put that there or well this this this is my personal experience? Why I didn't use any methods in data classes. I Had a really bad bad experience when we put a lot of logic inside our SQL alchemy models And it ends up really badly when we couldn't modify anything in our application And what was worse when we decided to move outside of SQL and to use some non SQL Database so it was really painful back then So we decided to also create this small border that we will not put any logic to our models it's because When we do so it's sometimes We sometimes want to do some hacks and add some extra methods Astrologic to to data classes because it's easier right now, but but but after Doing so we can end up with Very familiar with us leg legacy code That's how we create legacy code by Creating more methods in our data sets Time for a last question. Okay Hi does object not found exception exists in swagger declaration It by connection Now this This you ask about this. Well, this is the exception that I wrote and all the other Exceptions inherit from it. Yeah, and it is exist in swagger declaration. No, no, no, it is also it is in defined I believe in Yep, this is also defined in domain. So how our Exceptions are is this also the part of the mind domain and it's not connected to connection at all Okay, thanks Thank You Greg We can proceed with