 People from Germany might know you from the that you're the organizer of the Pi Berlin Meetups and that you've been around Picon DE as well a lot and of course we've seen you at other Europe items as well Today you want to talk about how to make your code production ready. Yes that's a super interesting topic because When you do stuff in production like now things can break all the time So how to make sure that that works is interesting for us. Are you ready to do your screen share? Yes So actually I'm going to talk about production ready code and whether our code is ready to be deployed somewhere or not A few words about myself. Thank you for a nice introduction. I'm basically Connecting from Berlin and I'm pi Berlin organizers since already a year and a half. I have 10 years in software development And seven years in Python. Actually, there isn't very interesting story, which I will quickly share before to start I'm here in Python because of the community most of the time On the conferences and I'm super happy that once our customer I was a former C++ developer and One once customers said that now you are a Python developer. You have to learn Python. I Didn't understand how great was it because back then I was in Ukraine and there was not much of a community thing in Python at all. It was like C++ Java and Different meetups, but not Python at all. And then when I moved to Berlin I realized that there is so much to learn and to share with the community. Everyone is so supportive So I'm super happy that I'm here right now and thank you for all the support. I know it's remote I don't see anyone here right now, but I can feel this support. Thank you so much You're all invited to the next pi Berlin meetup And if you're willing to speak for us, just feel free to drop by just a message or maybe just join the pi Berlin channel So what exactly does production mean to you? What do you mean by production ready code? What's the difference of production ready and non production ready code? Actually, I read the book release it it's great and there is a favorite quote of this book as You work so hard on your project. It looks like all the features are actually complete and most even have tests Can you can breathe a side of relief? You're done. Or are you? There is no checklist What to Go through on your code if it's ready for production or not. Can you deploy already to your customers? So I prepared some checklist for you And before starting that I would like to emphasize that the only difference between Production and not production code is that there is a customer behind the computer or behind of the device Who is trying to run your code? The difference is that? We are developers. We know how our code is supposed to work We know where to click we know what to do on the website or the application or whatever we are building But the customer doesn't know the only difference that Code could break and it will break. We have to make sure that we know how to investigate how to find the problem and fix it how to provide a solution to our customers and Here are the Checkpoints what to check before going to production. So first of all, I would talk about exception handling There was also a very good talk yesterday about exception handling handling and I will also share some tips about that and then how Actually not to become a detective to find your problems in the code how to make your login meaningful then How to set up your CI CD pipelines I prepared some beautiful examples And then how to secure your Docker images After that, I will show some links then you can follow later on So let's start with exceptions We don't really want to show to our customers 500 error because it's like you can see this error and then you know that something is wrong on the Server side, but as you're a customer you cannot follow up on that you can contact the Administrator of the website Usually there is an email to contact and then what to do with that. We don't really want to share this information with the customer We want the customer to see some Proper exception handling if something went wrong then they would know what to do how to follow up for that We are usually trying to handle exceptions in The manner that we are either silencing them or maybe just using the try accept block To catch the exception and then printing something on our side on the server side to see So what's the proper way of handling exceptions? We can try to catch exceptions and the basic exceptions catching exceptions by official Definition of exceptions in the official documentation is not a really good idea Because it's too broad and catching the basic exception is even dangerous So it's highly not recommended to catch any base exceptions So guess what will be printed here if you are trying to get some input From the terminal you run this code and then you're trying to catch exception and the base exception Then you are interrupting your script and then what would be printed next? You will get the base exception printed on the screen that keyboard interrupt Happened why is it happening? Let's take a look into hierarchy of the exceptions. It's way longer. I just cropped the first top section So basically base exception has more than exception. That's why it's not recommended to handle to Try to catch base exception because you might catch More than just exception which you're trying to catch you will get also keyboard interrupt system exit and generator exit So in this case we interrupted our script and then we got keyboard interrupt in this case if you will silent the exception then What will happen next? Highly not recommended to use base exceptions. So how to actually handle exceptions? Let's try to take a look at this example We're trying to catch exception of the exception class and then we will print some exception on the screen and then We need to change message of the exception to have something different like my custom message And then we will see on the screen that you're handling the above exception another exception occurred That's not the ideal way because you were trying to handle exception and then you failed by while handling the exception That could happen also if you're trying to do some actions Which might throw the exception on the handling, but this is super high high not recommended So what can we do if we really need to change the exception message we can use from At the last line you can see that we are trying to rise the exception with the custom message So it's gonna be a different exception, but we're rising it from exception with which we already got So at the screen we will see the above exception was the direct cause of the following exception And then you can see your corrector is back You will see the previous exception and the current exception and then you will see your awesome printed message The best way to handle exceptions is to be more specific on the exceptions For example, you can create your custom exceptions with the custom message And then you can rise your custom exceptions whenever it's needed You can try to catch your custom exceptions or specific exceptions to be more specific not to use a Broad version of just exception handling of exception and then you can still print them on the screen Last year at Europe item in Basel. I saw a talk From Mario Cajero exceptional exceptions. It was really really good. There are more tips to learn And you can see the video from pike on us in here highly recommended to watch it and to go through all of the tips The next step is logging how to make your login meaningful Let's take a look. What are the logs in the 12 factor app? Which was super popular a few years ago. Everyone was talking about 12 factor app Nowadays, I heard some docs about that. Some people mentioned that but not that much anymore but actually they have really good content and Feel free to go to the 12 factor app and check the locks section. So they're saying that We have to treat logs as event streams and logs provide visibility into the behavior of a rainy application So let's take a look at the basics of logs. What should log and clout I Did some research and then I found out that the main login attributes are when or where what who and the outcome itself so When would be the just a timestamp when the log entry Happened then where did it happen? Maybe passed to the file Then what actually happened if it was exception or maybe just information Then who did this action if there is a user then maybe some kind of ID information But no customer information at all I make sure that you're following all the GDPR principles and you're not logging any customer information and Then also important to have an outcome the message itself. So how can we improve our logs? Usually we are having this type of logs We are using the standard logger and then we are putting everything into a one lock message So I added here some important information about the conference name the talk name and also some random key ID After printing this we will see something like this. So then we have a log level We have where this log message happened We can also add the timestamp super easily and then we have the entire message This is not super useful if we're trying to find out what actually happened on our system So, how can we improve that and how can we make the log message? more Parcable for devops team if we have a devops team or we are our own Devopses and we're trying to build some boards on the top of the log messages in this case. We cannot really check by the conference name because For example, if we separate by commas the log message, then we will still get Some random stuff if for example in the talk name we have comma in the inside of the talk name if we try to separate by I don't know some other ID or maybe we will have ID friendly Then we need to make sure that everyone uses the same structure of the locks the same Quality of the locks, which is quite hard to establish So what actually I tried in my teams? We use the struct log for that There are also great talks about the struct lock and there are lots of examples in official documentation I will show you some examples Which I use myself So super easy you just get the logger instance and then you try to use the same We are trying to use the same Information from the previous log. So we will have also key ID conference name and talk name But the structure in here is a key value pair What will we get actually we will have the lock message and then Keywords we can easily parse by the name of the keyword and then we can get the value To build the dashboards on that which will show us way more than just Going through millions of logs From just our servers So yes, definitely recommended to use the struct lock And let's take a look at the features of it For example, here we can Bind some important keys to every log entry in this case For example, I have the entire big file of with all the logs log entries like info exception Whatever we need to log and then I need to have conference name talk name and key id in every log entry Obviously, I would not write like just copy paste in the entire line To every log message It's possible to use just bind and then in every log message we will have the same keywords already there So we just don't need to specify them over and over again through the all log messages Then we will get The same exception handling and then we will see our beautiful log message For the development we can have this highlight on We can use colorama for that and for Servers we just don't need that Also, we can use chase and format for logs Then we will see event as a message of the log Then we will see the euro python custom log level I will show that in a minute and then the name of the logger and the time step and many more So let's show me let me show you some code I hope you can see my scream with the pie charm So This is a basic idea Then let's take a look at the processors So what do we have in here? We have Some password In the log message which we wanted to print, but actually we don't want to have any customer information in here Because of that we can use a super custom Processor we can write the processor ourselves and then after that we will have the password censored We can also use tracing from the struct log we can For example Pass the trace ID from one service to the other one and then just add it to the log message Which would show us How this actually error transferred from one message to the other one And also many more features you can add so many processors and features into this one But we are running out of time. So I want to show you a bit More of the front features. I wrote everything in the blog post. I will share links later on so you can follow all the ideas later on And this is a really good talk About the struct log with more examples Effective CI CD. I prepared also some examples for you and I will share them after this talk Continuous integration and why do we need it? It will provide you the test coverage, reliability, fault Isolation, transparency, code quality, faster development and code review improvements. You can automate everything There are different examples of how to integrate your checks into the CI And I will show you some super tiny example with the github actions, which is super easy simple to set up You just go to actions and then you click on the Specific CI which is already proposed and then you can edit it easily. I have chosen the python application setup Here's the link to The test setup you can check it out afterwards and I will show you how it looks At a moment. So basically we have github workflows and then we can have our beautiful workflow I added everything in here. The only difference between Those different entries is this continue on error if you want to make sure that developer cannot really merge the code Before the CI is passing you have to delete this line continue on error But if you don't really care you just want to see how it looks on the CI then you can just Continue on error and run all of the checks before they're fixed But then we cannot really make sure that CI is actually passing So in here we have the CI Part and then here are all the jobs Basically, this one is the only one which is mandatory for my code and all the other ones are not really passing because I didn't really care to fix Just to show you the difference Uh, let's check the py lint. Py lint is not passing because there is something which is missing in here Also, I sort is not passing but actually my code was already merged because I didn't make that mandatory So make sure that those steps which are important for you to check are mandatory and there is also really nice too, which is called Documentation coverage tool interrogate You can check that a bit later. I don't have any documentation right now So it's obviously failing because actual result is zero You can use it like this interrogate and then just specify what do you want to check if there's enough documentation Then the docker file Just a couple of words about the docker file how to secure your docker file and docker images I also prepared a lot of examples, which you can check later on after the talk so The basic advice do not you use the root user Then I use trusted well known images just check on the docker hub if the image has enough downloads And it's an official source Then use copy instead of add um, although they have super similar functionality, but then Add can have extra functionality, which you will not expect. So make sure that you're using copy And then try to link your docker files. You can use that on the ci. It's super easy and simple I use one tool which is called Haddolint Uh, it's a docker file linter. It will link your code super easily. It will tell you what's wrong in the code This is just an example for them. It's super easy to set up on the ci Uh, so in this docker file, I made the mistake I did the peep install and then just a package name. I didn't pin the dependencies and because of that my linter um Advised me to change that whenever I changed and it was passing also, uh, this linter will let you know whenever you're using um add instead of copy so you don't need to worry about that as well and, um, the last but not least you can also check vulnerabilities in your code. I didn't try it myself um But I will definitely try and I saw that there is already interesting tool which is sort of for free and I'm Going to give it a try um, and a bit more hints how to use Not the root user in the docker files. You need to create a user. You need to create the group and why do we actually do that all? Because we would like to follow the principle of the least privilege here and that means that we should give access only to resources To perform their required functions. So if they need to perform Perform the just a function with no root user. Obviously, we will not give them root access um, and I have Some recommendation of the books For the further reading And also you can find all the links here in in my, um personal blog So thank you for listening and I believe we don't have so much time for questions No, actually, we are very good on time at the moment, but I don't see many questions at the moment because uh in the chat Uh, no one has asked any. I think you just gave them an overview so that they have the time to think about this Uh, I really liked the docker tips that you gave and there was also another docker talk yesterday So if somebody is into docker, then they should look at the other talks as well So it was a really good talk. You highly recommended to check it out as well. Yes So, uh, the only thing that's left to me is to give you a round of applause