 All right, so give a big hand for Ira. Hello everyone. I hope you are all enjoying this amazing conference. I'm sure I'm enjoying all the amazing talks. Today I'm going to speak to you about elegant exception handling. My name is Eyal Travelsi. I know it's hard to pronounce, so my Starbucks name is Jimmy. So about myself, I'm a software engineer at Salesforce. I have big passion for Python and data, and I love to share my knowledge in medium and on Twitter. So today I'm going to speak about how we can write safe code without compromising over maintainability and readability of the code. So I will show an example of a restaurant recommendation function. As you can see, we have a get restaurant recommendation function, we are extracting the configuration from a path that we are getting, then we are extracting the user. We are calculating the relevant restaurants, and then we are picking the best ones. So we can be proud of ourselves because we implemented our restaurant recommendation algorithm, and it's clean code, and it's highly maintainable. So why should we even care about exception handling? The obvious answer is that errors are everywhere. They can come from hardware and they will certainly come from your software. And it's simply unacceptable. We want our users to still be able to use our program. So the first lesson that I think you should take is that we should want to build a full tolerant system. So we have exceptions for that. And I'm going to briefly cover the exception anatomy before I'm going to show you the tricks and tips to write clean code. So exceptions as exception messages, tracebacks, and types. As you can see here, we have the type of the exception, the name error, we have the descriptive message, and we have the traceback, which allows us to understand from where the exception was propagated from, from which line and from which function. I will delve a bit more on exception types. So exception types help us distinguish between different exceptions, whether it's final found exception or a syntax error when our code is not syntactically correct. They are hierarchical in nature. There are dozens of built-in exceptions. And if they are not enough for us, we can create our own custom exceptions. So this is not the full list of the built-in exceptions. As you can see, it's hierarchical. We have in the top of the pyramid, base exceptions, and we have many exceptions like syntax error and so on. You don't need to remember all of these, but the common ones you probably should. So let's take the naive approach for exception handling. We are basically going to catch all the exceptions because we are catching base exception, which is in the top of the pyramid. We are then going to log it away and erase the same exact base exception. So now our code is clean and safe. But obviously, there are problems lurking. Since we catch the base exceptions, we are unintentionally catching many exceptions we didn't expect. For example, most programmers want their code to be at least syntactically correct. And obviously we are catching it instead of raising syntax error. In addition, exceptions are not distinguishable because we raise base exception no matter what. So if I'm going to try to recover from the invoker of the function, it will be hard for me to detect whether it was because of a missing file or because of syntax issue. And unfortunately, the naive code is very, very common. So we understand that the naive approach for exception handling won't be enough for us. So let's take the second try. So instead of catching the base exception, we are going to catch a specific exception. We're going to catch final found exception. We're going to catch base on the code error and key error. If I'm going to catch a final found exception and chase on the code error, I'm going to raise the same exact error. So here I'm going to raise final found exception and here chase on the code error. And when I have a key error, I'm going to use a default user as a recovering mechanism. In doing so, the invoker of this function can recover from final found exception and chase on the code error in a different manner. So the second lesson is we should catch relevant exceptions only and that different obligated exception should be distinguishable. This will make our code much safer than the naive approach. And then we make it a little bit cleaner. And obviously, the answer is yes. So in this talk, I'm not going to cover all the type of cleaning that can be done. I'm not going to cover the pay paid object oriented programming and so forth. But let's begin. And I will show you some useful techniques. So in our example, both the final found exception and chase on the code error are handled in the same manner, which mean they can share the same exception block. In addition, instead of key error, I can use dictionary get method. And by doing this, though, I'm avoiding another few lines. Lastly, I can use the else clouds, which call only if no exception is being obligated. And by that, I know that the exceptions can only occur in this function. So this is a bit cleaner than the previous code. And that's the teachers that we should use Python syntax to the fullest. But obviously, our code is still not elegant. It's dominated by exception handling, and the business logic is not clear. And we should really try to make it clearer in order to for the code to be maintainable. So I don't want you to give up. I really believe that error one link should not obscure our business logic. And I will show you what we can do further to get there. So we should separate the business logic from the exception and code. And we can do it by handling the exceptions in another layer. So I will remind you how the perfect code look like. And I will show you how we can reach something similar to it. So as you can see, we have to get the restaurant recommendation function, and we have to get config function. So if I want to make this part of code cleaner, the business logic, I came, I can simply push the exception and include different place. And by that, I'm making the business logic much cleaner. So obviously, the recovery mechanism depends on the use case here specifically. I put it in the get config function. And if I want to make it a bit further clean, I can make the get config to a get user function. Doing something like this. So obviously, this is much cleaner without compromising on the safety of the code. So we should really try to pick the right obstruction level to handle the exceptions. But are we completely safe, we have some exception and encode, and the code is quite clean. Obviously, we are still not completely safe because we have silent errors. And silent errors are errors that do not crush our code, but they do deliver incorrect results, which make them harder to detect and make them much worse. So the question is, how can we detect those error? And the answer is simply by using validations. We have many type of validations that we can check. We have output and input type and values, we have preconditions and post conditions, side effects and invariants. And when we find an issue with the code when we validate and fail, we should simply fail fast. So we won't find the issue later on. So I'm going to briefly cover some techniques for validations. We can use vanilla exceptions, we can use typents, and we can use contract testing libraries. Obviously, the best solution is to combine all of these depending on the use case. So vanilla exceptions, we look from thing as follow. We have our function and we are adding an if clause that validates the type of the input and the type of the output and raise a type error if there will occur. And in addition, I could have checked if it's valid at three parts and by that raising a value error. So obviously we can validate everything with vanilla exceptions. To use validation will occur only on runtime and not on compile time. And unfortunately, it's not the cleanest code. So I'm going to briefly speak why I didn't use assertion here because it would save me a one liner. And basically because it raises the wrong exception type. So what the issue is, we're going to raise an assertion error, which will make our life much harder to recover. So it will be hard for me to understand if the issue was a type error or a value error, for example. Another small issue is that they can be compiled away. The second technique is typents. So this is an example for the same code. As you can see, simply, I can add typings which tells which type and which type the output is. And just by doing that, we can validate the input and output type, but we can do other validations. We can check it both on compile time using NumPy, MyPy, and on runtime using this decorator, the type checked of type guard library. And this is quite elegant. Another technique is contract testing the libraries. As you can see, it's pretty similar to the previous slide. I can validate if it's a valid S3 path just by using this decorator. So obviously, all the validations are supported. They occur only on runtime and not compile time. It's clean and elegant, but unfortunately, to my knowledge, there is no mature and maintained option for contracts testing library. So you should be really careful when you pick your own solution. But there are still problems lurking around. Even for a simple function like to get relevant restaurants, which basically a get request to an imaginary URL. There are a lot of exceptions that can occur. So our app might live in an unstable environment. And by that I mean that my network can be down, the server network can be down, the server can be too busy, and many more issues. So one technique to make our code more resilient to flaky environment is to do a retry mechanism. Here is a naive implementation of retry mechanism. Simply I'm going to iterate five times on the same request. And if I'm getting an exception that is corresponding to a flaky environment, I'm going to retry unless I reach the maximum allowed retries. So obviously this is much safer again, but it's not clean and it's obscure our logic. So there must be a better way. And obviously there is. So Python has a lot of patterns that are really common. We have decorators and we have context managers. I'm not going to cover them too much because it deserves its own talk. But obviously the most common use cases are already implemented in the standard library or in other third party libraries. So in our example, we can use the tenacity library to do a retry mechanism. And by just using external library, we are reducing the amount of bugs and we are getting more features. As you can see, this is much cleaner. So we should use patterns for better code reuse. So what I'm going to speak about next. So I'm going to delve a little bit more into exception types. As a reminder, exception types help us distinguish between different exceptions. They help us emphasize our intent and we have both built-ins and custom exceptions. So I'm going to speak about when we should use custom exceptions or built-in exceptions. So in my point of view, and it's very controversial topic, the entire slide is, is that we should use default exceptions by default. Because they are familiar, they are well documented, we can do some Stack Overflow magic with them. But obviously we have custom exceptions for a reason. The reason is that sometimes built-in exceptions does not suffice. And these are the cases when there is no built-in exceptions that emphasize our intent. For example, if I want to emphasize that I have a configuration file and it's malformed, I might create a malformed exception type as opposed to use some value level. In addition, if I want to distinguish between different exceptions, for example, if I want to recover differently from a value error that is too big and too small, I might create a new exception type. And when I want to group together different exceptions to make recovery easier. So I can use multiple inheritance to make the recovery much cleaner. And lastly, when I'm going to wrap third-party APIs, and I'm going to show it in the next slide. So when we are wrapping third-party APIs, we want to minimize our dependency. I'm going to show you an example here. So let's say I have the Get Relevant Restaurant function, which can raise requested timeout. If I want to recover from this function, this means I need to import a request in this function. So in this case, I know in this function which implementation was used in the Get Relevant Restaurant. And if in some case the Get Relevant Restaurant will change the implementation, I will need to change my code as well, which is obviously bad. Another cool thing about that it has cause. So the exception cause indicates the reason of the exception. And we can override it to replace the exception that is being raised. I will show it. It will be much easier to understand. So let's say we have zero division. And I'm doing some amazing recovery mechanism, and I'm failing. So by doing so, I can raise the zero division error. But let's say I want to emphasize that my recovery mechanism actually failed. So I can do something as follow. And as you can see, I will have the two tracebacks and two exceptions here. If on the other hand, I just want to show that my amazing recovery mechanism failed, I can do something as follow. And as you can see, I'm raising the exception. Obviously raising the exception as like this is a bad practice. And you should pick or create your own custom exception in this case. So we should really thrive to pick the right exception type and message to emphasize our intent and to help us recover. A small thing about sensitive information. So our exceptions gonna be spread far and wide. It's gonna be in the login login system in the monitoring system in the incident system and even on Slack. So we should be really careful when we are handling personal data due to many reasons like compliance. And we should never reveal our own weaknesses because there are better actors everywhere. So you could never be too careful. A fun example for that is that if I'm going to do a login and I'm failing because of a two common password. I don't expect anyone to write the password in the exception message. There are some common gotchas for exception and link in an item that I'm going to cover. So exceptions as block order. And by that I mean that when an exception is being raised, it's gonna try to match the exception type from top to bottom. By the way, value error is both exception and a value error. So in this specific example, we would expect to get a value error because it's the most specific exception. But obviously we are getting an exception. So we should write the exception exceptions from the most specific ones to the more generic ones. Another common gotchas is the not implemented and not implemented error. So not implemented error is a valid Python exception and not implemented is simply a constant in Python. So when we are raising and not implemented a constant, it tells us that it's a type error because we are trying to raise something that is not an exception. It tells us that we should raise something that derives from base exception. As we said, it's in top of the pyramid. Another catcher is return in the finally block. So let's say we have this try block, we have some exception and link. So when we see the return, we think we don't need to read further the function, but obviously we are getting the surprising result. So finally occur no matter what in the try accept block. So you should be really careful when you are returning from the finally, if we would have a return block, it would work in the manner that we thought. And that's about exception and link catchers. But this sounds like a lot of work. Should I do all of these, no matter what type of application do I have. Obviously, not all programs are made equal. We have some programs that need to be extremely reliable, like airplanes and electricity factories. We have systems that we want to be highly reliable, like our drive less cars. We have reliable systems body systems and basically car. So for the first lessons that we learned, we want to build a full tolerant system to a certain degree, depending on the use case. So let's remind ourselves the lessons. We want to build a full tolerant system to a certain degree. We should catch relevant exceptions only. We should make different exception distinguishable. We should give Python syntax to the fullest. Error handling should not obscure our business logic. We should pick the right obstruction level for handling our exceptions. We should validate and fail fast. We should use patterns for better code reuse. We should pick the right exception types and messages to emphasize our intent and to help us to recover it from it. We should not use sensitive information. And there are some exception handling gratchets. So there are some topics that I didn't cover. Error codes and when we should use error codes as opposed to exceptions, the functional approach for exception handling and its benefits. How we can avoid some error using domain driven designs. So we should make sure that we have the right exception architecture patterns for resiliency. Feel free to ask any questions now or later. And I hope you enjoy the rest of the amazing talks in this amazing conference. Thank you. Thank you for this all. All right, so we got a question here. You mentioned earlier to use the built-in exceptions when possible. And so like what are the advantages of using one of the customer, one of the custom exceptions instead of one of our specific ones. So I briefly covered it. Obviously, if I want to recover, I will give you an example. Let's say I'm an account manager and I can only work on companies that are between 100 and 200 people. So if it's bigger, I might move it to a different team. And if it's smaller, I might just ignore this customer. So basically, if I want to distinguish between different exceptions, this is one reason that I would pick to create a custom exception. Another one is third-party APIs that I showed. And lastly, if I want to emphasize my intent. So if I want to emphasize that my config file is one form, I can't use any of the built-in exception because it's very specific. Right, cool. Thank you. Is the iContract package reliable for production? So probably not. What I do specifically is just to write my own decorator because the iContract is not, it's not very stable and mature. Unfortunately, if any one of you know about a contract testing level that is more mature and more stable, please share it with me. But as I said, it's pretty easy to just write this specific decorator and just simply use it. Alright, thank you very much. Those are all the questions that we have. Amazing. Thank you very much. Thank you very much.