 Thank you and hello and welcome to my talk about solution-oriented error handling, which is a set of a simple set of guidelines to use the mechanisms of Python for an efficient way of error handling. It does work with the standard libraries. You don't need any extensions. It has, it yields a small amount of dedicated error handling code and most of your code can focus on producing the results on producing the stuff your users actually want. It yields helpful error messages that are easy to derive from the code and it consequently simplifies support in a way that users might be able to deal with more errors themselves without opening tickets or even if you get the ticket and the error messages in the ticket you get more information and better information to fix it yourself and support the user. What I'm talking about now is I give a short introduction about errors in general. Then I give an overview of errors in Python. How does Python represent errors with exception handling? How does Python clean up resources after errors? Then I'm going to focus on the principles of solution-oriented error handling with a dedicated sections on error messages and error hierarchies or exception hierarchies and I'm summarizing the whole thing at the end with a couple of recommendations and guidelines you can apply to your code. So about errors, what are errors? Errors in the context I'm talking about are conditions that prevent a program from producing the desired result. So where do errors come from? That was a user error. Oh, it quit. That's bad. Yeah. They don't have any error handling or no proper error. No, that was the system error handling. So I don't know what's wrong. I don't know what to report now. All I can do is restart it. Okay. So at least it could recover from the error. But now it's talking German to you, which is inefficient. And Here we go. We have a program that's supposed to produce a result. Where does the program come from? The program comes from you to developer. You write the program. You code it. But you don't usually don't code it for yourself. You code it for a user who wants the result. That's why he comes to you and says I want something useful please make it for me. However, the user usually doesn't only want to program. He wants to provide input and the program should create the result depending on his input. Additionally to that you need an environment for the program to run in. And the environment sometimes is provided by users. Sometimes it's provided by dedicated people. We usually call operations. So that's the whole thing we do basically. So how can errors creep into the whole concept? Well, the users can provide broken input. You might use invalidate formats. They might use values out of range. Operators can provide insufficient environment where their access rights missing where you cannot connect to a network where a disk is full and developers can write the program that is broken in the sense that it misses corner cases that it has incorrect results or as we just seen that has insufficient error handling like LibreOffice. So how do errors get fixed? Well, in case of LibreOffice if this ever gets fixed somebody will sit down and write a better program. So it's pretty much the same with input data and the environment errors get fixed by people. They add more memory. They add valid or they change the input data so that they are valid. After that the program can continue and finally produce the desired result. For people to actually fix errors, they need information. And ideally the program provides this information, which LibreOffice didn't. Actually, ideally the developer writes the program in a way so that it provides this information and this is what this talk is about. Solution-oriented error handling is a way to make it easier for you to provide the information to fix errors. Okay, errors in Python, the widely used way to represent errors are exceptions. You can do funny values, you can do global variables, but essentially and consistently the standard library uses exceptions to represent errors and I encourage you to do the same. The major benefit of exceptions is that they cannot be ignored accidentally. You don't have to check if they are an error. Once an exception is raised, it just traverses up the calling stack until some routine decides to deal with the error. The Python program cannot simply continue after an error already occurred. In C, this is pretty simple. So how do you detect errors? Well, you use a if and the error condition and if the condition is true, you know, you have an error and then you just raise an exception. Here's a trivial example. I have a height parameter and if it's less than zero or it is zero, the exception tells me that the height must be greater than zero. What can I do once an exception is raised? Well, I can handle it using the accept clause. In this case, I try my routine, pass an invalid value, get an exception and if I print the exception, I get the error message on the screen. In this case, I would get the result height is minus three, but must be greater than zero. When I want to clean up after errors, Python has several mechanisms. Here's a simple one or the first one. In case of a file, I try to open the file. If it works out, I try to process the data I read from the file and finally, I just clean up by closing the file. The interesting thing about that, finally, closes the file independent if there's an error or not. It will always close the file. I could also use the with statement for that. It has a little bit different syntax, less code and gives the same result. In order for the with statement to work, you need a context manager. So the class you use, in this case, file object, needs to provide a context manager. I'm going to rush through this a bit. I can simply add a context manager on a fly using the closing routine, which is very useful in Python 2, less in Python 3, because most of the standard objects already have a context manager. But in this case, I can add one by calling the result in a closing. When you want to write your own context manager, then you have to add basically two standard routines to a class. Here's a very simple example. This is a task to copy a file. The initial part just allocates the source file and the target file to open it. There is a run part which runs the task and it simply copies the bytes from one file to the other. And there is a cleanup routine called close, which closes the two files. I said naïve variant because it doesn't do any error handling yet. I could use this naïve variant the way it is with a try and finally to open a copy task, run it and execute it and clean up the resources afterwards. If I want to do that with a context manager and the with clause, I just have to add two routines. Enter and exit. And they will always look like that. The context manager can do a lot of things, but if you just want to use it in the with clause, there's nothing more to it. As soon as you have a close and an initializer that allocates all the resources and the close routine that gives all of them back, that's the context manager. These four lines of code. And once you edit these, you can use it with the with clause. What's happening here? The with clause basically calls the init function at the part where you create the object. Once you assign it to a variable, it calls the enter function and assigns the value of the result of enter to the variable. Once the whole with clause ends, it calls the exit function. That's basically what context managers do. And I think it's a very simple way to not bother much about giving back resources. Here's an example of improved context manager because the initial variant basically only allocated the resources. And you can run in a case when you allocate multiple resources that the initializer already breaks after allocating the source files, but it cannot open the target file and you want to make sure to give back the target file. So you basically remember all the resources initialized and with none and make the close routine a bit smarter, only close those files that are not none. And the init also checks for errors and if there is an error, it calls the close beforehand. It's a pretty simple principle. And then there's another statement to detect errors. We already talked about race. There also is assert. How many people know the assert statement? I have used it already. How many people think they know when to use assert and when to use race? I would say it was about a third who know about them and about 10% who think they know when to use it properly. What is the assert statement? Well, I could also detect an error using assert. It's again the previous example with the race. I can do the same with assert by just specifying condition. In this case, it's not the error condition, but the condition for the correct state. Hate must be greater than zero and provide an error message, which is optional in this case. There is a couple of differences with assert. Internally, it just raises an assertion error. So to some extent, it is comparable with race. However, you can deactivate it. If you run Python from the command line with the minus O switch, not the digit zero, it deactivates the checking of assertions, which might increase performance a little and might remove consistency checks. Functions called by assert therefore must not have any side effects because otherwise the program starts to behave differently whether minus O is specified or not. And the question now is when do we use assert? This is something I'm going to talk about a little bit later. So in summary, the gist of Python's exception handling is that you can detect errors with race or assert. You can deal with errors using try and accept, and you can clean up using finally closing or context managers. These are the tools Python gives you when you install it. Everything of this is already there. So the interesting part is now how do I use all this in a solution-oriented way? The principles of solution-oriented error handling are that you focus on the solution and not on what's wrong. You have clear responsibilities between developers and the user and operator. You have helpful and specific error messages, and error conditions and error messages can be derived from the code. So it is a developer-centric approach. When I talk about responsibilities, let's take a look at the graphics that we had before. The responsibility of the user and the operators is pretty clear. They have to provide sufficient, complete and valid input for the program to produce the result. The operators have to provide an environment where the program has everything it needs, concerning access rights, memory and so on. And the developer has the responsibility to write the program. So that's something important to remember. All we can do is write the program. We cannot add memory at the user's site. We cannot add or change input so that it works. The user has to do that. However, we also have another responsibility. If the program cannot produce the result, we have to provide helpful information to the user, or we have to make the program to provide helpful information to the user so that they can fix any errors on site. How can we do that using Assert? Well, Assert is a tool to detect errors by checking the internal program state. Ideally, it detects errors before the program is delivered. So it's a tool for the developer. And if an assertion fails, the developer has to fix the error by changing the code. Maybe he has to govern new requirements and whatnot. But at the end of the day, he will have to change the code for the assertion to go away. So the target audience for the assertion clearly is the developer and assertions are there to detect bugs in the program. Assertions are a very useful tool to establish clear responsibilities between caller and calling routines. You can particularly use it to validate parameters and point out invalid parameters. So that's something in Python where I use it a lot. This is called preconditions. Assertions provide execute table documentation. So it's better to write assertions, which actually check if a parameter is correct than to write documentation nobody reads, and that might get out of date. Assertions are executed every time the program runs. So once the assertion gets out of date, you have to fix the assertion. On the other hand, rays, you can use to detect errors in the data and environment in things the user and the operators can solve. So they can provide valid data, complete input, or they can provide a sufficient environment with enough memory disk space and network connections. Rays is something to point out conditions the developer cannot deal with at the time he writes the program. He has no influence on these conditions. The target audience clearly are the users and the operators. So a couple of words on wording error messages. Well, you have a lot of usability guidelines and books about it written by usability people, but very often they are too vague for developers to actually turn them into code. So error messages should be polite, precise, constructive. Well, how do I actually do that? And in practice, you see that many error messages talk a lot about the things that are wrong and not about what should I do, what's actually precisely wrong, what can I do, what are my possibilities. So the solution oriented approach to error messages is that they describe the actual state and value and the one that would have been expected. They describe the actions necessary to solve the error and they describe the actions that led to an error to provide context where the error comes from. So how does it look in terms of code? Oh, I was one slide ahead. How does it look in terms of code? Generally, code to detect errors looks like that. If an actual value is not the expected value, then I raise some error with the message, actual must be expected. So I already have these in my code. Here's an example. We already know it. If hate is less or smaller than zero, I raise a value error, which says hate must be greater than zero. How do I describe the solution? Well, I already did. I already told them hate must be greater than zero. So from this error message, the user just has to do whatever is necessary to establish the expected condition. For that, of course, he has to understand the program and how to use it. A couple of more examples for such error messages. Date of birth must match a certain date format. The option color in the section layout of the config file must be one of red, green, blue. So how do I describe the action and the value that led to the error? When I raise an error, I just specify the value I used in my error condition. So in my case, hate had the value minus zero. And I just add it to the error message. The error message says hate is zero, but must be greater than, hate is minus three, but must be greater than zero. When I get an error already from a calling routine and I catch it with the accept statement, I don't know what the initial condition is. So I just have to rely on the initial message. However, I can add the context in which the error cured. So I just describe the action that could not be performed. Typically, I cannot process. I cannot connect to a host. I cannot write to a file. I cannot use the value of a form in a certain field. So I describe the item. And in this message, I have to use the terms, the user knows. Not my variable names, which ideally are the same, but something the user knows. And then all the information in the error message is something the user should be able to deal with, provided he has enough knowledge about the software and how to use it. Let's take a look at an example. We already know this routine. It is process something and gets a hate variable or a hate parameter and raises an exception if it is not in the valid range. Here's a routine that would process hate values. It gets from file. So basically it opens a file, reads a line, converts it to integer and checks for errors. If everything goes wrong, if everything goes right, there is no exception. If there is an exception, it writes an error message, which says cannot process data in file and the file name, line and the line number and the actual error message it got in the value error. And in summary, the output of this, in case of error would be, cannot process data file in some text, line 17, hate is minus three, but must be greater than zero. And basically you get all this for free from your code you already wrote. So in summary, a pattern for error messages is cannot perform an action at a certain location. Something is actual but must be expected. And you should have all of this available in your code. A couple of words on exception hierarchy to make it easier for you. Exceptions are classes in Python that are organized in a hierarchy, which allows you to collect similar exceptions in groups and deal with them in the same way. So we can use this to assign responsibilities. It's already pretty clear that developers should fix assertion errors. In my experience, it's pretty clear that users and operators should fix OS errors. However, there is a quite large amount of exceptions where you can't really say who's responsible to fix it. So it's your task to make this more specific. In my experience, I usually just define a data error because that's the errors about user input users can fix. And when I detect something invalid, I just raise a data error. Exceptions raised by other routines, value error, index error, lookup error very often indicate at the data error. So I just catch them with accept and re-race them as data error again. Here's an example how to define a data error pretty easy in Python. How do I convert value error to a data error? I just catch the value error and raise the data error where I leave the original message intact. In Python 3, you can also preserve the stack trace by using exception chaining. Here's a little example for that. So I use a solution oriented exception hierarchy basically comes down to users fix data error and OS error and the developer fixes everything else in particular assertion error. A couple of recommendations for accept and raise. When do I use accept? You have to use it at the utmost layer of your main routine and print an error message or do something useful with every exception. In graphical applications you should use it for every enclosed user operation or action pattern as developers call it. If enclosed operation fails, you can say this operation failed and you have to establish this or that condition. So I use exception to data error as I've just shown to make it clear who's responsible and you can use accept to convert errors into valid states which is a bit of a risky principle but where it works reasonably well is for example to provide default values for things that are simply not there. So generally you hardly ever use accept and when you use it in a tightly focused way which means little work for the developer and which also means that the error handling in your code becomes very easy because most of the time you just clean up if there's an error you don't care because you can't fix it. The user has to fix it, the operator has to fix it. So you just clean up and it traverses upwards the stack. Eventually the caller one of the calling routines will decide how to deal with it and you don't have to care about producing the result because the exception mechanism already stops the productive part of your code and the benefit of all this is you have a clean code focusing on producing the result and it doesn't intermingle with try accept and what not. When do I detect an error? Well it's pretty easy if you use a natural naming in your code it means the routines in your code have the same name as the things they are supposed to return because then an error condition is when it cannot do the thing it's supposed to perform as the name suggests or it cannot return the result it's supposed to return that's an error condition and in that case you just raise or as a okay here's a small template for a command line application how to apply that the most interesting part is here you have a production code if it cannot do it handles the data error and OS error five and two and if there's anything else it just logs the exception including the stack trace and that's all there is and if it's a simple command line application there is not a single accept statement in it there's maybe a couple of raise statements where you detect certain error conditions but other than that you don't have to bother about error handling and it gives exit code okay so in summary solution-oriented error handling is a set of guidelines to efficiently use Python's exception handling the core consideration is who can fix the error users can fix errors during runtime if they are data errors or S errors developers can fix errors during the implementation if there are exceptions or assertions or other exceptions and you always clean up what you've introduced with finally the with statement and context managers so error handling code basically consists of detecting a couple of conditions that indicate errors, raise and assert them most of the time just delegate the exceptions and at a few specific locations catch them with accept and the pattern for helpful error messages cannot perform action at location something is actual exactly describes the solution to the error okay questions thank you Thomas any questions there are two microphones okay go ahead do you know if there is a way to emulate the from error from Python 3 and Python 2 sorry I can understand you is there a way to emulate the from error syntax from Python 3 and Python 2 only by hacking around there is no clean way you can use the original stack trace in your old exception but I think you have to go down into dirty metaprogramming to actually do it I've never done it myself okay thanks I think we've run out of time for questions so if you could give one final round of applause to Thomas thank you very much