 Thanks for sticking around this late. I'm not sure if you're actually sticking around to see me or if that's because this is the room where they're serving the happy hour after my talk. Hopefully it's to see me, so thank you. My name is Brad Urani. I work at Procore. We make software for the construction industry. It's a huge suite of tools. Procore is a giant suite of tools. So many tools and so many features that I don't think one person knows at all. It's one of the oldest, largest, most mature Rails apps on Earth. I think it's about two million lines of code and it has, I think, 90 or 100 people working on it every single day. We deploy it four or five times a day. And we get a lot of errors and that is not because we're not careful or anything. It's just because we release a lot of really, really advanced business tools and we're constantly shocked at the way our customers use them. They use them in all sorts of really interesting, creative ways that we can't often predict. So we often find ourselves surprised by some of the exceptions that pop up in our logs and stuff when customers do things that we're not expecting them to do and we have to react. We have to react very quickly because we love our customers and our goal is to make our customers happy. Quite frankly, they pay a lot of money for this software so they expect us to be there right there on the phone answering their support questions and we are. And a lot of the content of this talk came out of some techniques that I've learned and that our team has learned over the years in order to make errors more useful in order to create error architectures that make it easier to troubleshoot potential problems and just make our lives a little easier. I really like this subject. I'm kind of weird. I get really excited about error handling. Maybe it's because when I go to sleep at night, at least I know if something goes wrong, at least we'll be able to troubleshoot it easily and figure out kind of what happened. By the way, so this is RailsConf, right? And everyone kind of makes their presentations kind of funny and puts like lots of memes in their talk and I'm not very good at picking out memes. I don't know, like the technical content comes naturally to me but not like finding all the humor and putting like the funny cat photos in so I put some funny like memes in here and then I didn't like them so I took them out. So I just put them all at the end. So stick around to the end and there's just like a random selection of completely irrelevant memes. But like I said, actually I left a couple in and hopefully I don't blow the timing. Anyway, let's see. This talk starts off, it's kind of about fundamentals and it starts off with some basics, some real simple things but it gets advanced pretty quickly so if you're here for some more advanced architectures and stuff stick around because it ramps up. Okay, so a little word of caution like before I dive into this is that when you start talking about what you can do with errors, exceptions and I'm gonna explain the difference in a second, it's really easy to overdo it. I'm gonna show you a lot of kind of like tools and things that you can add to your application that quite frankly add complexity but you want to make sure that that is necessary complexity that you're not doing this for no reason. If you're doing vanilla Rails like MVC and a lot of crud, don't go using the things I'm showing you here. These are architectures that are best done sort of reactively when you discover gaps in your knowledge or that your airports are coming through and you don't have the context that you need to go and add them after that becomes a problem. This is a good thing to do reactive architecture because inevitably I've seen it from junior engineers, senior engineers who were like, wow, I've got this new power from the stuff Brad taught me and they go and create something really elaborate that's not necessary. So don't overdo it. And by the way, most of these techniques apply to any language that supports exceptions. In fact, a lot of the best literature on this if you want to study deeper into this comes from the Java world. It really originates all the way back in the C++ days. And so here's just like a simple just to get started. We've got raise, oh no, and this is how we raise an error in Ruby. We can just pass a string, right? That is the same as this runtime error.new, right? Runtime is a class just like any errors are just a class in Ruby. So those are equivalent really and we actually have three syntaxes. So all of these three do exactly the same thing. And then, oh, where's my clicker? No, I lost it, nevermind. So anyway, in errors in Ruby are a hierarchy, right? And you can see up at the top of the hierarchy we have exception, right? So exception is the root class and everything below it kind of extends exception including what you see there is standard error. Standard error is the class we're mostly gonna be dealing with today. So I mentioned exception versus errors. The difference is that error is a subclass of exception. So for the more visually inclined among you, right? This is a hierarchy. This is a tree. So and spoiler alert, we're gonna be creating our own tree. We're gonna be creating our own tree exception hierarchy a little bit later, but you can envision it like this, right? Standard error extends exception and a lot of those like subclasses, you know a lot of those other errors that are in the framework like zero division error, they extend standard error. Okay, so when we rescue them, we can rescue just like this and that is actually equivalent to rescuing standard error. Now we said standard error is a subclass of exception, right? That will not rescue, for instance, if you just raised exception that would not rescue it. It only rescues standard error and subclasses of standard error. So just like recap, right? When you raise, you raise runtime error, right? But when you rescue the default level of standard error and since runtime error extends standard error, just using rescue will catch runtime error and anything that extends either of those classes. Okay, never do this. Notice I've rescued exception, which is the root case, right? Which is the root at the top of that hierarchy because you will rescue all kinds of crazy things like system stack error, which is what happens when you get a stack overflow. If you really wanna see what happens when your program runs out of memory and you continue executing, right? That's what happens with no memory error, signal exception, that's the one where you press control C, that's what gets raised, so it will prevent you from actually quitting your program and script errors actually gets thrown if there's a syntax error in your program. So don't rescue exception because it includes all of those where a standard error does not. You can make our own, right? Like a really serious error, like out of nachos error, that would be terrible, right? Just by extending standard error. Okay, and it's probably worth mentioning that like inside the framework, inside all the popular gems you use and inside rails itself, most of them have their own sort of exception hierarchy. So this is what the exception hierarchy, the error hierarchy from ActiveRecord looks like. Okay, so let's talk about rails. Oh, this is one place where like I had a meme and it was actually a good meme, right? So yeah, check this out. I didn't choose the chug life, the chug life chose me. By the way, if any of you are thinking of having children, don't do it because you'll have to watch this insufferable cartoon called Thomas the Tank Engine which will drive you bonkers, right? But if you do have children, make sure they watch it because it's about these trains on the island of Sodor and they cooperate and I swear to God it is the perfect parable for like a good functioning dev team. Like if you want your child to be a software developer, Thomas the Tank Engine will like teach them all the proper principles about like cooperation working together. So it's a good deal. So anyway, here's like a typical rails controller. This is just like straight from the scaffold, right? No surprises here. We're calling user.save which triggers the validations. Will this ever throw an error? Turns out it can, right? For some very, very unexpected things. Like what if you just can't play and can't connect to your database, right? There's some completely predictable error. It can happen, even though .save, it doesn't have the bang, that won't throw errors, right? It is still possible. So if you really want to rescue anything, you might be tempted to do something like this where you just kind of throw a rescue at the bottom. This is an example of like not good air handling and I'm gonna get into, I'm gonna slowly explain like as I go on the kind of why, what the problems of this are. But you might be tempted to do it. For instance, like what if there is like something in your save process there for a new user that does occasionally air? Like what if like in an after save callback, you're contacting a third party service and occasionally that third party service isn't available and you kind of want to recover it from it. You might be tempted to do this, but this is lacking in a lot of ways and I'm gonna get to why. Okay, so rule number one here then is don't rescue just because you can. In some cases, right? What I'm showing here is a lot of what I'm showing is what happens and where and when and why you might rescue airs and when it's good. But the first rule here is that sometimes you don't want to do that at all. So just remember that then in a lot of cases, if totally unexpected airs comes up, the best chance is just to let them come up and show the air page, right? That generic 500 air page. But when we talk about all this, right, we have to like focus on who are audiences and we really have three audiences, right? We've got our users and we want to sort of direct them in the right way. We want to show proper air messages, redirect to the right pages if possible. We got developers and computers, right? So on the users, like I said, right, we've got messages we have to worry about. We have control flow, whether they get redirected to the right pages. For developers, we want developer facing descriptive air messages so we can figure out what's going on. Additional metadata which is I'm gonna get into, right? We wanna make sure that those are reported to our air logging system. We wanna make sure that the airs show up in the logs so that we can troubleshoot them. And for computers, right, those are like they could be consumers of our API, in which case we wanna use errors to properly return the correct status codes from our APIs, et cetera, so people can program against our APIs. Or in the case of your own program, we're gonna show class hierarchies for showing control flow. So we have a few goals here, right? Just like I mentioned, user facing air messages. And then we want to control status codes, air messages, add contextual data. And we're gonna talk a little bit about notifications, how you can make sure that emails and SMS messages in Slack go to your team in proper ways given the proper things. We wanna make sure that it ends up in logging and we're gonna talk about like how to change that on HTML versus JSON. So, here's our method again, right? And I said this has problems. Well, what's one of them already, right? We can already see one of them which is that one of our audiences is developers. And as developers we rely on tools, air reporting tools and logs and you'll notice that this does not end up in any logs. So if I do this and I just rescue it, I never know what happened. This gets swallowed as we say, right? Because there's never any evidence that occurs. So here's another rule, do not swallow airs ever, right? Make sure that if something bad happens that there's always a trace of it somewhere in your system. I didn't think this like snake was like kind of scary enough. So like, oh, there's a really scary snake, right? Just to rub it in, do not swallow airs. And honestly just to like belabor the point, the scariest like kind of snake at all, right? Do not swallow airs. Not sure where I was going with that one. Anyway, but like under reporting, right? So one of the things, one of these awesome great tools we have is reporting and there are all these wonderful solutions these days, these third party systems. And the old days before these, like we used to have to build this stuff ourselves and it was always awful. Now you can just buy one of these. These are third party SaaS services that you can integrate into your program, right? They're awesome and they do really, really wonderful things. And I'm gonna show you how to take advantage of these a little bit more than you might already be doing. So here's like an example of like an air report. This is bug snag. We use bug snag and it's been good and reliable for us, we like it. They really should be paying me to do this talk, but that's another story. So this is like an example of an air message that comes up when something goes wrong. And you can see that they've got these great graphs and charts and wonderful tools. They have also like a really, a lot of nice search features and filtering features and things like that. Out of the box, they're really configurable. They know whether you're in staging or development or production and behave differently. Most of them you just drop the gem in your gem file and it just kind of works and you don't have to do much config. And like for your hobby projects and things like that, there's usually a really nice free tier. So take advantage. But there are a couple power user features that I think most people don't realize are there and people don't utilize quite enough that can really fine tune your usage of these products in ways that you might not be aware of. Severity level and custom metadata, I'm gonna show you both. So back to this again, right? We know that this is not going to our air reporting solution, but there are some cases, this may not be one of them, but there are some cases where we want to rescue an air but still get it into our reporting system, right? So what I recommend first is you start with a class specifically for that. So whenever I start like a new project, right? One of the first things I do is create sort of like a utility class which just has a class method called handle. And just to start off, it's just gonna look like this, which is just calling out to that gem for whatever your air reporting solution is. It calls that gem. We're gonna build on this. We're gonna put more in here in a bit, but just to start, right? All we're doing is using it to notify our reporting solution, okay? We've improved that a little, but let's go like a little bit further. As I said, most of these systems report some kind of like severity level. So we've added an optional argument here to pass in severity, okay? This one is a particular, this error, right? It's on a user create. Some of these errors might be predictable. Like in the example, like I said, we're a third party services down. We know that's gonna happen occasionally. So it's kind of a error, but a slightly predictable one. We may not wanna be bothered every time it happens. So the default severity level is error. We can turn it down a notch down to warn. And what's nice is that in those air reporting systems, right, they have all kinds of configurable filters on like what shows up in your email, right? Some of them have like SMS and Slack alerts, but you can generally filter them out by severity level. So we're already using this tool in a more powerful way that gives us a little more control. We pass that down to the gem. All your usage may vary, but almost all of them support something like this. But the other thing is it never showed up in the log, right? And sometimes we depend on log files. Sometimes we have to get in there and grab the logs to really try to see what happens. We wanna make sure that we end up with our error in our log too. So we're gonna add to this class just some logging. And you can get a lot more sophisticated with that. You can, you know, based on severity levels and other things, channel it to different logs. And there's no end to what you can do, but we're already starting to build sort of a powerful framework here. Now we've got it in our air handling and our logs. So that's nice. Okay, now I've added something else here. How many people work on a big department with more than one team, more than one squad, right? We have, I think, 20, and like I said, like 80 developers, and guess what? Not all errors are all relevant to all developers. When we pull up our, you know, our bug snag in the morning, it is a nice way to filter on these. So I've added something else. You notice I've added a metadata property and I've said tool onboarding. Well, this gives me another really convenient filtering option. This is something that I can use to tailor alerts, to tailor emails, and various other things. It makes working on a team a lot more powerful. And metadata is like an arbitrary property. It's a free for all kind of hash where you can pass in whatever you want. But believe it or not, almost all air reporting tools handle this, right? It will actually like show up in your report, and some of them get crazy, like in bug snag, you could actually like use the metadata, like add new tabs to the report, and doing all kinds of things. But check the docs for yours, because you can probably do this, and it's probably searchable, filterable, and can be used to tailor alerts and things like that. So there's a very powerful feature for you. So whatever you accomplished already, we're adding contextual data, and we've given us flexibility notifications, just with a few lines of code. So getting a little more advanced here, right? I mentioned we wanna build our own hierarchies. You don't wanna do this for no reason at all, but there's some cases where creating custom air classes and things like that can make your whole architecture a lot more powerful. The most common case, not the only case, the most common case where I see this comes useful is when you have applications with very deep call stacks. So like in this example, like imagine like a fictional online shoe store. You might have like, you know, if you're doing MVC, this is not, if it's just like the controller calls a model and that's it, you don't need this. But in more advanced flows with like a lot of business logic, and Procore certainly has a lot of business logic, you might have this kind of scenario where you have like a purchase class which like, you know, calls a few methods, it calls a method which reaches out to an item class, which reaches out to a warehouse class, and like all the way down at the bottom there, you've got this like check available. So this is a shoe store, we're checking to see if like the shoe is an inventory, and there's a lot that could go wrong there, and we're way deep in the call stack and we're way far away from the controller. So it's possible that someone checks out and wants to buy a shoe and the shoe does not exist. So we know way down at the bottom of that call stack that we need to display an error message now, we need to tell the user what's going on, but the problem is we need to show it way up in the controller is where we have control over that and we can actually affect change and show the user the message, right? Now, what's nice about this is like, it's either a feature or something a detriment is that all those layers in the middle don't need to know about that error. Some say that's good, some say that's bad, right? That's what exceptions allow you to do is allow you to raise an error way at the bottom of the call stack, ignore it in the middle, and catch it all the way in the top. Other languages don't work like that. For instance, Go takes it to the extreme, it does not support exceptions. Every function that might have a possible error in it returns two values, a result and an error, and then there's like a conditional, and you would have to pass errors like all the way up the call stack, which is a lot of extra boilerplate, right? But also a lot less magic. So it's just a trade-off, I'm not gonna say one is better than the other. The good thing about errors is convenience and less boilerplate, the bad thing about them is that sometimes they come out of nowhere when you're not expecting them. Elixir takes a slightly different take on this where they encourage this sort of pattern-matching take, which is similar to the Go approach in which you have to pass the error all the way up the call stack, but it's got this nice pattern-matching syntax that makes it a lot more pleasurable to work with. Weirdly, it also supports exceptions, I don't know, maybe they couldn't make up their minds about what's better, I wouldn't be surprised to see them take them out, honestly, but in Ruby we have exceptions, regardless of what you think about them, we have them. So what we can do is raise here and rescue way up in the controller. Okay, so for this, we're gonna create an error hierarchy and an example one might look like this. You can see that I've started with just a generic store error, right? I've got a purchase error, which is anything related to the checkout process on my shopping cart, and I've got certain inventory errors, inadequate inventory or like an out-of-season error, I just made that up, this is my example error hierarchy, and you can see that also extending purchase error, we've got a coupon error that might include, that might also have subclasses, for instance, coupon expired or invalid coupon, right? Okay, so here's a controller for our purchase, so pretend this is the controller from when the user clicks purchase shoes, right? We've created this error handler class, like I said, we've changed the severity level down to warn, why is that? Well, actually some of these are kind of expected, we would expect sometimes to someone to enter an invalid coupon, we would expect sometimes someone to try to buy something that's out of stock, ideally they might, they would probably know it's out of stock before they click purchase, maybe there's a race condition, maybe two customers checked out at the same time, right? And one of them got the shoe and one of them came just a little too late, so these are sort of unlikely, but these are unlikely errors, but unlikely things happen and we know that, right? Now, I want to show you something else, right? Which is cpe.userMessage, notice that I'm not using it's user message and not message, this is a little trick I learned over the years, a little bit of wisdom, which is don't use error.message for users, user facing errors, it is a lot more convenient to have, because they're really not the same thing, right? error messages are for developers, those are computer language things for us software engineers, whereas messages that are supposed to face users are completely different, so we're using user message instead of message and that's something I implemented myself, which I'm going to show you in a second. Okay, now we've got a lot of flexibility in this controller, because notice, let me go back, notice what we caught here, we caught purchase error, and if you remember purchase error is only at the second level of that hierarchy, that includes inventory errors and coupon errors, but that might not be enough control for you, because what if you want to do two different things, depending on what kind of error it is? If it's an inventory error, you might want to redirect back to the shopping cart path, if it's a coupon error, you want to redirect to a coupon code page, so now I've changed the one I'm rescuing, I'm rescuing a subclass of purchase error, in this case inventory or coupon error, and now I can take different actions. You can already see that the error hierarchy I created is giving us more power to control flow and do slightly different things depending on what kind of error occurred. Like I said, user message, right? Don't use error dot message. In fact, it might even be a security alert, you could potentially leak information about the internals of your program if you expose user dot mess or exception message, right? The standard just message, it's unlikely, but it's possible. It's better to have that piece of totally separate thing. Okay, here's another slightly different example. Also part of our error hierarchy, if we're rolling a lot of our own, like authentication, or not authentication, but authorization really features and things like that to allow permission to certain pages, we might have to create an error hierarchy for that. This is a little different in that it could occur on any page where those purchase errors are only on checkout, right? This could occur on any page. So what we can do here in this case is up at the top of the controller we can rescue from store unauthorized error, which is up near the middle of the hierarchy. And in this way, we can capture errors no matter what controller action they're on. In fact, we might even take this out of this controller and move it all the way up into the application controller so it applies to all pages. So now we have the power to not only control on a per action basis, like what the result is when something unexpected occurs, we can now apply this application wide. And you notice where it says status unauthorized, we're also changing the HTTP status code when this occurs. That is awesome for our API responses, right? I talked about having one of the audiences we have is not only us as developers and users, but also computers, we're now affecting the status code too. And it's worth saying that the framework does this quite a bit, right? Okay, so the framework does this quite a bit. Active record, for instance, automatically changes the status code for you. So if you like find a nacho that's not there, right? Because they're not a billion nachos in the world, unfortunately, it will automatically change the status code for you to a 404. And if you do like cheesiness to a million, but the max cheesiness level is only 100, it raises record invalid, which gives us a 422. So the framework's doing this, we're just doing the same thing that the framework is doing, which is using errors to control HTTP status code responses. And if you're doing JSON APIs, it creates this nice, it does this for you, which is pretty slick. Or if you want something different, you have the ability to control this now. Okay, and you'll find things, like this is source code from Rack, for instance, where you can see that Ruby on Rails or Rack at least is automatically matching up error classes to status codes. Okay, so I mentioned this hierarchy, this is back to our purchaser one. I'm gonna show you what these classes actually look like. I'm gonna show you down the left side, their error purchaser inventory error and inadequate inventory error, what an implementation of these might look like. So here's the top of the hierarchy, right? This is the generic store error. You can see that I'm passing in a message. I have that generic metadata, a hash called metadata and a symbol called severity. Wait a minute, we've seen this before, haven't we? These are the same parameters that I passed into that custom handler class I made, only now I'm passing them in and construct your arguments to the error. Dun, dun, dun. So what we're doing here actually is now we have the ability to not only add these things at the point where I rescue the error, but also where I raise them. Also remember, I said use user message for customer facing errors. Not only have I put that into effect, I've also got the ability to internationalize it for non-English speaking people. So we've already got new abilities here and then see how I've got adder accessor at the top with metadata and severity. Believe it or not, for whatever error reporting solution you're using, at least most of them, just the fact that I have those adder accessors up there will automatically add them to my error reports no coding required. It's usually built into the gems of these things, of these solutions you're probably using is the automatic ability to just have that just work. So that's really cool. Here's the next one in the hierarchy, purchase error. There's no code in it. It's basically an abstract class, right? It's just inheriting, but that doesn't mean it's not useful. We already saw an example of me rescuing this in my controller because I wanted to treat purchase errors different from other types of random who knows what errors. And then that had a subclass which was inventory error, which is also completely empty. This is also useful even though it's completely empty because remember I had an example where I rescued inventory error separately from coupon error and do two different things. So just because the class does nothing but extend my purchase error doesn't mean it's not useful. It ends up very useful. And I may never actually instantiate this class and use it by itself. It may just be an abstract class, but it's still a very handy tool. So here's down in the bottom of this tree, inadequate inventory error I've passed in some actual useful data here, things I want in my report. Product name, requested quantity, and available quantity. And I've added them to that metadata hash. So now it's gonna automatically show up in my airport which is cool because when this happens I might wonder what's going on. Like it shouldn't be possible for someone to order something that's not in inventory, right? Usually they should get warned before they check out but in certain cases, race conditions among them it might happen and I kinda wanna know what's going on because I might get a support call about it and this will help me now that this is in the airport and I've overridden user message for a better more descriptive message. Okay, so back to our controller here. I'm catching purchaser, I'm rescuing purchase error and displaying user message. Okay, so what have I done here? Remember, down where I raise I added some metadata. I added product name and I had requested quantity, realized that that's all the way down at the bottom of this stack and I did that with constructor arguments in my error. User ID, actually that's out of date, but user ID or whatever else I want I can pass it up at the top in my air handler so I've created an architecture now with an air handler class and a custom error to add additional metadata, additional things I want whether it's at the site of the raise or at the site of the rescue. What if I want something from the middle, right? Maybe there's a coupon code being applied in there and it's somewhere in the middle of the stack in my business logic. I can basically rescue it and re-raise it. I didn't show you this but presumably I've added an add metadata field to that error class, right? I can always rescue and re-raise if I want to add additional context on the way up. So why rescue and re-raise to add metadata? Worth pointing out there's something else going on here, right? In addition, like in this purchase process, I may be removing items from inventory, applying a coupon code, clearing the cart and processing the payment. All of those involve database inserts, all of them can fail. What if I get through the first three and done a process payment, it fails? I've got new inserted rows in my database, right? That are not valid. What do I want to do? Roll back, right? Realize that the errors are affecting this. By having those errors bubble out from the base class, I don't have to do anything special except for use of transaction that I'm also getting that rollback behavior based on my airflow. Okay. There are other times, I showed you raising and rescuing the controller. There are some places where we want to raise an error but then also continue the program execution where something unexpected happened but we can recover and show the user a page as normal. Here we have a shipping price, right? And I'm getting a quote from UPS. Guess what? The UPS API goes down. I used to work in e-commerce, it happens. Do we still want to take the user's money and purchase their transaction and send them their shoes if UPS is down? Yes, right? So I'm returning $6.99, just blanket charging them $6.99 for no reason than that's just a random number. So this is a case because I'm just going to charge them $7 for shipping because I want the order, you know what I mean? So here's a good case where we're going to rescue and you might continue. We have that special handler class we made because we also want to know what happened. So when someone orders like 50 pounds of shoes and it's like $100 and we're charging them $7 for shipping, we want to know what went on. So we want that in our air log, don't we? So we've got that custom handler class that gives that to us. Okay, so we raise and forget, right? We've got three basic behaviors. Raise and forget, which are unknown, system errors, connectivity errors. Raise and rescue and controller that we do that so we can change the status code, customize the error message, soft error messages which I'm going to show you. And sometimes we raise, rescue and continue which is anything that's a problem but not enough to stop us from continuing which is my shipping price example. So three kind of distinct ways to sort of handle these things. Let them fly and just show the 500 page, recover and show something nice or continue as if nothing went wrong. Okay, wrapping errors, right? Oh wait, I think I have a meme for this. Yeah, oh, wrapping error, get it? Because it would be an error to wrap a cat in wrapping paper. Like RailsConf, right, cat memes. Cool. So, all right, so here's our call stack again. Check available down at the bottom. This is exactly the same what we saw but imagine instead of doing like querying our own database this hit a third-party API. Maybe it's one of our own microservices or maybe it's a third-party service, right? Things are gonna be a little different because we might have something like this where we're using REST Client to make an API request to something else. Well, that is from a gem, right? REST Client's from a gem. We're gonna get, we might get back in some cases request timeout. We could just let that one fly all the way up to the top but if we caught request timeout in our controller we're not really sure what to do with it. We're not really sure where it came from and we don't have a lot of data to go with it, right? And we might not be able to direct the user to where they should be going. So what we're gonna do is rescue that one and raise our own inventory unknown error, right? Okay, if we rescue that, right? Ruby, by default, has this great property called e.cause without us doing anything the cause in that case will be the original exception the REST Client request timeout exception. It does that automatically. The Ruby runtime gives you that automatically and most of these error reporting solutions also will show that. So we've got a call stack on top which is the call stack of our own exception and then it says caused by and there's the inner exception and we have two call stacks for each error and these reporting solutions already show this. So now we can really track things down. So that's really useful and most you don't really have to do anything to get that to work. Okay, so why wrap exceptions, adding metadata. We might choose to catch one error and raise another so that in the case of if this is an API for instance we can change the HTTP status code by using logic on our controllers we get better control flow. Here's a real world example of a case where we're doing something like this like using some pretty advanced error logic. So this is from Procore. This is a construction budget. This is like one of the world's most complicated webpage. Any button or click you make on this page sends this cascade of business logic. You know, it calls a hundred different methods from a hundred different classes. It's really, really complex to calculate someone's budget. See they're over on the left. There are two orange buttons in between one. There's a gray button. Just in order to know if that button is enabled whether it's orange or gray we have to first run a query that joins across six tables. Then based on the result hit a third party API service in which case is an integration partner get the result back from that and then decide whether the button should be orange or green. This is a real world example. Guess what? We're not doing all that in the controller. It's 25 levels deep down in the call stack and like way deep in our libraries and guess what? Third party services can go down. Can't they, right? So what we do in this case is we do kind of what I showed you which is capture the rest air, right? Wrap it on our own air which is like third party service unavailable air and then we have special logic way up in our controllers in the application controller in fact so that it affects every page so that instead of just showing a blanket 500 blow up we can give someone someone nice. This is like a soft air page, right? Where it actually gives people like a little nudge on where to go, a link so they can check their settings. You know what I mean? So this is a good real world example of why you use custom air hierarchies in deep call stacks. Little extras, these are things that I didn't have time to add but I'm just gonna give you like a little teaser for this stuff. Some people like to add a custom, like a unique identifier at the raise site. So if you just add like a randomly generated string or some kind of identifier that allows like easy searching through logs, you know? You could like correlate certain airs with pages in your documentation or maybe you could add a unique identifier and have some kind of mapping file that maps a unique ID to an air in your documentation so you could provide someone in an automatic way links to, you know, pages that help them figure out what went on. And we've accomplished quite a bit here, haven't we? Let's just go over what we did. We've changed the user-facing message and we now have complete control over what we say to the users in a way that's internationalized and totally configurable. We've better program or facing messages by raising our own hierarchies, by creating our own hierarchies, by creating our own air classes. We make our logs more intelligible to us so that we can actually comb through there and more quickly figure out if this is actually a real problem that we need to address or if it's something we can safely ignore. We've changed HTTP status code for our computer consumers of these APIs, right? They need to act on proper status codes. We now have full ability to control those. We've added contextual metadata which is everything related to what was going on when the air was raised. We now have the ability, both at the site where we raise the air through passing things to our exception and the part where we rescue it through our custom air handler class to add anything we want to that air report because we've added severity levels and various metadata that we can filter on. We now have completely configurable notifications so that we can make sure that the right team gets the right message and thank God we don't have to build all that ourselves that you get that with these solutions. We've got it in the logs, we have full control over the logging and we have this control flow. We've gained these control flow abilities where we can either continue execution, we can decide whether we just wanna fail and let the page blow up or we can show some kind of soft message or some more appropriate message to get to the users. You might wanna, if you get really complex, generate some kind of diagram like this because it gets complicated and hard to keep track of. This is a fictional one but if you really get into this, if you need it, right, and I'm gonna reiterate my original warning which was don't do this stuff if you don't need it. It's only in really complicated big apps that this stuff becomes important and that you need it but you might wanna diagram it like this just to show, just to keep track of it all because it's hard to keep track of sometimes. A little bit of further reading, there's some great posts online that I've linked to here. I'll tweet these slides where you can read more about this stuff and learn a little bit more about some of the capabilities of these tools and how you can build this stuff yourself. I'm Brad Urani, I love Twitter, I'm kind of an addict, follow me, I'll follow you back. I'd love to connect with you on LinkedIn. If you're into Mastodon, which is like this new hip, cool, decentralized, farm to table, social network, feel free to connect with me on that. Like I said, I work in Santa Barbara at Procore, we make construction software. We have one of the coolest, awesomeness teams of all time. I love my job and we're hiring like crazy, growing like crazy and I'd love to talk to you. This is the view from the office, it's tropical paradise, it's sunny all the time, it's gorgeous, come visit us. And oh, also check, my coworker Derek is giving a talk tomorrow about how we built out a really complex and really powerful API structure. Be sure to catch that one. And then here's some memes, like I said, they're totally irrelevant, right? Oh, there's some pigeons with sunglasses, yeah, great. Oh, live longer prosper, oh, it's Spog with a cat, right? This is, uh-oh, Brad's getting political, be careful. Oh, and there's some more, something, something, something, Rails didn't, and there's the train, oh, and this cat, it looks like he's got plastic legs. And I think that's all I've got. And really that was just a ruse so that I wouldn't have time for questions. So, thank you, but I'd love to take questions afterwards.