 massive group hug, that's awesome. So as I mentioned really quickly in the lightning talk, I'm Doc Norton. You can find me pretty much anywhere as Doc on Dev, whatever site, WAC, Doc on Dev, that's probably me, and of course that's my Twitter handle. As I mentioned, I am global director of engineering culture at Groupon. And they can't give you a raise, they add a word to your title. A little bit more about me, I am a husband, I am a father, and there's actually only one reason that I even mentioned this, I love sharing this one. I'm also a grandfather. So enough about me, I've got a question for you. What is technical debt? It's hard for me to see folks in the audience, but I am looking for an answer from someone. Feel free to shout it out. What's technical debt? Stuff you've got to clean up later, stuff you're not sure you want to keep, things that slow down your team. These are all pretty common definitions of technical debt, do you agree? So Ward Cunningham is actually the guy that came up with this phrase. And in a paper, oops, the paper in 1992, Ward said, look, shipping first time code is kind of like going into debt, right? Little debt speeds development and as long as you pay it back promptly with rewrite, you're pretty good. It says, the danger occurs when the debt's not repaid. And every minute spent on not quite right code counts as interest on that debt. Now a couple of things about this. First of all, while it was written in an UPSLA 92 paper, a year or two prior, Ward used this exact same explanation in an email to his client. And he was working on a financial system. And so the metaphor that he chose fit their domain, right? It made sense to them. That's why he chose the debt metaphor. I don't know if he had been working on medical instrumentation, what the metaphor would be today, but the debt metaphor fit for them. Now, if you guys know Ward, he's a really nice guy. He's very gentle, he's very soft-spoken, and I think sometimes he's a little subtle and we miss it. So I want to hone in on exactly what it was that Ward said here. He warns us of danger, right? He doesn't say the challenge, the concern, the potential eventual problem. He says danger and danger should actually invoke fear in us, right? And he says every minute, not quarter, not month, not iteration, not release cycle, not day, every minute spent on code that's not quite right. Not crappy, not awful, not horrendous, not hideous, just not quite right. Every minute that we spend with code that's not quite right is dangerous. But I'm here to tell you, technical debt is good. How's that possible? Well, technical debt, as Ward described it, is basically two things. It is either a strategic design decision, something that we've done to allow for rapid feedback, so that we can, rapid delivery, so we can elicit quick feedback and we can correct our design, right? So we're not exactly certain what this thing should be. We have a pretty good feeling about it. We get something out there. We see what happens with it. We adjust as we go, right? Or it's an indication of learning, right? Actually, in either case, it's an indication of learning, whether it's intentional or not, because now we know what it is that we need and we understand that our implementation doesn't match and we can change that implementation, right? So when Ward was talking about technical debt, this is what he meant. But remember, it's a metaphor, right? It's a metaphor for here be danger. The thing about metaphors is they absolutely rock, right? They allow us to talk about a complex problem that maybe we don't all understand in terms of some other solution or situation that hopefully we have a common understanding of, right? We want to talk about this, but because that's hard, we talk about this instead. It helps us, right? It helps us to reason, it helps us to communicate. But something happens. Something goes awry eventually. Every one of you has been in a conversation where you began with some metaphor that seemed to make sense. And then the next thing you know, right? So you say, like, well, you know, we got to change the engine while we're driving down the road. And the next thing you know, you're actually having an argument about transmissions and suspension systems, et cetera. You're actually arguing about the thing that is not what you were talking about, right? That is what I call metamorphosis. This is when a metaphor goes wrong. We begin to talk about the thing to which we are comparing, not the actual thing, right? And eventually, in our industry, the debt metaphor turned into phrases like this. It was simple. People understood, well, I know what debt is. It's like a credit card. It's like a home loan. It's like short term, right? It's intentional. It's prudence, pragmatic leverage. There's a loan shark. There's inadvertent reckless debt in the third quadrant, which, by the way, we will talk about. This is an actual phrase that came up. This became so pervasive in our industry that our leaders adopted this same kind of thinking. And eventually, eventually, Martin Fowler came up with a technical debt quadrant. Well, dude, now it's official, right? As we learned earlier, there's a quadrant, right? Now you can throw tons of money at the consultant who came up with that. The way he describes the quadrant is he says, well, look, there's a few different types of debt, right? You got to think about it. It's either reckless or prudent, and it's either deliberate or inadvertent, right? And there's these different types of debt that actually occur. On YouTube in 2009, Ward finally spoke up again. So this is a few years later, and he said, yeah, that's not what I was talking about. A lot of you have confused this with the idea that you could actually write bad code, right? You've confused the debt metaphor with the idea that you could write code poorly. But the ability to pay that debt back depends upon you writing code that is clean enough to be able to re-factor. Now what is re-factoring? No one in the audience knows what re-factoring is, right? So we've changed the design without changing Apple implementation, right, without actually changing the behavior, right, the internal design changes. So how do we do that? What's the safest way for us to be able to do that? Well, the safest way for us to be able to do that is to have good tests around the code, good coverage, right? And the cleaner the code is, the better shape that it's in, the easier it is for us to actually re-factor it. So here it turns out that in order for you to actually have technical debt, you have to have code that is clean. Clean code is a prerequisite for technical debt. The last thing Ward had to say on this was Twitter 09. He said, dirty code is a technical debt as the pawnbroker is to financial debt. Don't think you're ever going to get your code back, right? Fine, folks, you want to go with this metaphor? Here's how it goes, right? You're never going to get it back. So if you're looking at your code and you're asking yourself, well, is this technical debt? Check these, right? Is the code actually clean? Is it tested? Is there a learning objective or event, right? Did we move this out into production because we want to learn what it is the users actually need? Or because we want to see how the market actually responds? Or because we're waiting to see if that federal relation actually goes through or not, but in the meantime we can generate revenue doing this thing, right? Is there a actual plan for the payback of that debt? And is the business truly informed? And this is important. See, us saying in a meeting, okay, but it's going to be a problem later, is not the business being truly informed. We've actually got to get them more engaged. We've actually got to get them more involved and help them understand the cost of the short-term decisions that we made. If you say no to even one of these things, you don't actually have technical debt. What do you have then? You have cruft, right? You literally have cruft. From the dictionary, an unpleasant substance, the result of shoddy construction, or redundant, old, or improperly written code, right? You basically just have a mess. You've just got crufty code. You don't actually have technical debt. So I've given this talk a number of times. I've had conversations over dinner with folks, over drinks with folks, and one of the things that comes back to me often is like, hey, you know what? Chill, man, it's just semantics, right? It's just a word, and it's not that big of a deal. But if we agree that technical debt is good and we also agree that quick and dirty is technical debt, then we're agreeing that quick and dirty is good. And I can't abide by that. I've been on enough projects that started off quick and dirty and ended in a horrible death, right? If we're lucky, we start off quick and dirty, and at some point we've generated enough revenue that hopefully we can convince the company to do the grand rewrite, and this time it'll be better and we'll get it correct, right? So I don't think it's just semantics. And you know, and I was thinking about it. I looked at this technical debt quadrant again. There's some things about this that really kind of agitated me. I mean, one, just the fact that it's a quadrant, right? But we must ship now and deal with consequences. To whom in the audience does that sound prudent? So this is deliberate and prudent and it still sounds pretty reckless to me. So I actually think that that should say let's deploy and gather more information, right? That's different than dealing with the consequences. But then there's some other aspects of this thing that bother me. And I was having a hard time articulating why does it actually bother me? What is wrong about this, right? So I started thinking, well, what if we looked at technical debt, this metaphor, this concept, and we applied it to other fields because it's so pervasive in ours. If it actually makes sense, it should be able to make sense other places. That's not supposed to come up. Yep, so let's look at automotive, right? Technical debt and automotive. Now imagine that you took your vehicle to this mechanic and you came in and the mechanic said to you, hey, listen, so we encourage the mechanical debt to stay in budget. We should probably get some metrics around that and make sure we pay that down in the future, right? What about medical? And this is not photoshopped, by the way. So end of the surgery, you're talking with the surgeon, your loved one is still in recovery. And the conversation is, you see, we incurred some health debt during the surgery. It's kind of like you paid for the surgeon with a credit card instead of a home equity loan, right? When we apply this to other industries, it's obviously ridiculous. So how is it that it's okay in ours, right? What are these things actually? That's reckless and deliberate. I don't think that was an accident. I'm going to give the benefit of the doubt and say that this was reckless and inadvertent as we apply them to our technical debt quadrant. So to me that says reckless and deliberate is just irresponsible and reckless and inadvertent is just incompetent. So on our technical debt quadrant, we have irresponsibility and incompetence. The only thing left that's actually technical debt is that which is prudent, right? We wrote good code and we moved something into production that had a design that was not, we weren't sure this was exactly correct but we knew that it was close and we wanted to learn or we moved something that we were highly confident was correct and when we got it out there, we discovered that, hey, actually, that's not what the users wanted and we made some adjustments, right? That's the only aspect of this quadrant that's actually technical debt. So I want to play a game. Does anybody here want to play a game with me? We're gonna play Croft or Debt, right now. This is usually a much longer talk. We're only gonna do two rounds of this. I'm gonna apologize right now. None of this code is Ruby. It is also, by the way, not Java or C sharp or any other language you've seen. It's all kind of pseudo code and that's sort of intentional. So let me take a look at this. And I want you to tell me if this is Croft or Debt. It's Croft. How do we know that? It just looks like it is, right? It just looks like it is. I mean, ew. So there's a couple of things that we could do with this. Just taking a look at it, first of all, I can read through this and every time I kind of read through it I started to get a feel of like, okay, what is this doing, what is this doing? It's not really clear what this does so how can I clean that up? Well, you know what? Maybe if the person that wrote this in the first place had just commented it because that's the universal solution for unreadable code, right? Isn't it? No? If customers federally regulated. Well, there's something else that I noticed, right? So if you look at this, I don't know what this class is. I don't know what this method is but I do know that it's interrogating the hell out of my customer. It's asking my customer an awful lot of questions in order to then be able to actually make a decision. So maybe what I've got here is aside from sort of the psychomatic complexity that's very obvious, I've also got some feature envy, right? This responsibility is probably in the wrong place so I've already got a comment that tells me sort of what this should be but what if we just did this? That's what that code should be, right? And then that actual logic should be over in the customer object and hopefully it doesn't look quite as heinous as it does here. One, there doesn't have to be as much interrogation and two, I think we can just write a little bit cleaner, right? So yeah, definitely cruft. Now what about this one? Is this cruft or is this dead? And bonus points to anyone that gets the joke that's hidden in here. What do you think? Cruft, dead? Dead. Some would say cruft. But it's hard to say, right? This is borderline. So some, if you follow some of Uncle Bob's stuff then really the fact that you've got three cases here means that you've got a problem, right? If you've got it, basically this is just nested ifs flattened to look sexier through a construct but it's really still high psychomatic complexity, it's still nested conditional, right? So if I've got one conditional, that's okay. If I've got two then maybe I should be looking at my composition. Maybe there's some polymorphism, maybe there's something else that I could be doing to actually get rid of this, right? So let's try that just to see what it looks like. Now obviously there's some other code that's hidden here, right, so kind of introduced a couple of methods that maybe we didn't see previously. But it's hard to say that this is really better, isn't it? My point is only that sometimes it's super obvious and sometimes it comes down to personal choice, right? If that case statement had had like 27 different options in it, everybody in this room would agree that it was dead, that it was cruft, sorry. But right now, borderline. So we're still gonna make some personal judgments, right? So cruft is a bad decision every single time. Why, why? Well one, we're all professionals. We are professional software developers. This is what they pay us to do, right? And you're gonna create unintentional cruft. You know why? Because it's unintentional. You're going to make it, you didn't mean to, but it's going to happen. And you're gonna have to clean up that existing cruft sooner or later, you're gonna have to pay that down. So the title of this talk is the technical debt trap and so far I haven't even talked about the trap. What is the trap? Well the trap is cruft begets cruft. When you start down this path, it gets harder and harder and harder to get out of it, right? First of all, you've set a precedent for speed over quality. So you made a compromise early on and now the precedent is set. If you're in an agile shop and your management has read the book and they are excited about having the agile installed, it is very likely that they expect you to be moving faster because in at least one of the books that they skimmed over, it indicated that teams velocity will increase as they get better. So they expect you to go faster and you've already set a precedent for speed over quality but the cruft slows you down. So you actually have to write more cruft to keep up. And eventually what you have to do is ask permission to do your job correctly. Now we may hide this in some way, what we might do is we might say, well you know, this whole thing started and we were on Java five and now we wanna go to Ruby or we need to get to Java eights or blah blah and there's some rational reason why we have to completely rewrite the entire application. Some technical reason why we have to do that but the odds are the reason that we really have to do it is because we can't keep up anymore. I'm telling you, I've seen systems that were written in RPG 30 years ago that are still running just fine today and are still being maintained. It's not always the need for the technology in a lot of cases we kind of make up the reasons why we need to rewrite. So what do we do? Well, avoid the trap, avoid it in the first place. So there's this technique of writing, doing incremental fixes. So maybe some of you have these, you're cruising along, cruising along, cruising along and then you do some kind of a debt iteration or some kind of a cleanup, right? Where every so often it's, you just focused on cleaning up the code. Well, it turns out there have been studies that have been done, I'm gonna zoom in on this. There have been studies that have been done where they've followed teams that have done this and basically here's what happens. So this black line is traditional application with no refactoring, right? And what's happening here is the actual cost of changed in the application over time. Keeps getting higher and higher and we can see that this is the curve on this that somewhere right around about here it just goes, right? I mean, it gets to a point where there's, you can't do anything anymore. You can't write new features because the cost of change is so high. This is a code base where we're constantly cleaning and refactoring. You'll notice that its cost of change also increases. Well, the code base gets larger. You can't hold all that context in your head. I mean, there are rational reasons why this happens, right? This is what happens to teams that do these cleaning sprints. You go through this period and then you spend some time cleaning it up and you lower the cost of change. But what are you doing now? Well, the odds are you're trying to get caught back up to the schedule that the business expected and even if you aren't, you've basically jumped right back into the old habits and over time there's no way for you to clean as much debt as you're creating with those habits. And so it turns out that really you're on the same path. You're just kind of fooling yourself as you go along. So you've got to completely avoid it. You've got to clean constantly, right? Don't make an intentional mess. Monitor your technical debt. Follow the Boy Scout rule. What's the Boy Scout rule? Leave it cleaner than you found it, right? It's not that hard. If everybody on the team is doing that, the work that you're doing that's new is good quality and you clean up one other thing. Over time it actually does get better. It can feel futile. It can feel like you're spitting into an ocean of debt. But everyone making the effort, it can actually help. The other thing is that if you're following the Boy Scout rule, what you're gonna be cleaning is the code that you're most often changing, where the highest churn is, which turns out that's where your highest risk is. So just like coverage, if you've got a code base that has zero coverage and you start writing tests, write tests around the stuff you're actually working on. If everything else is ugly but working and you don't need to touch it, right? Remember that quality is your responsibility and never ask permission to do your job correctly. This is really hard, right? It can be very scary. We're under a lot of pressure sometimes to deliver. We're also very fortunate. We are in a field that is in extremely high demand. People who stand up for what's right and do their jobs excellently can be fired for that very thing. And I guarantee you, you will find a job the next day. In fact, if that happens to you and you do not find a job the next day, call me, you will have a job the day after that. So there's a few key metrics that we can look at for monitoring our cruft and debt. Code coverage, code complexity, coupling, and then maintainability, which was only a .NET thing, but I believe there's now a maintainability index that is available with a couple of the Ruby libraries. I know that metric foo, which is kind of old, was working on a maintainability index as a heuristic along with all the other stuff that it did. I'm not gonna get into details on these things in terms of what tools should you use or not use for each one of these things. What I do wanna talk about a little bit is, we can have these arguments, and I often hear them about how code coverage can be gained and it doesn't really mean quality. And cyclomatic complexity can be gained and it doesn't really mean quality. And coupling can be gained and it doesn't really mean quality. And you know what? You're right. Put all three of them together and I challenge you to game them, right? I would like to see someone come up with a way to game coverage, complexity, and coupling all at the same time and not accidentally improve the quality of their code while they're doing it, right? And then the maintainability index is basically just a heuristic that uses several of these together and gives you a number on a scale of like zero to 99. It's kind of nice for graphing. The other thing I watch with these is don't set targets, right? If your coverage is at 30% but that 30% is covering the stuff that has high churn, that's awesome. If your coverage is at 100%, that's a smell as far as I'm concerned. Somebody's probably gaming the system, right? Just monitor the trends. Is it getting better? Is it staying the same? Is it getting worse? And based on what you guys are doing as a team, is that what you expect? If you have made the decision to rush to hit a deadline because there's a very large financial opportunity that's available and you're okay with the code quality staying flat or maybe taking a slight dip and that's what happens, that's fine. That's what you planned on having happened, right? For the most part, I'd say, hey, try and get it trending up but just watch the trends over time. Don't watch any single point on any of these. So I wanna make sure there's time for Q and A. Do a quick review. Technical debt is a strategic design decision. It requires the business to be informed. It includes a payback plan. Cruft happens. It needs to be monitored and cleaned and it is not technical debt. And my final reminder, never ask permission to do your job correctly. Comments, questions? Somebody call bullshit. Come on, it happens every conference. Yeah, so how would I recommend reversing the debt ratio in a large legacy code base, right? So I would take, I kinda mentioned it before, I'd take the same approach that I would take to introducing testing to a code base that doesn't have it, right? Really focus on the areas where there's high churn, where the team is actually right now working is that's where the greatest risk actually is. The coupling metrics are actually very helpful there, right? So look at classes that have a lot of efferent coupling, where a lot of other classes are dependent upon them. That's a good place to actually take a look. It turns out that the Sonar cube, which Sonar is a tool that was available in the Java space originally where it would actually grab all of the static analysis information and would aggregate it and show you reports over time. There's now been plugins that have been written so that you can actually run static analysis tools against your Ruby code base or your Python code base. I believe your JavaScript code base, et cetera. And Sonar has this very cool feature where it actually will show you a heat map of the code base and say, hey, this is a space. This is an area of the code that you can make an easy change to that will have a high impact. So it basically shows you quick hitters and it shows you high riskers, right? So you can actually make some informed decisions about where should I focus. But for the most part, focus on the stuff that's currently changing. If you've got some old, crafty set of classes that nobody touches anymore, that just, you know, that do their job, even if they're flawed, but you're not touching them, don't worry about it. When it's time to actually touch them, then consider cleaning them up, right? And the way I would do it is if they don't have unit tests around them, I profile them first and then start cleaning. And profiling for me is I write unit tests that indicate the behavior that I observe. And I write them as if that's the unit, the behavior that I expect. And one of the fun things about this as a consultant, I went into an organization, I actually wrote a bunch of these tests. And one of them said that it should allow you to change the state name without adjusting the abbreviation. And one of the developers said, well, that's not what it should do. You know, I've got very upset that I would write a test that said that. This is what the class does. This is what the class has always done. It's always been broken. You just didn't have a test to tell you that, right? So I like profiling. Yeah, so what are some techniques to use to actually like maybe inform or push back on the business, right? I mean, so one of the things that I actually advocate for is getting the static analysis around the code. And one of the reasons that I do that is because it is objective, right? It may not be perfect, but it is objective. We can actually show that the quality of the code is getting better or it's getting worse. We can talk about the impact of if we contrast that against our velocity, right? We can start to show that, hey, when the team is pushed to move this fast, quality goes down. When the team is allowed to slow down a little bit, quality comes back up, right? And so now we're starting to make an informed decision that isn't based on the opinion of different developers, right? And that's part of the problem, especially in an organization that's very large, is one team will say, man, that code is crap. And another team will be perfectly proud of that code, right? Because it's very subjective. But if we can actually provide objective measurements, we can change the conversation. Cool. Well, thank you very much, everybody.