 Before coming to our camp, this is my first year. I'm really excited to be here. Before coming to our camp as an American, I didn't know very many things about Belgium. One of the very few things I knew is that you guys are obviously much, much better at soccer than we are. Not even Tim Howard could save that. But when I got here and I started walking around, I was amazed by how beautiful the city this is. And yesterday, after getting in, was just kind of wandering around and I was looking for food and looking for a place to get cash so that I could buy food and all those kinds of things. And I felt like everywhere I went, I kept seeing things that I had never seen anything that looked anything like that before. Maybe this is very normal to you guys, but what an amazing place. What a beautiful place to have a conference. I'm really glad that I get to be here and get to speak with you guys and get to meet a lot of you. And so the story that I wanna share today, it's a story of high adventure and a story of danger and intrigue. And this story ends in happiness and delight. And that's why I wanna share it. But like many stories of high adventure, it starts in the darkness. So a long time ago in a job far away, I was working for a company that among other various activities that they did, they put on these educational events, right? They were like seminars. And like you might expect, the event has a date and a time, it's connected to a venue and then it has a list of attendees. And really early in this company's history, they had realized that if you were gonna invite people to something, then like you need to be able to answer the phone and tell people, like answer questions for the people who are coming to the event and you need to be able to cancel their registration for the event and all that kind of stuff. And that was way too much work for a startup. And so they made that somebody else's job and this other company would just send an email once before each event with a list of here's all the people that we should expect at that event. And eventually that got written into an API and this is the part of the story where I got involved. And so the person who had written the original implementation of the API, he had left the company and it had been a little while and my boss one day walks into my office and he's like, hey, we need to start accepting some registrations from a new vendor. They're a little bit different than all of our other vendors and we wanna make sure that when they send a registration to us that it's marked differently in the database so that we can compare how the attendees that we get from them compared to other attendees and decide if it's worth it to continue working with them, et cetera, et cetera. And as all developers do as we all do, I kind of did the quick checklist through my head. I thought, okay, what's gonna be involved here? How hard is this gonna be? Well, the API already exists. It's already live. It's already taking registrations. He told me that they are sending the exact same field names and same types of values as all the other vendors and I'm like, okay, well, this should be really easy, right? This is sure thing. I mean, this will be done in a couple of hours and the next thing you know after I clone the repository and open it up, I see something like this. And at a moment that I'm looking at code like this, I tried really hard to garner some empathy for whatever developer was in some circumstance that this is what ended up coming out. And in reality, this code is actually just a piece of the Zen framework. The real code was slightly worse than this even. The main method that I needed to fix was a little over 350 lines and it contained loops and conditionals inside those loops and in one case, a loop inside the conditional inside of a loop and I don't know what circumstances led someone to create something like that, but please don't ever do that, ever. And so as much as I tried to think back on the mistakes I've made in my life and I still felt cheated that I had to work on this code. And about this time that I'm coming to this realization that this code is in a really terrible condition, my boss walks back into my office and he's like, hey, I actually found out. They started sending us registrations last night and I'm gonna need you to fix all the ones that got marked in the database incorrectly and it needs to be done by then and day to day. Okay, thanks, bye. And this is probably a position that all of us have found ourselves in at some point or another. And at this point, it was pretty obvious to me that if I wanted to work effectively with this code base in the future, I was gonna need test coverage. This code had no test whatsoever. And I was petrified of the idea of breaking this thing, right? Cause it's just a big ball of PHP code and I had no idea what the data looked like as it was coming in. I just knew what was there in the procedural code in front of me. So I was gonna need to have some test coverage. I was gonna need to have some sample data of what data really looked like as it came in from these different vendors. And I'd also probably need the help of some sort of domain expert from the business who knew why there was all these edge cases programmed into the API in the first place. And it was also very obvious to me at this point that given my deadline, I did not have time for any of this. And so I went back to my office and I cursed and I swore for about eight hours and after eight hours, I shipped something that I wouldn't say I was confident that it was going to work, but I was hopeful that it was going to work. And after I pushed it live, no one came and yelled at me. So I went home and that was a day of work. It wasn't a good day of work, but it was a day of work. And this is the point of the story when I hit my first really big break. I accidentally got very lucky. And I got lucky because something interesting happened which is that nothing happened with this system for about two months. And so for two months, I had this terrible memory rolling around in the back of my brain about this traumatic experience I had had with this terrible code. And whenever people in the office were swapping war stories of like really terrible code they'd seen, I'd be like, oh, I got you topped. This thing actually exists in our system right now and it's so terrible. And I would find myself trying to explain to my coworkers and other developers on my team how I would write it, what should it look like if I was going to be writing the system again? And I found myself using words like, it would be a pipeline of operations. It would be the data would flow in one direction through the system. It would have really, really great logs. One thing I really hated about working on it the first time was I didn't really know what the shape of the data looked like as it came in. And I didn't really have good way to connect it to something that existed in a database. So it was very hard to know if I had made a mistake. And I was like, it would definitely have very thorough logging so that I could go back and audit bugs and find things that needed to be fixed. And then I also really wanted to have this idea of instead of having all the edge cases from every vendor that ever sent us registration in one big file, like let's put those in like per vendor files somehow so that at least you only have to deal with one set of edge cases at a time. And something really interesting happened. When I was giving this description of the system to people on my team, people who were somewhat familiar with the business case at hand, they would look at me like I was stupid. They would have this look on their face like, oh, this is so boring. I can't wait until this conversation is over and I can get back to whatever I was thinking about before you started ranting about how terrible this PHP code is. And it made me realize that I was having, I had a very poor description of my system. I wasn't doing a good job of describing what was in my head. But I didn't yet know how to have a better description. And I got lucky once again. At the end of the meeting, someone, another person from our team came up and they asked me, hey, I heard you're thinking about a new registration system. What are you, like, what's gonna be different about it? And this person was, had all the business experience, but no technical experience whatsoever. So I couldn't use words like pipeline or logs. And I struggled, I kind of hummed and hawed for a few minutes before I finally started by saying, have you ever registered for a university class? It'd be kind of like the registration office for a university. This is something that everyone in our team had attended at least one semester or at least signed up for one semester of college at some point. And so everyone had some experience of what this thing would look like. And I said, okay, you can imagine that inside that registration office at the university, there's gotta be filing cabinets on filing cabinets on filing cabinets, just full of paperwork. And you could always trace back through when someone registered for a class and who's registered for which classes and stuff like that. And if someone wanted to register for a class at the university and they didn't natively speak the language of the university, the university would provide some sort of a translator to help them fill in all their forms correctly. And when I gave this description of the system, the person listening was kind of like nodding his head. Like he had some mental model already there that he could latch these new ideas onto and it made sense to him. And this relates to a really well known software design principle, right? This is something all of us kind of know about. This idea of putting abstraction into your system. I think it's probably generally true that if someone wants to pay one of us in this room money to write a system, there's probably enough inherent complexity that you don't want it just to be a big soup of things that all talk to each other with no sort of organization. You almost always want to enforce some sort of organization on your code. But what's interesting is that knowing about this software design principle wasn't actually what got me to a better description of my system. What got me there was talking to human beings, specifically talking to different audiences. If you talk to another developer about something that you're thinking about, it's gonna, you have to see the world through their eyes. And if you go talk to someone else who's not a developer, maybe someone on your team who's a designer, maybe someone on your team who's in marketing, you're gonna have to try to find a way to describe your system in the language that they know. And it's gonna teach you a different set of things about what you're building than talking to another developer. So at this point, it's been a few months. I've ranted to lots of people on my team. I've stumbled my way on to a better description of my system and I'm starting to form this idea in my head, the shape of a system that I wanna create. And now I'm getting antsy to work on something more concrete than just words. So I drew a picture and this diagram, this was one of my early attempts and this diagram is basically a sin against diagrams. So please, for the love of all that's holy, don't ever draw anything that looks like this ever. And there are many reasons that it's terrible but we'll talk about just a few really quickly. One thing that's really terrible about it is it jumps between levels of abstraction. And so for instance, it talks about JSON, like what data format are we gonna use to talk to these other systems? And it has very fine-grained details like when does a call to the database happen? And in the same diagram, it's talking about very high-level concepts like mappers and mappings and registration processes. And these are obviously those are concepts that would require lots of details in order for them to happen. So it's a little bit all over the map. Another thing that's clearly wrong with it that I noticed when I was trying to draw this for other people on my team is once again, when I would draw with just like shapes and arrows, people would get that same glassy-eyed look of just like, oh man, this conversation needs to be over soon. And so I knew that this was a bad representation of my system, but I didn't know yet how to make it better. And once again, I got very lucky. I was in a meeting, including the founder of our company and as is probably very common amongst founders, at some point in the meeting, he launched into some 10 to 15 minute tirade about how our startup was the best startup that has ever started up and blah, blah, blah, right? And you can kind of tell when those conversations are coming and I had been taking notes and I only had a little bit of space left at the bottom of my page. And I decided to try to sketch my system again, but I realized that with such a small amount of paper left, I was gonna have to only pick two or maybe three things to draw. And so I stopped and thought, if I can only draw three things, which three things would I draw to represent my system? And my next attempt looked something like this. And this is way less symbols, but kind of did a better job of conveying what my system was really trying to do, right? I don't know what a 109C is, but all government forms from all governments everywhere always have letters and numbers in them, right? So like anyone who looks at this knows that some sort of standardized form. And the idea of a university, you know, like they always have pillars somewhere, right? I think that's like a rule. You can't be a university without pillars on your campus somewhere. And then I imagined that there would be like a vault, some underground like secret layer where all these filing cabinets lived with copies of every piece of paperwork that any bureaucrat ever touched at this university. And then when I went back to my office and I started to draw this for another developer a few days later, I found myself being able to take those three concepts and I could dive into any one of them and I could kind of draw a story about what was happening inside of that piece of the system. And this made a lot more sense to me and it made a lot more sense to the people who I was trying to share it with. And I don't know why this receptionist is holding their own head on their shoulders, apparently it looks like, but apparently that's critical to their job. And I could start to see some stories, some narratives that would be happening within the system. And it was pretty clear to me that if I was gonna start writing the system tomorrow, I had at least kind of my top level of abstraction mostly figured out. I knew how I wanted to represent the data. The data should look like some sort of standard object that had the same interface no matter who had sent it in and that all of the mapping and edge cases would have been taken care of before whereas a part of constructing that object, that standard interface. And I had some idea of the major process types of objects, the sort of verb type objects in my system and what they would be and what I might call them. And this once again relates to a really well known software design principle, intentional abstraction. Like we all know that we should put abstraction into our systems and not just have everything one level of abstraction. But once again, what's interesting is that even though I knew about this principle, that's not how I arrived at this concept of my system. The way I arrived there was by drawing and drawing under different sets of constraints, drawing, trying to draw with kind of big chunky objects and very few and then trying to draw it with smaller, more fine grained details but being intentional about what I was trying to draw and what things I wanted to convey with this sketch. And so at this point, things finally got off the ground. Once again, my boss one day walks into my office and he's like, hey there's, looks like there's a little bug maybe with one of our vendors sending us registrations and they wanna like send us some AB testing value and we're gonna send it back to them or something. And so I was like, okay, here's the deal. You can either pay me eight hours of swearing and cursing in my office which is right next to your office so you get to hear it all day long or you can pay me for 16 hours of work and I'll get you the first prototype of the new system. And the first prototype is only gonna support that one vendor but it's gonna have test coverage so that we stop breaking this thing all the time and it's gonna be written in a way that other developers on the team can actually get their hands on this thing and make some progress. And that was my pitch to him and probably mostly out of pity for me, my boss said, go for the new prototype, let's start to build it. So I had these two days to start working on this thing and I thought, this is awesome. I'm back in my comfort zone. I know how to write code. Red, green, refactor. I got started and I was like, okay, the receptionist, this is like an important piece of the system. This is sort of like a router or controller thing and all right, it gets going and my tests are failing and then they're passing and I'm feeling good and I get going on the next thing and I get to the vault. Something terrible happened. I felt so awful because the code in my vault was terrible. I mean it was probably slightly better than the PHP code that had originally started me down this path and I thought, how could this happen to me? I had such a clear vision in my mind. I knew about this beautiful next iteration of my system and when I went to write it, it was terrible and I was like, okay, well, let me try and refactor it and I took a first pass at refactoring it and when I refactored it, it broke a bunch of my tests and so I threw away those tests and now I only had like two tests and my two tests really were just kind of like exercising the code from outside the vault. It was just kind of like the vault level testing and my refactoring was like maybe 5% better, maybe 10% better, not much of an improvement and I was feeling terrible but I looked at the clock and I was like, this thing's got a ship. I don't have time to clean it up and so I got really lucky again. Sandy Metz has a really great talk which probably lots of us here have seen about leaving a mess in your code and she talks about how there's some messes that when you leave them in your code it's gonna cost you a lot down the road and there's other messes, she calls these omega messes, they're kind of on the edge of your system somewhere where it only gets interfaced from a few places and that mess cost you a lot less and I got really lucky that without knowing about this I just happened to stop at a time where the boundaries of my mess were pretty well-defined and had some test coverage. So I had some basic confidence that my system wasn't doing anything totally crazy but that was basically it and I just left that mess and I shipped it and it went into production and it was taking registrations and things were okay but I went home not feeling very good and then I got really lucky again. Late at night one night I go to sleep a little too late and after a little bit too much soda and chips and I had this dream where I was looking at the code in my vault and all that code became characters from the TV show Firefly and when I woke up it didn't make any sense at all but what was interesting is when I woke up I realized that when I was watching this story of how my code was talking to each other that there was kind of three main characters and that like Mal and Zoe pretty much made up the bulk of what was happening in that story and even though I definitely didn't name any of the classes in my system Mal or Zoe I thought about it but I didn't do that but it did help me to start to see some patterns in that mess of code that I had created and so once again about another week later my boss walked into my office and was like hey looks like maybe we have a little bug and I was like great these are your options today eight hours of swearing and cursing in PHP code or eight hours of I can just move them to the new system I'll make sure all their names and everything are the same supported the same way they were they can use the same password we'll just tell them to swap out their URL once we're ready to support them on the new system and he was like do it let it be done and so I got to work and it actually turned out that the only thing I had to write was a translator because they were sending and doing the same use cases with the data as this other vendor the first one I had supported and so I got those tests written in passing like two hours and I was like oh I have some time and so I took my ideas of Mal and Zoe and their crazy adventure of code and I went back and started doing some refactoring in the vault and at the end of those couple of hours of work it wasn't great code but it was a lot better and instead of going home feeling like a crying unicorn I went home feeling like this I'm a leaf on the wind watch how I saw I felt amazing I had created a monster and I had tamed the monster and I was feeling like an amazing programmer once again and once again this is like a well-known software process or design principle sort of thing right iterative refinement the idea that you're not going to get the design of your whole system right on your first try it's basically never ever going to happen even if you got it right your business is going to change and then you got it wrong and but once again what's interesting is that all this emotional turmoil I went through I knew about iterative refinement we were a fairly agile sort of shop I should have known better but emotionally it still felt like this terrible thing to go home and leave a mess in my code and it wasn't knowing about iterative refinement that got me to a better state of my system what got me there was telling a story so it turns out the human brain is really well adapted for solving social problems and if you can frame a problem that you're having in terms of a social problem there's already a whole bunch of heuristics and shortcuts and all sorts of things that your brain already has to do all the time that it can apply to that problem if it can see it like a social problem so telling yourself a story about your code can be a really powerful thing the other thing that I lucked out on was that this project happened to be one of those projects that kind of came up once a week or so and we kind of all know this but the bottleneck in programming is not characters per minute that you can type it's not about how many Vim shortcuts you have or bash scripts or anything else like the question is how many good ideas are you gonna bring to the table how many times can you get creative about the problem you're solving and find a simpler way to express that problem something that someone else will be able to make use of in the future so after all of this did it work well a few months after that part of the story it was time for me to move on I was gonna leave that job and I was going to a new job and I called one of the junior developers on our team into my office and I said you are now going to be the caretaker of Mal and Zoe and he really didn't understand and then I said it registrates the system and I spent about 30 minutes I kind of drew the diagram I told him the metaphor of the system and kind of gave him a quick intro and gave him a task to work on sent him on his way and he worked on it for about an hour and I kind of stopped by his office he had run into a few questions we talked for about another half hour and at the end of that half hour he never had any more questions for me he got done with his task around the end of that day and ended up working on two other poll requests before I had actually left that company and never really needed to come and ask me about it again and right about this point in the story was when I realized that I had stumbled on to something pretty great I was leaving this company and there was a piece of code in their ecosystem that I was proud to have built I was like really happy that I was leaving behind a project instead of telling the next developer like oh hey it really does this and this and this but when you read the code it's gonna say this and this and this but just kind of ignore that I didn't have to say any of those things like the concepts that were in my code matched the way that the business talked about that problem and that was something really powerful and Sandy Metz again has from her talk on Ruby on Ales she had this slide and this quote made its way around Twitter quite a bit and it's a pretty strong statement but one that rang true with me and she was talking about drying up your code and basically her point was don't be too aggressive about trying to dry your code if two objects sort of look similar but you're not really sure what the similarity is yet or you're just kind of guessing it's cheaper to leave the duplication than it is to get that decision wrong and I am totally unqualified to mince words with Sandy Metz but I'm going to anyway and in my experience so far I would actually say that almost anything that you can do wrong with your code usually ends up being cheaper than having the wrong abstractions because usually the business problems that we're trying to solve are complex enough that someone couldn't do it in Excel or whatever else they were gonna use they needed some tool that had a little bit more power behind it mostly that's true and if that's the case then when someone has to go in and read that code they both have to try to figure out what is the business trying to do how does this make us money why is this important to our business and they have to try to figure out what the heck were all these abstractions that are in your system right now and why is this thing doing like metaprogramming inside of a method to generate a module and include it and like if they don't have to think about that they're gonna be able to get a lot more stuff done and another thing that I learned as I reflected on this experience was that I had spent a lot of time reading design pattern books and object-oriented design books and reading and following people like Kent Beck and Sandy Metz and Avdegrim and those things were great they gave me a great set of tools that if I looked at two possible implementations of a problem I could understand the trade-offs that I was gonna be making I could see that this one was doing a better job of adhering to single responsibility principle which is important under some circumstances and maybe this one was a little bit more readable but mixed up some of the concepts a little bit more and sometimes I could take the best of both of those and come up with a third solution but sometimes those things end up being at odds and it's important to be able to make the trade-offs knowing what trade-offs you're making and knowing about software design principles helps me to do that but knowing about software design principles doesn't help me to generate those two ideas in the first place and at least in my own experience when I'm doing red-green refactor I'm getting just enough feedback to make a few small design decisions but I don't generally think of 20 possible implementations maybe I think of one or two or maybe I'm pretty much sure what the structure of my code is going to look like and I'm only really debating should this code use a select or a filter and I'm just thinking about readability level design decisions but the creative process is where I actually was getting these new ideas that had drastic simplifications from my system and made it much more understandable for the people on my team and this made me realize that that I had been so lucky that I had worked on this project that it didn't need to be done in a week what it needed was to have a small thing done in a week and then another small thing done a week later and then another small thing done a week later and that had given me the time to sleep on a lot of these ideas I had been able to generate a lot of ideas and then before committing them to code I was able to realize that they were actually bad ideas and throw them away and come up with something better and at this point I feel like usually this ends up shaking out to look something like this that if I'm starting a new project I'll usually end up spending the scale here horizontally of time is not linear by the way but near the beginning of a project I'll spend a lot of my time thinking about the design of my system and just trying to get creative and I'll only do enough writing of code and refactoring to try to get an idea of do any of these ideas actually make sense and could I put them into code and then once I get sort of past that initial stage it gets to maybe about 50-50 I'm writing a lot more code but I'm still taking a lot of time to step back and go this part of my system feels terrible do I need to rethink it a little bit am I choosing the right abstractions for my system and usually once I get past the first few weeks on a project then it gets into a stage where my only design work that I need to do is just kind of that red-green refactor it's just about saying is this method readable is it understandable, does it have good names those kinds of decisions and all of a sudden my design time will go down really low but it's important to remember that every once in a while a feature request is gonna come up that just doesn't really fit inside the set of abstractions that you picked initially some features gonna get requested that requires a new concept in your system and whenever that happens I try to force myself to step back and take a little bit more time to say should this be just one more special case or should I introduce some new abstraction should I change some abstraction in my system and how much would that cost me to try to change that and then usually the design time drops off again after that and the other thing that I realized after all this is that I had spent a large part of my career up to that point being a first draft author I thought about so I love to read science fiction books right I love authors like Asimov and Heinlein and those guys and I thought I wonder how many times how many of the books that I've read were their first draft and I'm pretty sure the answer is none of those right I'm pretty sure none of the classic books that I really loved were like their first go at that novel and yet what percentage of the code that I shipped to production was my first draft was just like the first way that I thought of solving that problem and I didn't want to keep doing that I had this I had a gut feeling that if I took more time to design with more care of my system that I would actually end up being able to ship code faster on a more reliable schedule but until I actually had some more experience of doing it that was just a guess it was just a hunch right it was a hunch sort of backed up by experience of other people who I heard talking in industry but I needed to be able to experiment and actually do it myself and in this case for me that meant making a change of teams and I went to work for a company called MX and that's where I'm at right now and all this design stuff is not for free right Sandy Metz talks about this a lot too that like trying to do good design it does cost you time up front there's gonna be some extra time that you could have just done the red green refactor and only gotten that much design feedback about your code and you're choosing to spend some other time doing some other activities in the hope that it pays off for you later and in my experience thus far about a year later it totally does and the timeline for it paying back so far on all the systems I've worked on looks like something about three to six months that when you're about three months into a system if you've been taking time and care then your velocity ends up much more maintainable beyond three months whereas if you're working really hard in the system and just getting out the door over and over and over again then I usually see that tail off start to happen and there was a really interesting tweet about this pretty recently about go carts in 747s right so when you're driving in a go cart like well first of all it probably just feels like Mario cart right I'm like thinking about shells and what shell I should shoot at the person in front of me in another go cart but it feels really fast right like when you turn in a go cart you feel it in your gut and you feel the G force of like and you feel your tire start to slide and all that stuff and it's exciting and it feels fast and when you're sitting on a 747 you do not feel that way but you're cruising you're making really really good time and you really really wouldn't want to ride a go cart to our camp from America and so in this case the person tweeting this was talking about documenting and testing but I feel pretty strongly that the same thing applies to design and being able to pick the right abstractions in our systems and as a final thought of reflecting on all these things I was thinking the other day about Rich Hickey his really great talk about hammock driven development and he was talking he asked that question at some point in his talk he says when was the last time that you thought about a hard problem for a whole hour when's the last time you thought about a hard problem for a whole day or for a whole month or across this time span of an entire year so all of us in this room someone pays us money to take abstract ideas and make them so prescriptive and so concrete that a von Neumann machine can do them over and over and over without making a mistake so our stock in trade is about taking abstract ideas and making them executable we are paid for our thoughts and ideas and yet I spent an awful lot of time thinking only the minimal amount of time that I could about how to make something possibly executable before I got it out the door and that is I don't wanna come off as elitist right there's definitely a time and place for making those trade-offs and for saying hey sometimes that three month window is all I need to be able to really know if customers wanna pay me for this or maybe this thing just needs to get out the door because the servers are on fire and our customers are furious right it's not that this trade-off never makes sense but as a general rule I didn't want for my career to be about that rough draft mode I didn't want my career to be about how fast can I write code I wanted my career to be about how high of quality software could I create what was I capable of if I had the time to stop and think so what I really wanna advocate with this time is somewhere in the spectrum of your feelings of passion and enthusiasm maybe they range from something like the love of your family down to like getting an oil change in your car right and if software on that spectrum falls somewhere closer to the love of your family if it feels important and exciting and meaningful to you then you should probably find a space a team or a set of circumstances where you can write code that you get the time that you want to think about what you're writing and how you're gonna build this thing that you're building and for me at least I found that it leaves me at the end of a story feeling happy and delighted and leaving pieces of code behind me that I'm proud to feel like I worked on and I don't have to make excuses for that's it that's all I got thanks guys