 My title is kind of odd, ThoughtWorks lets you choose your own titles and mine has gradually grown over the years as the things I do has kind of grown over the years. So my title is director and software architect. That kind of makes sense, but the last one is a meme wrangler and that probably makes no sense at all. But it's a kind of an abstract title, but it's really, so a meme is this term coined by Richard Dawkins in his book Selfish Gene and it's a basically a meme is a viral interesting ideas that come up on our projects and do something with that beyond just the project that it's on and I go into this explanation because this talk is a direct result of that role within ThoughtWorks of looking at interesting things across projects and trying to derive conclusions from them. And so that's what I'm going to do in this talk. The origins of this talk came based on conversations I was having with the technical folks at one of our projects called OVE, which is online vehicle exchange. What they do is an important for our purposes. They are actually an auto auction site for used cars. So if you're car dealership in the US, you can get a log on to ove.com online vehicle exchange and do an online auction. But I'm much more interested in the technical aspects of this project. This is a Ruby on Rails project that was started very aggressively almost five years ago now. In fact, the very first check-in was in October of 2007. So we're coming up on six years now, I guess, in October of this year. And one of the goals of this project was to try to the the client had been number one in their market. They'd fallen number three and they were falling more. And so they desperately needed something to revitalize their business and get them back to the top of their market. And so we went into this project very aggressively with this idea of emergent design. We'll start with as little as we can and then grow as the code base and the problem domain grows. About three years in, I started having conversations with some of the tech leads here. I've worked on this project on and off for its entire life. And they were concerned about some of the things that we're seeing in the emergent design space, some challenges. And that spread to some other projects. And I started chatting with people on other projects, bringing the challenges that the OVE folks had identified. And seeing if those same challenges existed on other projects, it turned out they did. And so what I've done is consolidated a bunch of those things into this talk. So this is an ongoing R&D effort. I'm going to keep building on this IP within ThoughtWorks. And I'm right now in the process of reconciling this with the product innovation practice within ThoughtWorks. And you'll see some aspects of this within this talk. Because one of the things that I do, I'm a purely technical geek kind of person. So I'm very concerned with agile engineering practices. So when I attack this problem, I attack it from the bottom up, from the architecture up toward things like product features. But our product innovators attack it from the top down. They care about features and how do you manifest that all the way down to something like architecture. And so that's what I'm trying to do now is actually reconcile the emergent design space with this kind of product innovation space. And we'll see how well I'm doing. I'm going to give a brief introduction to this idea of emergent design. I'm actually doing an entire talk on just this core concept on Friday. So if you're interested in the kind of core, some of the tools and techniques that are used in emergent design, I'll talk about those on Friday. I'm just going to point out a couple of things here in terms of definition. So emergents, of course, very closely related to the root word emergent, which has a couple of definitions. According to Webster's dictionary, one definition is riser and emerging out of a fluid, or anything that covers or conceals issue and coming to light. Well, most of the time our software does not come out of a fluid. And so I've edited this definition just a little bit to make it more appropriate for software. This is actually pretty appropriate for software, though, rising or emerging out of something that covers or conceals it and coming to light. And the other definition for emergent is suddenly appearing, rising unexpectedly, calling for prompt action and urgency. It turns out that these two definitions actually closely match two things that we see in the software world. And let me talk about those two things. So one aspect of emergent design is that conceptually at work you have this problem. And you shouldn't stare at it while you're at work, you can't figure it out, and you think about it while you're driving home, when you're mowing your lawn, you're thinking about it, when your spouse is talking to you and you're supposed to be paying attention to them, they're still part of your brain that's still working on this problem, because you know that this is not right, this jumble of stuff that you have here. You finally think about it hard enough until it finally resolves itself into some sort of structure that makes sense. This is really the exercise of finding abstractions and patterns that are already there in code. So if you've got some code, these are techniques where you can go and find reusable chunks that you can pull out and make use of them. Not quite design patterns, kind of patterns, but little things that work well in your application that you'd like to leverage and use again. So that's one aspect of emergent design, is finding abstractions and patterns and existing code bases. The other aspect is this concept borrowed from the lean movement of this idea of the last responsible moment to make decisions about things. Very frequently in software projects, we make decisions up front when we have the least amount of possible real data about the real problem we're going to face. And we run into problems down the road when the real problem manifests as not the problem we thought it was when we were making important decisions in our project. The idea of last responsible moment is try to defer decisions as long as you can because the longer you can safely defer to the decision, you have gained more knowledge and context about the true nature of the problem, not the problem you thought you were solving. It never turns out to be the problem you thought you were solving. It always morphs into something else. And so if you could put off those decisions, you're working from a place of more information. But you can't put it off too long. That's this idea of not the last moment, but the last responsible moment. And the really tricky thing, of course, in the Agile Engineering world, is how do you identify exactly what that last responsible moment is? That's the kind of stuff I'm talking about in my emergent design talk on Friday. But the point here is that the longer you can safely delay the more real data you have to be able to make a decision. So those are the two aspects that I'm really going to talk about in depth in this talk. And it turns out that these two aspects very nicely correspond to the two kinds of systems that we frequently see out in the world. Greenfield systems and Brownfield systems. Finding abstraction patterns works really well for existing established code bases because you want to harvest things out of it that are reusable whereas last responsible moment works best for brand new projects or Greenfield projects where you're thinking about these kind of architectural concerns. And we'll see this split show up in a bunch of different places. Speaking of splits, there is a, I'm not sure even what to call this, a design aesthetic that I've kind of identified. And part of that design aesthetic comes from this document. This is a white paper that our chief scientist, Martin Fowler, published several years ago, 2006, someone would say. It's still available for download from that link. This is basically Martin's take on the role of architecture on agile projects. And so it's a discussion of what do agile architects do. And along the way, he kind of accidentally gives my favorite definition of the distinction between architecture and design and software. In doing so, he says that he gives actually a lot of different definitions of what architecture might be. But the only one that he can really say with certainty is true, is the one that I agree on that this is my favorite definition of architecture are the things that people perceive as hard to change. The architectural elements in your piece of software are the things that are hard to change later. And in fact, this led me to this design aesthetic that I was talking about that I'm calling the jeweler's hammer. So the idea of the jeweler's hammer is if you can take one big complex subject, just like a diamond cutter can find exactly the right place to hit it and break it and potentially create two more valuable things from the large thing without losing anything, that's the idea of this jeweler's hammer. And I'm going to apply this exactly with the help of Martin to distinguish between architecture and design and software projects. So let me talk about this a second. So architecture, according to Martin, are the things that are hard to change later. Which leads me to another distinction that I need to make here. And that's one about evolutionary architecture versus emergent design. I've actually written extensively about this subject. There's a link here at the bottom. You can also get it off the slides that you can get off of the website. I've got 20 articles about this subject at that link. But I made the distinction here between evolutionary architecture and emergent design. Because you cannot have emergent architecture. You have to have something in place. And that's where this really abstract picture comes in place. Here, the architectural elements are the gray boxes. And the design elements are red boxes. Clearly, if you start shuffling the gray boxes around, the entire thing is going to get unstable and fall down. Whereas the red boxes, you can shuffle around some. And so this is my jeweler's hammer metaphor for architecture and design. The architectural elements are the things that are hard to change later. And the design elements are relatively easy to change later. And splitting it like this gives you all sorts of interesting ways to think about these two different aspects of design here. And in fact, this idea of evolutionary architecture start very small and build it in such a way that it can evolve. But then emergent design is much more flexible because you don't have the same kind of bedrock constraints on design that you do in architecture. The other thing that he says about this definition of architecture, the architectural elements are hard to change later. And you should try to have as few of those as possible. Because notice what architectural elements give us. They give us scaffolding and frameworks. Things that we don't have to write by hand. Because it would take you forever if you had to write everything by hand. But they also represent constraints because they keep you from doing some things. And so the real key here is how can you minimize constraints and maximize flexibility. And that's what we're shooting for here. That's one perspective on architecture. And that picture is good from a kind of shuffling things around standpoint. But it's a little misleading in that it tends to suggest that you have a lot of architectural elements and that's not true. So let's look at another picture of this. Where you have architectural elements at the bottom, which are hard to change later. And you have as few of these as possible. Above that, you have design. And this gradually becomes easier to change later as you go. But then on top of that, so in the past, this is basically where I've stopped. It's architecture design and then there's some other stuff that happens from some business people or something up about that. But in trying to reconcile this with product innovation stuff, I've realized there's yet another design layer here up at the top which really manifests as features that end users can consume and do things with. And this is exactly the dichotomy I was talking about. As an engineer, I always start from the bottom up and the product guys always start from the top down because they care about what discrete features they can deliver to their users. Another way of thinking about this is that you have these two layers and this is the realm that I have primarily been dealing with up until now is this design and architecture space. And in fact, I'm splitting these out into technical architecture and then product architecture up at the top. And so you now have enough to understand what my agenda is for this talk. I'm going to talk a little bit about how Agile architecture differs from traditional approaches and then I'm going to do lessons learned. There are nine different lessons but they're actually split in two because it turns out this transition from green to brown, from green field project to brown field project is a really, really important inflection point that most people ignore to their peril. And I'll talk more about determining what that thing is and then the rest of my lessons. The first thing I'll talk about is traditional architecture versus Agile architecture. The traditional role in architecture tries to come up with a cohesive vision for everything upfront and maybe not soup to nuts but they've got a pretty good idea of all the things that they're going to try to build. They're trying to establish the architectural constraints upfront and basically the master use case list or story list or however they're managing that sort of stuff. And there is basically they're trying to create a concrete vision of what this thing is going to look like for its life. The problem is trying to create a concrete vision for something like this before you've written a single line of code is a fool's errand. You'll never make this work well because software is way too nuanced to ever make this work well. And so we all know the doom and gloom of trying to say this is the only thing that you need and then never change this once you start trying to write code. That never works in software. And so this is really just perceived concreteness of vision. It's not really a concrete vision. The Agile role is very different. We don't have the luxury of thinking about a lot of those things upfront and in fact what we try to build are the kind of basic architectural things that we might need and a very thin vertical slice that shows all the way up to features and functionality. We have to show features to users and so that forces a different kind of constraint on us. We can't build the layers all the way up to the top. We can build some infrastructure a little bit but mostly now we're going to be building the infrastructure and design and carrying on all that effort as the project is ongoing. And so over time your vision becomes more and more concrete and it should be more actually concrete because it's based on actual facts. But the consequences of less time up front to think about this stuff is that Agile architects have to have a really good feel for current events in the code base which is why Agile architects must be coding architects. You cannot be an Agile architect and never touch the code because it's changing all the time. You need to have a really deep awareness of those changes because that is the direction that your architecture is going. So it needs to be a hands-on role and it needs to be able to switch very frequently from macro to micro. Macro being the large-scale architectural concerns has this component reach this last responsible moment does this need to change and then switching down to micro concerns at the coding level. You tend to have fewer boat anchors. That's the classic anti-pattern, the boat anchor anti-pattern that this is on the C2 anti-patterns wiki. The boat anchor anti-pattern is the anti-pattern that says you have to use this expensive piece of equipment or software because we paid a lot of money for it. It makes no sense for your project but every project has to use it because we paid a lot of money for it. You tend to get fewer of those on Agile projects and hopefully you can create an architecture that allows you to pivot and I'll talk more about pivotable architectures in just a second. There's another observation I can make here. In the traditional architect role, what you're trying to do upfront is come up with the best possible solution. So you're shooting for a global maximum in this problem space. The problem though is that a global maximum is very hard to find when you're standing on the ground. So for example, there's one global maximum here and lots of local maximums here but notice that if you're standing on a street and look up at tall buildings, which is the tallest building here? You can't tell because you don't have enough perspective to see. You need to get up on one of the buildings and look across the city to tell what the tallest one is and you can easily see what the tallest one is and that's exactly the case here. When you start at the ground and try to envision the global maximum, you can't tell which is the global maximum because they all look like tall buildings only when you get up onto one of the maximums can you actually see in the distance what the real global maximum is. And so in the agile world, we're tending not to try to shoot for the global maximum which is what the traditional guys do because they rarely achieve it. I would say it's disappearingly impossible to take something as complex as a multi-year software project and nail exactly the right design upfront before you've written any code. Agile, instead, shoots for a good local maximum. We're not going to try to do the ultimate. Let's just try to get up high enough so we can actually see the global and then maybe pivot to a better local maximum and maybe even the global one, but we're not even shooting for that. That's just too ambitious. You could actually get the global maximum if you could write the piece of software and then throw it away and completely rewrite it from the ground up with no changes whatsoever. This is Fred Brooks talked about this in Mythical Man Month that if you want a really great piece of software, write it once, throw it away, and then rewrite it, and it'll be really good because now you know all of the mysterious things that you didn't know the first time. But of course, we tend not to throw things away and rewrite them, and so shooting for a local maximum is good enough in almost every case. So let's start talking about the lessons learned from doing this on projects for a while. And the first one is really obvious, but I put it up here because it seems like no one stops doing this. The number one lesson is don't do stupid things. Don't do things like staff projects inadequately or ramp up projects too quickly because the problem space is really aggressive. Don't assume good initial velocity on this brand new framework that you've got, particularly if you've got three or four new frameworks or using a new language and a new framework. Don't assume that you're going to get good velocity on those things right out of the gate. Don't indulge in resume-driven development. I've seen a lot of projects that were clearly resume-driven development where I need to pack my resume with as many technologies as I possibly can so that the next job I get, I've got a broader chance of getting it. It doesn't do any good for your project, though. And pretty much anything on this list. This is the Anti-Patterns Catalog, and this is a very old site, so much so that I don't think a lot of people know about it anymore. This is the Anti-Patterns Catalog site. This basically lists all the crazy things that you're probably doing on your project that you think are crazy, but you can't get justification for saying they're crazy to somebody. This is a list of anti-patterns. There are a few of them. I'll be willing to bet that your project is doing at least five of these right now. So it's worth browsing this list and reading it and hopefully not getting too depressed about your project. So, number one, don't do stupid things. Okay, that's pretty obvious. Number two is the aggressive use of metrics and monitoring on projects. One of the things that I'm very passionate about in the Agile world is the engineering practices in Agile. I think that's where we still have huge innovation left to take on because we're at the very, very early stages of the engineering practices. That's really what the continuous delivery stuff is, are good Agile engineering practices. And one of the parts of continuous delivery is this idea of metrics and monitoring. Unfortunately though, there are a slew of metrics out in the world. And there's some challenges with metrics that I'll talk about in a second. So, I really just wanted to highlight a couple of very simple metrics here that are directly useful for this kind of emergent design exercise. And I actually talk about different applications of these same metrics in my talk on Friday. And two metrics I'm going to talk about are psychometric complexity and the two coupling metrics, afferent and effort coupling. Psychometric complexity has been around for a geologic age. This metric has been around since the 70s. It started out in structured languages like C, but it works pretty much for any structured programming language. This gives you the numeric complexity for a function or method. The idea being it assigns a numeric value to a function or method based on analyzing the number of decisions and loops and things like that within the function. Assuming that that makes it more complex. Here is the formula. Thomas McCabe created this. This is sometimes called McCabe's complexity. The formula is edges minus nodes plus two where the edges are possible paths through the code and the nodes are the lines of code. So, for example, if you have this little method in Java, you can sketch it out in this edges and nodes kind of view. The lines are a little bit hard to see there, but these are the edges. I have four edges and four nodes, so applying this formula will give me a cycllement complexity of two because two minus two plus two is two. It's kind of boring when the numbers cancel each other out like that, so here's a slightly more complicated function or method, and I can chart it out using this kind of edges and nodes view, and you'll see that this has eight edges, seven nodes, so the cycllement complexity of this code is three. This is one more complex than the code that we just saw before. So, having this metric lets you say which are the most complex methods in your code base. That's a pretty useful thing, is being able to find the most complex things because, presumably, probably some of them are accidentally complicated and not complicated for good reasons. So, it's a very useful metric for that, and I'll show you some aggregate metrics based on that in just a second. So, that's cycllement complexity. The second one that I talk about are these two coupling metrics, afferent and afferent coupling. These refer to how many incoming and outgoing connections exist for a particular code artifact. This code artifact could be a class or it could be something like a package or a namespace. The metric works both ways, and in fact, whether you're looking at packages or if you're looking at classes, depends on what you use the number for, what value the number has. Afferent coupling is the number of incoming connections to a particular code artifact. So, in this particular case, the afferent coupling for this package or class would be five. Afferent coupling is the number of exiting references. So, these reference other classes or packages. The way to keep this straight, the two terms are very, very close together, obviously. Afferent, the first letter of it is E, which stands for short for exit, and these are the exiting references. So, that's the mnemonic used to remember which one is afferent and which one is efferent. Once you have those metrics, you can now apply a couple of more, and these are metrics that come from this tool called JDepend. JDepend is an open source Java tool for doing afferent efferent coupling analysis of packages in the Java world, and one of the metrics they define is instability. Instability is an indicator of the packages resilience to change, where you have efferent and afferent coupling, so the instability is basically the efferent coupling over the sum of efferent and afferent coupling. So, if you have i equals zero, you have a completely stable package, which means that nothing from the outside world references it. It's completely self-contained. It's completely stable, meaning that changes in other packages cannot possibly force me to change this package. Whereas, if you have a one, then you have a completely unstable package. Everything in this refers to something else in another package somewhere, so the chances are extremely high. If you make a change in another package, it's going to break this. This is a coupling metric. This tells you, if I make a change here, what's the likelihood it's going to have a side effect change over here? That is a function of how coupled they are with afferent efferent coupling. That's the instability metric. The other metric they have is abstractness, which is the ratio of abstract classes to concrete classes. And notice that abstract classes here may be interfaces as well. Things that have no actual code in them versus things that have code in them. And a completely concrete package has zero, meaning it's all just concrete classes, and a completely abstract one is one. Taking those two metrics, they now derive a third metric called distance from the main sequence. And distance from the main sequence is this idealized line that basically goes from here to there. The distance from the main sequence is where your class ends up here is how far it is perpendicularly from this main sequence line. Because ideally it should be somewhere on this main sequence line, meaning that it has a good stable ratio between instability and abstractness. And the further it diverges from that line, the more likely you are to have issues with it. That's the goal of this distance from the main sequence metric. So it's the perpendicular distance of a package from the ideal line of abstractness plus an instability equals one. Squaring on the main sequence means that it's nicely balanced between abstractness and stability. And if your ideal package are either completely abstract and stable, meaning they're just specifications for things, or they are completely concrete and unstable, meaning that they actually do useful work in the world. Yes, and the tool that does this is J-Depend, so about Clarkware. So Clarkware, this tool has been around forever, and that's exactly where this comes from. They've implemented a tool that does these metrics for us. J-Depend, the letter J-Depend is J-Depend.org. Actually, no, it's not. I keep thinking it is. It's actually Clarkware slash J-Depend. If you do a Google search for it, it'll be easy to find. The other cool thing about J-Depend, and one of the reasons that I like it so much as a metrics tool, is that it's actually written in Java, meaning that it has an API that you can interact with. So, for example, here is a way to determine dependency constraints on your project. So, one of the hazards you run into, and we'll talk about this in a bit, we talk about component stuff in just a bit, is component cycles. This component talks to this component, which talks to this component, which talks to this component. That makes it very hard to do really intelligent builds because you have all these self-referential things floating around. One of the things that J-Depend allows you to do is actually define rules at the architecture level and let J-Depend check them for you. So, for example, let's say that we have an architecture where we have this util package, and it's okay for everybody to pull stuff from util, but we never want util to pull stuff from, for example, the web package. That would be a violation of our architecture. You can use J-Depend to come in and say, I have all these packages, and here's what my dependency should look like. Analyze it and make sure that's true. So, this allows you to proactively create a unit test that as soon as somebody does an import they shouldn't, it fails a unit test, and you don't have to chase down that circular reference six months later where it's going to be really hard to fix. There are actually some commercial tools that do this, but you can get by perfectly well with J-Depend to do this. You can also look for cycles. There's actually code in J-Depend to go chase down cycles for a particular package or for all packages. Notice there is down here you can say, give me whether cycles exist throughout all of the code that I have here. And if you like this main sequence distance test, you can also programmatically verify that, set a threshold for this, and then see how well you're doing in terms of abstractness and instability. There are two uses for metrics on projects that I find. One of them are as information radiators. I'm going to give you just a very quick sampling of some tools that I tend to use for both these things. The first one I'll talk about are information radiators. And the first one of these I'll talk about is a graphing technique that you can get through several tools. I produce this one with a tool called Source Monitor, which is a freeware tool. A KIVIAT graph allows you to set up ranges of acceptable values for things. So for example, you see complexity there, max complexity at the bottom. That's psychomet complexity, so you can say the maximum complexity I'm really comfortable with in my methods is between two and eight in this case. Whereas the minimum complexity shows up there as well. Actually, the average complexity shows up there as well between two and four. And so basically what you're doing with all these different metrics is creating this range that says, if I have a perfect class, it will be a perfect circle inside the center of the donut. So it'll be exactly here and around the corner, exactly in the middle, that would be a perfect class. And the further it deviates from that norm, the further away it is from the ranges that I find acceptable for this various sets of metrics. So this is a nice way to look at a single class and say, okay, what is the most dysfunctional class we have in terms of metrics, so I can go chase that down. And this tool gives you some options like go find me the most dysfunctional class so I can go and chase it down and try to fix it. Another open source tool that is very useful as an information radiator, something that you can set and run and produce grass and results for you so you can look at later, is a tool called iPlasma. And this creates this pyramid, this iPlasma pyramid. So let me explain what this means. These are all different metrics, some of which we've seen before. So this particular class, this particular project I should say, has 224 packages and 1,289 classes, which means that the number of classes per package is 5.75. We have 7,905 methods and the number of methods per class is 6.13, 45,000 lines of code, and so the number of lines of code, psychometric complexity per line of code is 0.26. So there's five, five and a half lines per method and the psychometric complexity per line of code is 0.26. So looking at this, the average complexity on this code base is about one and a half, because if you take five and a half times 0.26, then that's about the average complexity per method in this code base. And the ones on the right are things like the number of calls that this method makes and the fan out, which is how much it fans out to other responsible classes to do work. This gives you one comprehensive view of a bunch of these metrics and the relationship between these metrics. But the dimension I haven't talked about here is color, because it also color codes these things and this also lets you set acceptable ranges for things. And so you'll notice that my number of methods per class is green because that falls nicely within the range here, but my psychometric complexity per line of code is actually red. It is slightly over what we say the high value, which is 0.24 and I'm 0.26. Eric also has some other metrics on his site like test-to-code ratio, which is always an interesting metric. And this one is a very useful metric, psychometric complexity per line of code. This is that 0.26 number we saw. This is actually the same code base that produced that 0.26 number. That is this red line right here. I believe that's 0.24. So that's the threshold that iPlasma had was 0.24 and we're mostly over that. But the interesting thing about this is you can see this is psychometric complexity per line of code with the releases of this piece of software overlaid in gray numbers. And you can see the interesting characteristics here of the amount of volatility early on as they were rushing to try to get things done. And 3.0 was the first real release of this code base. And you can see the trend is kind of disturbing here because it's trending upwards in complexity per line of code. And Eric used this chart to convince the project manager we need to do some structural refactoring. And that's what 3.0 and 3.1 were. And the effort of that refactoring smoothed things out a bit and carried on to this trough down here at the bottom. So this is actually a very good graph that shows things like technical debt and had technical debt increasing and refactoring efforts to pull that technical debt down. This is a really good chart to show that. The other typical use that I have for metrics are as probes. I really like information radiators because you can wire them into things like continuous integration. They can produce reports for you. You just have to remember to look at them. But there are also situations where as an agile architect you want to be able to specifically probe to find some things. So let's talk about some of these. One of these is a tool. It's based actually on a very old small talk tool called Code Crawler. But this has been written by a graduate student for Java as an Eclipse plug-in. It's called Xray. And this gives you a slightly different view of a code base. This is a code base in Xray inside Eclipse. And these all represent classes, these lines. And in fact, the length of the box is the number of lines of code. And the width is the number of methods. And then the lines indicate inheritance relationships. So you can see just by glancing at this that this is a very complex method with lots of lines of code and lots of methods in it. It had better have corresponding responsibilities. And if it's not, then it may be overly broad. You can also use Xray to find things like dependencies. Xray also reports afferent and efferent coupling. So here's the afferent coupling coming into that class in red. And the pink are the efferent coupling outgoing from that class. And so you actually get the class and some details about it and its coupling characteristics at the same time. Another tool that is actually written in small talk but still downloadable and against a lot of different kinds of projects is a little bit like Xray but a 3D version of Xray. It's this thing called Code City by Richard Weddle. The idea of Code City is that it takes that Xray view and creates a 3D cityscape from it. So this is JDK 1.5 done as a cityscape. And it's the same kind of characteristic, the width of the base is the number of methods, the height is the number of lines of code. So you can tell some interesting things about a code base just by looking at this. What's really cool about this is it'll actually let you do flybys from your code base. So you can rotate your code base, look at it as kind of a virtual city. You can flip over to the overhead only view if you just want to see the overhead only view, which is that. You can also turn on some flags. So there's one that says brain class only and God class only. And so the brain classes there are in yellow and the God classes are in red. So this gives you a really good target to go find out should these things have all that responsibility because these God classes should be controllers. If they're not, then there's clearly something going on there. This gives you a really nice kind of cityscape overview of your code base. The other cool thing about this is time sequence. You can look at your code base through several different snapshots and watch it grow over time. Now what's interesting about this, I want to go back just a second between two of those transitions. Notice the transition from here to here, between those two transitions. I just went back and forth between those two transitions. So there were a lot of changes to code base between these releases and so these tall spindly structures, what do those mean in code city? Well, they're classes that have a lot of lines of code but almost no attributes and so they're all static methods. And so apparently a functional programmer joined this project because they created a whole bunch of packages that have nothing but a bunch of static methods in them. I would want to know that as the architect of this project if that's not what's supposed to be going on then this will show me right away, hey, wait a minute, somebody's creating some buildings that don't look right in my city. I need to go poke around and find out what's going on there. This is a really great investigatory tool that lets you kind of see the evolving landscape of your code base. I don't want to suggest that metrics are any kind of a magic weapon against complexity and software. They have some limitations. One of them is a signal to noise ratio. This is one of the difficulties a lot of metrics is that the metric is actually telling you something interesting but there's so much noise around that metric that it's drowning out the signal so you have a really bad signal to noise ratio on some metrics which is why you frequently need to use several metrics to make a case about something because one metric won't really tell you anything because it's getting drowned in noise. The other problem is there's no one true metric that tells you anything magical about your code base. It's almost always some combination of metrics to make a case about something. Another common limitation is that they are gathered and then ignored. I've seen a lot of projects get very excited about metrics wire a bunch of metrics into continuous integration and not look at it for a year and a half. A year and a half later they look at it and go, oh look, our code base is a wreck. Maybe we should look at this before that. And also inaction and overaction are problems here. And interestingly, one of the biggest problems with metrics is overaction. And that's due to this phenomenon that's known as the Hawthorne Effect. The Hawthorne Effect was an experiment done by a company called Western Electric in Chicago. They were an electric company. And this experiment took place in the Hawthorne Building, which is why it's called the Hawthorne Effect. And the idea was Western Electric wanted to see does the amount of light in the room have an impact on worker productivity. This was done in the 1920s when electricity was still kind of a new thing. And so they gathered a bunch of workers together and said, okay, we're going to do this test. So they measured their baseline productivity and they raised the lighting level and people got a little more productive. They got really excited because they're on the verge of proving the more electricity used, the more productive you are, which is fantastic for an electric company. And then just to make sure that they knew what they were talking about, they lowered the lighting levels to the original level and people got even more productive. So it turns out it had nothing whatsoever to do with the lighting level in the room. It had to do with the fact that they told them we're measuring your productivity. And that is the Hawthorne Effect. Measure and let it be known that you're measuring and people will instantly start gaming that metric to a pathological degree. And so if you get really excited about something like code coverage as a metric and you tell everybody on your team, we're going to go hard for code coverage, you'll eventually see people doing crazy stuff to try to get code coverage to be perfect when it's not really adding a lot of value to your project. So you have to watch out for overaction as much as inaction because people get too caught up and what a single metric is showing them and let that drive your code base into a ditch where that metric is perfect but a whole bunch of other ones are not. And you'll notice that several of my metrics here tend to prefer trends over discrete values. Those are almost always more useful in the agile world because you have a cyclomet complexity on a class of 12. Is that good or bad? I don't know. It depends on what it does. But if it's 12 now and you check it a month later and it's 20 and you check it a month later and it's 35, clearly it's going in the wrong direction. It's getting much more complicated very fast and so the trend there is really interesting. And in fact, a very popular open source tool, Sonar, gives you not only trend graphs but also things like trend movies. This is actually struts. Complexity per method over complexity and you'll see that when this gets to September of 2009 something horrible happens to the struts code base where it gets way more complicated and it turns red and sprints to the upper right-hand corner. This is because they merged another code base in the struts and it trash all of their metrics all at once. Seeing that trend line is kind of interesting because you can go back and see exactly where things start going bad. So that was number two, this idea of metrics and monitoring. Yes, I've used a CPD pretty extensively which is the copy-paste detector. Is that the kind of thing you're talking about that you don't have any experience with? Yep, those are really terrific. In fact, there's a tool called PMD and it comes with a tool called CPD, the copy-paste detector, which is a very valuable tool. It tries to go find chunks of code that you copied and pasted to reuse. You should run this on your code but drink a stiff drink before you look at the results because you won't be happy. They busted the JDK in 1.4 for about a 50-line copy and paste duplication in the JDK from hash map to weak hash map. Instead of deriving it, they just copy and pasted a bunch of code from one to the other and that's clearly bad. So that's another great metrics tool to chase after. Absolutely. I'm literally just scratching the barest surface of metrics. I've actually a 90-minute talk on visualizations for code metrics and it's crammed full and there's way more. There's actually a lot of interesting innovation happening in this space and I think more to come as we learn more and more about what is quantifiable about code and what's really just kind of trivial knowledge. My number three lesson is be careful of smearing across your architectural boundaries. So let's say that we have an application like this and this is what it's like when you conceptualize it. It has this framework that everything it relies on and then it has these components and you have an application that relies on these components. Unfortunately, the reality of your project probably looks more like this where you've got things smeared between those layers. A lot of frameworks make this much worse. Ruby on Rails, for example, for all the productivity gains it gives you suffers from smearing a lot because you have to... It's domain model. You actually extend Rails pieces into your domain model so you smeared the framework into your domain model and that causes some issues sometimes. It's not to say that that's necessarily a bad pattern but it's something you have to watch out for because it blurs the distinction between architectural elements or the things that are hard to change later. And the way that this manifests, the way that you can find out if this is happening in your code base or not, is look at the afferent coupling between components in your code base. If that's really high, then you probably have smearing between the things that should be orthogonal from one another because you've got a lot of crosstalk between these two elements that should be isolated from one another. And this is a particular concern for frameworks, like I said, Ruby on Rails, or Backbone or Angular in the JavaScript space which does things to the DOM that's mixing up your domain model with your framework stuff so you have to be very careful. It's not to say you shouldn't use tools like that but going with your eyes open. Number four, and this is kind of obvious but nobody does this, and that's stay in sync with upgrades. Frameworks, drivers, operating system patches, yada, yada, yada, all those things because what we have found frequently, is certainly true, more true the more kind of aggressive cutting edge you are. So one of the things that happens a lot in Rails is you end up future porting features. So the next version of Rails, we need this feature, and so we implement it. The next version of Rails already has this feature and it's implemented a different way. And so now it's really hard for us to upgrade to the next version of Rails because we have to change the way we're doing that to match the way they're now doing that. So the more the work, the more likely this is to happen. In fact, this happened a lot in the very early Java days. You had people writing their own MVC web frameworks, writing their own application servers, etc. Because they're basically future porting versions. But lingering upgrades basically amounts to technical debt. But it's a very special kind of technical debt. It is technical debt with a gun to your head. It is technical debt times two because this one is a ticking time bomb because if you don't upgrade now, you're going to end up skipping a version. An upgrade is going to be a lot harder and then you skip another version and upgrades can be virtually impossible and then they're going to end of life support for the version you're using, which means no more security updates or things like that. You've got to be aggressive about this. And depending on the technology stack you're using should temper your aggressiveness toward doing upgrades. So if you're in something like Ruby on Rails, you've got to be really aggressive on upgrades. If you're in something like Angular Backbone or something even newer, you need to be even more aggressive. If you're in a much more stable state like spring, hibernate kind of stack, you can afford to be a little less diligent because the changes are not likely to be as abrupt in those mature frameworks. But the age of the code base times number components gives you an indication of the ongoing effort required to get stasis back in your project. And this is especially true for smeary frameworks like I said, something like Ruby on Rails, because it smears into your domain model some and now that will have an impact on your upgrade too. So what we started doing is trying to budget these relatively early as technical stories. As soon as we know of another Rails upgrade that's going to come out, we put a story card in place for that and start budgeting for that right away because the faster and more aggressively you can do that, that problem is now solved and you can move forward, linger for long periods of time, it gets worse and worse and worse the longer you let it linger and it makes it less and less likely that it's going to be feasible to actually even get done. Number five is trying to keep your domain logic as non-repeatable as possible. And I'll talk about some aspects of this that are kind of interesting. What I'm talking about here, when I talk about domain logic, are things like business rules, validations, data rules like inferential integrity constraints, things like that, workflow kind of behaviors because functionality always has to exist across all these tiers. Validation is a classic example of this. You have to validate in the web tier and you have to validate the application server tier because you want to make sure that they didn't bypass the validation through the browser, which you can do from various tricks. So that functionality now exists in two different tiers and a lot of frameworks favor rapid application development over don't repeat yourself and this is really just a manifestation of smearing, smearing those architectural concerns to make it easier to do rapid application development. This is always both a goal and a challenge to keep your domain logic dry and you can do some crazy stuff in trying to achieve that, which we try to avoid. But this brings us to an interesting place, which is the middle of my lessons and you transition from green field to brown field. This is a transition I don't think a lot of projects identify correctly, but every project hits an inflection point where it's no longer a new project, it is now an existing project and it's basically an ongoing maintenance and upgrade or update mode. And your engineering practice should reflect that inflection point and so what we started doing on projects is add semi-permanent technical debt cards. We now reserve a chunk out of each iteration to work on technical debt. Change, go and refactor some things that we now know are not quite right. We didn't get it right the first time, but we were close so we can now go and refactor those things. And we also switch emergent design styles. We switch from the last responsible moment kind of world into the harvesting abstractions and patterns kind of world. But once your code base gets to a certain place, you're not adding architectural elements at a very fast rate and it's hard to change later. Those are the bedrock things that don't change much. And so at that point, you're now mostly in a kind of refactoring phase and you're now in kind of design. I'll talk a little bit about this harvesting model for reuse in just a second. And we gradually escalate the refactoring effort over the life of the project and so the bigger the code base gets, the longer it's lived, we'll gradually ramp up the amount of effort required to do that. I have a graph here in a second about that. Which brings me to an interesting philosophy that one of my colleagues is espousing now which he is calling No Design. And here's what this is in reaction to. No matter how much you try, requirements evolve chaotically sometimes and bedrock assumptions become changeable. So on the OVE project I was talking about before, it was an online vehicle site. And when they first started building this application, since the beginning of time, if a car is listed and it is then sold, it is no longer listed. It's basically a state transition and that was a core thing that they built into all the reference and integrity in the database and they built it into the rails and the validations and then one day decided, you know what, we'll sell it but then let them immediately list that sold car so it's both sold and listed. That caused no ends of nightmare in the code base because now you've got to track down all the places that are enforcing that little rule and it's in the JavaScript, it's in the Ruby code, it's in the database, it's all over the place, it's hard to chase those things down. That's what he's saying is this No Design philosophy is don't hard code anything anywhere because it's going to change. And the question is how can you lessen the impact of these kind of chaotic changes. Let me give you a good example of this. This is an example, I was chatting about this last night, somebody brought this example up and I thought it was perfect. Does anybody recognize this? This is iTunes 1.0. And iTunes 1.0 did one thing, well two things, let you buy music cheap and sync it to your iPod. iTunes 11 just came out and it's more complicated than an operating system. It does books, it does movies, it does TV shows, it does music, all these things. There is no way that when they built iTunes 1.0 they had enough architectural stuff in place to build iTunes 11. Bedrock assumptions like we're going to connect a one device and sync music to it, change over time, sometimes really radically over the course of time. So Steve Salkin is the guy who came up with this idea and his point is a lot of times we kind of think of an application as an equation we're trying to solve. Let's put all the constraints in place and get it all working and we have the equation solved and that's in production. But it's not an equation, it's a process, it's a snapshot of a process because those things that used to be mathematically certain become uncertain over time. So the idea here is to make rules and policies design rather than architecture. Here's that distinction. Easy to change later versus hard to change later. So for example, he's suggesting don't put referential integrity constraints in the database. That's just going to cause you headaches later. Have that in code so that you can change that code later without having to go all the way down to the level that you traditionally do. And this goes into this dry idea too. If you keep that stuff in one place, not in the database and the browser and the code, find one place to put all that stuff, no matter where it is, then you've got one place to go change those things. And one of the things that you can think about one way to solve this problem is to actually incorporate a time element and say this rule is valid between these dates. So this reference to integrity is true, but we're not going to say it's true past a certain date because we don't know how the problem domain may change past that date. And so you can actually add something like a time element that says, well, these rules are in effect for this time period. And to do that, you have to add more infrastructure at the architectural level. It's kind of ironic because this no design philosophy I'm talking about actually requires more design. But it's not really design, it's architecture. Let me talk about what I mean by that. So let's say that I want to do this no design thing where I don't put any kind of hard-coded stuff anywhere. But to do that, I have to have architectural support for that. I have to have a place in code where I have referential integrity constraints and validation rules and things like that. And those are going to be relatively hard to change later. So this is a trade-off adding architectural elements there so that you can have flexibility and design and toward the top of the stack later on. And I'll attack the top of the stack part of this in just a bit. The point of this is there's a very important transition between a greenfield project and a brownfield project. Now I think most projects miss that and don't react to it appropriately. Which brings me around to the second part of my lessons. Number six is malleability. This is actually a characteristic of metal where when you pound metal, it makes it so that it's very easily bendable. And this is very much in this kind of no-design idea of never hard-code anything. Data relationships, validation rules, referential integrity. All those things should be in a place that you can easily change as design rather than architecture. But please not in something like a rules engine. That's in the jumping from the frying pan into the fire because that imposes its own really draconian constraints on the things that you can do. So I'm not really suggesting that you go to something like a rules engine to do this. You're actually much better off keeping it in code or doing something like a domain-specific language which doesn't have the kind of architectural constraints that rules engines typically have. Number seven is loose coupling and integration points. A lot of the early integration efforts focus really heavily on this RPC style interaction, remote procedure call style interaction. I'm just going to go through a very, very quick kind of comparison between soap and rest approaches to do this. It's maybe more technical than this particular audience wants to get. But I'll go through these relatively quickly. Oh, I meant to say this up front, but I didn't. The online schedule says it's a 90-minute talk. The paper outsize says 60 minutes. I've got 90 minutes' worth of material. I'm not the hour mark, so if you're tired of listening to me, feel free to go and look at one of the other talks if you want. But I'm going to keep talking for another half hour or so. So Leonard Richardson, several years ago at QCon, basically created this scorecard of how different integration styles, how mature they are from a restful standpoint. He defines four levels here. And I'm really only going to talk about two of them, which is level zero, single URI and a single verb. And then I'll talk about level two and then briefly talk about level three. This is basically, though, let's talk about level zero first, which is the very typical remote procedure call-style interaction, where you have some code. It gets serialized down to a transport layer. You move the XML over here. It gets deserialized. You react to it. Serialize it back on a payload. Deserialize it and get something back over here. Basically, when you're doing this, you're using HTTP just as the transport layer. This is typical in SOAP and XMLRPC and PlanoXML. Those are all exactly the same kind of architectural style. The only real difference between SOAP and XMLRPC, et cetera, is the protocol that you create, what the format of the XML looks like. But notice that SOAP was designed to be transport agnostic. And so even though they're using HTTP here, they don't use any of the HTTP status codes or anything else built into HTTP, because it's just a simple transport layer, because SOAP was designed so it could work over HTTP or JMS or a bunch of different formats. Mostly we're using HTTP now, but there's protocol transparency in SOAP. But the only real difference when any of these RPC styles is the wire level protocol. What do those messages look like to get passed around? RPC architectural styles score a zero on Richardson scale. They are very lightweight, and they're universally interoperable. It's super easy to create SOAP stuff. And any IDE, right-click, create me a SOAP endpoint. But they're not particularly robust, and it ignores the web as a platform. There's really a lot of useful stuff within HTTP that REST takes advantage of. And of course this pre-stages the rise of REST and representational states transfer. This is Roy Fielding's thesis on using HTTP as an integration strategy that focuses much more on resources rather than procedures. The remote procedure call concept started back from mainframes and kind of made its way into the modern world, but REST is much more resource-centric. And in fact, he also coins this term hypermedia is the engine of application state. I won't get into that. That's more than we need to get into here. But the idea here is that you're going to take advantage of HTTP as a transport protocol, and when you post an order, what you get back is a status code as it was created at a location header set, but no actual XML coming back. What you're doing here, rather than calling a remote procedure, is asking for a resource or putting a resource there. And so when you get a resource, what you're getting back is information about a resource. Frequently this is in XML, or sometimes in JSON, but it's still in XML. But CRUD services here embrace both HTTP and URIs. This gives you a really nice solution because it's resource-centric. It's resource transitions instead of a remote procedure called endpoints. It's best for these kind of CRUD operations. It still, though, doesn't solve the fundamental problem that you run into with RPC-style frameworks. And it's this. Typically when you have a client that's integrating with several other server components out in the world, the client is the one that's driving that workflow because even though they're interacting with different applications, it's managing some sort of workflow. The workflow is going to have decisions in it. You're going to do something, and then the user's going to have to decide what I want to take this path or this path. And so then there's something else that comes up they have to decide. So there's a workflow being driven by a client somewhere. Which couples the client and the server together because you can't easily change one without changing the other one because the client has to have intimate knowledge of what workflow is available and applications can own business logic but don't manage their own workflow which is usually driven from an external source. A lot of vendors will try to tell you well the solution to this problem is something horrible like BPEL or BPM but that's even worse because now your workflow isn't in code which means you can't test it, you can't diff it, you can't refactor it and you can't really use it. It also suffers from the hairball effect that once you get a project big enough you can't even see all the connections to solve the fundamental problem. That's where this idea of hyper media is the engine application state comes in because it tries to manage this interaction at the level of processes rather than procedures. So the idea here is when you make a call to something to a server rather than the client saying I want to do this then I want to do this then I want to do this all it's going to do is say I want to start this process and what comes back is going to be okay here's the first amount of data that comes back at a bunch of links in there and the subsequent steps you can take in the process. That's hyper media is the engine of application state because now as the client I just choose one of those resources to go to through a verb and a transition and now it can give me the next set of my workflow. So the classic example here so you have an e-commerce site and you're doing your own credit card processing and at some point you decide I want to switch to a third party credit card processor. Well if you're writing the client you'd have to make changes of the client to do that in a hideous environment you don't because you just have the server send back a different link to the new credit card authorization and the client knows to authorize credit cards I follow the link that comes back from that server that decouples intimate knowledge of the client from the server. The client still has to know what workflow is available from a user interface standpoint but the implementation details of those workflows are hidden from the client and can change underneath in as many ways as you want. That's the idea of hypermedia when you make a call to something it brings you back the next possible steps you can take. You choose one of those which brings you back next possible steps you can take and you keep navigating that graph of resources until you reach the end of that business process. So we're talking about resources rather than calling remote procedure calls and this is what it looks like you just get links back as part of the payload and now your application knows it can follow that link. Notice that this adheres more closely to the no-design philosophy than SOAP does because in REST you can decide how stringently you want to do things like validate validity at endpoints or diversioning at endpoints etc because REST doesn't try to validate every single thing that comes through the endpoint like SOAP typically does you have more flexibility in versioning and some other things like that which is why we tend to prefer REST and that's this idea of loose integration at coupling points we much prefer resource centric things in RPC style interactions RPC style interactions are great for an equation but lousy for a process because that's going to change and it's going to cause breaking changes throughout your integration layer so we prefer resources over remote procedure calls we prefer service choreography over orchestration and that's really just a topic in the REST world. Number eight lesson are course grain components and services and this interestingly ties nicely into the continuous delivery space which is really a summary of agile engineering practices and in the book continuous delivery Jez and Dave define a component as a reusable replaceable part of the system that implements the same API independently deployable encapsulates a coherent set of behaviors and responsibility to the system this can be either binary components like jar files or gem files or DLLs or it can be resources like in a REST based system now in the continuous delivery world typically what you see when people describe a deployment pipeline you see something like this and this kind of assumes a complete application is going through all these stages the commit stage this is all for the complete application itself but one of the more advanced parts of continuous delivery is why can't we do this at the component level instead of the entire application level one of the things that you do if you build an entire application particularly if you have body components is you do hyperactive builds where you build absolutely everything required to make that project and a lot of that effort is not actually necessary and so one of the things that continuous delivery tries to do is to do continuous delivery at the component level rather than at the application level because this gives you finer grain control over things like a life cycle of components it allows components to evolve more independently of one another particularly if you can programmatically verify if they're not breaking one another I'll talk about that in just a second so this is where we'd really like to get to from an engineering standpoint is have some sophistication around the pieces that make up our application one of the things that you really need to make this kind of component based system work is a directed graph that looks like this that has no cycles in it cycles are poisoned for this kind of thing because if you have cycles in a component one component relies on another one that ends up relying on this one again then you're pretty much stuck doing hyperactive builds you can't really build those easily at isolation from one another so let's say that you have these components that make up your application this is where by the way the stuff I was talking about chasing out cycles earlier really comes in handy because you really need to get rid of the cycles before you can do this but notice that we can't just say that each of these is an equal citizen everybody relies on this guy all three of these rely on him and this in turn relies on this guy and so setting up a deployment pipeline for this you have to have some knowledge about the dependencies between things within your system so now if the framework changes we trigger a change on all three of those dependent components which then triggers a change on the portfolio management application itself but if just one of those components changes we don't rebuild the framework and we do a lot less work which actually cuts the cycle time down the amount of time it takes to actually complete the task on something because you're building less stuff so how do you mitigate the challenges in component based systems you lose architectural coupling one of the things that we started doing a lot is something we're calling semantic monitoring particularly in our REST based systems we have several applications now we're building basically REST services that run and handle just one thing they own their own data, etc and they're in a large integration ecosystem one of the things we want to make sure particularly as we bring out new versions of services that need to support new functionality breaking functionality from old ones we want to make sure that things don't break and we have these really brilliant tests we run at development time we've written some of these tests that can run as a monitoring function as well so you can run those same tests to make sure they still do the same things you thought they did so we're now taking some of our tests that we would have traditionally run at build time and running them as a monitoring tool as well to make sure that things still work so basically as soon as a new REST for service is deployed we run the test as a monitoring facility to make sure that it's existing correctly in its ecosystem another thing we do a lot for this is this idea of consumer driven contracts the terrific article it's actually on Martin Fowler's site at Johnson who's at DO4J now about this idea of consumer driven contract when you have a component it basically provides some services for a consumer the idea of a consumer driven contract is the consumer also has a contract with the producer the provider that says you will give me at least these things I need at least these things to be able to keep functioning that way you can actually now test to see if I roll out some new functionality let's look at all the consumers that are talking to this is this going to break one of those guys because they are stating their assumptions up front here are the things I need from you as a provider and now this lets you check well I'm going to make a change is it going to break this relationship between these two guys so last one number nine is know what your product goals are and it goes back to this product management paradigm for applications and components we are big fans of this we are big fans of having someone own components within an organization enterprise because if someone acts as a product owner then they are going to chase it and make sure it actually gets deployed and is prioritized etc but there is a tension here between Yagney and pivotability and this is really the origin of the conversations I was having with Dan McClure who is in our product innovation space because it turns out he is very frustrated with our architects bludgeoning him with Yagney he keeps saying we are going to need to support this feature at some point in the future we know it is going to be there it is definitely part of the application and the architects keep telling us we don't need that yet but what that does is limit his ability to change features at the end because he doesn't have the infrastructure in place to do that so this actually manifests differently between Greenfield and Brownfield projects so you hear a lot of people talk about minimal viable product and I found out last night there are really two uses for this term excellent in the wild one is the original meaning for this term that came from lean software but the more pretty common software version of this is what Jeff Patton calls a minimal viable product experiment where you put just the littlest thing out there or maybe several different variations the littlest thing and see what people like and start using that is this kind of idea of get the littlest thing out there and get it working but Dan's point about this is that minimal viable product doesn't have enough to become an actual product most of the time a lot of the things that are going to be required to make it a real thing get yagneyed away by the agile architects who say no no no you're adding too much stuff too early and so he is calling this term a minimum feasible product what is a product that has the minimum stuff required to make it sell or appealing or useful but also the support that you can easily change features later that's what we're really after here this idea of a pivot friendly architecture this is the idea of a pivot friendly architecture this goes back to my global minimums and maximums that I was talking about before because remember I said what we're trying to do in the agile world is find a pretty good local maximum once we've achieved that we see the global and higher maximums the idea of a pivot friendly architecture is have the infrastructure in place to allow you to pivot from here to here without too much effort it was pivot up to a higher local maximum getting more toward the ideal solution to this problem probably not the global maximum but you can kind of stair step your way up to a better more elegant version of this application so a pivot friendly architecture is designed to allow choices late in the development process for features that you may or may not be there or features that may have different implementation details and you want to try two or three different user experience pieces for example this is more architecture and design work than just a minimum viable product and more than what is traditionally said in the agile world you should do which is as little as humanly possible but what this means is that all layers create tight feedback loops so it really comes down to how lucky you are because let's say this is my picture again let's say that the product guy decides he wants to pivot away and change some major functionality that touches three features in some design and architecture and we were lucky so here's the example let's say that you know the application has to have some sort of scheduling component so we built an architectural element to handle scheduling and they decided they want to change that around completely that's easy I just pull out this architectural element and this design element and put another one in place still supports the same kind of features we're good to go we knew that architectural element was going to be there it was easy to pivot to a better one because we had planned for that but a lot of times you're not lucky a lot of times what the annoying product guy is going to do is going to break your beautiful architecture design partitions that you have and so in this situation you can't really rely on architectural refactoring but you can use other engineering techniques to tackle this problem these tend to pollute your code base a little bit more because they put more code into your code base but things like if you didn't guess right things like feature toggles where you add code to say this feature is on or off and you can feature toggle things in through architectural layers or you can do branch by abstraction if you need to change some internal functionality by implementing the strangler pattern you can do branch by abstraction so there are engineering techniques that handle the situation where you didn't guess right because there are a lot of situations where you're not going to be able to get that architectural constraint exactly right this is mostly on greenfield projects on brownfield projects a lot of times you see this characteristic in fact this is kind of a fake version of the graph I showed you earlier of cycle man complexity per line of code a lot of times on engineering projects this is complexity over time and what you'll see is it will reach an inflection point and then there will be some effort that pushes it down an inflection point and some effort but what you've noticed is that every time you do this as the code gets more and more mature you have to put more and more effort into these refactoring exercises to get the same kind of bump that you got before in lowering your complexity so refactoring and restructuring exercise require increasing effort for the same result over time and so plan escalating effort toward remedial architecture and design this goes back to what I was saying earlier have technical cards that you start playing automatically as part of your iterations that say this is an architectural or technical debt attack you're always going to have things to do for those technical cards there will never be a time when you're sitting twiddling your thumbs because you don't have something to do that you can make improvements in your code base this is actually the trade-off for the reduced upfront effort that upfront effort that traditional architects do is not wasted time they actually discover a lot of interesting stuff they just don't discover everything they don't discover enough to be confident that they can build something that will last forever but there's a trade-off there this is part of the trade-off a lot of times, particularly in enterprises you start talking about some of these practices particularly the idea of pivot-friendly architectures and the common response you get back is we don't have time our application groups have got to deliver features at a breakneck pace we want to see these new features as quickly as possible and so this is a strategy that we've used a couple of clients that have worked pretty well if you can manage it correctly so let's say that you have three different development teams and they all have their own release cycles they're all releasing interesting stuff that all of it gets integrated together into one big release maybe using trunk-based development the idea here is that rather than tell these people to slow down and create more reusable assets because that's going to cut their velocity and a lot of times you want your application developers to have really high velocity we've done this harvesting model so these guys all have releases what we do is have a group of typically senior-ish developers who come around and visit these projects and look at the things they've done purely from an application standpoint we've got to get this function out into our application but then looking to see wait that's really, really re-useful at other parts of the enterprise or in other applications and so what this team is going to do is harvest the results of those individual application development teams and incorporate it into the overall ecosystem so if you're doing a rest-based development somebody will create a one-off restful service that does something this harvesting team will look at that and go oh that's really useful for other applications let's formalize that do a little bit more architecture work around it make that a first-class component and deploy it so that everybody can use it this is this harvesting model of reuse this works well in companies or enterprises that are very concerned about cutting velocity but still want to achieve code reuse this is one way to do it is to have someone kind of following behind and harvesting the really re-useful elements the last thing I'll talk about is annealing I talked about malleability earlier I have two metal metaphors in here this is a less common term annealing is to heat metal or glass and allow to cool slowly in order to remove internal stresses and toughen it this is how you make metal tougher is by heating it letting it cool slowly you anneal it this is exactly what you're trying to do in your code base so greenfield projects tend to emphasize malleability we're going to make it flexible so that we can make changes to it as the storm of new requirements unexpected things come in but on brownfield projects you want to try to maximize your annealing efforts maximize your efforts to take what you have and make it better now that you've seen what it does you can a lot of times see what it really should be in a better sense a better local maximum use that refactoring exercise to get toward that better local maximum this is the price you pay for having emergent design for bullets in the software development world all that time that regular architects spend up front they are building some value we lose some of that value this is the price you pay for emergent design you need to invest that time back on the back end of the project to anneal and harden the stuff you have and ultimately I think you can achieve a better end result design than if you started shooting for a global maximum like the traditional guys do so there's a bit of an epilogue here early on I said that this conversation started at OVE over concerns about emergent design and OVE is currently in the process of refactoring toward loosely coupled rest services with narrowly defined orthogonal behavior they're doing exactly this what I was talking about which is where this subject came from they're doing a lot of work with consumer driven contracts we're doing a lot of our projects and they're reducing a lot of the hardcore smearing artifacts that exist in their code base they're being very aggressive now about making sure that they have important code in a single location and how they can reference it and they are now actually very happy with the state of the code base that just passed the five year mark I asked them to ask a bunch of people in this project independently give me an honest evaluation of this code base and the kind of common response was nah it's in pretty good shape nobody's like oh this is the best code base ever on earth but it's in pretty good shape because it's constantly under change this is five years of a better than 12 pairs Ruby project doing emergent design very aggressively the entire time and at the end of this stage we feel like we're in a very good space and would in fact do it pretty much the same way again that's the summary of all the things I talked about and I have a couple of minutes for questions if anybody has questions about any of this this is probably a tad more technical than some of you were expecting but that's ideas I'm wondering what you'll begin off with making the language being architectural or anything that people are hearing I actually like DSLs as a way so his question is I was talking about smearing between architectural elements he was asking me so how do DSLs exhibit this I actually like DSLs a lot for this because they're testable they're still code so you can write tests for them frequently particularly if it's an internal DSL if there's something it won't do you can dip down to the lower level abstraction and do it there and so I think DSLs are actually a really terrific way to capture these dry things that you want to capture these things like reference and integrity and relationship between things capture those in a DSL and then interpret it in the various ways that you know the JavaScript wants to interpret one way and database the other way DSLs a lot as a solution to that much better than things like business rules engine which are terrible you talked about avoiding future proofing features like Rails 4 comes out and whatever you thought was a way to do it that has changed now so how do you really avoid it because how would you know that you don't know that you don't know because how would you know that what would come in the next version you don't so you build something and then it comes up in the next version it's like oh they've got a version of that now we're going to have to pull our hand written version out and start using theirs a lot of times you know up front so I can't remember the exact feature in Rails but there was a feature in Rails that they had known was coming for like two years they couldn't wait two years so they went ahead and wrote their own version of it and then when the upgrade came it was actually a giant hassle to go in and rip out all their code the paradigm was different slightly different from what the Rails guys implemented sometimes you know what's coming but you usually get a few months notice that an upgrade is coming something like Rails as soon as we see that now we put cards insert story cards we've got to do this right away as soon as it comes out we'll attack it so you're not saying that don't do future proofing you might have to do it but when the actual thing comes please change it to that yes yes we're off using framework features if they're pretty close to what you need because that's going to be in the framework forever and if you're using your own hand written stuff you're going to have to maintain that forever this was the death of a lot of companies who wrote their own MVC framework and wrote their own application server because they get used to it and then commodity web frameworks and application servers come out but what happens is these smartest people in the company work on your internal application server and your internal web framework they're not actually building business value they're building plumbing that's not a good thing to have your very best developers doing is building plumbing somebody else has already built most of that plumbing for you set them for you on innovative solutions the actual thing you're building rather than the sophistication of the plumbing underneath probably got time for one more question anybody have another question about anything nope well if that's the case then I will give you a question so the real difference between evolution and architecture and emergent design that's your question right architectural elements of things that are hard to change later so for example the database is an architectural element that's hard to change later have you ever tried to switch from Microsoft's SQL server to Oracle it's a massive change huge amount of effort versus something like the validation bits that come with your web framework that's a design element you can change the way you do validation within your application trivially that already supports that that's really easy to change later so you can be really aggressive about how you change things like validation because it's not going to have these bedrock impacts like these architectural changes you're going to have so the web framework itself is an architectural element but the pieces within it like workflow and validation are really elements of design I'm sorry some of the things that you mentioned up here some of the lessons learned you could apply to actually when you actually do design as well yes, this talk is about evolutionary architecture and emergent design those are all things that you do this is really lessons learned from applying emergent design over several years I have one more question I think you've possibly referenced about rules engines yes about rules engines, why don't like them so I'll give you kind of a long answer to this question but this is a broadly applicable answer I recently wrote a blog post called Why Everyone Eventually Hates or Leaves Maven it actually explains why don't like business rules engines as well and it has to do with I used to be in a consulting company the Microsoft Access Projects for clients and we eventually shut that part of the business down because we realized that every single Microsoft Access Project started out really successful and ended in total failure and we tried to figure out why and we figured it out and my colleague Terry Dietzler figured it out and my book Protective Program is Dietzler's Law Dietzler's Law says that anytime you use a framework like that that 80% of what you want to get done is super easy and quick because the framework supports it directly the next 10% you want is possible but difficult because you have to hack underneath the framework or do something that wasn't designed to do or bended in a weird way but the last 10% is impossible because you can't get far enough underneath the framework and users always want 90% of what they want they're never satisfied with 90% of what they want and that's the problem with business rules engines one of the problems is it suffers from Dietzler's Law it works great for simple rules and simple relationships but then when you try to model business rules that it doesn't like you might be able to get it to bend it to do that but you're going to reach a point where you absolutely cannot express that in that business rules engine now you're stuck because now you've got to rip the entire thing out and the problem with business rules engines is people think I can get rid of those pesky developers and let business analysts build these rules but the problem is they're not developers so you're a business analyst you're doing something in a rules engine and this rule is almost identical to this other rule with one minor change as a business analyst what are you going to do you're going to copy and paste and make that one change and so over time your rules engine becomes this unnavigable huge thing with giant copy and paste chunks and you have no idea what's going on in that thing versus developers doing that they're going to say oh these are related let's inherit one or compose one from the other and reuse that behavior just because you've given a tool to somebody doesn't mean they're going to use it correctly so I've never once I'm a consultant so I see a lot of broken projects I've never seen a project that was happy with their rules engine it has passed about a year so that's why I don't like rules engines one of the arguments about rules engines the vendors give is that vendors performance they say it's performance I can I guarantee you can hand write that business rule code and blow the doors off what that silly rules engine is they're trying to sell you something that's exactly what they're trying to do they're not trying to make your life better they're trying to sell you products and they're actually trying to sell you products that are not well suited to your business they're actually well suited to the widest possible businesses because they want the broadest market possible which by definition means there's going to be a lot of junk in it that you don't need at all that your competitors need that's another downside so that's it I'm over time thanks very much for coming hope you enjoyed it thank you