 to the systems that you're building. Incidentally, to the extent that it's readable, these presentations are all written in code. As a little experiment for this conference, I tried building the presentations using Reveal.js. Has anybody used Reveal.js before? It's a very simple JavaScript library that allows you to set up a web page. And each slide is effectively an HTML section. And the JavaScript does slide transitions and that kind of thing. So if you want to have the experience of writing, building your presentations in software, it's a good way to try it out. And so incidentally, this presentation, all the other presentations that I've given at this are up there on my GitHub account. So if you want to check out the source, you can do so and follow along. And if you want to dabble a little bit in Reveal.js yourself, this is an example of something that you could use. If you have some suggestions for improving the presentations, please feel free to fork my repo. And you can send me a poll request and I'll evaluate it and potentially integrate it into future versions of this presentation. So this is the presentation that I'm actually most excited to give at this conference. And so I hope you guys enjoy it as well. Let's just see how many more people are going to come in before you get started. So I started by asking a question of how many people code. I think pretty much everybody put up their hand. What language do you use? How many people here use Java? OK. And how many people use Ruby? What about JavaScript? Or server side, like doing development in Node.js? Anybody use Node.js? OK. JavaScript, yeah. So the code examples are in Java, so at least the Java people will be happy. And I think that the non-Java developers out there will be able to follow along pretty easily as well. So my Twitter handle is xortech. I encourage you to please tweet about the presentation during the presentation. A lot of people have been doing that so far for the presentations that I give. I really appreciate it. If you have questions, I can respond via Twitter. So it's definitely a great way to pass feedback to me directly, and I encourage you to do that. Let's get started. So hi there. First, a quick little blurb about me. So I already mentioned the Twitter handle. Thank you for bringing that up. I'm the product lead at a company called Pulse Energy, which I've spoken about at the other two presentations that I've given so far at this conference. I've been practicing Agile and XP for a while since 2000. And for about five years, I was a consultant at ThoughtWorks. About a year and a half of that was spent here in India in Bangalore, during which period of time I was involved in setting up the first Agile India conference together with Nuresh. So it's nice to be invited back. So I'm not sure how well it's showing up on the slide deck, but how many people know what this is? All right. What is that black rectangle that's in the middle of the picture? Some people put their hands up. And I'll give you a hint. It is not Apple's new iPhone. It's the monolith from 2001. How many people have seen this movie? OK. I wasn't sure what to expect. It's an old movie, Stanley Kubrick, 1968, I believe. There are a lot of references to 2001 Space Odyssey in this presentation. So if you're a sci-fi geek, you will hopefully enjoy this. And for everybody else, I think you'll be able to follow along. So this presentation is really all about dealing with monolithic systems. How many people have built a monolithic system in the past? Now, by monolithic system, I mean basically one big application that you have to maintain. I think pretty much everybody put up their hands. Great. OK. Good. So this is for you. And so what is a monolithic system? I mean, it's characterized by it being massive, uniform, single thing. It's rigid and invulnerable, difficult to break into pieces. And what we're going to do is talk about how we might break up a system like this. So where does the monolith come from? Well, nobody knows. But where do monolithic systems come from? Well, actually, within 2001, in the movie, it's revealed that the monolith appears on the planet Earth in the Olduvai Gorge in Africa. And it is attributed by the author of the book, Arthur C. Clark, as being the key that led to the evolution from apes to man. So the monolith is what encouraged the apes to pick up tools and start to use them within the context of the movie. And I think that there are similarities here. Really, monolithic systems are about evolution. They are produced as a product of evolving systems. You don't start out with a monolith. You start out with something relatively small. And over time, that system evolves. And it turns into one of these great big monoliths. So anybody know where this picture is taken? Sorry? So is it at the Taj Mahal? OK, I don't know. I haven't been there. This is actually from one of my favorite blog entries by Michael Feathers. How many people here know Michael Feathers to read his work? Anybody read this specific blog entry? It's from, I think, 2009. So he coins this phrase about canalizing design. It was based on an observation that he had made from looking at software systems that when changes of a certain type happened, they tended to be clustered together. So let's say I go and change one area of the code and several other, let's say one class or one method, several other methods, similar methods, will often be required to be changed. And those changes will happen over and over again. And so it's based on an observation that software systems, the way that software systems are built up, kind of like running water. So you have water flowing. And it starts to carve out a bit of a canal. And over time, more and more water flows into that. And so you end up with this path through a software system. And so to look at a specific code example, so can you guys read that? You might want to come a little bit closer. You guys want to come closer because there are going to be a bunch of code examples up here on the projector. And so it'll be easier to read if you're a bit closer. But effectively, what we have here is we have a function called ask howl, something, an action. And within that, we've got an if block and an else if block and then a return statement. So it says, if the action is read me, then return affirmative, Dave, I read you. Else if is read message or returns the message, it's dangerous to remain here. You must leave within two days. Or it returns the default howl message of I'm sorry, Dave. I'm afraid I can't do that, which is what howl seems to say a lot to poor old Dave. So if you were tasked with this bit of code and you needed to instruct howl to say something else, what would you do? You add another else if, right? And I mean, what we have here is we have the path of least resistance. The easiest thing to do is to go in and add an additional statement. So what's been laid down as a canal by this set of statements. And then we're just going to keep going. And that functionality is going to grow. That else if block is going to grow. And then to a certain point, maybe somebody will make the decision, OK, I'm going to. There's too many conditions here. Maybe I'm going to change it into a case statement. And then maybe a few more get added and say, OK, well, then we need to introduce some sort of an abstraction. But that tends to be the way that systems grow and evolve over time. And by default, only natural, natural law, people tend to follow the path of least resistance. Here you can see the path being carved across the grass rather than following the sidewalk. So really, functionality accretes within systems. So it gets added on incrementally, incrementally over time. So we end up with this evolution that ultimately culminates in what's often been called the big ball of mud. It's an anti-pattern. And to a certain extent, we are, and our teams are, that dung beetle pushing that ball of dung and it's accumulating more and more dirt and cruft until we end up with a monolith. So what are some of the problems associated with monolithic systems? Well, one is what I call code encumbrance. So this is a pretty geeky talk. So I'm going to ask the geeky question. Anybody here play Dungeons and Dragons that will admit to it in public? OK, the rest of you probably played it a little bit and you just won't say. But D&D has this notion of encumbrance, which is basically you have a character, an adventurer, and they accumulate some items. And in order to make the game seem more realistic, your character can only carry so much weight before they start to slow down. And then they become more vulnerable to being hit by other monsters and that kind of thing. So really, as code accretes, I think of it as being encumbrance. It makes your avatar, your player, move slower and slower. Obviously, there's the increased cost of regression. You've now got way more code paths to areas of functionality to test as your system grows. There are problems associated with scaling. You can't scale bits of a system independently. If you've got some functionality that's more resource intensive, but it's embedded inside of a monolithic system, it's difficult to scale that. And ultimately, because all these pieces are coupled together, if any one of them fails, then you can lose the entire system. So there's a loss of resilience. So these are just a few of the problems associated with ending up with a monolith. So what's the alternative? Here we are. We're Dave. We're looking at the monolith in our future. How can we avoid it? Well, so here we have the promise of service-oriented architecture. I think a lot of you were probably here in Fred's talk before this, where he talked about the promised land of tens of thousands of little services that were all talking together, and they were all loosely coupled, and everything was basically unicorns and rainbows. And you end up in SOA nirvana. And that's where we want to get to, without question. But the reality is that we're not in that place. And to get to that place is going to require a lot of work. We have built a monolithic system that we have to maintain, or we're in the process of building a new system that is maybe headed in that direction. So what are we going to do about it? How are we going to be able to at least start to move incrementally closer to that land of unicorns and rainbows? Well, the reality is there's actually quite a few barriers to SOA. There are good reasons why we end up with monolithic systems. One, there's many design barriers. What's a service? What's the boundary around a service? What makes up a service? Not sure. Fred made the argument that services are really small, and they're basically just an individual class. That's kind of an extreme perspective. Normally, services are regarded as being a little bit larger than that. But let's say we're working on a new system. Our domain model is evolving. Our understanding of how the system should be structured is evolving. What is the right unit to break out into a service? It's often quite difficult to know. Until we've built up enough functionality, you'd be able to say, OK, these features, these classes, they belong together. They're interacting together. And they could be pulled out and turned into an individual service. Another barrier, Rebecca Parsons talked about Conway's law. This is not a great basis for service-oriented architecture design. It is structuring the design of services around the individual team. Just because you have a team that's focused on one specific area, it doesn't necessarily mean that the optimal architecture is to allocate a service to that team. Because these services tend to be long-lasting. Again, unless you're in the SOA Nirvana world where services are effectively disposable. But more often than not, they tend to be quite long-lasting. So you want to make the right decision when you decide, OK, this is going to be a service. Because there's overhead associated with splitting out a service. And then if you make the wrong decision about what the contents of the service are, then you maybe need to merge it in or merge it in with other functionality. So they're design barriers. They're technology barriers. So the languages that most of us use, I mean a lot of people put up their hands for doing Java development. Java's not that well-suited for communicating with external systems. You've got the overhead of needing to serialize Java objects. You need to decide on what your message protocol is going to be. You need to design what those messages look like. You need to handle now versioning because you're passing or consuming messages from different versions of different types of systems. And so there's not necessarily great language or library support for this either. Within my company, we switch to using Node.js. I think Node.js makes building and consuming services much easier because of the fact that it speaks JSON natively or speaks JavaScript natively, which makes it very easy to emit and consume messages that are in that same format. But for many other languages, there's quite a large impedance mismatch. There's deployment barriers. So now you need to worry about instead of just deploying your one application, you've got many different applications that you need to be able to support, potentially a more complex build process. You need to manage the versioning of interdependent components. I know you've got multiple things to deploy, many things in production. So there's now more things that can possibly go wrong. So there's lots of good reasons why we end up with monoliths and then we are not able to immediately jump to SOA Nirvana. But how do we get closer? How do we go from a monolith to that magical land? Or even how do we go from one monolith to two smaller monoliths? That would be quite nice too. So we want to split the monolith, but how do we do it? Well, on the surface, it seems pretty easy, right? I mean, basically what we do is we take a bunch of related functionality out of our existing system, we package it up in a module, we turn that module into a service, we deploy the service, and we're done. Easy, no problem. Anybody tried to do this? Anybody worked on trying to decouple, break up monolithic systems into smaller services? It's easy, right? It's totally, it's a trivial matter. Walking the park, we do it all the time, yeah? No, I don't think so. So yeah, not so fast, Dave. But by the way, OK, so most people put up their hands. They haven't seen Space Odyssey 2001. Within Space Odyssey, the main character in Astronaut, his name is Dave, and he's dealing with this intelligent, semi-intelligent, or highly intelligent computer that has got its own opinions about things, and is often telling Dave that he can't do things. And so here's the system telling us that it's not that easy. So what happens when we try and do this? What happens when we try and pull these bits of functionality out and turn them into a service? We end up with what I'm calling here alien goo. You end up with these long strings attached to that functionality that you're attempting to extract. And it ends up being a big mess. Sorry, this is actually not from 2001, Space Odyssey. Who knows what movie this is from? Sorry? Yeah, which one? No. Sorry? Three? No. Sorry, hand up at the back. No, not three. Prometheus. This is from Prometheus. But it's the same idea, right? I mean, they just recycle the same concept throughout the entire series. Alien goo, though. Not something we want to get on us because it will turn us into aliens as well. First, a quick digression. Who? This is an agile software conference, right? Everybody should have read this book. Who has not read this book? OK, you should go read this book. It's quite dense, but it's got lots of great concepts in it. Specifically, in domain-driven design, Eric Evans talks about package design. And so as we've got people here that are familiar with different programming languages, package is obviously a very central concept within the Java domain. If you're from the .NET domain, it would map on to a namespace or a module, et cetera. But more or less, it's just a unit of grouping stuff. Often they'll live together in the same directory. So here we've got the complete works of Arthur C. Clark packaged up together into a single volume. All those things are, all those books are related, so they're packaged together. Makes sense. So what are some of the principles associated with good package design? Well, we want to maximize cohesion within a package. So this should be, I'm assuming, is a familiar concept to everybody in the audience, right? Notion of cohesion. Things that are related belong close together. So what we want to do is put those things that are related together in the same package. And then the other side of things is we want to minimize coupling. So if we've got, if these red arrows that are spreading between packages A and B are potentially unwanted coupling, so we want to be able to remove those. So maximize cohesion and minimize coupling. Is this a familiar concept to everybody? Yeah, great. Okay, so does this look familiar? Have you ever built an application that has this layered architecture where at the top you've got a web layer, a controller layer, below that a service layer, a data access layer? Who's built a system that looks like this? I'm assuming everybody who's a programmer, right? So, okay, it's very tempting to then just take that layered architecture and then use that as the basis for our package structure. How many people have a package structure in their system that looks like this where you've got a package or a module or a DLL or something like that that contains all the controllers and then all of the services and then all of the data access layer? Anybody got this architecture? All right, so what's, so most people didn't put up their hands, so then you should be able to answer this question. What is wrong with this package structure? Okay, and what's wrong with that? I don't know if you probably couldn't hear that, so it's not partitioned functionally, it's organized around functional layers. What is wrong with that? Everybody said that they knew about cohesion and coupling, right? What's wrong with this? Okay, and so, well, let's actually, let's move on. Let's look and see what's inside these packages. Unfortunately, it's still, it's pretty small to see, but what we've got in here is we've got the foo controller, the bar controller, and the baz controller, and the foo controller is talking to the foo service and the foo service is talking to the foo Dow. And so we've got the foo, the bar controller talking to the bar service, talking to the bar Dow. So what, maybe this makes it a little bit clearer. What's the problem here? Exactly, we have maximized the amount of alien goo in our system, right? We have maximized coupling between the different packages, so not something we want to do. So what Eric Evans talks about, he doesn't talk about alien goo, but he does talk about designing your package structure so that it's organized around different domain concepts. So, and how many people have this sort of a system where you've got a module or a package that is all built around a certain domain concept and maybe you've got all of those different layers in your architecture represented within that one package. How many people do that? All right, so not very many, interesting. Having this sort of structure makes extracting services much easier. So for you guys who didn't put your hand up, the next bit is some tools that you can use in order to be able to move more closely in this direction. So to go back to our overall objective, we've got our monolith, we want to be able to split into two smaller monoliths. What's the process for doing that? Well, now we talked a little bit about package design, then we've got a process for achieving that. So what we want to do is we want to start by pulling related functionality into a common package so that we maximize cohesion. And then we want to sever any dependencies that are deemed to be unnecessary, so that's a minimizing coupling. And then we can take that package and we can extract it into an independent module, which we can serviceify, and then we're done, right? Right, we're done, no problem. Yeah, great, but this is also a hard thing to do within any sort of a complex system. So hence the purpose of this talk, which is to talk about test-driven decoupling. So what is test-driven decoupling? It's a very simple concept and some of you may be doing the same thing within your system. If not, then hopefully this is something that you can go and try and do. Basically, test-driven decoupling is using tests, unit tests, in order to enforce package structure within your system to support your decoupling effort. Does anybody do this? Does anybody have unit tests in their system that enforce their package structure? All right, sometimes. Okay, so for most of you, this is a new concept. So how do we do this? Well, we want to use a few tools to support this. The good news is that they are free and open source tools, many of which you're probably already using. Like, okay, how many people use a unit testing framework within your system? Hopefully everybody put up your hands, yes. Okay, the other ones you may not use, but they're very easy to introduce. Anybody use JDepend? So you Java people, JDepend? Or endepend for .NET people? All right, we're doing a lot of development in Node.js these days. There's a module called MAG, which is quite nice. It takes JDepend and endepend a step further. It allows you to generate directed graphs of your dependencies. But these are all useful tools, and I'll go through an example of how to apply it. So one question that often comes up, when we want to be able to do this sort of extraction operation, people say, well, what about external dependencies? So here within my Java system, I'm using this Apache Commons library, and I'm using different versions of it, and I've got log for J, but I'm also using slip for J, and I've got all these external dependencies, and it's a mess. So do I care about these as well? What do you think? Do you care about your external dependencies if you want to start to decouple your system? No, no, don't worry about it. Don't worry about your external system. Here's Hal that's saying, I'm sorry, Dave, this mission of breaking up your system is too important for me to allow you to jeopardize it. It is a distraction. Don't worry about it. Just focus on the code you own. Don't worry about the external dependencies. So what? So you break your monolithic system out into two systems, and they're each using different loggers, or they're each using log for J, and slip for J, and whatever else, and log back or Apache Commons logging, or whatever it is. So it's a mess. Tackle that as a separate task. Don't attempt it, but it doesn't have any impact in terms of your decoupling effort, because the decoupling is based on your code. It's your code that's coupled together that's making it difficult to pull things apart. Just focus on the code that you own. So I know you're dying for a specific example, so I'm gonna get into that. Do you? What if you're inheriting from your framework? Yep. I'm not saying don't care about it. I'm saying, but as part of your decoupling exercise, it's not necessary. So yeah, okay, so you end up with two systems that are each dependent on spring. That's fine, especially if you're following this, that's probably also gonna be inevitable, especially if you're following this domain-centric packaging approach. So now I've got a reporting service and I've got a charting service and they're both dependent on spring web MVC. Fine, that's okay. I mean, if I was attempting to decouple on a layer by layer basis, then okay, yeah, I wouldn't want hibernate to be in my web service layer. But that's not how we're talking about breaking up our systems at this point. So what about the infrastructure? So the question is about what about infrastructure services? Can you provide an example of what you mean? This, the same way as other things. I mean, it will be more difficult in terms of, okay, so the question is potentially, how do you decide where to start? Like why? You could start with this money conversion service. It seems to naturally live on its own, but you realize in the process you've got, it's actually got many different connections within many other things and so pulling that out might be complicated. So maybe it's best to start with something that's actually on the periphery because you can pull that out a little bit more easily. And then so now you're left with fewer things and that's the approach that we took with starting on the periphery and working your way into the center of the system. But the same principles would apply. So let's go through a specific, we're gonna look at some specific examples of test-driven decoupling in action using JUnit and JDepend. And again, if you're having trouble seeing the screen and reading the code, please come a little bit closer. So here we've got a very simple model of our system. We've got three packages. We've got a system package, a core package, and a util package. The system package depends on core and util. Core only depends on util. So here we have a unit test that asserts the relationship between core and util. So what we do at the start is we say, okay, what are the valid dependencies? The valid dependencies are util. So I wanna assert that the dependencies from this core package are valid, basically that they only contain a reference to my util package. And so in this process, I asked the question about external dependencies. I filtered out all the external dependencies. Anything that does not fall within my company's package space I don't actually care about. So can you guys read this and follow along? If you can't, please come a bit closer. I encourage you to come closer because there's gonna be more code examples. But does this make sense? So I mean, there's a lot of details that are abstracted, but basically the key idea is that I have a unit test that for a specific package, I am asserting what its dependencies are. Sorry. Yes, so for those people that are curious about what find internal dependency looks like, I've got some slides at the end that will go through it, but I wanted to at least give everybody a sense of what the key idea here is because what's backing this is maybe 20, 30 lines of code. So it's really not very complex. It's using tools like J-depend or N-depend in order to be able to extract the effort dependencies of a specific Java package. So here's a slightly more complicated example. System depends on util and core. So I'm saying that core and util are the two, I just form a set of those two packages and I assert that the dependencies for the system package are equal to those two packages. Same as the first example. So let's look at a slightly more complicated example. So here we've got, we've added in this security package and security is dependent on many of the other packages that we just looked at, but it's got this invalid dependency over here. It's dependent on the web package, right? Security shouldn't need to know anything about my web package. So what's going on here? So what I do is I write my test where I have a set of the valid dependencies. I have an invalid dependency here. So the comment says that it's invalid. That's depending on web. And I assert that, so I assert that, so what I wanna do is I wanna be able to get to a passing test as quickly as possible, right? So I'm asserting that this package, the security package, depends only on the valid and invalid packages that I've specified, okay? And then so now I've got a test that is in place that asserts the structure that I want. I can then go and take this invalid dependency out. I end up with a red bar, failing test, and then I can go and attempt to break that dependency within the actual code. So here's our process. And this is like just your standard TDD, but apply to system decoupling. So we start by writing a failing test to capture the existing dependencies. And I'm actually, I'm pretty lazy. Like what you could do is you could go and try and figure out at the start for this specific package, what are all the dependencies and I'm gonna encode those into the assertions of my test. I'm too lazy to do that. What I would do is I would write, actually I'll show the specific example, but write a test where I will actually assert, like write an invalid assertion. I'll assert that there are no dependencies for this package. My test will fail and it will say, I expected there were no dependencies, but actually there are these dependencies. I'll say thank you very much. I'll take those dependencies out and I will add them to my assertion. And so now I'm at a green bar. I will assert those dependencies. So in this case, foo is dependent on bar and baz. So they get added into my assertion. I'm back at a green bar. I split foo, I split those dependencies up into valid and invalid ones. So here bar is valid and baz is invalid. And then I go and remove the baz entry because that's the dependency that I wanna break. I get back to a failing test. I'm back at my red bar. And then I remove that, I break that dependency between foo and baz within the code. So very simple step-by-step process. Yes. Sorry, in the previous step, where did I, from the code? Oh, where did I remove it from in the test? So here, I've got my, this line here doesn't set invalid line. All the changes, I removed that line from the code. And I changed this over here. So it's just the val, I'm just asserting the valid dependency. Does that make sense? So it's like a super simple change to the test to get me back to a red bar state. And then the work begins of actually going in and changing the code in order to get me back to a green bar. Makes sense? Any, is this unclear to anybody? Okay, good. So here's the reality though. I mean, trying to break these dependencies can take a long time. And we don't wanna be stuck in a red bar state forever. So what I fully encourage is just check in your test with the invalid dependencies listed already. So they're just specified there in your assertion. That's fine. I mean, it allows you to, the test is passing. But all you've done now is you have added a test that enforces the existing structure of the system so that you know that you're gonna know that if there's any backsliding, so whether or not an invalid dependency is now added to that specific package in the future, you'll know that. And it allows you to continue your working. Also, because of the fact that breaking dependencies can be pretty hairy, it's nice to be able to revert. Like sometimes I'll go in and I'll start to pull one specific dependency out and I'll realize that it's way more complicated than I anticipated and I'll just go, okay, bail me out of here. I'm gonna go and pick on a different dependency and try that one instead. So I can just revert back to the state I was in before. So run these tests as part of your build process. Obviously key. So here's how telling Dave that you broke your build. But this has been fantastic for us because it's very easy. Even in a very well-structured system, it's really easy to incidentally introduce coupling into the system that's unintended. And so having the test in place prevents that from happening. The build will tell you when that happens. Especially with modern IDEs, where import statements will just automatically be created for you and you don't necessarily pay attention to which package has now been imported into my system. The test will tell me if I've done something wrong. Done something that does not reflect the desired architecture of the system. And so that's it. I mean, basically you go through this process, you decouple your modules, and that gives you enough of something that you can start to then pull out, put into a separate component, and then serviceify, and you've started to break up your monolith. So that's it for the presentation. Any questions about this? Super simple idea. Actually, before I take your question, how many people now that I've explained it would go and actually try this out on your systems? Awesome. All right, cool. So that's a pretty good endorsement, please. So the question is, he's not clear why following a domain-based package structure would lead to less coupling than following a layer-based approach. It doesn't. It does not lead to less coupling. But if our end goal is to extract a service, do we want to have a service that is layer-based or do we want to have a service that has a domain-specific responsibility? Because that's really where cohesion is going to be maximized, is by having related domain concepts put together. So it's not necessarily going to be less work. It may be more work in order to move into this sort of a package-based structure, but it will provide you, I believe, with a better result because of the fact that your system is not growing this way, right? If you've got this three-layered system, the most you can do is get three monoliths out of your system that are coupled together. Your system is growing this way, so you want to be able to split your system into these vertical chunks and then deploy them separately. And so the first step there, I think, is pulling those things that are related into a common set of packages. Other questions, please? So I mean, I have done some Ruby development in the past. I'm not an expert, but the same principles should apply in as much as you're still going to have related domain concepts that you would want to put together, whether they be into a separate gem or what have you. There are probably other Ruby developers in here, right? How would you answer this gentleman's question? And so I think that what you're describing as an engine is similar to what I'm describing as a service. Yeah, but I mean, that's a great tip. So take a look at engines, but also take a look at ways in which you can use tests in order to facilitate decoupling your system to move them into these independent engines. Exactly, exactly. Other questions, comments? I don't know if you guys can bring the mic out, but it might make it a little bit easier for the people who are raising questions for other people to hear what the question is. Yeah? The dependencies that you're talking about, are these compile-time dependencies? Are you talking about even runtime dependencies? I think, you know, Jdepend, for example, will probably only give you the compile-time dependency, right? Yeah. So I'm only talking about compile-time dependencies. Okay. Yeah, I mean, that's the easiest thing to be able to track through these sort of static code analysis tools. And it depends on what type of a runtime dependency you're talking about. For example, for us, we use AspectJ, and it does, because of the fact that the tests run after the aspects have already been applied, it will detect whether or not those, like the addition of those aspects introduce unwanted coupling. I was just saying that Eric Dornenberg had done quite a bit of work on using AspectJ to enforce architectural rules. And if you just search for his blog, you'd find some stuff on that. Well, let's go down here, because here's the mics down here. Maybe the mic can come back to you so other people can hear the question. So early on, I'd used MACR, which is a build tool, which just does that. But I see this approach is in line with the TDD, so you get feedback more immediately than waiting until the build to happen and give you the feedback. Yeah. But during a greenfield development, I don't know which approach would you recommend. Just do the same TDD-ish or you use MACR, because I've used MACR before, so it's a build tool. Sorry, what's the name of the tool? MACR. MACR? Yeah. I'm not familiar with it. But the principle is the same, and as much as it allows you to explicitly specify your structure in a way that you could run during your build process who would provide you with feedback if you had invalidated it, then go with that. The only advantage I see with that is it's one place where you define all your rules from two and so on. So that's the advantages, just one place where you have everything. Well, here there's one place as well, which is it's within a specific test suite or test case. So it is embedded in code, which has got some advantages, and as much as if you're using a refactoring browser, then it will find references. But I saw strings. You saw strings, but I mean the strings will be found by the refactoring browser. Or if you wanted to, depending on the language that you're using, you could use some sort of a statically typed reference to that specific package. One thing that we've done, I'm not showing the latest version because it's more complicated, but we've refactored this approach using theories from JUnit4 in order to be, in order to express the dependencies. And if you want, I can show some code that shows what that looks like. But I thought for this presentation it introduces an additional level of complexity. Yeah, it's up to you in terms of whether you want to have this encoded within your code, or if you want to have it living outside, either is an option. The key thing is keep it, just start out, keep it super simple. The nice thing is JUnit, most people are using some sort of a unit testing framework. It's easy enough to add it in there. One thing that we debated about was whether this belongs in our unit test package, like in our unit test build, or in our integration test build, because I'm assuming, especially if you've got a monolithic system and you want to have some integration tests, your build may start to take some time. And this is a bit of a slower test to run because of the fact that it's using class introspection. I mean, it still runs in the order of like a couple of seconds, obviously depending on the size of your code base. But we found that this is something that we wanted to get immediate feedback about whether it failed or not. So we have this as running as part of our unit test build. Please. Does it change your continuous integration strategy with this approach, or it pretty much means the same? It's exactly the same. So as I was saying, this runs as part of our unit test build. It's not split out as a separate build. And it's no different than any test failing. And the nice thing there is because the unit test build is, or the unit tests are what the developers are more likely to run prior to committing. And normally this type of a violation, like a change that would cause this test to fail is related to something that they've done at that point in time. Often it's incidental, and so it's an easy thing to catch before you end up committing. Other questions, comments? Can you show us the find dependencies code? Sure. How many people want to see what that code looks like? Sure, go ahead. Okay. Do you mind switching the slides back on, please? Okay. So this is for serious geeks only, so I'll go in the room. Again, apologize for the relatively small font. I'm going to show two methods. The first is called Analyze Package. And so it's a good idea. Let's see if I can just go like this. Any bigger than that? Does that make it more readable? Okay. So first line, instantiate jdepend. Then we want to be able to point jdepend at the directory that we're going to analyze. We tell jdepend to analyze all the packages that exist within that directory. And that's based on the target package. So here I passed in, you know, whatever it is, com.food.baz. So it's just going to the appropriate directory that contains the class files that we wanted to inspect. So we tell jdepend to analyze that. That gives me back a collection of all of the packages that are in there. jdepend is a pretty old library. It doesn't support generics, at least not yet. So we have to deal with some, like, non-generic related stuff. And we iterate over the packages. We find the package that we're looking for. So all this code does is it says, just give me the jdepend representation of the package that I want to analyze. Make sense? Okay. So then what we want to do is get the invalid dependencies from that. So for that specific package that I want to analyze, I want to be able to get the efferent dependencies. And then I, because that's going to give me every dependency, and as I said, we don't want, we don't care about any of the external dependencies. I'm going to filter out the ones, I'm going to filter out all the ones that are external. So only keep the ones that are internal, and the way that I'm defining internal is being that they fall within my root package space. So, you know, for me it's com.pulseenergy, right? And then it's like everything that fits underneath it. That's the stuff that I care about. Anything else is going to live outside of that package space. So it's pretty simple. And then that's just adding all of those packages to, based on their name, so just returning the string, to a collection that's returned back, and then I'm doing an assertion on that basis. And one thing that's nice is to, which I'm not showing here, but within the assertion it's nice not to, you can have a lot of dependencies within your packages, right? And so it's nice not to have to hunt through all these packages. It's nice just to be able to see, you know, what's the difference between the set of valid and invalid dependencies versus what's actually there in the package. So you can just look at the disjoint between the two sets. The code within the test, actually this is a way back, for those people who do Java development, it's using Google's Guava collections library. So if you're in Java, definitely use it. It's very, very nice. Makes collections much easier to work with. Does this make sense? Does this provide enough detail? I mean, you could see that there's not a lot of code here. And a lot of it is unnecessary complexity given the fact that Java is a very verbose language. So it could conceivably, I don't have our node implementation using match, but it's a lot fewer lines of code. This only handles single level of dependencies, doesn't it? You don't want it to go, you know, transitive dependencies and so on. That's correct. So I'm really just looking at the direct dependencies. Now, okay, one thing that might be of interest is, so let's say what happens, I've identified a specific invalid dependency that I want to be able to remove. There are probably better ways to do this, but within Java, what I would do is I would search for, within a specific package, I would search for that string in all classes within that package. And so what it will effectively do is it will flag all import statements and then so I can see, okay, which classes within this package have this invalid dependency? I can then go and look at those specific classes and see, you know, why are they dependent on this invalid dependency to try and break it up. But one thing that I think is kind of nasty about Java, Java imports is that by default, it does, the import statements refer to specific packages, sorry, refer to specific classes rather than packages. Ultimately, I think that what you care about is not that this specific class depends on some other specific class. I'm more interested in the fact that this specific class depends on this specific package. Because that's the coupling that I care about, is class to package rather than class to class. Because class to class, I think, is normally too granular. You want to operate more at a package level. I mean, at least from a design perspective. Please. This might sound like a very stupid question, but I know you're doing TDD, and my question is at what point do you start writing these tests if you're on a Greenfield project? I mean, on day one, what do you know what an invalid dependency is? So... Generally not. So generally, I would not write these sort of tests at the start because of the fact that it depends on how well you know what your domain is and how confident you are and how your domain should be structured. More often than not, I find that you don't know where things need to go to start out with, and you could attempt to enforce that structure upfront, but often it's just easier to let it evolve. You can start by... I mean, often just start by having everything in... I mean, how do most people do it, right? Start by having everything in one directory, and then over time you go, ooh, there's too many files in this directory. I'm going to start to split it up because now I've got enough examples that I can see that, okay, well, these things are actually related, so I'm going to put them in this directory over here, and these things are related to something else. I'm going to put them in this directory over here. Oh, I've got some structure now. Now I start to understand my domain. I'm going to write some tests that will... I think that this is a sensible structure, so I'm going to write a test that enforces this structure, and if there are any surprises that come up, I'll get notified about it, and maybe those surprises are okay, in which case then I'll adjust my test to reflect the new structure that I've learned. That's the approach that I would take. What do you think? Does that make sense? Okay. Any other questions? By the way, it is a great question, you know, it's... It was not a stupid question then, yes. I don't think that there's any such thing. Please. Currently we are doing this, whatever the story we are picking up, related module, we are refactoring, and we are making it as a pure JSP pay, removing all the Java code and all. Do you suggest, like, is there any tool is there, or is there any best practice we can follow? Otherwise, I can better way to do it. So I haven't tried doing this with JSP pages. Conceptually, you could point Jdepend at the directory. If you pre-compile your JSP's before deploying them, you could point Jdepend at that directory. Or if you wanted, you could do something even more primitive and effectively instead of using something like Jdepend, you could write some string matching within that... Because, I mean, your package name, that's going to be, like, a distinctive string within that JSP. So you could just effectively load the file as a string, search it, and then... and then do an assertion on that basis. So it's... you could be creative with it. I mean, it's not... obviously it's not nice to end up in that sort of position where you've got a lot of Java code embedded within a JSP. It would have been nice if things had been done differently, you know, that's what you got to start with. But at least the tests will prevent, you know, that from happening in the future. I mean, if you wanted, what you could do is you could make it even... you could add whatever sort of rules you want in order to enforce the structure. Maybe you want to add a rule that for every... every JSP that has had the Java code removed, you could have an assertion that ensures that no Java code gets added in the future based on some string matching. Really, I mean, this opens the door to a lot of other things that you can do, basically writing unit tests that enforce good practices within your code base. I mean, you can use static analysis tools for this as well to use things like PMD or... sorry? Yeah, check style, exactly. So there's lots of these ones, but you may not have specific rules that enforce the kinds of things that you want. So there's no reason why you can't just write your own as a unit test and have it run as part of your build process and fail to build it if somebody does the wrong thing. I mean, more often than not, it's done because people make mistakes. You know, we're Dave, right? Might as well get Hal to tell us what we can and can't do. No. I'm not sure who's done match, but I don't know. I haven't looked... I have not looked at the internals. Well, no, I believe that it's based on... it's based on directories. No, it's based on... it's based on require statements. Yeah, okay, so they just pause with the require statements? Yeah, exactly. So they're able to parse the require statements probably from the source code rather than... because, I mean, it's all dynamically compiled. It also depends if you're not using requires or if you've used Google Closure, right? So it doesn't really... you can use require, but if you don't use requires, then yes, you have to base it on the directory then basically some kind of a namespacing way wherein you map the namespace to the folder and then ultimately kind of write a Python script or whatever. Okay. Sorry, it was quite hard to hear you. And were you talking about Closure with the J? No, Google Closure. Yeah, JavaScript Closure Library, compiler, yeah. So you may not necessarily have a module pattern which requires JS1's YouTube so you can have a typical class Java-ish way or a C-sharp-ish way to organize your code. Yep. So in that case, yeah, this makes sense to... I mean, you could... but if you use... it depends on what what sort of... what's it called? I mean, there's AMD versus requireJS versus... what's the name of that... the syntax for managing dependencies between JavaScript files? Anyway, there's one that will use... the common JS syntax uses the require statement so it's the same as what is used within Node.js and that allows you to use a tool like MAJ on your client's side JavaScript files. If you want to do this client side as well, you can use the same tools. Maybe I can share a bit of code and then maybe we can... We started out using requireJS and we recently switched to using... I forgot what the name of the framework is, but it uses commonJS on the client side because it allows us to have consistency between our client side and our server side objects. This is kind of an aside. I don't know how many people... It's an interesting topic, actually. Managing JavaScript dependencies and if you're building a complex client side application, you're invariably using a large number of JavaScript libraries and upgrading libraries as a pain. If you're having to do it manually, you're having to go, okay, is there a new version of jQuery UI or some specific JavaScript component, if I have to go and check the web page of those specific plugins every couple of days to see whether or not there's a new version, it sucks. Whereas it's much nicer to use something like NPM, which is the Node package manager in order to tell you whether or not there's a new version. That's now supported and using the common JS syntax makes that easier to manage. Managing JavaScript is a whole topic. I know, it's a totally separate topic from this. That's one thing that's quite nice. There are libraries like Ender. We don't use Ender, but it was one of the ones that we evaluated. I can't remember what we ended up settling on because this part of our application is pretty easy, but I definitely recommend looking at something like that because it makes managing your client-side dependencies much, much easier. Thank you.