 Just out of curiosity, how many people here had to come down from like higher altitudes for the conference? Okay, how many people had to come up from like sea level or so? Okay. Thank you. Thank you. Who else got winded like walking from the hotel? I'm gonna say that was the altitude and not the fact that I spend most of my life sitting at my computer. God help us if we ever go below sea level. I'll have no excuse for anything. Hi. Welcome to Drupalcon Denver. Tame the burrito. It's the name of this session. You probably know it since you're here and And I'm beeping. This session is technically it's about mastering Drupal's layers. The idea that there's you know Drupal isn't just a giant monolithic wad of code. Generally. But the idea that Drupal is divided into layers actually almost any major software project can be divided into different kinds of functional layers that work together. Each of them have different purposes and understanding how they work together and interact is a really important part of mastering that software and Drupal's no different. Also, I'm really fond of bad metaphors and burritos software what's not to love. There's a lot of chipotle around here, so I figured I'd work that in. My name is Jeff Eaton. I'm a senior architect at Lullabot, a Drupal consulting and training and Drupal stuff company. I've been working with Drupal for probably about Six years something like that. Yeah, you know, you're a Drupal nerd when you actually start thinking in terms of major core releases rather than actual years. It's like I remember installing image module and everything else that started with image and everything broke. So that's probably around four six. But yeah, I've been doing Drupal stuff for a fairly long time and it's interesting that a lot of a lot of large complex systems anybody who got in sort of on the ground floor or at least when they were simple enough that somebody could understand them in a long weekend of just poking around the code and stepping through with the debugger. There's it's very easy for for those of us who got involved with Drupal at that kind of time period to still have this mental model of Drupal that's anchored in a much simpler age. You know, when it was all like, you know, when we didn't even have deeply nested arrays and everyone used lightsabers and, you know, it was a simple time. It's not really that way anymore. Things have gotten pretty complicated and that's what we're going to talk about coping with the change in Drupal's complexity and what we have to do and what we have to understand in order to really work with it effectively. So the first thing that's important to notice there actually isn't any burrito in this presentation. I know, I know, I know, it's it's actually a terrible analogy because I actually went through an Emma Jane, Ha, Hovgen, awesome, awesome Drupal person, the author of I think the Drupal users guide that just came out. She went through a couple early versions of this presentation. Her comment was, you know, it the slides that attempt to explain which part of Drupal is the rice and which part is the guacamole. It really feels like it's just labored and so a lot of the burrito analogy has actually been stripped out due to merciless natural selection in the slide. So all you're getting is the meat, not burrito meat, but the, yeah, never mind. The important thing to keep in mind is that what we're talking about is layers, layered burritos. Yeah, that worked. But we could use anything as a metaphor for this. Ogres have layers, onions have layers. If you're a super, super nerd, you could talk about like the OSI network model, anything that has layers, even tacos. All of those things are basically about different kinds of things that are distinct entities and they work together to form a coherent whole. Well, I guess an ogre isn't actually made up of distinct separate entities. That would really be complicated, but you get the idea. The important thing to hear is we're thinking about the layers of Drupal, specifically with an eye towards how can I make my code work better? Like if I'm a developer or, you know, you're a developer. Okay, so quick show of hands, actually. How many people here write Drupal code in some capacity? Okay, awesome. That makes sense, because this is technically a coder track. So, yeah, that works. The idea here is figuring out how I can make my code better, how all of us can, and specifically how we can do that in the broader context of how all of our code works together, not just as a freestanding.ink file that's been polished to a wonderful brilliant sheen. Because, you know, that's cool, but in the real world, it's all of our code working together. So it's tricky. If you've ever seen one of my presentations, you probably are aware of the fact that I'm highly opinionated. I will say highly opinionated things. And if you disagree vigorously with me, that's excellent, because it's the start of a fun conversation. The neutrality of this presentation has been disputed. So, like, if you want to, you can start, you know, tweeting with the DC burrito hashtag on Twitter or just throw things at me or like a cost me in the hallway afterwards and say, you're wrong, Eaton. It's all good. But this also comes out of a fairly long chain of experiences working with Drupal and working with relatively large scale sites and small sites and individuals and core and contrib. So I'm not just making it up. Most of it. So it's about how we can make our code better, which is also funny because my first Drupal module ever was called comic and it consisted of code I ripped out of book code I ripped out of image and three sequel vulnerability. He's thrown in for fun. But time heals all wounds. We're not going to be talking about coding standards right now. A lot of times when we talk about how we can improve our code, we focus on the very, very fine grain stuff. How do we, you know, how do we properly make sequel queries? How do I generate a form properly and use this one API correctly? How do I not have sequel vulnerability? You know, lots of stuff like that. It's very, very fine grain. It's, you know, that polishing up that include file until it shines and, you know, web chick would not file a syntax bug against it. If I tried to, you know, submit it in a core patch, you know, that kind of stuff. I honestly think we pretty we've nailed it as a community. If you go to Drupal.org coding coding standards. There's a lot of really great information about that. And one of the interesting things is the important part about those kinds of coding standards is that in a large community of software developers, the important part isn't that two spaces is the correct and right way to indent things. It's that everybody in the community is doing that to indent things. So our code works smoothly together or at the very least can be read when you put two files next to each other. And you don't have to learn different ways of looking at and reading code. So we're not going to be talking about coding standards, but that kind of style that that the reason coding standards are important is very close to the reason why we need to get on the same page about how we think about the Drupal stack. We're also not going to be talking about gates like the there's a concept in Drupal as of I think about late in the Drupal seven cycle that started to become a big discussion topic. The idea of core gates so that major patches or major changes to core had to pass certain sort of, you know, markers before they could really be considered, you know, for inclusion, things like accessibility, you know, UX impact, you know, code complexity and stuff like that. Those are good things and they're useful to read up on because it tells you about sort of what priorities and what what things we consider blockers for major changes Drupal core. But that's also not what we're going to be talking about right now. It's kind of a solve problem or at least one that's, you know, nailed down close enough that, you know, just go to the page read about it. All we're going to be talking about right now is context, not not context module and not see tools context for this idea, the, you know, the Drupal world that we live in the layers of the Drupal stack and what that means for us when we're writing individual pieces of code that kind of context the actual dictionary definition of the word, because I don't think actually any of our Drupal contexts apply to the dictionary definition of the word. So, whoops. How many people have been around long enough that they remember like, you know, before form API. Now I feel old. Thank you. Thank you. At least there's there's some people in there way way back in the day Drupal really had a small number of layers. We still thought about it in terms of functional layers that had to work together but we're distinct and separate, but they were very, very simple. You had Drupal core, you know, we still think about that, you know, the core functionality that you install fresh copy of Drupal bam it's there and there's a certain set of expectations around what kind of stuff Drupal core does and is responsible for but on top of that you had contrived modules you could write your own you can download them you can drop them in we had 300 of them it was amazing who could you know you take a weekend to install all of the Drupal modules. And but you know there was a fairly good understanding of what functionality belonged in like our database layer in core versus a third party contrived module so that that kind of separation made sense and then on top of that we had themes you know if you actually needed to change the markup for something in Drupal well obviously that's the themes job and you know via successive altar hooks we've muddied those waters a little bit but still you know the idea stands you had Drupal core providing baseline functionality. Modules adding new functions and capabilities on top of it and then you had themes responsible for rendering things into HTML and you know always right with the world and everything was happy that actually didn't stay that way for long it's not that you know there were magical new crazy layers that got added. But what actually happened was that middle layer the modules layer where all of the crazy things got added to Drupal got really really really complicated we started accumulating radically different kinds of modules at that layer you know does anybody remember like the rise of modules that whose names ended with API. The crazy new you know oh something something API I'll install that it's got no UI but okay. You know the idea that there were modules whose sole purpose wasn't in fact to add a new feature that a user would actually see on the site but rather to add some sort of new baseline capability that other modules could build on top of you know there there was a time in Drupal that that was kind of crazy you know. I think one of the image module I'm sorry James I'm going to pick on image module a lot over the next hour so image module was one of the interesting early examples because it was a module that basically did image manipulation and scaling and stuff like that in Drupal. But it also came with a bunch of functionality like image galleries and a node type called image and it made a taxonomy vocabulary for you called like I think galleries and you know so it like set up all kinds of stuff like you know module you dropped into a phone install features. But there were other modules that if they needed to do stuff with images also said yeah go install image module because I'm not going to rewrite all that code and then that meant congratulations you've got an image gallery and you've got an image node type called image. And if you don't want that well you could go write your own image handling code. You know and that's where sort of you started to get that separation between different kinds of modules and what their purposes were. One of the other major major C changes was the emergence of a module I think probably a few people in the crowd have used it it's called views way way back in the day. If we wanted a new content type and we wanted a page to list it on we took story module and we copied it and we renamed all the files and then we did a search and replace in the module and we were happy and we walked uphill both ways. And you know that was commonplace it was considered acceptable but then CCK will technically flexi node came first that nobody wants to talk about that. But CCK emerged and the idea that you know between views and CCK instead of going off and writing some sort of custom module to you know build a simple news content type or something like that you could just use the UI to start clicking through and building stuff. That was a real sea change it was like in addition to just pure API modules suddenly we started having this layer of pure construction kit module stuff that you could assemble that functionality out of rather than just downloading and dropping in the function module. So you can start seeing how these different kinds of layers already started emerging in what was previously a wonderful pristine three layer system where everybody new designers live up there and nerds live down here and people who don't want to think their nerds live in the middle. Sorry I say it because I love and because I'm a huge nerd. Yeah that we started accumulating a lot of stuff and then you had the rise of modules like features or if you were there during the during the crazy days modules like patterns which was basically features but with giant XML files and stuff like that you know the idea that well now we're building sites out of all this config stuff but we need some way to like capture that other than writing a really really long list of instructions for the person who has to duplicate this work somewhere else. So it just got really really crazy and then even internally inside of Drupal you know one of the ways that we started dealing with all that stuff was just adding more adding more layers of complexity to how people could poke around and alter the internals of Drupal which got kind of crazy. We do in fact have a hook to alter hooks now. You can change the order of hooks from a hook. I'm sure there's a hook hook hook hook somewhere. It's kind of crazy. And I'm pretty sure that everybody has run into the kinds of pain that emerge from that level of complexity and not just that level of complexity but that kind of complexity. Has anybody here ever installed a module that provided some sort of functionality that you liked but the UX that was baked on top of it was just absolutely unacceptable for the end user functionality that you were making? Does anybody? Okay a few people. Yeah. You know that's the themes problem is an okay answer for a little while but eventually when you realize you're basically rewriting every single tag on the site and your theme because you need to finesse everything to actually look coherently. That's tough. Has anybody ever run into a module that just completely reinvented the functionality of two other modules because it didn't want to have a trivial dependency? I've done that like four times. Modules that can't be installed in an install profile. They break if you try to actually have really nobody's run into that. Is that just me? Is there like anybody from phase two in this room? The idea that all of these different layers that we've got are there and they're important and we've got a lot of ecosystem that relies on all of those different kinds of layers working together. But we aren't necessarily really thinking about those layers and how they need to work together and the result is problematic. Now if it were just my code or if it were just your code that wouldn't really be too much of an issue. You just say oh well okay I'll just you know carve out three extra weeks and I'll refactor all that crap and it'll be fine. But that's not how Drupal works anymore. We've got you know okay how many people here have built a website with 25 modules turned on? Anyone who says no is lying. Let's say 50. 100? There's still a fair number of hints. Okay who here has a site with 200 modules? Okay who here has a friend with a site with 200 modules? Okay yeah that's what I thought. Hell is other people. That's sort of my theme for thinking about how code conflicts happen in Drupal. It's not disparaging. It's not me. It's just when you have 18, 20, 30 people all working on completely disparate projects that just by happenstance and convention and luck all work together and you're building a major website out of it. That's complicated or I think it might have been Larry Garfield who actually clarified this and said that hell is other people's cash tables which is also true. But I mean the caching issues in Drupal are an excellent example. Caches are great. They solve all kinds of problems but suddenly you've got every single module on the planet implementing its own caching strategy that works great when it's used in isolation and suddenly you've got a major engineering task to figure out how to get your entire site's caching strategy to work effectively with all of these different tools. Those are the kinds of things that we're actually running into on a daily basis in the Drupal community even if we don't recognize that it's those boundary collisions between modules doing different kinds of work not necessarily playing well with each other. So I guess the heart of this is basically that we have to figure out how to work together effectively without necessarily getting every single Drupal developer in the same room to agree on every single function name and every single data structure because that just ain't going to happen. That was hard when DrupalCon was like 300 people and we just added a zero. I think we actually officially topped 3,000 actual attendees this year so, ooh Denver, I think that means we can afford to pipe in more oxygen. So one of the issues, one of the consequences of Drupal's growth is also that we are no longer just hackers and hobbyists building a tool that's interesting and fun to work on. We're also responsible to end users and clients who aren't necessarily in a position to be able to say, oh yeah that's not working, sure I'll rewrite token module this weekend. I'll crank that out, that'll be fun. We're responsible to people who may not have a dedicated Red Bull fueled room full of hardcore Drupal people to throw at every conflict that they have and we've got to figure out what to do about that. My way has to work with your way and your way and everybody else's way if we're going to continue sustaining the kind of growth that we've got. And this is where we actually come to the idea of five layers instead of just those three that we talked about earlier. I really worked hard on a five layer burrito metaphor but it just fell apart. I'm still bitter about that but there's going to be guacamole somewhere. You may actually be familiar with the concept of Drupal as a five layer stack. Nathan Ketchpole catch one of the core maintainers for Drupal 8. Actually wrote a blog post quite a few months ago about looking at Drupal in this five layer stack form I guess and I ended up writing a post on lullabot.com about it. So this isn't a completely new idea. This five layer model for Drupal has been talked about for a while and also ogres have layers. But it's important for us to all get on the same page about this not because I want to be right which I do but because we all want to work together on this stuff. And I think these layers are relatively well thought out by a large enough number of people that we can be pretty confident that this is an OK way to look at things. The web framework. Has anybody heard that word tossed around a few times in Drupal discussions lately. The Drupal web framework. No one. Does anyone had coffee. Are you just like you're all watching YouTube right now. I can tell the web framework layer of Drupal is sort of like the absolute bedrock. One of the reasons it's a little unique for Drupal is that it's actually one of the least Drupal portions of the entire application. We're talking about things like the ability to handle an HTTP request or the ability to cash responses so that they only need to be built once before they are delivered. Things like database access or session handling and stuff like that. These aren't even things that are Drupal in any way. We may have implemented our own code to handle these things but not because it was somehow inherent to the nature of Drupal. It's just if you want to build web stuff you got to have that so you end up writing that code. And that's a layer that I'm going to go out on a limb and say that relatively few people in the Drupal community actually live that day in day out. Some of us may like find ourselves bitten by a really obscure bug that lives somewhere in there that only happens in a particular edge case. But very few of us are actually really mucking around at this layer. And moving forward as Drupal evolves this is actually also one of the places where we have the most promise for integrating code from outside projects. In Drupal 8 we're actually just gutting some of the chunks of Drupal's internal pure web framework code and replacing it with code from the symphony framework. Has anybody heard about using symphony code in Drupal? It's cool. I'm going to go out on a limb and say this is good news. As we reuse code from other projects especially at this kind of universal ground work layer we're benefiting from the same kind of open source models that we tell everyone are a great reason to use Drupal. We get to benefit from the work that symphony developers do. They benefit because there's a much larger set of eyes in the Drupal community who are flushing out odd edge cases in it and everything. It's great and because this is not Drupal specific stuff the penalty for us swapping stuff out is very low. But that's the web framework. That's like I'd like something to be restful. Those kinds of things generally happen here. The next layer up from that is basically building blocks. When you're talking about building blocks we're not even talking about user visible functionality necessarily. We're talking about the code and the architecture and the conceptual models to handle things like say nodes. The idea of some sort of canonical chunk of content that we will use throughout the site. Nodes are a Drupal idea that form one of the basic building blocks of I'm going to say probably most Drupal sites. Things like fields added by the field API or CCK in Drupal 6. Those have become sort of an elemental core building block of Drupal. It's not like you know you can go to your client say oh man you're going to love this. It's got fields but they still form a really important part of what you're putting together when you build out a site. Things like views or even Drupal regions and blocks stuff like that. Those all live at this building block layer. They're pieces that we assemble into coherent functionality that end users end up seeing on websites. End users do not care if we implemented something using a view or using custom code that we wrote from scratch. The only thing they care about is whether or not it works well and it gives them what they need. There's a lot of stuff that lives there. Things like flag module. Even I think rules module might well live under there. They're the pieces that we assemble. Does that make sense to everybody? This isn't necessarily crazy stuff. But again in the olden days of Drupal when we just had core modules and theme. This stuff all was just sort of muddled into that one middle layer. End user functionality. Sure I'll tell my users you've got nodes. Congratulations and they liked it. What else did they have? PHP Nuke? Come on. The layer on top of the building blocks is the building tools. It's actually kind of hard naming this layer because it's a little abstract if you're just thinking about it. But has anybody ever heard the story? I think it's about two fish swimming around in the water. And they're like hey it's a nice day, it's a nice day. And an older fish swims up and says hey how's the water? Two fish look at the water. That's water. We're just here, we're chilling. That's what the Drupal building tools are like. When we as site builders or even developers tend to think about using Drupal. A lot of our thoughts immediately go to tools that are used to assemble those building blocks into functionality. Not necessarily the end user stuff that people are actually working with on a day in and day out basis. Catch in his post about the five layers of Drupal called this the Drupal application. Like when you download Drupal it ships with the building tools application that you as a site creator can use to assemble stuff. It's like the field UI module which is separate from fields and the concept of fields being attachable to nodes. Field UI, the actual forms and interfaces that you use to go in and assemble fields and configure them. That's part of this building tools layer. Views UI module as opposed to the views module that just builds and lists pages. That's where these distinctions between like the configuration and administration interface and the actual module that does the heavy lifting work. That separation it's relatively new in Drupal's life history and not everyone has gotten on board with it. But I think it's really, really important for us to emphasize this. Like views was one of the first contrib modules out there and it just did it because of the size of the code base. Earl Miles wanted to make sure that he was able to actually manage it effectively. But the views UI is a totally separate module that you can turn off separately from the actual views module that builds listing pages and blocks and stuff like that. That separation is critical because it's very easy for us to both ignore the value of really good high quality building tools. Which sort of leaves anybody who is in a hardcore nerd who would rather be just, you know, writing raw SQL queries. You know, anybody who isn't in that camp is sort of left out in the dark if our building tools aren't high quality. But it's also very, very important to realize that these kinds of tools are not something that matters to the end visitors. The vast majority of people who will actually use a site. These are important because they're what we use to build sites. But once we've built a site, they're sort of like the booster rocket for a Drupal site that falls away and is only used when we need to go in and change something. And if it doesn't get us to something high quality for end visitors, it's all a moot point. So there's a lot of interplay between these layers, but keeping them separate and understanding what their purpose is is really important. One important thing to keep in mind is that it's not just administrative screens that live at this configuration layer. Drush, like the command line tool for Drupal, I would probably argue that it can be a building tool. You can use it to do things like import and export, you know, view definitions and sling them around. Greg Dunlap and I had a heated argument last night about whether or not the features module constitutes a building tool. Because you don't actually build things with features, you just build them and then sort of capture them in features. But I'm going to boldly argue that it does count and should be in this layer because the features module is not something that end users care about. It's a tool you use at this point in the stack when you're crafting functionality out of those raw building blocks. And I think it's also important for us to remember that this isn't just happening via the web UI. We've got to keep that in mind as we move forward. And the final one, well, no, the fourth, there's five layers, we're on four. Yeah, I'll catch up, is features. Not the features module, but actual end functionality that someone who's coming and visiting the site might want to use. Examples would be something like discussion forums or like the My Dashboard feature on a given website or something like having a photo gallery or newsletters or surveys or a wiki or something like that. These are concrete kinds of functionality that someone working with the site on, you know, a visitor or an end user basis cares about and hopefully finds compelling. And these aren't necessarily just for like, you know, customers or visitors or whatever for the people who manage a website day in and day out but aren't building the website. People like content editors or moderators who have to keep track of user generated content or like the staff of a company that has to use the actual back end of the website to like manage users or whatever. They're not thinking about it in terms of building a website. They're thinking of it in terms of just a task I have to perform on our company's website. So even if as developers or site builders, we think of the admin section as, oh, that's where all the site building tools go or whatever. For the end users internally in a company, the admin section could just be the really confusing and crazy place where I have to go to add new people to our company directory or something like that. And that's actually one of the areas of the most, I think, serious blurring that we currently have. Administrative tools for people who are just managing the website or managing its content and users versus administrative tools for site builders and developers who want to muck around and stuff like that. You know, I'm going to go out on a limb and say that, you know, the HR people, Lullabot, do not actually want to access the develop section in the admin page. They do not care about running simple tests, but, you know, doing things like managing user roles is very important to them. So understanding that features are both for internal users who are managing the site and for external users who are coming in and, you know, visiting or browsing or whatever is important. The idea is that these are distinct functionalities for people who aren't creating or building the website itself. Does that make sense? How many times did I say features? Nine, ten? I'm aiming for twenty at the end of the presentation. The final layer and I think, again, the farther up in the stack we move, the more controversial the actual names of the layers get. So we can start a bike shed discussion about what the layers should be called later. But the very top layer in the Drupal stack is products. Now, I do not mean like marketable sellable products that we're going to like, you know, sell on a Drupal app store or box up or, you know, license or something like that. I mean finished products, a website that can be handed off to someone. If I make my blog really cool and I feel like I'm done building it in Drupal, which, who here has ever actually done building their Drupal blog? Anyone? No? Okay. But, you know, once it's actually a finished product, that is the thing that we are aiming for. That's what we are trying to build, like bobsflowers.com. I don't actually know if that's a website, but I'm going to guess probably. That is a product. You know, even if his nephew built it for free, it's still a product that is a discernible thing. Originally in my convoluted burrito metaphor, this was going to be the actual tortilla itself because it wrapped everything up. But again, I'm going to have to leave that. I've got to let it go. Distributions like open public, you know, a high profile site like the Grammys or even like, you know, like software as a service, you know, type offerings like Drupal Gardens. Those things are products because they're a finished end tool. Now, Drupal Gardens and other sort of, you know, Drupal sites that let you build Drupal sites are an interesting sort of middle ground. But I still think it's important to keep in mind that, like, the tool that we're building, like Drupal Gardens in that sense, is a product in and of itself separate from the individual sites that people build. Because if I'm a user coming to, like, Drupal Gardens, I see a website that allows me to accomplish something I care about. The fact that it's a Drupal site letting me build other Drupal sites is irrelevant to me. It's a product I could use because I want to accomplish something. So this product layer is often ignored by a lot of us or at least not talked about a lot. It's what we say, oh, well, you know, that's the client spec if we're working for clients or if we're working internally at a large company, it's just, oh, it's the website. That's what we work on. But at the end of the day, this is actually the part that matters. This is like everyone who's not us. This is what they care about. And as much as we work hard on things, there is a lot more of them than us. And this is what we've got to shoot for. When we're working on those other intermediary layers, at the end of the day, we're just working on building blocks and tools for ourselves and reusable functionality and stuff like that so that we can get to this point faster and better and easier and with less money. This is the end goal. That's important. It doesn't necessarily mean they have to be super complicated. Sometimes a product could be a single really compelling feature that just you go and you use it. I think webform.com just launched. It is a really, really tightly themed version of webform module at webform.com. It's open source. You can use it for free. You can go and make like survey monkey style surveys, but it's a Drupal website and they have the maintainer of webform module. He launched that and it's cool. It's a feature and a product simultaneously. But a lot of the tweaking and the customizing and the finessing, the stuff that we end up using to tailor the experience of a website really finally to the people that we're trying to reach. That lives at this product layer because that stuff is very, very hard to reuse. It's not like you can download the branding module that will apply great branding to your website or something because that's not a reusable chunk of code. That's what you do at this layer. So stepping back for a minute and sort of looking back at this stack of layers. We've got the web framework at the very bottom, the stuff that barely even qualifies as Drupal. We just need it to be able to make web pages. And that's where we can leverage a lot of big, big gains by reusing code from other projects if it makes sense. We've got the building blocks. This is a super, super hot layer in Drupal because you can also become super famous and beloved or reviled by building a module that sort of adds new, compelling, useful building blocks to sort of the toolbox that we've got when we build sites. Things like flag module. Wow, I love that module. But it's basically one of those things. The building blocks are also things that you find yourself coming back to over and over and over. When you look at wire frames that someone has handed to you or crazy napkin scribbles about the new site we have to build or whatever. One of the first steps of doing Drupal architecture and stuff like that is usually starting to think of breaking that down into what building blocks can I identify that I know are going to be a part of this. So that's a really important layer. The construction tools again are whatever we use to assemble those things and sort of craft actual functionality out of all of those building blocks. Things like a user karma system put together out of user points, flag module, voting API, two node types, things like that. Those are us assembling building blocks using the construction tools into compelling functionality. And that's where the distinct features live. Stuff like feature module exports that we put together or if we're going to go old school even just pure standalone modules that without any other dependencies just implement a silo of functionality that is useful. The old image module example, you downloaded one module, you dropped it in and you got galleries and a photo content type and you could do stuff like that. Just because something is a distinct feature doesn't necessarily mean it has to be a giant ball of configuration and 18 dependencies. Sometimes it is, sometimes it isn't but like that layer of end functionality that you could drop in and turn on and maybe theme but like that's your functionality. That's an important layer and it's also very important to keep that layer separate from building blocks. If you think you're making building blocks but you're baking in lots of assumptions about like what the final feature that this is going to be used in will look like. What you're doing is actually putting a lot of work into coding a reusable building block but actually making sure it only works as a distinct feature. So that can be problematic. People may still love it at that layer but it may not realize the kind of big bold future that you want it to if you wish it were a building block. So that's an important distinction. And then there's products and websites, Drupal distros, hosted Drupal platforms, the website that I made for my mom in Drupal because she wanted to use Drupal because I worked on Drupal. That kind of stuff. Those are our five layers and I think they work together pretty well and if we're careful when we're building stuff out it can really help. But at the end of the day, well actually at the halfway point of the presentation, we're still sort of left with this why do I care question. You know it's a lot of fun to sit around and sort of wax philosophical about Drupal and how the internals of Drupal well okay some people think it's fun to wax philosophical about the Drupal internals. But why does this actually matter to us? The first part and the first answer is really simple. The more complexity we have the more chances of collision there are the more different pieces we have floating around the higher the odds of some of them just plain not being on the same page about how they're supposed to work together and that means pain. Also when you start working on a new project or you go to write some custom code understanding where in that stack you're going to be working. So you have a clear vision without you know necessarily going through tons and tons of you know sample code and figuring out where the problems are. Being able to go into it and say okay I'm making a reusable building block I don't have to worry about the tools right now because I can just dummy up some records for it. But I'm going to make a building block that does X and Y and Z and that's what I'm working on right now that both frees you up and like helps you focus and I think that's awesome. I love it when I'm able to say I'm doing something that lives at that slice and I know where I'm looking and I know what I'm working on and I also know when I'm crossing those boundaries and I need to worry about it. And focus ultimately improves quality because if you know what you're working on and you know where you can concentrate your efforts on it means you don't have to necessarily get distracted. And if you're releasing the module it means you can say hey assembling this into end user functionality is your problem not mine. Well that's probably not the right way to approach it but I do. In addition this kind of separation really helps us future proof the Drupal framework and the entire Drupal stack. When we have that kind of separation of our layers it means that things like swapping out portions of that very bottom slice for symphony without necessarily breaking the theming layer is actually possible. So when we separate those layers we actually gain or we actually make it possible for future us to make better decisions about how each one of those layers could be improved without having to gut everything else. And that's great news. And finally this is I don't know if I like the way that sounds but I think it's important if we look at that layer as like as going from very specific at the top at the product level to very generic or usable at the framework level. We can realize benefits from both ends of that as new features are added to the low level framework like say the ability to use different kinds of database systems or better caching or something like that. Those can bubble up improving all the layers that live above them without necessarily having to complete to add new functionality to those layers everybody gets improvements that bubble up from below. And in addition the horrible ugly hacks that we have to do at the very top high customization levels because you know honestly it's like you know the client really really wanted the cursor to turn blue when I moused over input fields. God knows why but it needed to happen you know so sometimes there's just ugly hacks not even necessarily cosmetic ones but even functionality hacks that we have to do with that layer. And a lot of times some of the really interesting evolutionary functionality or even revolutionary functionality in Drupal is something that we find and we discover when those weird odd special case hacks need to be made. And those things can slowly trickle down into other layers maybe they become a reusable feature or maybe somebody decides hey this is actually a reusable building block I can share. But the idea that all of these layers are interacting with each other and benefiting from the stuff that's going on in each of the other layers is I think important. So stuff hacks can trickle down from products and features can bubble up from the web framework. That's actually a very unpleasant metaphor let's move on. So I'm going to try to keep things rolling but I think it's important when we start talking about philosophy to actually ground it in actual concrete ideas about what specific things we can do differently or what specific things we can do to improve like our own personal code. So I'm going to try to go through a couple of quick bullet points on what concrete things we can do and what rules we can keep in mind in order to sort of leverage the real benefits. I think one of the important ones to keep in mind whoops that no they aren't but that was a bit too early. Okay projects that you download on Drupal.org do not have to correspond one to one to a module if you've used something like views it comes with you know views UI and the views module itself if you've used something like the date module. It's like nine modules or something like that if you download the commerce module it's like I think you know 40 or 50 modules. I mean I don't know I lost count but you know sometimes that can be frustrating because when we actually properly segment out our functionality into different kinds of reusable chunks like that it also reveals the fact that the module administration page needs some desperate help. But barring that that's that's a real that's a basically a problem for site builders to chew on and you know nobody else has to worry about that. But the idea that when you release a project on Drupal.org or when you work on a large chunk of functionality it doesn't have to be tons of stuff jammed into a single module file and you know pile of includes. Figuring out well can I make a sort of reusable building block layer that lives at you know the API you know at the API level and then I'm just going to build a lightweight UI module that separate from it depends on it. Those kinds of separation steps are really important because they leave the door open for other people improving each layer independently. Like I wrote a module called simple views a few years ago that was a super lightweight you have five options to choose from front end for building a view it built the same view that the full views you I does but it only gave you five choices. Because Earl Miles had separated views UI module from views module that kind of implementation of a new building tools layer on top of views was way way easier than if those two modules those the building blocks of views and the construction tools of views UI. Those have been baked together that would have been much tougher so that kind of separation is important. And this is something that I think everybody working on core patches at some point has yelled angrily in a bar forms are not API's if the way that your module actually does data manipulation is jammed into a form submit handler somewhere. I don't want to go over the line but you're going to hell. I am not actually a spiritual authority of any kind so you shouldn't take my word for it check with you know your clergy or whatever. But forms are a front end to doing like post requests against a web app. We need to actually really internalize the idea that we need API's to do our heavy lifting work and form submits and form validation and stuff like that needs to call those API helper functions to do things like that. Check whether this is valid data or actually save this or something like that it doesn't feel like it matters when you're writing a quick and dirty module. But again this is the kind of stuff that immediately breaks if you ever try to do it in install profile or if you ever have to automate it because hey it worked great on dev now I've got to do it on the live server something like that. You want to be able to have the code that actually does the heavy lifting live in a way live at a lower like building block level so that you can automate it so that you can write Drush commands against it so that you can stick it into an install profile so you don't have to manually click through the stuff. That's critical it's really important if you've got critical logic in a form submit or validate handler move it to helper functions call the helper functions from those submit and validate handlers and that that's just at least a very first step that way it can be called from other places without them having to fake form posts because that way lies madness. The other one and I don't want to spend too much time on this because there's actually a lot of discussion going on this year about configuration management and how that's changing in Drupal 8 but configuration is more than just like crud. It's more than just doing reads and writes to a configuration table it's not just doing variable sets or whatever it's the totality of how a bunch of stuff works together and has been assembled into coherent functionality that's why it's a hard problem. And this is one of the reasons why providing API if your module is configuration heavy say like something like OG or whatever where you know it doesn't actually do anything until you really sync a lot of work into assembling the pieces and stuff like that. Thinking about it as a configuration workflow that you have to figure out how to model and figure out how to capture not just people can just write records to the database if they want to set it up or whatever. That's important again it's a little fuzzy it's a little abstract but if you're interested in really chewing on complex problems of how configuration can be isolated to you know the right layer. Check out sessions that are going on this week about the configuration management initiatives and deployment tools and stuff like that because that's where people are talking about this crazy wacky you know slice of stuff. Another good one when you're starting to think through like how to actually separate the UI of your module from the functionality that it embodies is think about operations that your module can perform and actions that your module can respond to in events that it listens for and stuff like that. Rather than just thinking of pads that my module exposes and clicks that my module will respond to and stuff like that just you know altering a button onto another form can sometimes get us there in the short term. But if we're just again jamming everything into special submit functions and alter hooks and stuff like that or really living at that bottom layer we're focused on a very this will happen in a web browser I'm mapping different URLs and different clicks to the stuff that needs to happen. Just stepping back and starting to think about like you know separate API you know separate API chunks that can handle key operations for the module and are triggered by the UI or say. Breaking out important operations that your module can perform into reusable actions that other modules can wire together via the configuration tools. Those really really make a huge difference when you start trying to reuse your own code in a different context because what you find is oh sweet and I need to change the UI entirely but these six key operations my module performs are exactly what I need on this additional project. It's it's magical and wonderful when you can have that feeling I want everyone to experience it sometimes I can't it's great. And when you are building something for reuse and to share in the community an important principle to remember is that examples of how it can be used always beat assumptions for how it should operate out of the box things like you know a module that when you install it. It may be full of building blocks and configuration tools but by golly the minute you turn it on it's going to set up three content types and give you six taxonomy terms to you know populate demo data so you know how it works. That tends to be bad it's making assumptions about how the module is actually going to be used rather than providing useful examples. The views module actually very early on in its life cycle did an interesting thing you could bundle you know canned views with your module but you could also set them to be disabled by default. So people could say oh your module has added three custom views they're not turned on so I have it like accidentally added four new pages to my website without realizing it but I can go into the admin interface and dissect them and decide whether or not I want to reuse them on my site. Examples that come with your module are always better than sort of baked in assumptions about how it should be set up right out of the box. I'm a big believer in that there's probably some people who disagree with me because the idea of a turnkey module is great. But if you really need your module to do something out of the box have that be a separate module that contains all of the set up stuff or some sort of separate step you can you can go through to say OK I also want the defaults for this module to be active rather than just always assume. Somebody wants that that also makes install profile writing much easier because otherwise the install profile has to undo all of that stuff as part of the setup process that's not. And finally thinking again about like this layered approach to things the more generic of a solution you're making the harder it is the more different kinds of use cases you need to think through the more different edge cases and different needs you need to take into account. So if you're building something that you think will need to be reused or you will want other people to be to reuse small bites are very important. It's easier to solve small problems in a generic reusable way than it is to solve really big problems. And honestly that's one of the reasons I think we've experienced such a module explosion. Lots of small modules that do something well and get assembled into big chunks of functionality. Turns out that's actually a much easier way to accomplish reuse than having the El Grande infinitely configurable things I want to do with images module. I don't even want to imagine what kind of checkboxes would need to be in that for it to really work for everybody. But we've learned that you know image cache presets image fields you know jQuery you know scrollers you can assemble those things into the functionality that you want. So solving small problems in a generic and reusable fashion is much easier. Sometimes there are just big problems like configuration management. God bless Greg Dunlap Hey Rocker for taking that on. He's like the senator of Drupal at this point. Previously it was Karen Stevenson for maintaining date module but they're neck and neck now. Sometimes there's just big problems you have to solve but when you can chunk it down into smaller ones that you can solve independently reuse becomes much easier. I talked a little bit about this earlier when discussing forms not being appropriate API's. But one of the best ways to determine whether or not you've really really magically separated things out into great layered like systems is if you can if you can control your module from an input form Drush simple tests to ensure that it actually works because everyone's writing simple tests for everything right. We'll just not tell Larry and and also can it be set up from an install profile without exploding in flames. You know can I use input forms in the standard admin interface can I control it with Drush can I do it from an install profile and can simple test another automated testing tools actually exercise my modules functionality without jumping through bizarre hoops of like pretending to send post requests so my module knows it's supposed to do something. You can like have these four points hit you're doing a bang up job so kudos if you do and if you're not well none of my modules do either. And there's a lot of emphasis in the Drupal community on reuse and how important it is but it's also key to remember that we can't always reuse everything. Sometimes you have to just do something to optimize a particular website or maybe you only have three hours to get a critical piece of functionality out the door for your employer's website or whatever. That's that's an opportunity to optimize you're either optimizing for speed or your time or something like that and that's when sometimes mixing the boundaries between those layers is important. Sometimes you have a module that cuts through four of those five layers and you know you bit the bullet and you just had to do that because frankly we had a long weekend to get this feature working and it works and we'll deal with it later if we need to. There's nothing wrong with that. These aren't moral imperatives despite my comment about going to hell if you put key code in the submit function. These aren't moral imperatives. They're tools for reuse and even though we can isolate these layers to reuse things sometimes it's absolutely necessary and absolutely appropriate to cut across those boundaries because we need to. The key is understanding that you are doing that and realizing what's implied in crossing those boundaries and what you're sacrificing when you do that because you know isolating out every every chunk of functionality for clean reuse isn't without its tradeoffs either. It takes more time to write and test that code. It's somewhat more complex. You have to configure everything rather than just dropping one thing in and doing it but it's a series of tradeoffs you know time and you know speed and performance optimizations versus the ability to reuse things effectively. And one last note about this is there's one thing that actually was in that original three layer stack, the theming layer. What people used to say was that's where design happens. This five layer model doesn't actually have that anywhere and I think that's useful and important because we're starting to get beyond the point where we can just punt off questions about design. Not necessarily like color and font choices but issues about how users interact with Drupal as a system or the features that we're building. That's not really something that we can afford to punt off to some other person anymore. If you're writing an API you are making design choices and you are making user experience choices. Maybe not for the end visitor who's going to browse it but for the other developer who's writing code against your code. You are making design and usability choices that affect their code writing process. If you build configuration tools you're making those kinds of choices for site builders and so on up the stack. There's nowhere in that layer that we can afford to say I'm just writing code for me that other people will figure it out. We're making design and UX changes even if it's not about CSS rules or something like that. And keeping that in mind is something that can help us bring a certain level of care I think to every one of those layers no matter how deep in that stack it goes. So those are the five layers. That's how Drupal works. The bad news is this presentation is a lie. This is not in fact how Drupal works. It is not a burrito. It's more of a KFC famous bowl sometimes. If you've ever seen one of those it's not a pretty sight. In the real world we mix layers vigorously. We cut across those things. We say holy cow user module does that. It's not a pretty scene. There are modules and there are portions. I got it. I got to take that down. There are portions of Drupal's code base that do work with the kind of layered model that we're talking about. But the important thing to remember is that because Drupal is a giant distributed crowd of crazy open source developers who build stuff and work with each other's code to build even bigger things, you can change the way that Drupal works by changing the way your own code works. If you decide that I'm going to approach building my code with this kind of layered model, hey, you agree with catch. So kudos. I shouldn't put words in his mouth. He's not going to like that. You're getting on board with a kind of approach that has begun to really prove itself over the years. So your module can benefit from it in and of itself. But in addition to that, you're starting to incrementally change the world of Drupal by doing it that way. You actually can change Drupal. You can sort of like leave Drupal cleaner than you found it. Every time you write code and write a module or find a module that doesn't separate its UI or leaves important code in the submit handler, sometimes just breaking a submit handler out into a helper function that does the meaty work is a really simple patch. And if you do that like Smokey Bear, you can prevent forest fires, always clean up the campsite kind of thing. Little changes can incrementally make a really, really huge difference in the code that we have and in the ecosystem that we all work in. So the recap. Drupal's got lots of moving parts. The code that we work with has tons of potential for collisions, and that's where a lot of our pain lies these days, even if we don't consciously realize it when we're encountering those bugs. Better separation between the different functional layers of Drupal aids in that cooperation because it makes sure that we know where those collisions are most likely to occur, and we can help make sure that at the worst case, we're only colliding with other people at this layer, not with everybody throughout the stack. The five layers, you can call them whatever you like, but the idea that we've got a framework, we've got building blocks, configuration and construction tools we use to assemble them, reusable functionality that we use to assemble sort of coherent sites out of, and then final finished products that either we capture in distros or handoff to a client or whatever. Those five layers I think are a useful model for us as we begin to think about where am I working, where does this code live in the stack, and what should or shouldn't it be doing or thinking about based on where it lives. And finally, you can mix those layers and cross them when you need to optimize, but when you want to reuse, that's when the isolation is really, really valuable and important. And finally, there is no spoon, there is no burrito, but you can make it be that way. You can. The power is yours. If you're looking for additional useful resources, there's a couple of interesting blog posts. I wrote one a while back that rounded up some of the posts that were going on about this. It's at lb.cmlayers. lb.cm slash initiatives includes a link to all of the currently in effect Drupal 8 core initiatives where a lot of key refactoring work is going on. And keeping track of that is a great way to sort of learn about what the latest thinking is about how these Drupal pieces work together. And just because it's always fun to have a fun booker laying around by Martin Fowler. lb.cm refactoring links to the book refactoring. It's like 15, 20 years old, but it's a great guide for people who are serious code heads about the process of iteratively refactoring a large code base to improve it on an ongoing basis. Rather than saying, yeah, let's toss it out and build it from scratch again, which is always fun, but never ends well. So that's it. If you want to fill out the survey, you can go to denver2002.drupal.org slash program and click the take a survey link. I hope it's been useful. If you hated it or you loved it or you want to talk about these kind of ideas, talk amongst yourselves, find me in a hallway, tweet about it. It's an important conversation. And the more we start looking at things this way, I think the better things can get for us. That's it.