 You are at configuration management for humans There's a reason I didn't call this configuration 101 or basics of configuration and that is because What all started with a story of a client of ours very recently We inherited a large College project from a major land grant University. It was the College of Science and Engineering with 28 sub-colleges and departments all running on a Drupal 8 Install and we inherited this project They had been working on internally and quickly realized that they needed more resources So they brought our team on and my very first question to them was how are you using? configuration management and The answer was I were not we're doing single file uploads every once in a while like if I need to add a view So I sort of sighed but then I thought back you know six months before that our company Which I'll tell you a little bit about in a minute was also not using configuration management well over a year after Drupal 8 Launched and it dawned on me that there are many people that are not using it Don't understand it and that it's still this very complex system And so my hope is that this session will explain it In a very human way that makes sense Where I'm not going to cover a lot of advanced commands or topics if you want those I see Mike Potter there You can time travel back to yesterday and see his advanced configuration management session or watch it on video But he I believe covered Some of the more advanced commands. This is sort of an overview session that hopefully you will leave here with a Fundamental fundamental understanding of what configuration management is and why you want to use it So with that said let's get going My name is Tim broker. I'm not very active on social media, but you can find me on Drupal.org at slash broker Also, that's my Twitter. I don't tweak very often, but that's me I have been building websites professionally since 1994 and I did account. I think I've built Large-scale systems and over 17 distinct content management systems over those years In 2007 when Drupal 5 was released. I switched to Drupal and I've been using Drupal Pretty much exclusively ever since them. I Am currently the co-founder and technical director of a company called electric citizen We are a full-service Drupal agency in Minneapolis We have a strong focus on the public sector so most of our clients are large institutions colleges government agencies cultural institutions, etc And quick shout out we play a big role in organizing the Twin Cities Drupal camp, which is June 7th through the 10th this year It's a great camp with great people great parties and great food. So Worth the trip if you're in the area Okay, so with that said I want to give a quick sort of overview of where this session is gonna go I want to start with some sort of background and context on who I am or who I was in 2015, which is when Drupal 8 was released I want to tell you a little bit more about our journey into configuration management and understanding it then a quick config 101 where I sort of Give a quick overview of the basic fundamentals of how this is working Then we're gonna do a demo where we take a site from absolute zero to full configuration Which is great, but as we will see there are Handful of problems that come along with that and we're gonna look at some of those and some of the solutions Then we'll do a sort of a recap of the best practices that we hopefully have learned over the course of this session And then finally a look into the future and what might be happening as Drupal continues to evolve and grow Okay, so me again, I'm a long-term developer Drupal developer since 2007. This is 2015 the year that Drupal 8 was released We have a large client base at this point of Drupal 7 sites complex sites We have multiple team members frequently changing and updating those sites and We are heavily using features to manage all of this. How many people here know and have used features Just about everybody. Okay, great That was our as with all of you that was our way of getting things done In 2015, I know what configuration is. I know it's a difficult problem And then it's complicated and I'm a huge believer in Drupal 8. I've been following along With the configuration management initiative. I'm very very excited in 2015 for Drupal 8 to arrive You maybe you're like me. You were maybe you were an experienced Drupal 7 developer Or you could have been more of a casual hobbyist a WordPress developer, maybe you are new to CMS's completely in 2015 and you heard about this thing called Drupal 8 and you're You're there to check it out or you're just some random human being who who wants to use Drupal Drupal itself he we could potentially consider Drupal the antagonist in this story in 2015 Drupal 7 is a king It's been around for five years a very long time for a system to build up a Ecosystem of best practices and ways of doing things And then in November of 2015 Drupal 8 is released And as everyone here probably knows it's vastly different than any previous version of Drupal That we have seen and I'll talk about this a little bit more But I could I would argue that the configuration management system was arguably its most anticipated feature for me It was the one big thing. I was sort of waiting for and hoping for and that sort of sets the stage for us here. I Am going to show you my first experience just a Short version of it of what happened the day I first installed the the first release of Drupal 8 and encountered configuration management So configuration that seems like a good place to start but right away this looks like old Drupal I've got configuration all my stuff here. I don't see anything new yet Scrolling scrolling and finally configuration synchronization not configuration management, but it clearly says important export Okay, this must be what I'm looking for Okay, I click in there's nothing to import. Okay, that looks good. So let's check out some of these other options This is literally the first time with me exploring the system. I'm sure some of you have been through this yourself Okay, sync directory. I don't know what this is yet at this point configuration archive a tar ball Now this has me puzzled because we've long since moved past downloading tar balls So now we're going to go check it out a little bit more single items pasting a yaml file It's structure. Okay. I don't know really what this is Configuration there's all these options. Okay, I certainly don't have anything to paste here So I'm not quite sure what's going on. So let's check out the export option Okay export and download a full configuration of this site as a G zip tar file Again, this surely can't be what I'm looking for and what I've been waiting for it for so long So let's check out the last option here Single item export. Okay YAML structure again content type See what this is. Okay. We've got an article and Here is my first real exposure to YAML. I have no idea what this is Why it's there or what's going on at this point? Depending on who you are you're either a little bit Overwhelmed right now or underwhelmed. I was a little bit underwhelmed thinking surely there must be More to this system So what does an intrepid developer do but we're gonna go read the docs right? We've explored the UI We want to check out what the actual official documents tell us so I first encountered these documents in 2015 you can see here. These are our 2018 versions. I Went through all the reversions to see how they've changed they have changed some but this largely represents the documentation that was there when Drupal 8 launched and that is still there today and This is going to be a little overwhelming But I'm going to go through this quickly to show you what I read and if you haven't used configuration Before you're going to be confused by some of the things you see here, but we will work through that So this is the main page on Drupal.org The system is designed to make it easy to take live configuration test changes locally and export them to files Perfect. That sounds exactly like what we want So this is the the overview Now here's my first sighing of confusion in this document configuration can be exported and imported Either in its entirety or a single piece of configuration using drush and or drush console config commands or The configuration management. So this is my first sighing that this could be a little bit more complex and confusing that I was Then I was hoping for Scroll down. Okay. Either a single object can be imported using a copy paste workflow Copy paste that does not sound right to me and again or the full site and again This is today right now on Drupal.org or the full site can be dumped as YAML files to attire.gz file Okay. Well, it doesn't seem right to me, but we'll keep going and keep reading It's strongly read and this is the last page of this this main document It's strongly recommended to do a database dump. It could save your life. So now at this point I know we are in for some for some real trouble. So let's keep going Workflow using the UI again tar files. Okay, this can't be right. I quickly came through the rest of this I see the next one Workflow using drush this page assumes you are familiar with drush. I am this sounds like what I'm looking for here This is surely going to be Where the good stuff comes Install Drupal 8 will call the site live perfect make a copy. We'll call it development perfect Then I read down and this is still there today on the site a link to the issue queue and something that says until this issue Was fixed you will need a full copy of the site Turns out not to be really relevant, but it has me nervous that right in the docks. We have a link to the issue queue Okay, so we'll change our site name. That's good. So I'm gonna keep reading down Okay, da-da-da this exports the configuration to your sync directory I still do not know what this is. No one's told me what my sync directory is it wasn't in the docks I didn't see it in the UI When I do this it will be deleted the contents of this folder will be deleted whoo the sun's nervous I'm back to that database dump to save my life And finally you may want to change the location of your sync folder. Okay, maybe this is something So I'm sort of overwhelmed at this point, but I'm gonna go into this next article Changing the location of my sync folder Okay, by default Drupal places it in sites default using a hashtag thus Sites default files config hash All right. Let's go find this folder It's not there. So I'm immediately thinking Something has gone horribly around with my install because I don't have my config sync hash file Let's keep going as I read more I realize that I have to actually create this directory Make make a change in my settings file and that's it Yes, we're done and I actually did this and it was honestly my first taste of victory with configuration management where I felt like I have I did something I have a sync folder. All right, perfect Now here's where I get into real trouble and for people who have been following along with Drupal seven and eight The whole idea of configuration management was to store Configuration in files and version control so that you can manage them and move them and deploy them So this is now marked as incomplete. It wasn't when I first came to the Drupal.org dex But I'm thinking at this point. This must be of what I want because Files that's what configuration management is all about getting configuration out of the database and into your files So I read on for it to work. I need to modify settings and services YAML files. Okay You should do this before installing Drupal as it's complex to revert back to database configuration management once you've done this Okay, I really messed up at this point. I did not do this scroll down add the following code From this comment a comment 104 to so I'm now adding comment code to my non-existent install from a comment Open the services YAML file another comment add add this YAML from this comment to your services dot YAML file So I'm in real trouble now. I I'm really in for it. I don't know what we're gonna do There's one last article. This was also there in 2015 and it's still there today It's now deprecated, but it was not when I got there. So I'm thinking, okay, maybe this is gonna be the final Piece that ties this all together for me. So I have a Drupal 8 site and want to put it on my server. Yes Up to Drupal 7 this was a rather straightforward process In Drupal 8 CMI gets into the mix and has to be taken care of Okay Keep reading and now here we are talking about tire files again exporting my database from my local server Importing my database exporting an importing database. This does not sound right, but voila We have now gone through every single configuration Management doc on Drupal.org and we're done as far as I'm concerned. That's that's configuration management Oh So at this point I have no real indication if especially if I'm not me if I'm a new new to Drupal or new to CMS's I have no real idea of what Configuration even is I don't know why I'm supposed to use it how I'm supposed to use it or management and You know, I've done all I can I've gone through the UI I've read every doc there is and I've actually done some blog research This so and I want to preface that I had no way intend to criticize the documentation team and the the work That's out there and all the bloggers But the fact is this is a very difficult topic to talk about and write about because it's using concepts that are new It's using concepts that are somewhat difficult to understand and even the words used to describe them This is the last active comment on a very long thread in the issue queue arguing over calling it active configuration versus stage configuration Active currently in operation knew it will be it will be imported people are still arguing about this and This is a quote from a still seminal blog post by Matthew Tift who was very heavily involved in building configuration management When you Google Drupal 8 configuration, this is still I think the fifth Article that comes up. It was published just in November 2015 when Drupal 8 was released But yet it still is a very high in the Google rankings and one of his points was there is no recommended workflow Some small sites will not ever use it For the sites that will use it. There is one key question The truth is there are many many questions, but it's true that today even today in 2018 there is still no recommended workflow Okay, so now we're gonna get into What happened to me and our company electric citizen at this point? How are we gonna figure this out? So, you know, I wish I could say that I was determined and really ready to charge to this But the truth is there was so much new in Drupal 8 that we were I said, you know what? We're gonna we're gonna worry about this later. It's optional clearly. I went through the the config UI There was nothing wrong. There were no errors. There's no messages telling me what to do Let's learn some other things, you know, we have to learn composer and symphony and object-oriented programming and we have features that work and As Mike Potter who is sitting right there again a maintainer of the features module Was actively not just Mike everyone was actively pushing features and Making sure that features were gonna work with Drupal 8 and in conjunction with configuration management And everyone else was doing the same thing including most of the early and I think current Drupal 8 distros for example lightning I believe still comes packaged with a bunch of features and a lot of people were still using features because that simply is what we knew so we charge forward and Rebuilt our first Drupal 8 site with features the same way we did in Drupal 7 and quickly ran into No small amount of problems especially when we Got further along and tried to combine our features with configuration. Suddenly. We had this real mess on our hands and we sort of Stopped even thinking about trying to use configuration and we literally sort of carried on this way for over a year until one particular day July 26th 2017 I'm browsing Drupal Planet and I see this headline stop using features Hmm. That's that caught my eye and if you look down at the bottom there by Mike Potter software architect Mike has devoted a large portion of his career over the past X number of years Promoting and working on and developing features Including in Drupal 8 when I saw that name and that headline I had my heart started racing a little bit was like what what do you mean stop using features? So I continued reading Drupal 8 sites still using features for configuration deployment need to switch to the simpler core workflow Oh Okay, so I'm feeling we're probably doing something wrong here. Let's keep reading point seven where he says the list goes on Points one through six were detailed explanations of every single problem We had been fighting for months and months and months with features and configuration The list goes on and it did go on so why why were we using and why is it still being used as I? Reference before up until a few months before this it was the default workflow Aquia BLT was using features and then here was the one that really resonated with me my old D7 workflow using features Still seems to mostly work. I'm used to it and just deal with the new problems And I don't have the resources to update my build tools and processes. That was us. That was our company on this day So on that day it was a Wednesday. I looked this up. I went home This was a late in the afternoon when I found this article and I stayed up that entire night devoted to Learning what configuration was all about and and try and find a way to solve our pain and the problems we had been dealing with and by the following Monday When I came into the office. I had my first site using complete configuration management I ripped out all of our features and very quickly I brought that to our developers and And that has been our life ever since I have not touched a feature since that day we've Confession our corporate website electric citizen comm still is running features because of the cobblers children Situation and that was the first droop a late site. We did but other than that. We are now Fully devoted to using configuration management. So we are now going to and some of this might be basic for some of you but I I've in preparation for this and through You know just over the past few years. I've seen so many sessions and posts that start off with this premise that you already know what configuration is and You know immediately they say okay just export and import and you don't really know So I'm going to just give a very basic overview of what even is it? The very first thing you do when you install Drupal or or WordPress or Jumla or any other CMS typically as you configure it In Drupal's case, you're choosing your language You're setting up your database. You're configuring your site name The first thing you do is configure it and then in the Drupal world You can figure things like your content types and your related fields You configure your block layouts Your menus you can figure if you want a main menu and maybe a secondary and a tertiary menu Your taxonomies all of this configuration your views how they all work configuration Some people don't realize this one, but when you are enabling a module you are configuring Drupal to say this module is enabled and Then basically everything you see on this page, which looks again a lot like Drupal 7 did is your configuration If you install a module that might say, you know configure stage file proxy. That's all configuration It's distinct from your content I think most people get this but a lot don't and I say usually because there are some gray areas like web forms But configuration is not your blog posts. It's not your menu items. It's your menu structure It's not your taxonomy terms. It's your vocabulary where those terms are stored It's distinct from your theme kind of or usually It has nothing to do with your CSS files or your JavaScript There are a few places in Drupal in the appearance where you do configure some things and I'll show a quick example of that later And it's stored in your database historically just about every CMS in the world You install a database you configure it and that's where it lives Okay, so now we have a real brief overview of what configuration is why why do we want to manage it? So this is actually sort of an ancient problem in the software world that has been around for a long time outside of the CMS world And the idea is how do we come up with repeatable dependable processes for making frequent and complex changes to an existing system So why do we want to manage it? So technically because of that you don't most websites today? I would I would venture to say 90% run pretty close to this model. You have a developer making changes, maybe not right live and I'll show you a Little bit more on that later, but generally the idea is you change something on your live site Your users are seeing that one-to-one relationship. That's how most CMS's work without things like features etc But the truth is that the real world is quite a bit more complicated that than that You have a team typically Maybe not but often you do especially as websites get more complex You have a client That could be your boss. It could be someone that's paying you to build your website You have the client typically has content editors people who are actively also doing things on that site and your users of course and So with those added complexities Over time the expectations around a website start to grow your team wants to be able to make changes easily The client wants to approve changes and see them somewhere else before they affect their live site. They don't want things to break They don't want downtime. They don't want to have a content freeze or a code freeze And then over time the expectations start to grow as things get more complex and With that typically comes more complications You start adding more modules more More complex theming you're adding things changing things new features fixes and That also tends to grow over time and this problem of how are we going to manage it becomes harder to answer So how do we do it? historically This is the model with that most people start with you know the idea is okay. We're not going to do stuff Right on our live server. Let's set up a Local development environment a dev server and a test server the client up there on the right interacts with that test server When they approve it it goes to live That's great. That's sort of the de facto standard these days with aquia pantheon platform.sh the idea is You can set up these servers where you can do different things, but it does not solve our problem. How do we? Deploy how do we get our code from our local site to the dev server to the test server and back to live without breaking anything so this this you know tiered architecture doesn't really solve the issue It sort of is the first step that helps us get there though So, you know historically the typical approaches have been content freezes you tell your client, you know what? We have this complex thing to do So your team needs to stop working on the website for one week Our team is going to work on it and then we're going to re-upload your database and then everything will be great Back to my earlier slide that expectation these days does not usually go over so well Code freezes same thing or features, which is what a lot of you probably did and what we did And a lot of people in the Drupal 7 world would write You know using C tools exports or custom modules would find a way around this You know a much more a higher level of technology and strategies than then something like a content freeze So that worked, but in fact it was you know still quite painful features As you probably know could be very difficult to work with sometimes They were error prone to create because you had to go through a checkbox of about 400 individual items to to get there Not everything could be feature eyes. There was a module called strong arm etc So and literally it felt like half our team's life sometimes was was devoted to working with features and figuring out why they Were overridden and figuring out how to get them reverted. It was Very painful. So how do we fix it? So I Don't know how many people were in Drupal con Chicago in 2011 But Dries gave his state of Drupal address Drupal 7 was still quite new and everyone was excited about it But almost his entire State of Drupal address was about how painful certain aspects of Drupal still were despite how great Drupal 7 was and the number one Point that he brought up was this idea of configuration and features and how difficult that is and Right at that conference and shortly after he declared the configuration management initiative as the first Sort of step forward with Drupal 8 and I he Appointed Greg Dunlap. Otherwise known as Hay rocker as the lead of that initiative and one year later at Drupal con Denver, I think 2012 the first iteration of this had been committed to core and we were on our way So today so Drupal 8 2.5 years old As everyone who's worked with Drupal 8 knows it's still sort of evolving in front of our eyes every six months. There's big changes People are still confused. I'll go back to my earlier story about our client who Was not using configuration management despite this very complex system and they just didn't know how or why to use it They were new to Drupal and and they were afraid of it. So let's figure out how it works Now I've seen several charts and and documents trying to explain the basics. This is my attempt Apologize if it doesn't work, but we're gonna quickly run through this so We're back to our first model where we have a website and Drupal When you configure it that is called your active configuration that lives in your database Everything that your user see is coming right from that active configuration in your database That's how most things work before configuration on day zero You're not using configuration management until you need to take this step You export your active configuration to code you are now Technically using configuration management. It's your first step You have a one-to-one state between what's in your active config and what you've exported they match perfectly Now typically the next step is your team or you takes that exported code and imports it to a local Server or local development environment and now your entire team is in a one-to-one state with your active configuration Then this is what we're all here for you actually get to go do your work You build content types you add fields and views or maybe it's a small bug fix or a change that's that's sort of the Key point of all of this is what why we're here is trying to get work done right so your team does your work They then take that code and export it again. It now does not match Your active configuration on your live site it matches the work that your team has done Now you then import it again to Typically a staging server or a dev server where your client can then review your new work No one has touched your active config yet. Your users are still seeing exactly what is in your database Your client is looking at this and getting ready to approve it and Finally you we know it's okay. This looks great. You then import it into your production database Those changes that were made over in the work stage have now been imported and they are now your active config Your users are seeing those changes. It's in the database. It's your active configuration This cycle then repeats over and over and over again throughout the lifecycle of a project where Okay, now it's tomorrow and someone else wants to make a change. All right Let's export the active config import it make our changes Export it import it for review and then finally import it back That's the basics so I Get that that could be confusing for some and it's it is it's a hard concept to talk about sometimes But let's sort of walk through a a demo now of how it actually works in Drupal 8 0 to full configuration so for the purposes of this I'm starting with a a Fresh Drupal install out of the box in Theory you could have an active Drupal 8 website that has been being worked on for the past year But no one was using configuration management the same principles I'm about to show Apply to both but for the purposes of the demo we'll start with a basic site so step one install Drupal I Don't I won't talk much about this, but typically you'll want to use composer Get for version control and drosh I consider those sort of the cost of entry these days for for Working with a Drupal 8 site Then your next step is to install a local copy a Cloud copy and I say cloud I don't mean necessarily cloud it could be any hosting infrastructure or you could do this in reverse That's those two arrows. It doesn't matter which one starts first The idea though is you get two sites in a one-to-one state that are identical so Here I have my local site running Lando, which is a great local development environment if you haven't used it and Then on pantheon, I have an exact clone of this site They're identical Perfect next step define your config folder. So this was that document that I referenced earlier where I felt my first taste of victory Still the very first thing you should do It is optional, but highly recommended not to use in my opinion the default hashtag version I like to have more control over that and in particular I like to keep it out of my project route You know technically doesn't matter but the idea is you want control over where your configuration is going to live So in your settings file, you'll actually have an entry like this This is in that document on Drupal.org if you're ready to take this step where you're defining Okay, where is my configuration going to live? Then we're going to export it. Okay, that was a step one in our in our earlier chart So until you export You can set up your config folder. You can have your two sites You are still not using configuration management if you go into your configuration synchronization Synchronization screen it will tell you there's nothing to do. So this is really when it comes into effect. So we're gonna do this so I'm gonna do two examples here The first is a tire ball which I'm not gonna follow through on because I think this is the wrong way But you can do it this way. The idea is you export your configuration save it to a tire ball Download that file and unzip it into that folder that we just defined But I think there's a better way and that is with drush So drush C ex is a shortcut for drush config export It takes a little bit of it's a little bit of an intensive process here because it's comparing Our active configure actually right now. It's just exporting it. So okay success We've exported to our config sync folder that we defined. So let's look at that and now I Think by default there's somewhere over 725 yaml files that have been created and dumped into this config directory So, you know, I'm scrolling through them Nope, my screen is cut off there. But what I'm doing right now is Opening up one of these files to see it because I have never seen one before And there it is there. Oh, we have some screw. This is gonna potentially cause a few issues moving forward, but Up above there is my name configuration management for for humans. This is my yaml file great Okay, so we've configured our our exported our configuration. It's all there now. We're at that work step. We're okay. Let's get to Well, I guess I have a slide here. Oh, I wanted I forgot about this this slide but in your database Is an exact it's called the config table. There's a one-to-one Correlation between every entry in this table and those 725 files stored as a blob file. So I'm gonna download it Just to see what this one looks like Sorry I'm editing it but there is the version in the database and there is my title configuration management for humans Which matches again exactly in a one-to-one state with the configuration that I have exported So now we get to the work part the site building part This is again where the fun stuff happens. So I'm gonna actually just quickly make a few changes. I'm on my local site here configuration Say my client has said okay, this isn't actually for humans. Let's make this configuration management for robots We have a new target demographic We're gonna automate everything. This is no longer for humans Okay, we're gonna save our can save this this piece of configuration in this case our site name and our slogan Okay success great And while we're doing that. Let's really spice up this site a little bit and Change our theme. They think it's boring. We're gonna go to a hot firehouse red here How many people have ever actually used the color module? Really, okay great talk to me later about how and why Okay success Let's go check it out. Perfect. We've now got this great fire firehouse red We're all about robots perfect So we now have those changes and we need to export them again. Normally in Drupal 7 this is where we'd start creating features But no we're gonna export Okay Now again, I could use the UI and download a tar ball But in this case that you can't see it, but I'm typing drush cex right now and shortly. It's gonna tell us success I hope This is the sort of intensive process where it's comparing what's in the database and what's in your file system So there they are. Okay, there's our new Bartik settings System site, okay. Yes Success, okay, we have exported our configuration into that cold code folder that we looked at earlier Our next job is to deploy it. How do we get it from here to there? Okay Again, I apologize. You can't see my commands, but I ran a get status here. I can see that I've got some changed files got some new files and So what I'm gonna do now is do a get add and make sure that I pick up any new files that were created So that will happen anytime you are Making a new field or a new view you're gonna have a new and you export you're gonna have a new file that somehow needs to get into your repository So I'm now I added it. I'm now doing a commit message Making some sweet configuration changes And once that's done, I'm gonna do a get push and again this could be tire balls You could be Unzipping them and then manually uploading them to a server in this case though. We're doing a get push Deploying these back to Pantheon perfect. They are now there export Deploy So let's go check our changes They've been pushed to Pantheon Okay, that's about local site just to review robots read. Let's go to our Pantheon site. Hmm. They're not there yet Let's refresh Still not there Okay, let's see here. Let's go check out the config UI and see what's going on clear cash. Maybe this will help Still not there. We forgot something import our changes. We can't just deploy them You now have to do the next step which which is to actually go into your system and import So we're gonna go do that in this case. I am gonna use the UI This could also be done using the Drush config import command, but for visuals I think it's nice to see this so in our config UI we can look at the differences between what we have here And now here's where things get confusing Active. Okay, that's humans Robots that's staged a very confusing term Back to our configuration page Import all organizing. Okay, it imported successfully now Back to our site perfect. We've just pushed changes from our local deployed them to Pantheon imported them and we are good to go so At its most basic level, this is just absolute magic for anyone that's worked with features or Tried to do this using other methods. My example was very simple But that same principle could be used for adding, you know countless views fields content types all the configuration We talked about anymore. You don't have to think about it. You export it. You import it You're not working with feature madness and reverts In my description of this session. I said 30 percent time savings I think it could be even more in some regards