 Oh, thanks for coming everyone. Ooh, that mic's a bit loud. Oh, well, nothing wrong with a good loud mic. So we're going to talk about standardizing website builds with Drupal scaffolding. So my name's Nick Dickinson-Weld. I've been working with PHP for around 15 years and Drupal for about a decade. I mostly do backend with a bit of DevOps. I'm currently a Drupal backend manager at Toyota Creative based in Washington, D.C. Although I'm actually on the far opposite side of the continent. I'm based in Victoria, D.C. Just north of Seattle for those who don't know Canadian geography very well. So I do have code samples and I will be sharing the slides and link for the code samples here and then on Twitter and LinkedIn for those of you who don't want to type a 30-character string into your phones. So before I get started, who here knows what Drupal Composer scaffolding plugin actually is? That's nice. So back in Drupal 7 before Composer then really existed and certainly wasn't in use by Composer. Composer wasn't in use with Drupal. It was often hard to standardize sites once you've done the initial site creation. Then with Drupal 8 there was a different solution as everyone figured out how to actually work with Composer in real life, not just in theory of Drupal works with Composer. Now what do we do? So in that time there was a community plugin, the Drupal Composer Drupal scaffold plugin which copied files individually via curl requests from Drupal.org or from GitHub depending on configuration of this specific scaffolding plugin. It worked okay. It had performance issues because it was curling 20, 30 files individually. That's not ideal. So then in Drupal 8.8 the Drupal Composer scaffold plugin got released. That's part of core. It's always available and it allows you to have full management of core and contribute non-site specific custom files. For example, if you're on Apache host you could scaffold your .htaccess files. If you're as an agency standardised on Lando you scaffold your Lando files or DDEV files or similar. Personally I use it for lots of files. The specific files aren't really important. They're just noise. I will be having examples of specific files that I use to kind of give you ideas of what things you can use it for but that's really based on your own personal needs. And basically anyone who has been using Drupal since 8.8 even if you're not consciously aware you are using this plugin because it's part of the core recommended project that is if you're not using a totally custom Drupal installation. So with Composer Scaffold when you run either Composer Update or Composer Install you get the updated files in your local. It avoids having to commit index.php in similar files usually. Settings.php is a one-time file creation in most implementations of the scaffolding because usually you then change that file. Whereas for many implementations such as Pantheons they provide a settings.pantheon.php which you never change. They only change and then you update your scaffold plugin and you get a new version of that as their platform changes. On individual project level you can override basically everything that your scaffold plugins are providing. Any questions before I get into actual examples? So here's basically the very most minimal possible usage. You've required Drupal Core and the core Composer Scaffold plugin. You are allowing that core Composer Scaffold plugin to run as a Composer plugin which started in Composer 2.1 I guess as being a required thing to do for any Composer plugin. It's a great security feature but does add a little bit of noise. If you need to do no changes you can basically just define your web root and you're good to go. Ideally you're going to do more because even the core Composer plugin is not ideal for most people's usage. It adds a bunch of files that you don't actually need. So here's a fairly minimal but more normal usage. In this case we see as well as the extra configuration under Drupal Scaffold we have a few more detailed options. So we have allowed packages. Every package that provides data for the Composer Scaffold plugin you have to manually allow except for Drupal Core. Drupal Core is always allowed to provide the data. You have the locations which can be just web root. In fact almost all instances I've ever seen is just web root but you could provide other locations as needed. There's no specific limitation there. You could provide config root. You could provide my custom location. And then below here you see file mapping. This is where it starts to actually be really useful to customize your own install. So Drupal Core provides HT Access, HT Router, Example Get Ignore and a web config file. There's basically no use case where you need both an HT Access and a web.config file. Web.config file for those of you who don't have to deal with them is for Microsoft hosting which is not my favorite choice but it's better to have a default for those that do have to use it. So personally I usually use Nginx hosting so I have both of those files ignored. False just means that the file source is not copied over. You can do more detailed choices but in most cases you don't need to. Here's a more complicated usage which is actually one of the things that we do at my work. So you see we have the file mapping now instead of just single lines of this is the file location, we aren't copying it over. We also have file location within curly braces, mode append and the source. None of our sites have both of these files in them but we can define the details here in composer.json for project and whichever file exists get updated with our custom changes to the Pantheon or Mazie hosting files. Basically it means that we don't have to worry about conflicts unless they make very massive changes but we get our own changes all the time. So if you're just starting to use it, if you're using a new site and you use the Drupal Recommended project you have it everything as you need and it's not really a pain to, you can just slowly improve your usage of it, you can add customizations etc. If it's an old site it's going to be a bit more painful. I mean old sites, they're painful. Even if you've made them yourself then you know everything you did wrong. So if all the files are already committed which honestly I hope not but I've seen it many times then you can just add the relevant lines, require Drupal Composer to your scaffold plugin and it will start to work. You will then have a whole lot of files that have file changes that you will have to decide if you want those changes or if you're overriding existing changes which is a bit messy but it's just a natural result of having an old file system to deal with. If the files are not committed then you have the challenge of if you, where did these files come from? Do we have them in some form of versioning so that we know what they were like before this plugin but in most cases that means they are just the general default Drupal file, hopefully. If you are dealing with that situation you can add Get Ignore Faults so that you can, in general the plugin adds a Get Ignore file so that any scaffolded file is not copied over into your Get repository which is good for performance. If you add the Get Ignore Faults then Get will see the file changes. So Drupal Composer's scaffold is a great plugin but needs data. The common data sources that I use regularly is Core which most people do, Amazing for anything that's on Amazing Hosting and Pantheon for anything that's on Pantheon Hosting. There's a dozen or so other options on packages currently but most of the sources I'm aware of are from private agencies, individual people and they aren't really discoverable and in most cases are entirely private and non-shared. Yeah and that's about it for just the basics of using Composer Scaffolding. So we're going to move on to how to create your own data providers. Before we do that is there any questions about using Composer Scaffold? Okay, let's move on to creating your own. So it's the same Composer.json syntax as using it in your own project but it's a bit more verbose. So for starters we have a Composer.json with the extra section. We have assets in a specific position. They can technically be anywhere in the repository. It's very much the standard to place them at assets slash whatever file name just so that, well, something has to be standardized. As to what files should be chosen, basically is it broadly reusable between most sites that you're using and is this file likely needing to be changed? If the file is going to be changed for each site that's usually a bad choice. If it's going to be used just the same every site, it's a great choice. There's definitely a big gray area that's going to depend on your usage. In that case you do have the append option which I mentioned earlier that you can just append a specific extra bit to the file every time and then use mostly the same file. Regardless of the details, definitely make sure to use versioning. If you use something like dev-main for your data source versioning you will get to the point that you break your old sites, which is less pleasant. So here's the structure of the data provider. This is a very simple one. We have two files under assets. We have a default.lando.local.yaml and .lando.base.yaml. These are never needing to be changed in my setup, so I just have them copied one to one every time it gets installed if the file doesn't exist. The composer.json, I've simplified here to only include the extra keys. It's otherwise just a regular composer.json. You can include any files. You need any description fields, etc. Okay, so that was a very simple one. This here is a fairly complete example of my own data provider. I cut a few lines out so that would fit on one screen easily, but this does show kind of the basic primary options here. We have a lot of simple copy into place and then we have some files that I change for every single project or have a good chance of changing, like .lando.yaml. I put into overwrite faults. Therefore, if it exists, it won't get overwritten and you have your normal local setup the same every time from the base. And then I have a bunch of testing files. Again, usually per project, I don't actually change PPCS configuration but I want that ability, especially for older projects that I might add this to. And then at Toyota, because we have multiple different providers that we currently use, we have a single composer scaffold plugin that has then multiple packages that we can also use. We call them named after their hosting providers for simplicity. In this case, because we have allowed packages in the sub-package, we do not have to specify the allowed packages in each individual project as long as the first package in the chain is allowed, everything is allowed underneath it. Which is good and bad. Sometimes you want explicit allowance, but in this case, this is how a composer scaffold works and we decided it was much easier than manually specifying three lines in every single project. And, yeah. So there are a couple other options when you're creating the scaffold data source. You have simlink true or false. If it's not set, it's default to false. Otherwise, instead of creating new files, it will... Let's just get back to the right page there. If it is true, it will create simlinks for any new files that are being created. If the file is then appended to by a later configuration, then it will get changed into an actual real file instead of changing the simlink location, which you probably don't want to do. Simlink is most often used for Drupal Core development when you don't want to change the original files. And then, with the append option, you can also append to files that were not scaffolded. By default, it will only append to files that were scaffolded, but if you use the force.append option, you can append to anything. The file has to exist, but other than that, there's no restrictions. So, obviously, that could be slightly dangerous. If someone changes the file you're appending to, you're making assumptions, basically. So, when it comes to challenges, basically the only challenge is override issues. If you were expecting this file to never need overriding, and then you suddenly override it, and you and Composer install, your changes get overwritten. In that case, you can just ignore the change by saying append false, which is totally fine. That's basically the only challenge to using it. And yeah, I would generally speaking, if you know your projects, if you have a dozen projects, you can look at them and say, we use the same configuration for our local environment in eight of these. That's probably a good file to copy. If you're using different configuration in 10 out of 12, that's probably not a good file to copy. Yeah, that's the basics of the Composer scaffolding plugin. There's obviously a lot you can do with it, but it is fundamentally a very simple project that copies files where you need them. Yeah, do we have any questions? Yeah, this one. So if we're looking at, the question is where is it getting these files from? So each project, if you're talking about the project that is using the scaffolding plugin, it pulls these files from any data source that provides them. It doesn't matter if they exist or not. If they don't exist, it won't pull the file, which is good and bad. The assumption is you will only pull data that exists, but it won't have an error if you try and do something different with it. These are the default locations. You can change the location, but the data source, which this file is an example of, you're basically saying, in this individual Composer package, which can be any Composer package, you can include it as a private package, which is usually what we do, or you could have published it on packages. In this project, we have an assets folder with this default.lander.yaml on the right, and then in the Composer.json, we have this little bit of JSON, which says when it's installed, we'll copy this to the project root directory slash default.lander.local.yaml. The project root directory is, again, presumed to be the project root, but you could override the location on that as well. Yes? Yeah, so the question is if we're using that in the project Composer.json or in an external project. The fun answer is both. So we have the scaffolding data source, which this file here is an example of, or we have in your local file, here's an example where you have the, just here's a very simple example of your local project. It just requires Drupal core and the scaffold plugin and it says the web root is at web. It will use any file that the core Composer scaffold provides. In most cases, you will use more packages, like in this example, where we're changing file mapping as well. And something I didn't go over earlier here, but where it says file mapping on your local project, you can change that to be a different source. It's not generally what I do, but you can change the source on your local and basically use your local as a data source. I don't, I can't figure out any real use case for that, but you can do it. Right. So the question is if you can use a local file as a data source and you definitely can if you use a local GitHub package, like a local Composer package with a directory file instead. I don't know if you can just override the file mapping and use a file that is outside of the project root. I suspect that might not work, but I've never actually tried it, but you definitely could use a GitHub, not GitHub, a Composer package that is just to a file directory and use it that way. That's basically no date different than any other data source as far as Composer is concerned. Any other questions? Yes. This, you can't, the question is if you can deactivate or activate projects with this. So the Composer.json in the data source can do anything a Composer.json can do elsewhere. It could require another Drupal module. It can't activate that Drupal module probably. Like if you use post install hooks with Composer, it would do that, but that's very much out of the scope of the scaffolding plugin itself. That's within normal Composer usage or perhaps abnormal Composer usage. The question is, can you provide a file mapping depending on the environment? There's no way that I'm aware of to put variables into Composer.json. So there isn't a default method. However, the scaffolding plugin does add an extra Composer event, which if you're familiar with Composer events, you can run any script. So you could put a bash script into the Composer data source and then run that on post Drupal scaffold event, which would allow you to make changes based on the environment. But there's basically, it would be an entirely unsupported usage. There's the facility to do it, but there's no default methodology. You'd have to decide based very much on your own system and needs. Yes? Yeah, the question is if we're using custom Composer projects or using the Drupal Recommended project. We definitely started from the Drupal Recommended project, but we do have both Toyota and me personally have different custom Recommended projects that we base all of our sites on that have our own scaffolding files already. Required so that it's just, if you're doing the same site over and over again, it's less work if you do your own project that you base everything from. No, the project template, the thing about the project template is you never, if you change the project template, all the projects won't get those updates, whereas if you have the data source in a separate Git repository, you can just update that to whatever new version. Ideally, you version it something so that if you say change from Drupal 9 to Drupal 10, your old Drupal 9 sites, you continue requiring 1x, your new Drupal 10 sites, you require 2x, and you don't have to worry about conflicts there. Or any other breaking change, but Drupal Core is just one of the big ones. Yes? I have definitely used both of those. Personally, I used it for a while and then decided it was more overhead than I really needed. I just generally add the Git definitions into Composer.json. If I had a lot of them, it would probably get to be annoying, but the overhead to benefit ratio wasn't quite enough for me to use packages private or other options like that. Any other questions? Yes? You can scaffold only... The question is if you can scaffold entire folders or just individual files. The answer there is you can only scaffold individual files, but you can make folders if by copying a bunch of files over, you don't have to only go into existing folders. Yes, you would have to define each file, for example, here. The default folder doesn't have to exist. If it didn't exist, it would get created to put in these files, but generally speaking, if you're doing whole folders, you might be better off to use Composer installers to just put the whole project in a specific location, where Composer scaffold is very effective in putting a bunch of different files in various locations. You have your settings.php, default folder, you have your project scaffolding, your lando, your ddev files in the root folder, you have index.php in the web folder. If you're doing whole folders, it's probably more efficient to just either use Composer installers plugin to put it to a specific location or do a post install script. Yes, the question is, copying the files around runs into permission issues. I personally never ran into the problem because all of these files, how the scaffolder works, is it installs the project to its normal location. You could theoretically scaffold data source as a Drupal custom or contrived module that when you install it goes into web modules contrived, or you can more frequently use the vendor folder. In either case, the web host already has access to those files, so they should be the same permission structure anyways. Theoretically, if you have different permissions on your vendor folder and your web folders, you could run into issues, but no more than just installing a Drupal module into contrived folder, I would expect. Thanks. Any other questions? Yeah, the question was, can I think of any examples where I've used post install scripts to alter the scaffolded files? I have occasionally wanted to do that. Most often I've decided I should try and find a way that I don't have to do that since that's less predictable than you have more different things going on. Very common usage is to include a patch file instead of just append, and then include a patch file and then run get patch and put that file onto a specific scaffolded file or module file that you can't patch through composite patches. Usually, composite patches does everything I need, but if you, for example, you like the settings.php that Pantheon provides for, but you need to change one line in it. You make a composite patch for it. You apply it through get-p-apply. It will work. The problem is, of course, when next Pantheon updates that file, you may have to reroll the patch. It's not a big problem, but it's just something I try and avoid because I figure it's less stable. Thanks. Again, unless there's any other questions, that's about the extent of today. Thank you very much for your attendance.