 Welcome, everybody, to the Configuration Management Initiative 2.0 updates here in Seattle. I'm Fabian Birke. I work for Nouvelle. You can find me on Twitter or on Drupal.org, I'm Fabian Birke. And hi, everybody. I'm Mike Potter. I'm a software architect at Phase 2 Technology, and you can find me mostly on Drupal.org. Just a little bit of background. I work for Nouvelle, which is a distributed team in Italy, Belgium, and the Czech Republic. Most of our clients are international organizations or institutions, and so they often want us to work with multiple people on the same projects at the same time and do fast and automated updates to their sites. They need to have frequent configuration changes, so we need to have a safe workflow for doing that, so we can't do napkin stuff. My colleagues who are also pioneers with configuration management back in early days of Drupal, and I've just continued with this tradition, so to say. And Fabian, for that to mention, he's the maintainer of the config split module, so that's a little relevant today, too. And I work for Phase 2. Phase 2 is a digital experience agency. We do a lot of Drupal work. We have a lot of big clients, some of which are listed there. So we don't just do Drupal, but we also do strategy and DevOps and user experience. But mainly my experience comes from I'm the common trainer of the features module and the maintainer of config actions in Drupal 8, and was the open atrium person in Drupal 7, so I've done a lot of configuration management stuff over the years. So just a brief outline of what we're going to talk about in this session, and we just had introductions. We're going to have a brief history of where configuration management comes from, why there is a second edition of configuration management, or the configuration management initiatives, some of the key concepts that you need to understand, and then the new event subscriber that we're going to have in the next version of Drupal, or 8.8 as we plan, and the kind of next steps of where we're going, and where you can come in and help us make this a reality. So configuration management, CMI was probably one of the most successful initiatives in Drupal 8. If you've all been using Drupal 8, I see a lot of heads nodding. So in the past, we had features in D7 and in D8. We now have declarative configurations, so configuration is no longer code. It's actual data stored in YAML files and in the database, and there's a common workflow now built into core where when you're deploying from dev to staged prod, you do the config export on your dev, and you go to your prod, and you do your config import, and it solves a lot of the problems. And there are contrib modules that you can use to extend that, but core was very specific on the problem it was trying to solve. But in the contrib space, we've seen enhancements to what core does because there's more problems to be solved than just that standard workflow. So you have things like config filter and config split that allow you to have different configuration based on what split is active, and you typically have a split for each of your environments. Like you have a dev split with the develop module turned on, and you have a prod split with, you know, aqueous connector turned on, something like that. There's modules like config ignore that allow you to ignore certain configuration. There's read-only that's a great one for production where you don't want to allow your client or anybody else to make any config changes on production, so you can lock that down. And then there's still, you know, features out there with the little aspect next to it that people still use sometimes for configuration management, but it's now more intended to bundle configuration into modules rather than being used as a workflow. And if you use it for workflow, you have all the same problems that you did in D7. So that's some of the areas that contrib has expanded on what's in core by default. Yeah, and so we have the configuration management 2.0 because the first version of the configuration management initiative, they were done. They achieved their goal. But as we've seen that we needed to do more, and we've identified that core needs to do more as well, so that we have a standardized way of doing things and these problems affect basically everyone, so it makes sense. There's three pillars in CMI 2.0. The first one is documentation. Unfortunately, there's currently not very many people working on this, but all of these three things could potentially happen in parallel. We have a little bit of trouble with the manpower, so this is a great way to come in. The documentation about configuration management that we currently have is not very opinionated, which leads to, you can do this, but maybe we should have it a little bit better to say this is a good idea and having the active configuration storage in your files may be not a good idea. Then the environment specific configuration that's targeted now for 8.8, and that is essentially a slimmed down version of config split in core. And then we have the cross-site configuration workflows, so that's when you want to use configuration from one side and have them on another side. That's the distributions use case, but we don't have any consensus in the community of how this should happen. So from the initiative, this is one of our goals to solve, but unless the wider community and the maintainers of distributions come together and say, how we want to solve it, we cannot just come up with a solution for core before configuration has a solution. For the organization, Alex, me, Mike, and Mladen are the initiative leads. We have a bi-weekly meeting in the last few weeks. It was weekly on Slack, and the best way to get, we have links, is the project CMI2. If you have suggestion to, like, documentation to add more links and make this a better page, then we're very happy to improve that. So CMI2 started several versions ago and has already had some successes. So if you, you know, if you all go and come to North America, Drupalcon, since a year ago, there's already been developments here. So in 8.6 that came out last fall, we were able to get in a new piece of functionality called install from config. There used to be a distribution called config installer that you could use. So if you had exported your site config, you could then use that site config to install a brand new Drupal site that was basically a clone of that site. And so bringing that into core in 8.6, the way it works, if you haven't used it, it's actually really a cool feature. You can do it a couple different ways. You can create your normal config sync directory to find that in your settings.php file that's part of your site. And now when you install, if you use the install from config option, the UI will actually see that you have that configuration exported and it will use that. And if you're doing profiles, you can also put that config sync directory within your profile directory as a config sync and it will use that. If you're using Drush as of 9.4, we actually have this dash dash existing config option. So instead of saying site install standard, you say site install existing config and it will go install whatever site you exported your configuration from. There's still an outstanding issue. So one of the principles here is we want to try to solve some problems, but we're never going to do the 100% solution. We want to get something in and then incrementally improve upon that. So one of the issues right now is you can't do existing config using profiles that have hook install. Mainly because some profiles use hook install to mess with the config and that doesn't make a very clean workflow here. Standard is unfortunately one of the ones that has this problem. And so there's actually an issue that's linked to the bottom of the page and it's there to fix standard and it's actually fairly far along. So if you actually want to get in and maybe help do final testing or RTBC on an issue, they've moved the config stuff into the proper installed directories and it had cleaned up their hook. So that should help with that. Yeah, and then in 8.7, which is released soon, we have a new configuration memory storage. This is basically all the stuff that we were able to break out from the more complex patch that we're now targeting for 8.8. The memory storage is probably only for people who work with this and for tests. The same with the config storage copy utility trade, which allows you to make copies of the configuration storage. Like the same everywhere because a lot of previous implementations, this code has been happening in lots of places and it was incomplete including in one of the core cases. So if you don't want to know about how to copy all the collections and all of the intricacies, you can just use this trade and be done. For 8.8, we already have committed the config export storage, which allows us to have a standardized workflow for export and configuration. Until 8.7, Drupal Core didn't have any sort of API or any place to hook into the workflow for exporting or importing configuration. We will have storage transform events and the experimental config environment module that we will talk about in a second. Yeah, so really quick, just to review CMI one. The way Drupal 8 works is you have your sync storage, which is the green area, which is your file system. You have your active storage in the database, which is the blue box. You do your import and export between those two. And if you want to read configuration, there's two ways you can do it. You can grab the immutable kind of read-only configuration. And that applies overrides. So that's where you can override things in settings or in modules. Or you can get the read-write version, you can call get editable and get a read-write version of config which doesn't have the overrides. So that's what's the basic workflow in Drupal right now. And then the contrib solution, the CMI one and a half. Basically, all it does is just swap out the config sync storage. And applies all the magic there. So in core, it's a very clean, very precision, very incision of getting a different kind of workflow working. And so, great solution, let's put this in core. But even though there's many modules that already enhance the workflow through config filter, there's a couple of problems and I think we should do better. If you want to know exactly how this works, I had a session at Drupal on Vienna where I explained in much more detail of how this magic works in different situations. But if you want to do this, if you want to use this API, you have to create plugins that have very many methods. And you have to know precisely how the storage is actually used. And you have to infer what happens and all the different methods they play together in this import and export steps. So we want to make it simpler. So we have this config storage transformer. We will do this with events rather than plugins. And you will get the whole storage with the event. And you can just massage it into the way you want it to be. And so this is not only useful for where in the import export, but we will use this in any place in Drupal core where config storage is used. Though we will more make it as a recipe rather than an API. So we will tell you, because dispatching an event is fairly common, it's not an API as such. So we will do it that way. So to go back to the schematics, so we want the magic to happen for the export and for the import. Because then you know exactly what you're doing and what you're reacting to. Whereas when you do the config filter version, you react on reading and writing and deleting and reading all and it's much more complex to make it work the way you want it to work and not have bugs. So the export version, as I said before, we have this export storage API now. So this part is almost taken care of. We need to find creative solution now for the import one. But we're working on that during this week. So really quick, with config split, what happens now is you take that config filter and you have the new storage system and it's creating these splits that are different subdirectories of your config sync directories. Now you have a dev folder and a prod folder and a stage folder. And you decide what modules you want to be enabled or disabled in those different splits. You decide what different configuration you want, like maybe different solar configuration. And when you do your import, it merges your default configuration with whatever environment-specific configuration you might have, enables those modules, overrides those config, and then does that when you export it. So that's how it works right now. And what we'd like to do in core is provide something similar to that, but for a very specific use case. So in a way, the config environment module, which is going to be a new experimental module in 8.8, we may or may not have it stable by 8.8, but it's going to be started as experimental. This is going to be an example of using that config transform event system. So config environment will have event subscribers that hook into that import and export process. And it will allow you to create whatever environments you want, like dev stage and prod. You will list what modules do you want enabled, like I want Devel and Dev. I want Awkward Connector and prod. You can list what configuration you want to change, like maybe your solar server configuration. And then there'll be a UI and command line interface for things like Drush and Console to let you switch environments. So you're on dev, you switched to prod. Now the Awkward Connector module gets installed, Devel gets uninstalled, and you're on prod. But the difference here between this and config split is you're only going to be able to have one environment active at any given time. You're dev or you're prod. You're not a mix of both. With config split, you could have multiple splits turned on at the same time, and they could interact, and it was all very complicated. So again, trying to simplify this for core. Because changing environments is already complicated enough, and some of these issues already exist in config split. This isn't a simple matter of sticking a JavaScript flag on your site, in your top toolbar, to let you switch environments. Switching environments causes modules to get enabled and uninstalled. And like I said, given that example of Devel and Awkward Connector, for example, and installing modules has side effects. So you're basically, when you switch environments, you're doing like a config import. And that may change things on the site. So you need to verify the changes, like understand the changes, just like you do with a config import. And so the two workflows that we're looking at is the first workflow is really the 80% use case of deploying from dev to stage to prod, which is what CMI-1 was all about solving. So here, you would set the desired environment, maybe in settings.php, based on environment variables. Or you add a new command to your deployment script, like a drush command. So at the same time that you're doing kind of your update DB and your config import, somewhere in there, you say, hey, make this site prod. And then when it does the import, you get the prod version of that. And that's a fairly straightforward workflow. We'll talk about problems in a second. But then the other workflow is potentially you want to take your prod server down to your local development to have the most updated content. Well, your environment is part of the database now. So as soon as you spin that up locally, you're now prod. And you've got to be careful with that, because if you're using things like API keys with Google Analytics, now you may be polluting your client's Google Analytics, because it thinks you're prod. So you obviously, again, with a command line or something, you would switch to dev before you actually start hitting the site. So it's complicated. Yeah, it's complicated. And it's just scratching the surface. There's many complexities that come with it. For example, what about people who want to use the ZIP archive, the download from the config? And then you go to the other site and upload the ZIP. Drupal Core supports this workflow. So whatever we put in core also needs to support this workflow. With config filter, there was no hope for that. But in the new version, thanks to the API that's already committed to 8.8, we can simply hook into that. And it uses the same process as for exporting with Drush. As Mike just mentioned, if you use the database done from production, this has interesting side effects. But for this, we also have a sort of a solution with the switching command or the UI. Translations. I know in the US, it's often less of a problem than in Europe. Drupal has this great translation management and language and multilingual. But so the configuration can be translated. That means if you have environment-specific configuration, environment-specific translation also have to be handled in some way. For that, we already have a solution as well. But we don't have solutions about what happens with updates. So just to give you an example with the API connector and the development tools that are enabled in different environments, if you update your site on the development environment, you usually get, you do the Composer Update, you get the new version of the valent, the new version of the connector. You run the update hooks, but only the val is enabled, installed. So the develop config gets changed because of the update hook. You export the configuration. Then you deploy it. And the deployment, when you export the configuration, the develop config gets split off and it's in a separate way. And so it will not be imported when you import the configuration again. But what happens is you do the Composer Install, which updates the modules. Then you run the app DB, which runs the update hooks of Darker Connector. And I'm not sure if I lost anyone already. But the update hooks only run in the production environment. And currently, the API, or I don't know if we can call it like that. But we expect the update hooks to be able to do all sorts of things. And if you change configuration, because the schema, for example, changed in the update hook, and this update hook only ran in production, then there's not really a good way of then importing the configuration again, which doesn't reflect this updated configuration. So you base the import, but then revert what the update hook did, which obviously is not something that we want. Currently, with config split, there is a workaround because you can add, as a part of your deployment process, you can config split, export the production split, which will not touch the rest of the configuration, but just export the configuration that is bundled in the production split. And then when you import as a next step, it gets merged together, and the good configuration is there, and it will not be changed. But we don't control the workflow in core. We just have this API, and the workflow is up to everybody else, and to which order of trash commands you run is up to you. The same is what about changes to configuration on production, especially with this configuration in different environments? It's very suggestive that you could have environment specific configuration on production, and then you change it, and we don't really have any way of dealing with this. You could say, OK, we ignore it, and the config ignored us, but then you don't have any way of updating it anymore. So we need to find a good solution that takes care of all of this. And yeah, you're very welcome to come and join the conversation about this. Yeah, so the summary is we need some help. If you were at Dries's keynote this morning, and you saw some of the initiatives that have gone into core recently, like Layout Builder, which is awesome, and Media, we put up those slides about 312 contributors and all the nice names. CMI2 is like 10 contributors right now. And I think for people that run in real sites, none of the sites Phase 2 runs without config split. That's part of our standard workflow now. So we're basically dependent upon a contrib module to do really basic deployment functionality. It'd be really nice if that was in core. So if you share that feeling, if you deal with config a lot, and you're looking to get involved in some core initiative work, come join us. We're doing a sprint this Friday. So we're going to have a table in the sprint room. You can do documentation, we can bring you up to speed, maybe just talk about some of these issues, try to come up with some solutions, write some code. Config environment is out there. We have issues, we have patches. It's got a really basic UI on it right now. So where there's some UI work as well. So there's lots of things that you can help with. We have four months to get this in before 8.8 alpha deadline. After 8.8, then things are going to start getting frozen in core while they do the deprecation. And so you probably wouldn't see this until Drupal 9. So we've got a timeline. We'd really like to get this in. So please come help if you're interested. Yeah, and now the final slides. China's for the contribution opportunities. There will be a first-time contributor workshop again and mentored contribution as well. It's all on this floor. And don't forget to fill out the survey so that we have your feedback. It helps us and it helps the Drupal Association to select future sessions. So thank you very much for your attention. We should still have five minutes for Q&A. Please use the microphone. This is supposed to be a recorded session. So if you speak to the microphone, then your question also gets recorded. Do you guys have a way to solve people all content? Yes. We have talked about this a bit, but it's not really in scope for what we're trying to do right now. But if you're passionate about it, we can coordinate with you as well. Yeah, there's a whole issue around content as config and that weird middle line of some things, menu links and taxonomy terms and things that are kind of config content. Web forums is another one. But it's not currently one of our three pillars, because we need to talk a lot about that before we can put something like that in for. With regards to the production modules, what you want to do in GataMate is, would it maybe make more sense to kind of establish what they standard to not disable production modules in environments and rather enable a top-level of the modules that you set up? Yeah, that was one of the solutions that we probably will favor. And in any case, the name config split comes from splitting, and this sounds kind of dangerous. Because it's a powerful tool. And every time I recommend it to someone, do not have split for only production. And if you do have modules very sparingly, the whole point of the configuration management initiative was to be able to stage configuration across different environments and to have, there's a lot of checks in core to make sure it's safe to stage. So if you do config ignore, you void the warranty a bit and you say, I don't want to be safe. And so we need to find a good solution for this. And I think probably the best is to document that you should not have too many modules in production only. And if you do or have the modules that are production only kind of modules, make them have some sort of toggle or use config overrides. I think it's not in everybody's mind, but there's a fantastic solution for most of these things with config override that works in most of the cases in this situation. Yeah, I think it would make sense for modules that interact extra money to get back to the core to be able to quick toggle them on. Yeah, like the config read only module that we mentioned, it's meant to be installed all the time. And you have a toggle in settings of PHP, you say, config read only is on. And then it will not allow you to change configuration. And I think this is a great solution that more modules should adapt that have surface use cases that actually make the right balance. And I was going to say, it goes back to what Katie said about dictating your deployment workflow. And if we're going to allow config environment to let you create your own environments, like what I would do is have dev stage prod. Stage is really all of my prod modules. And so you do your testing, you verify everything works. But you've got different API keys and things. And then when you go to prod, you're not disabling any modules or installing any modules. You're just changing config, changing overrides. That's a safer thing. But that would be like, OK, do we want to dictate that everybody has to have stage and prod, and that's the workflow? Like, you don't really do that in core either. So again, it might be documentation. Sean? Do you still see features as a useful model to the narrative use case that was originally meant for which one of the configurations was shared across? Yeah, you know my answer to this one, Sean. Like, I've been telling people to stop using features now for a couple of years. Use it to bundle stuff. I still use it sometimes in distros to bundle things. But you still have the same dependency problems you did in D7 in a lot of areas. So don't use features for deploying configuration. You shouldn't have a feature revert all in your deployment script ever anywhere anymore in T8. Thank you. Thanks, everybody. Thank you.