 So this is the session about the configuration management initiative 2.0 updates I am Fabian Bircher. I am a senior software engineer and I work at Nouvelle. You can follow us on Twitter and My Drupal.org username is just my last name. I Work for Nouvelle and Nouvelle is a distributed team in Italy Belgium and the Czech Republic We have a lot of international organizations and institutions as our clients and We use Like we do fast deliveries usually so several developers are working on the same site at the at the same time and We we have automated tests and we frequently deploy things so we need a workflow that is safe and This slide is a basically copy-paste since 10 years or more I I found out in with discussion that this is actually my 10th Drupalcon and the first Drupalcon I attended my colleagues gave Session on the main stage about configuration management. So It's kind of a tradition so so to say The brief Outline of the session I the brief introduction Very brief history. I can talk about this topic for ages. So come to me afterwards if you want to know more about it Then the CMI to focus areas The achievements we we have now in in 8.8 The things that are still open For 8.8 but need work and and need your input and then future plans for Realistically more Drupal 9 and and beyond and finally some best practices on how you can be ready for for the future already today Even with previous versions of Drupal and and how you can get involved So Most of you or some of you are probably aware that the CMI the the original configuration management initiative was Before Drupal 8 was released the CMI stopped when Drupal 8 was released and its main target was to provide a declarative configuration workflow into Drupal core and and since 8.8 8.0 0.0 We we had that it it was targeted to the like most common use case which is Staging the same site between different environments so But all configuration is exported all configuration is important and there was an important distinction and an important decision to Specify exactly a limited amount of workflows So that it can be done and and it's I agree that's totally much better to have something that works Than to have many plans and none of them actually are implemented. So they left everything basically to contribute to figure out than filling the gaps for all of these things and and contributed and so now we have contributed solutions for for some of these problems and They work quite well the config split as 1 million downloads for example And so we created the CMI to to bring some of it to core We have three different focus areas all of them need a lot of help the first one is is the one that is probably Least active so to say and that's documentation so All the tools are basically as useful as the best documentation you have and there's not that much Documentation on both how to use it and also how to extend it and how to use the API so we need better documentation for both of the Developers and and also the end users because this is a feature that's meant for for everybody Then the the focus area for which we have made the most progress so far is to Enhance environment specific configuration between Different environments of the same site. So so this is The the development modules that you don't want to deploy to production for example This is where the most mature Contribs solutions are and we've gathered the best like ideas and made them into core and the third one Is the one that I get most of the questions about at these events, but where there is no current A best practice let's say so and that's the cross site configuration So is when you have several sites that where you share configuration between them Distributions that has many names So shortly about the organization how we work and how you can help out Alex pot and me are the initiative leads at the moment We have a bi-weekly meeting at The five in the evening on the slack. There's this config Channel and we meet there. There's also a triple the dog project page CMI to Which is like the start to to find the issues that are tagged for our initiatives and to to find Thing more so this is the best place to join and to ask config related questions Next achievements for 8.8. Well the highlight that probably the reason why you're all here Installing from existing configuration. This is not actually a 8.8 achievement, but the 8.6 achievement But it's still like one of the major and the first real tangible results from our initiative You can define the config sync directory in your settings php and Or add it to your custom profile and then just run thrush site install with the Existing config flag or of course via the UI There's there will be an option like you can select from your install profiles, but then there's also used existing configuration There's a small asterisk that I will get to at the end for later Then another important Update or an important update that affects everybody like all of you literally hundred percent of people who use configuration management is That there is now only one configuration directory in your settings those php and it's not no longer an Array and it's no longer that config directories variable global variable and The reason for this is It used to be an array because of an accident or or it was never meant to be an array It it was an array because of historical reasons because of when the first CMI made their progress and and There was a time in very unstable alpha of Drupal core where there were more than one key But since 8.0.0 There was only ever one key in this array and Me included I came was like oh, this is an array cool. That means more than one value, right? I mean, it's that's what arrays do and You think like oh, how can I use this cool new feature and till you find out that actually it hurts more than it should and There's no reason for that. So we it's a developer experience improvement But it means it limits you in the future because there's you can come up with and rush supports labels and all of these things but in an effort to Consolidate and to make configuration management easier to understand and easier to teach it's important to to make sure that this actually core only ever supported this one key and Nobody stops you from adding anything else But core only supports one and therefore core only has one setting and It makes everything easier When you get used to it when you let go of this idea that there could be more There are for sure certain stances and situation where more Can make sense, but it's not the 90% use case and so now we changed it and It's for most cases a single like scriptable Replacement of the old config directories to this new settings same place Everything works the same way But it doesn't mislead anymore and that's that's the big win the big win is like the PR aspect Not how the code works because it's the same way Next we have an export API We have configuration management, which is about exporting the importing configuration But until Drupal 8.8. There was no API to export configuration Sounds a bit silly, you know, but It was never necessary because all the tools were just exporting from from the same place From the active configuration store and every everybody did did it this way But the problem is there is no extension point That you can interact with if you export directly from the active configuration Then you cannot extend this system. You cannot you cannot say oh, but now the active configuration store Behaves differently when you use it normally or when you export from it It's it doesn't make a lot of sense. So we added another config storage And so the replacement for it is really simple. You just instead of Using config storage you use config storage to export and you it's the same code from there on And this allowed us to then do more We now and this is for me my personal highlight the configuration storage transformation API It's Relatively simple, it's a like glorified alter hook It's an event driven. So it's the symphony event They're dispatched both when you export the configuration So when when this other storage that I just mentioned is accessed Or when you import the configuration If if you're develop if you're the trash maintainer or a group of console maintainer or a module maintainer that Does the import with the like non-standard ways? Then there is a small thing that you need to do But otherwise as a user of this API It's an event that you can subscribe to And it contains a configuration storage the the same Type as as all of the other configuration storages are and you just massage it the way you want you you You can add configuration to it. You can remove from it. You can change the configuration. That's in it The event doesn't mean that the interaction happens like this event is also dispatched When you want to see the diff like what is going to be in import But whatever you you happen to do to this Configuration storage is then what is actually imported or what is actually exported? So far everything clear right Of course, we cannot just add a new API to core without using it in core and this was actually the The biggest hurdle so to say to to get this in into core to make core work this way so we added Functionality that used to be part of the drush command the skip modules, then it became Contrib module and now it's in core. You don't need to install any module for this. This works eight eight Out of the box All you need to do is you add in your settings dot PHP the settings with the key config exclude modules and an array of module names that you want to exclude and Provided your system for importing and exporting uses this new API. So for example the rush 10 or the UI When you export the configuration these modules will never be part of it. So the If you have the well installed on your on your machine And you have this setting Then the export will be as if you had uninstalled the well before exporting it It will remove all the configuration that depends on the well And it will remove the well from the core extensions from from the enabled modules And on the other way if you import configuration that Doesn't have the well as part of the package that you import the zip or the file system But you have it the well enabled on your site. It will not uninstall the well It will keep the well there and it will just keep it the way it is it will not change any of the configuration So your your development environment is not affected by importing the configuration That doesn't have the well installed And if you didn't have for example file stage proxy installed it will not install it for you and If you didn't have it installed it will also not remove it because there is nothing to remove and This system is made for development modules or modules that you want to be completely excluded from the configuration synchronization This does not work at all with user system language Probably node All the configure all the modules that have a heavy influence on configuration entities It it will work when you export it it will remove system and user and all of like language from your export But the import will not work anymore the import will just say like sorry can't uninstall system and It will not allow you to import which makes sense. It's that's That's the that's the reason why we use the import it gives you some safety But using this system means also that one of you you take off one of the safety belts It's this the configuration management from from the Drupal 8.0 has a lot of safety belt. It's has a lot of like built-in Ways of of making sure the configuration Synchronization is safe and and it doesn't just blow away your content and it doesn't do all of this using this Allows you to have development modules but at the cost of a little bit less safety it's it's still Relatively safe, but if you abuse it, it's your own fault. You have been warned I Mean people will get creative and try to use this for many ways for which it's not intended But also then you I mean it's PHP and it's your own Drupal site. You can do it Whatever you want. I'm just not recommending it At this stage a huge thank you for for all the people who were involved so far and all the people who are going to be involved because The CMI 2 is mostly Volunteer driven I every once in a while I I have a couple of minutes to spare to to kick other people to like push them a little bit or what was it? He knows He was on my list to Of people to to notch like hey, there is a patch that someone wrote and maybe you should look at because Most and and this is probably Like a reason for for which most of like the country solutions they they kind of work and you don't actually need this to be in core and so You can just take the country module So it's very difficult to motivate like why why why should I spend so much time on? On doing this when it actually works for for my side. It's like I don't have a problem. It's like You know why why fix something that's not broken, right? So it's really Like thank you very much to all the people who were involved to like test issues test patches Even even people who write new issues and like explain what what is not working? And and this way we can move forward There's still a couple of areas that need help So I said before one of the achievements from 8.6 was to install from configuration Except it doesn't work with the standard profile, which is a bit ironic So the reason for this was we needed to exclude as much complexity as we can to get it done It was better to have install from config work for minimal and your custom profiles Then to try to solve all the problems at the same time and not have it So we excluded profiles with the hook install implementation because When you install a module there's two ways in which you can install a module the one is you go to your site you install the module and the other one is You had your site with a module extoll you export the configuration and you go to your other site where the module is not yet installed and you import the configuration and The module will be installed so that the import process installs the module and these are different different way these different processes and The the order in which the hooks are called is different so when you install a module from like the UI then Your configuration that comes with your module is installed and then the hook install is called when module is installed as part of a configured synchronization then All the modules that will be installed will be installed and have their hook install Called and then all the configuration is imported but not the one from the modules but the one from the sync directory or that the sync part and This this is has always been this way for modules And so you you cannot rely on the configuration that you ship with your module in your modules who can stall for method function and And that's okay. That's a limitation. Everybody got used to living with this. The problem is that with profiles There's only one way to ever call the hook install of a profile It only gets called once the first time you install your triple instance But now if we have installing from configuration Then the same happens so installing from configuration is essentially install system and then config import and The profiles then also get installed as a config import and then None of the configuration is actually there when you expect it to be there. And so it creates this Problem that it cannot actually work. So we said, okay Well, we just don't do it for now and have config install for other for all the other profiles and Now it's a time to actually fix this because this is in my opinion is still a bit broken It's it's clearly a feature that needs to be there But how it's going to be fixed this is still an open question because there is many Different avenues we could go down on on how did we could solve this And we say, okay, we added this other feature in 8.6 point zero So now everybody had time to refactor Their hook install method because anyway, it didn't work. You couldn't use this feature Like standard for example now works Or do we say, okay, we be special case the modules and we have lots of code to Have the profiles who can stall work differently than a normal module and Yeah, there's there's that I don't know it's up to you as well you the feedback is welcome ideas are welcome Patches are welcome reviews of patches are welcome No Standard profile would work But currently there is a piece of code that says if a profile has a hook install don't allow it Because we don't know if it's safe to allow it The easiest is to just patch it and and remove the hook install or rename the hook install And then you can install from standard by not calling the hooks that like standard install There's a list of other modules Their links so I will upload the presentation somewhere Where you can then click on I will also have this list for the contribution day so Some of these are harder some of these are easier, but these are still issues that could make it in 8.9 Not in 8.8 anymore. I think because some of them are bug fixes and Some of them are like the the add constraints to all config entity types Would for example allow the chase on API and the API first part to to use the configuration entities but some of them like There is no indication on config forms if there are overridden values Need more input because If you override the configuration Then there could be many reasons for this maybe it could is an API key that you don't want to show on the UI But currently it's not very convenient to not know That your form is Like you show on the form a value, but actually another one is used and For fixing this one could for example make forms more connected to the configuration schema which is Related to the constraints for example, so there's a bit interconnected, but If you want to work on any of these then I will be happy to point you in the in the direction and Help you getting along The config environment the the module that helped us to get the other achievements done So we added the API to this module and then we added the use of the API to this module And then we removed the API from the module and put it in core and then we removed the use of the API and put it in core and and Then now the the module is empty again And so but the idea of it is postponed and and not cancelled so we still want to Allow managing environment specific configuration which which was the outset like the original goal That we agreed on was was to make this happen We we didn't We had to compromise and make a make put in a much simpler version in in core Again, it's easy. It's better to have something smaller done and then to dream of the the big fancy thing that Wasn't done so this this is not Rupa 9 only and I think in the meantime we will focus on on upgrading the contrib echo ecosystem We will not forget about this and you can still also help working on this. It's not going to be wasted effort Hopefully most likely It's it's always The the amount of effort that goes into it if if it gets abandoned then it gets abandoned but as long as someone is interested in in this and and I think there's still a clear use case for managing environment specific configuration then there will be a motivation to have this available So best practices and and how you can get involved so for contrib solutions Used a new API where it makes sense Help port all the I Forgot how many the other initiatives had like a nice slide with like the With their modules ecosystem and that there is a lot of modules in with config something some of them can be Consolidated because the new API is a different approach than all the modules that depend on config filter for example If if you want to be prepared for for this config environment module that I talked about that we're still going to work on for 29 then the best is to use config split But with only one environment per module with only one split per environment Currently config split allows for as many as you want But when we get the config environments into core we will need to make it simpler and and more But with less edge cases because config split allows for many different workflows and many different edge cases and some of them they work and and most of them work, but you can also corner yourself Especially with updating configuration example If you have a split and you have configuration For a module that is only enabled in production and then you update This module and you have an update hook from this module then this update hook only runs in production and therefore the The configuration that it might be changed from this update hook Only run in production and then you have nothing to import from sort of to still keep it safe. It config split you can Have you can adjust your workflow to export the configuration first like only the specific one But it all depends on how you use it And if we want to put it in core it needs to be solid like it needs to work for everybody the same way So we we need to cut down a bit the complexity so You will not have multiple environments at the same time active for example as one of the ways of Keeping it simple For distributions so like this third part of the CMI to There's three steps come together Join the initiative and find the consensus around the workflow because There's a lot of people who maintain fantastic distributions But Every distribution handles configuration updates in their own way and in in their separate way and If if you start a new distribution you you can choose one of the approaches and choose your own adventure And it would be great if we could find like a Common solution or like a common ground a common denominator that then every distribution can specialize and and Benefit from but there is like a common ground of how you do this because currently While Drupal 8 supports distributions, you can obviously build distributions on Drupal 8 but the configuration management of a distribution in Drupal 8 is Somewhat lacking Call it and be nice for it and Important note here The this new API that we added is for is designed for configuration synchronization So do not try to use this new API for distributions But and there's a very good Alternative the config distro the It's a module it named config distro because naming things is hard And I didn't really know how to better name it Ideally, this would be something that core would support but we need to have a consensus first We need to first know that this is actually what core should support and There's an ethics because currently there's the patch is not committed yet, but there's a patch That transforms completely how config distro used to work into working the same way as the new API does so There's a new event that is dispatched when the configuration is updated. So The way it works is you import the configuration But instead of importing it from the sync directory from from the configuration that you stage It imports from itself from the active configuration And then it applies like the event system like you you all all of the event subscribers can change the The configuration that you want to be active after you have updated your distribution so It's It's a way around Update hooks currently probably the best solution is To use update hooks for your distribution like you change your configuration way in your module and then you have an update hooks that makes the new configuration be the same as the old and you have to have a lot of logic around to make sure that the Configuration is updated in a such a way that it still makes sense because the people that install the site based on your distribution might have changed it or Might not have and maybe you want to allow this change and maybe not it's Yeah, we need to find a good a common workflow or some some sort of consensus around this And finally a word about multi site Last time at the developer days I like 90% of the questions were about multi site and My suggestion would be to treat a multi site as you would a distribution With like a multi site is a distribution where it just happens that person who uses the distribution and the person who maintains the distribution is the same team And So shared code, but only shared some of the configuration. It's a bit it can work but with lots of caveats the The configuration split has been used successfully by some I was told but if you use the configuration synchronization process which is meant for Deploying the configuration of one site to another environment You're not using the tool for what it is built for and and it depends like really on if this is a good fit for your workflow, so if Do you have a staging environment for each of the sites in your multi site? Installation or how do you develop? Do you have like a master instance or do you develop lock features for each of them? Do you switch locally between which instance of the multi site you have? All of these things also here Come together. We would love to have more input of people who use multi site with configuration management so Yeah, join the conflict conversation. It's important that you Contribute as well to the ideas and if you If your business model depends on it, then you have a very good motivation to help out. I think Join us for the contribution opportunities. I will be there To to help review patches and commit stuff to my Contribute things obviously I cannot commit to other things But also on core issues That I can also not commit but I can help with and great the session. Thank you very much And there's still a couple of minutes for questions There's too many of you for all of you to ask questions about that. Yeah, go ahead first hand For regarding distributions is there an issue that's currently discussing that how they like to approach it or It was on the first slide. Yeah, that one other question. Yes You know the client conflict careful No, I'm sorry. Yeah, we have developed it for our use cases to record the configuration changes which clients are doing on the websites Do not override them so we can yeah Import our configuration without overriding the client Yes, we find it very helpful. Yeah Exactly, so you would also really like what kind of system do you currently use to? To use this module like what how do you hook into the? workflow Yes, when you and you edit you it records, but how how do you make sure that the recorded configuration doesn't get reverted yet? entities it shows up which changes have been made by which user and then as a administrator you can check that this locks and When you would like to override the configuration changes from the client you can delete the locks No, but I think we need to discuss this later. So it's very interesting. Yeah, there's a detailed Description on the module. I will check it out. Oh question. Yes Which country modules are Abandoned so I Don't know exactly The config exclude module Is completely replaced you if you can already start using it on 8.7. It was it's not By by the Drupal.org statistics a widely used module But you can use it on 8.7 already or 8.6 and then when you update to 8.8 It will it will be a drop-in replacement it that's for sure then the other one the config filter and There is already a patch, but we need to update the test for it for config filter To make a 2.x that if you use modules that depend on config filter They will use the new API. So the config filter 2.x will be the the bridge between the config filter 1.x and the new core API One more question one more minute. Yes For example field UI Yeah, so all the modules that don't have like a huge impact on the configuration So if you have modules that provide third-party settings for example, they're dangerous But modules that that are just the UIs like views UI perfectly safe views UI doesn't have any like special configuration that affects all the configuration Like the well has also configuration, but only its own it doesn't go and change lots of configuration I think that's it. Thank you very much for joining