 I hope you still have some energy for a nice session. I'm here today to talk to you about Drupal recipes. Talking about how we went from the Dries note to the initiative and where we are today. So, first up, who am I? I'm Alex Bott, Distribution and Recipes Lead. I'm also a Drupal core committer. I work for two companies. I split my time between Acromedia, which is a Canadian Drupal e-commerce development company, and Thunder, which is a distribution that builds magazine websites. Both of these companies are pretty good to me. They let me review code, commit Drupal issues, and they've both been great for me during the pandemic. I would have easily been an overhead that they could have got rid of, but they didn't. They kept me on, but also, considering I'm saying thank you to people, I'm a core committer. I get to review people's work and look at their contributions, look at their reviews, and whenever I'm standing up here in front of people, I think it's on my head to say thank you for all the patches and all the issue comments and stuff, that I get to read, and all of the awesome work the community does. So, thank you all for your work that you put on Drupal.org, because I know it's a lot of time and effort because I do some of it myself. So, yeah, thank you. So, a disclaimer. This session is not going to have any fancy pictures or beautiful slides, really, because I'm not a designer. Some people might think design is easy, but it's really hard, and this slide took me at ages to do. So, if you like beautiful slides or expect that kind of presentation, it's not going to be here today. Yeah, I wish there was an amitybu or an item presentation that we could all go and enjoy and see something amazing, but unfortunately I haven't had time to prepare like UN calls to explain the group module or anything like that. In fact, a little bit of a disclaimer. I'm kind of bricking myself. Every time I get up and talk to you, there's a part of me which just wants to run to the door. So, and people say, oh, you've been doing this for like years. It's got to be easy for you. It's easier over time. Actually, it's probably got worse over time. So, if you think it's difficult to get up here and you don't know if you can do it, try. You might be bricking yourself. Maybe to everyone else you're looking really confident, but inside myself I'm out that door. So continuing kind of with the disclaimers. Obviously this talk is about the recipes initiative, the distribution and recipes initiative. We're building this now. We're under construction. Some of this stuff that I'm going to talk about, it's working today, kind of. Some of it won't be. Some of it will work on my environment and not on your environment. And pretty much everything is subject to change. But the good thing is, is if you're sitting here today thinking, hey, wait a minute. They're doing it all wrong. Now's the time to come and talk to me, talk to people working on the initiative and convince us that we are wrong and work with us so that everyone in the community can benefit from your ideas. And that's pretty much how I started. I was working on a massive project in the UK for the Royal Mail website. And we were reviewing thousands of lines of diffs of features. I was like, there's got to be a better way to manage config. And I went and sat down on the CMI table in some Drupal Meetup and met with Greg Dumlat, Hey Rocker, the initiative lead for the CMI. It's too many initiatives. Recursive acronyms galaw, but I met with him and I was like, hey, can I help? That's how I got started. It's actually with initiatives. If you want to get involved with initiatives, come and meet me and I'll help you get on board. But yeah, if you're thinking like, hey, we're doing this all wrong, now's the time to speak up because we're all under construction. So what am I going to talk to you about today? First up, I'm going to tell you why we're doing this initiative. Then I'm going to introduce you to what Drupal recipes look like today. Then I'm going to show you an example recipe. I'm going to take a little bit of a deeper look at something called the Config Action API that's new. And then I'm going to tell you how to get involved. Right. So, obviously nothing in Drupal is a green field. This initiative, the Drupal recipes work, is standing on the shoulders of giants. There's been things that have happened in the Drupal community that have changed the way that Drupal is. Starting off with Deane Space back in 2004 or 2005, that's kind of the initial distribution that almost made Drupal the success that it is. There were fantastic distributions in Drupal 7 called Open Atrium and Panopoly. In Drupal 8 there was Lightning and Thunder and the Thunder's continued into Drupal 9. There's Commerce Kickstart and we're trying to take all the things that they've... all the lessons that they've learnt and make things better because as we saw in the initiative keynote that we've been talking about how to make distributions better for a long time but we haven't succeeded with the current pattern so something has to change. But we've got to recognise that there is a lot of work that's gone on before. So, the thing about distributions is that creating and maintaining them is hard work. I work with four other fantastic colleagues at Thunder maintaining a distribution and we spend a good percentage of our time making sure all the contrib we rely on is up to date with the latest API changes in Drupal making sure that our own custom code is up to date with the changes in Drupal trying to make sure that all of our sites that are built off it can update successfully and it's a five person team working 365 days a year but all the days that you would normally work in a year to do it. It's not a simple task so if you as a community member think I'm going to make a distribution and just put it out there in the community it's not going to happen. And one of the reasons why that is is because distributions become kind of responsible for the entire site. When you install a distribution it's there. It's responsible for what themes you're choosing at the start it's taking over all of your configuration and you just can't get out of there and it's really hard to share that with others because you become responsible for all of these things and you have all the dependency nightmares and you have to keep up to date with your contract if they move quick if they move slow you have to help them update or you have to ditch them it's really difficult maintaining distributions and another big problem with distributions is that you can only choose them when you install. It's that first step you have to know about them and so we put a lot of effort into thunder but any existing Drupal site can't really benefit from our work and that's kind of annoying. And to misquote a very famous song distributions are kind of like you can get check out any time any time you like but you can never leave you cannot uninstall a distribution currently in Drupal and if you do all sorts of fun things can happen unless you know what you're doing so distributions have been great but they're not delivering us what we need but here maybe some of you say at least someone has said to me this week Eli, but what about features? So features are great you know they've expanded on what models modules can do they allow you to build themselves in the UI you can package related functionality together but they are modules they can have services they can have hooks, they can have dynamic functionality and that means they have a maintenance cycle and they become hard to share for some of the same reasons that distributions are and I think we've all had some experience with the features UI where you're like, do I press update do I revert there's a lot of complexity in there and when you're trying to share things with other people the more complexity you have the harder it is we want to try and make something that's simpler and another issue with features is that because it's so complex it's really hard to put into core it would be almost impossible to put features into core so I'm going to introduce Drupal recipes I'm going to tell you what they can do and then I'm going to show you what they look like and hopefully demo one right, what can the Drupal recipe do? It can install modules and themes it can create an update configuration it will be able to create content when we get to that and it can apply other recipes it can be composed of other recipes it's like a declaration of what that functionality looks like it doesn't actually have any code behind it there is no code it's not low code it is no code so here's what a Drupal recipe cannot do they can't have their own code we're not going to allow code in there they're not going to be dynamic in that way because the moment that they're dynamic you have to cope with more situations and that can branch into a thousand situations before you have even scratched the surface of what you can do and Drupal allows you to extend in so many ways and it's fantastic but in this case the less you can do the more powerful it's going to be the other thing that they can't do at the moment is they can't provide an upgrade path because one of the ideas one of the fundamental things that we've been thinking about is that when you apply a recipe to your site you've got that functionality it doesn't matter if the recipe maintainer in a year's time goes and adds some new feature to that recipe you've got that functionality your site's working that way it's not necessary you've chosen that functionality that time you don't need the new functionality from the feature or the new way of doing it because your site has evolved in that time and the recipe maintainer can't keep up with the changes you've made so you don't want to keep up with the changes that the recipe is making so everything that I'm about to show you is obviously subject to change some of it's working some of it's not but again if you're sitting here and you think this is what your idea is please let me know right so this is what a recipe will look like you'll have a folder with a recipe.yaml which will look a little bit like an info.yaml there will be a config folder that contains the configuration that will be created when you apply the recipe to your site and this is what a recipe.yaml file would look like and I'm pretty sure most of you when you look at this, you'll think I know what this is doing I can immediately see what this recipe is going to do but let's just go through it and look at it in a little bit more detail we have some metadata names, the type there's also a description but I haven't included because it's not necessary we're going to apply another recipe first so this would be a list of recipes that you would need to apply before applying this one or that the system will apply we're going to install some things in this case the datetime range module or make sure that they're installed on the site if they're already installed, the recipe won't error and the node module and we're going to perform some config actions and that's the new API that I was telling you about so yeah what are config actions so config actions is the one extension point that recipes have at the moment again, as I said, we're trying to keep recipes as simple as possible so we're not making them extensible from the get-go, we're not trying to have a pre-recipe step where you have to do things if you want a pre-recipe step, that's just another recipe if you want a post-recipe step that's a recipe that depends on that recipe, we're not going to have places for hooking in and doing magic things because the more magic there is the harder it is to maintain but there is one area where it's useful to have a new way of declaring things and that is the config actions API it allows you to do things to configuration entities in a declarative way this new API it's basically a new plugin type so it's a new plugin manager but it is completely decoupled from recipes it is a new API on itself recipes it doesn't have any reach into the current recipes code at all and it will never because there are potential other use cases for example modules could use this to update config in a simpler way and so I'm kind of excited about what that might become right, so this is the bit that is linking into the config actions so what we can see here is that we've got a kind of name of the action grant permissions going to grant those two permissions it's pretty obvious what it's doing but how does that work so what we're doing is we're mapping that to the user entity the user role interface and we're mapping that via a plugin called the entity method plugin and how this works is using new stuff in PHP 8 and 8.1 it's using something called PHP attributes and so you can on any method on a config entity you can say hey, expose this to the config action API so this will allow the plugin manager to detect that the user role has this config action of grant permission and we can add metadata to that action in this case we're labelling it but what you might have noticed is that in the recipe YAML we had grant permissions but the function name is grant permission and I want to explain how that works so what this would map to automatically is grant permission and as we saw on the API that accepts a single string of the permission name so you could put that in your config action and it will work but our driver is clever when it finds a method of grant permission by default it will pluralise it and provide you two actions one of grant permission and one of grant permissions but you can change that through the PHP attribute by changing the pluralisation so we can tell it to pluralise it to add permissions so if the method name was something like add data that doesn't pluralise easily then you can use this as a better name the way we make something a plural is we leverage symphony code they have something called the English inflector that will change a singular to a plural so if your method was add property it would automatically add you another plural version of that action called add properties and so here's what it would look like if we bothered to change the action label it would add permissions but obviously grant permissions make sense so let's leave it alone and what this is doing is when I first was trying to design this API I was trying to make everything work from the grant permission plug-in so what you would see is grant permission and then I was going to have an array of stuff and it would magically work out like oh you've got an array of strings therefore the argument here is a string and I'm just going to map it over but that led to problems because the magic was in the wrong place because we would have to like reflect every function to work out what was going on and that just didn't work because what happens if the first argument on the interface was an array how would I like detect it oh I've got an array of arrays but what happens if the array was an array of arrays it couldn't work it out so one of the principles that we're trying to use to develop the new config action API and the recipes initiative is to have as little magic as possible but where we do have magic have it in the right place so it's predictable and easy to understand and if it doesn't make sense to pluralise the method because it doesn't always you can just set that to false and that's all the nice attribute stuff from PHP 8.1 right we don't just have the entity method config actions we've already got stuff to update simple configuration so if you want to set the default theme you can do that via a config action or manipulate any simple configuration like node settings or whatever we can also create configuration entities if they don't exist one of the issues is that if you depend on a recipe then you then you need it to run but if you can say actually I don't care whether or not the role exists already I will create it if it does but if it doesn't exist I can create it and sorry create you can create entities if they don't exist what that allows you to do is to decouple your recipes from each other so you can have one recipe that creates the role but if the other recipe says that it doesn't exist to potentially exist and if it doesn't I'm going to create it then you don't have this tight coupling because one of the things we want to avoid is tight coupling between recipes and so this is kind of like dynamic oh only do this if it exists helps us get around that and we're also looking at adding a driver so that you can call methods on services via your config actions because there are some methods like static menu override service so that's a pretty interesting issue where we're going to be able to expand the API to do more things right so I'm going to tell you a little bit about the initiative now about how we're working and where we're working so that if you're really interested in all this stuff you can join in so how to get involved a primary place where we discuss all of this is on Slack in the hash distributions and recipes channel people are there pretty much every day talking and we have a bi-weekly meeting at 4pm UTC and our next one is on the 27th of September so if you want to get involved when you get back to wherever you live that's where to find us but if you want to check out stuff today the best place to start probably is our own project the distribution underscore recipes project and on that page you'll find links to the proper initiative page you'll find links to all the documentation that we've written and you'll find instructions of how to test recipes right now so that's what we're going to do we're going to try and do a live demo see how this goes right okay let's just install Drupal I look at my screen not the screen up there so I'm installing standard because it's standard away we go right let's log into the site and basically the recipe that you saw that was creating an event node I'm going to do that now so what we saw there is we saw that it was going to create a content type I can prove that I don't have the content type here now it's not one I made earlier and I can also prove that the role that it was referencing doesn't exist so how do we apply a recipe or what does the recipe look like let's just have a look at the recipe it is exactly like what we've seen up on the screen so how do we apply recipes as you saw in well as you might have seen in Lowry's presentation Drupal ships with a symphony command called Drupal it's in core scripts and so the recipe patch adds a new command to that the recipe command so that's what we're going to run and so yeah we just pass it to the event folder event applied successfully ready where is my mouse so hopefully we're going to see a role yay event manager you see the content type and new in Drupal 9.5 you can manage your bundle permissions from here and we can see that the permissions have been added to the event manager all nice there you go hi Jack oh how do I get that back now we have to come on there you are so the hope is that recipes well recipes are they're applied to Drupal sites the Drupal site that we just saw has no knowledge that that recipe was applied to it the config that it created the modules that it installed are there that's great but it doesn't know that that recipe happened it's not relevant anymore so because of this they're easy to share because they're a one time thing and then there's no lock in as I said like the site doesn't know if someone goes and changes that recipe and that would break your site it's not applied already and another good thing about them is that they're composable from other recipes great so next summer I'm planning on taking a break for a while or I am taking a break not planning so I would really really like to find another co-lead for this initiative because it's going to take a while to land at all and next summer will be kind of important so if someone wants to be a co-lead please come find me that would be great again how to get involved Slack our bi-weekly meeting but also on Friday come to the contribution sprint I'll be there there'll be a recipes table come meet me, come talk please tell me that all my ideas are mad or not and yeah thank you for coming to my talk contribution stuff thanks time for some questions there aren't any questions from the outside world so I'll bring the microphone out to the stand here and you can line up to ask your questions okay thanks for the presentation where was in your example your demo the action conflict plugin the code that made the changes the grant permission so where was it where was it located where is the plugin located yes so yeah where was that located so that's the plugin the entity method plugin it's derived from the PHP attribute so what happens when the plugin manager goes find all the plugins it gets all the config entity types in the system it looks through them and looks for any methods that are annotated and then creates the plugin from that and the code for that at the moment all lives in if that's what you're asking it all lives in config okay it's in core so I guess my question is if I want to do some custom which doesn't yet exist how do I ship it with the the recipe that's a good question so yeah so what you're asking is if the recipe needs to ship an action well then that action would have to be contained within a module and so the way that it works is that it applies any recipe dependencies first and then it installs modules and then you would look at your config actions and it doesn't validate the config actions until the modules have been installed so the plugin would be available at that point but it's a good question and there's something we should definitely test there's a dependency to a module that is custom that has the plugin and it's enabled before the actions are run so basically I can ship them together in a single atomic deployment yes I cannot put my hand on heart and say that works today but if it didn't work it would be a bug thanks I have two questions the first one is when you now kind of created these different content types if they already exist what will happen then? Will it merge or will it fail if they so that's a good question if the two config entities that are 100% match each other at the moment that would be like that meets my requirements I'm just going to go on if they don't match each other then it's going to fail there's no merge because at that point we don't know what to do obviously but the key to describe why it can't apply a recipe at the moment it's very I just can't do it, the config doesn't match but in the future what we would look to do is like ask you do you want to override that and continue with this because you've got, you the site owner has to make the decision because this configuration is going to become yours and the next one I'm doing some beta tests and other tests do you see this kind of as a tool for setup function for the different tests that you have a recipe for test data for example well yes I mean yeah of course it could be used in that situation where especially once we've built the default content stuff like populating a site with content is often very useful for the hat testing and if you can put that in a recipe and then share that with other people because you've built it off other recipes it's like here's my default content that I use for testing this stuff then yeah that would be great one of the aspirations that you illustrated towards the beginning of your talk was to reduce the amount of magic and therefore chaos that could happen when applying recipes does the ability to call methods on services introduce attention with that aspiration maybe it's why I haven't written the patch already I've been mulling it over it's tricky because it goes both ways there are some services which make config changes like this static menu override service that seem important and it would seem very useful obviously like all tools within Drupal they can be extremely powerful and thereby be really very good at causing chaos so I don't know but it wouldn't be like I don't think we're going to like create a symphony expression language where you'll be able to access any service and any method within config actions I think it's much more likely that we'll make it something that you opt in and you have to think about it's a proper important choice so yeah I don't know in reviewing what you demoed I see a couple of conventions that seem slightly different from modules and I just thought I'd ask if it's deliberate so for example in the recipe yaml the install syntax is just the module name whereas in a like a module info.yaml where it declares dependencies it does like the project and then the module name and then the other the other difference I notice is in the configuration module the config folder will have sub-directories for like install and optional and I wonder if like there was a thought to following that convention as well yeah sure so with the kind of like project colon module name I think yeah we will support that in recipe yaml because sometimes there's a difference between the two yeah we will have to this is just where we are at the moment this is what works and we haven't really started to think about contraband we're still just kind of like making this work for core with respect to like the config directory and optional configuration as I was saying like what we want recipes to be is a description of that exact functionality optional config doesn't make any sense because once the recipes been applied it's essentially is a fem on it goes away so if you have the dependencies to install the optional config at that point it would exist and if you didn't then it wouldn't but it's just not necessary so there is just no sub-folders in there so I've made some modules that basically do a similar thing but just using like config management and I find that that optional can be useful if like maybe as an example like path auto so you can provide a path auto pattern if a site is using that but maybe you don't necessarily need to like make that a hard dependency as an example right? Sure so in the recipes situation you could just provide another recipe that requires path auto and has that configuration and depends on your other recipe that doesn't Hi I was wondering the actions are done on config entities right? Yes Is there a way that we can do actions on multiple content entities? Let's say I want to build a layout builder recipe that adds layout builder to all content types that are available but there's no way to find out inside Ymol what the different content types are Is there something we can Yeah so that might be a case for having a special config action that does that and so what that might be is just like adding a method to layout config entity that is like add to all existing and then you would just attribute the method and it would just discover it and then you could pull it if the need is there we can add that kind of stuff and of course if your need is just there in your particular case because the whole system is a plugin you can add a plugin that does exactly that and call it in your recipe Thanks Hi I guess I'm having a little bit of trouble understanding these and what role they play as in terms of you developing a feature for sharing with other people if there's no way to upgrade it and for instance you develop a blog feature right and it has a blog content type maybe some fields and other configurations maybe a view configuration and if then later you have a different version of it everybody use the first version can't use your further version because there's no way to upgrade to it like say if you change your config entity now your view has got a slightly different field or filter and then you would fail to install because it has the same config name but different config details then they're cut off right they have to pick the optimal time to install this feature is when it looks really good and hopefully has no bugs so I guess the lack of a lifestyle management I'm just wondering how that works into this being useful for longer term sort of feature development Sure so the lack of the upgrade path and life cycle for recipes so one answer to that is like well you've got your site you've got the functionality you want the configuration is the property of the site when the modules are upgraded the necessary updates will be made to that configuration already it's just like if you went through the UI and did it yourself so the upgrade path for your configuration is necessary would be done as necessary they're kind of like the lack cycle for a recipe discussion is on one level it's like you have that need the need has been met if there's new stuff maybe the recipe author will do it in a way that enables others to kind of like go and grab it or maybe they won't but it's not you have the functionality as it was it's like okay it doesn't really matter however to all of this one of the things we've been discussing is because recipes are trying to be as declarative as possible it might be possible to apply one version of a recipe to a clean Drupal install export look at the changes it makes and then apply it to another apply the new updated one and then produce a diff it's all it's all inherent in it and then maybe we can have some kind of automated providing way of doing it so symphony does things like this for its symphony flex recipes which this is kind of partially inspired by and there's some similar tools in Dependabot that maybe but that's really advanced stretch goals that we're not trying to do because we're trying to say that we're trying to keep it simple and if we keep it simple then in the future those possibilities haven't up my question is pretty similar but just for the content if I understand we can import content after the configuration in case we have some updates on the content we want to push for everyone on the distribution how can we deal with the multiple version we published or can get this new content on the receipt again the answer is very very similar it's like if you've already applied the recipe to your site you own the content and the configuration that it's created so on one level how much do you care that there's something different there however it's if you wanted to you in your recipe could include a module that implemented updates and manage them that way but the recipe itself should try to not care for that ok so it's like a default content we have to at worst uninstall and reinstall the receipts