 Alright, today we will be talking about workflow goodness and more specifically about how to upgrade to use these exciting capabilities. We will start to talk about what the workflow initiative is all about, at least cover it briefly as a little bit of background to where this is coming from and why we're doing certain things here. We're going to talk about what, why and who needs to upgrade. What is this thing that we need to upgrade and who is affected and impacted by this. And then of course also talk about the how. We will be diving into a little bit of code to look at the very specifics. But first, my name is Dick Olsson, I'm a long-time Drupal Core contributor working at Pfizer. And I am the workflow initiative coordinator. Supporting me here today is also Andre Matescu, he's on the front row here. He's been the one carrying out a lot of this work here. I get to enjoy standing on stage talking about it. Andre, I'm sure many of you are familiar with his long-time Drupal Core contributor contracting for Pfizer and is indeed part of the workflow initiative. All right, so what is it that we're trying to achieve here? What's the background to all of these things going on? So the core of it is we really want to improve the content workflow capabilities of Drupal and of Drupal Core specifically. We want to do that by making full site preview possible, enabling content moderation and the ability to preview any type of content change before it goes live. Individual content changes as well as packages of multiple changes. Within the scope of the initiative is also to bring other types of major improvements to the very heart of Drupal Core being the entity revision and translation APIs. So that's our goals. Where are we at roughly? First, a very major achievement was to introduce the workflow module, kind of the foundation piece for moderation. That is now stable in 8.4. Content moderation, we're almost, almost there. I think within the next few days, we're going to have beta status on content moderation. The code is very much considered stable in that sense. The next step is what we call the workspace module. We're not going to dive into all of the details of what this is and what it means. I've talked plenty about that in the past. We're also looking to down the line in our roadmap introduce the trash module to implement very user-friendly features around undo, being able to recover changes and so on. And again, if you want more details on these capabilities on the roadmap where we're at, I had a session yesterday, so I would recommend you go and check out that session. The video recording will be there. If you also go to the workflow initiative issue in the issue queue, you'll find a comprehensive list of other talks that I've done in the past as well. So today here, we're going to talk specifically about upgrades. And kind of the background piece to this is revisions. Entity revisions and content revisions. Revisions are really the foundation to many of the things, if not most of the things that we're building as part of the workflow initiative. It's a very foundation to many of these capabilities. And we really need to or want to upgrade everything within Drupal Core to become revision aware. This doesn't necessarily mean that we'll put a revision log field, like in our editor's face, so to speak, it's the infrastructure piece, the underlying foundation for driving some functionality. So what specifically needs upgrading to really use these capabilities? Content entity types that are not revisionable, they need upgrading. So the scope of that would be contrib modules. It would be site custom modules. So if you have developed your own content entity types, it's actually quite common with our beautiful entity API that we have in Drupal 8 core to actually provide your own content entity types. So to leverage these capabilities, that's also something that needs upgrading. And there's also other code and configuration that needs to be adapted to become revision aware, so to speak. A classic example, perhaps not so common though, would be comment statistics. So comment statistics doesn't really know or refer to specific revisions. It doesn't always make sense, but in certain cases revisions are important. Revisions of URL, URL aliases, and so on and so forth. So it's not only entity types. Why do we need upgrading? We touched briefly on this already. You need to upgrade or a reason to upgrade is to allow your editorial users to be able to moderate certain pieces of content. Nodes, blocks, taxonomy terms, so on and so forth. You also want to upgrade to then use full site preview that we're working on as part of the initiative for all editorial type content. And also then to stage that content between different servers, between your stage server to your production server, or between the work spaces. That is one of the concepts that we're working on. And this is not only important for your starting off functionality on your site. If you have a site running with existing content, that content needs upgrading, needs migration to work with these functionalities. So we will be talking about that. So who is impacted by these changes? Maintainers of content-tripped modules, obviously. So if you're a maintainer of a contripped module that involves or interacts with content entity types, editorial content entity types to be more specific, then you are impacted by these changes that we're working on. Also, of course, if you are a site owner with previously mentioned types of entity types, and if you have existing content, you're also impacted by the changes that we are going through. So these are really our two target audiences that we're speaking to here, and that we want to help with this upgrade process. To give a few concrete examples of modules that needs upgrading in Contrib. Paragraphs module, I think most people have heard of this module. It's very popular to build your content model. In Drupal 8, with this module, it allows you to have a very flexible and a very good user experience around content editing. The paragraphs module introduces a content entity type. This content entity type needs to become revisionable and revision aware. So this is a module impacted by these changes. Another example would be entity queue module, also a popular module for managing, ordering, prioritizing content, or entity specifically. I also asked a question on Twitter about examples in Contrib. A couple of weeks ago, another example that came up is look module. It's also a editorial type content entity. It allows editors to modify the look and feel of pages. So this would be an example of a module that is also impacted by these changes. So just to give you a sense of the impact here. So how? How do we go about such an upgrade? We're specifically going to talk about content entity types. We're not going to touch on code and configuration that is impacted by this. We're going to focus on the very core of the issue here being the content entity types. So it boils down to three steps, really. The first step would be to change the entity type and base field definitions of your content entity type. The second step would be to implement an update hook that will take care of kind of the data migration. And the third step would be to implement something that is new in Drupal 8 and it's called the post update hook. The migration and the upgrade is quite complicated. So it needs to run in multiple stages. This is why we have two update systems. So the traditional update hook that's always been there since, I don't know how long time, but since a long time. And then the new system, which is called the post update hook. And we're going to dive into the specifics here, but I'm going to make sure to leave some time at the end for answering your questions if you have. I want to keep this as a conversation towards the end. All right, so we're going to look at a concrete example, kind of a demo use case. And we're going to walk through these three steps. Specifically, we're going to talk about converting the shortcut module in core. It's a very small module. It is used a fair amount across sites. It is, and this really applies to most scenarios. The change that we're going through here isn't that big when it comes to the amount of code that you need. The average patch for upgrading an entity type in this way would be around, roughly speaking, 13 kilobytes. And that's a small patch. That's a small change, shit. Another reason for why this is a good example is because when you install a standard Drupal 8 installation, an installation of this kind actually comes with entities all the way in there. So when you install Drupal, you don't have any nodes, you don't have any taxonomy terms, but the shortcut module actually has content from the beginning, so it kind of serves as a good use case for running this upgrade and testing it. And the process that we're going to go through here when we look at the examples and the steps involved, they are fully documented on Drupal.org, courtesy Andre here on the first row. So you do have good resources to refer back to when it comes to going through this upgrade. You with me so far? Makes sense? All right, that's good. So let's dive into the first of our three steps, which is changing the entity type and base field definitions. It's gonna be a little bit of code here, but I hope you'll stay with me. Is it big enough? Can people see on the screen? Yeah, good. So what we're looking at here is what's called, first of all, it's a patch file. So it's not the entirety of a file, it's just a patch to see what changes we're making in a specific file. We're looking here at the entity type definition of a content entity type, specifically the shortcut content entity type. So there's a couple of things that's going on here in this change. We're gonna start where we see the revision table. The revision table and the revision data table is the names of the database table names, essentially, that you have to give a name. If you've already defined a content entity type, if you're a contrib maintainer or a site owner that has built your custom entity type, you will have had to define the base table and data table. These two keys are the same concept, but for the revision data, all right? So we need to enter in those two lines there, give our tables some name. And then we need to add an entry to the entity keys. Entity keys, if you see that array of keys there, that's really kind of the primary keys that involve the ID, the UUID and kind of these most important base properties of an entity. So we need to add both the revision key and the published key. The revision key is kind of straightforward. This will be the key that annotates or that gives the information of what revision ID a certain entity is at. The published key is the key or is the field, the base field, that is responsible for the publishing status of your entity type. So when you moderate a content entity type, the publishing status is, for obvious reasons, very, very important. So those are the two entity keys that you need to add. And you just need to name them here. The process that will take care of really entering this into the database, that will be taken care of in the update hooks that will go through later. So you just have to name them here. You don't have to manually go into the database tables or add these columns or anything. That's what the update hooks are for. After the entity keys, we have a entirely new concept here. I think it's entirely new introduced by the upgrade path. Yeah. And this is a concept of revision metadata keys. So these are keys or base fields that are of importance to the revision API. So these are fields that don't exist in the base tables. They are important only for specific revisions. And the system needs to know and specifically call out the names of these fields so that we can handle these when we go through moderation process and so on. So you just have to name three fields. The revision user being the user who created the specific revision. That might be different to the original author of the original entity, right? Revision created is the creation timestamps specifically for the revision. And revision log message is kind of the change log or the log message that you might be familiar with that the node entity type has had for a long time. The fact that these three fields are here doesn't necessarily mean that they need to be put in the UI. They don't necessarily need to be presented to the end user. You can, let's say, decide that the revision log message should always be empty if it doesn't make sense for you to always enter in the log message. They are, however, part of the interface. They need to be in the database for some consistency. So those are kind of the main things going on in the definition itself. The revision table name, the entity keys, revision metadata keys, this new concept. And then the last important change here is at the very bottom. I hope that everyone can see here. But on the last line, we are extending a different type of base class. So it used to be the content entity base class that we're extending. But here, we're extending a new class that includes functionality that we will depend on. This base class is called editorial content entity base, which includes this functionality run moderation, revisions, publishing status, and so on and so forth. What's also important, this is not highlighted in the code here, but the shortcut interface, the interface of your entity type also needs to implement the publishing entity published interface, entity published interface as well. So that is indeed very important. You with me so far? Make sense? Okay, good. Along with this first step, we modify the entity type definition. We also need to modify some base field definitions. This is gonna depend on a per use case basis, really. Specifically for shortcut, the shortcut entity type, we will want to have the name of the shortcut and the location or the link of the shortcut. Those needs to be revisionable. As you kind of have to go through your use case, what fields on my content entity type make sense to have revisionable? Because not every field on your entity needs to be revisionable. Not every field on your entity type will change when your editorial users sort of go through and make changes to them. But in this particular use case, the name of the shortcut and the link location is what we want to make revisionable. So then, that's an easy addition that you need to make. You need to chain another method call to your definition, set revisionable parameter true. So that's fairly straightforward. And again, I want to emphasize that this is gonna be on a per use case basis. So you kind of need to evaluate this for your specific entity type. All right, so that's the first step finished. Next step is to entering code for your update hook. This is a fair amount of code, relatively speaking. The patch as a whole is still a small patch. But it's not gonna make sense to go through the details of the code here, because this is standard code that you need to include with very, very small modifications. And the reason why we haven't kind of included helper classes or helper functions for this is the fact that the code in update hooks are never, ever able, shouldn't be reused. It should be like a one-time thing. It shouldn't linger around as a available API in core. So it might seem redundant when you go in and read the documentation, but there are very good reasons why it's kind of a little bit of copy-pasting there. So the guy, this recommendation here is to essentially copy, paste. This is the only time when you're allowed to copy-paste code. You should never be able to do that, really. I give you permission. And you'll find this code in step two of the documentation on Drupal.org that we've put together here. So that's fairly straightforward. There's, it's annotated in the documentation. There's a few parameters at the very top that you need to change. The name of your entity type or the machine name. And I think the bundle, now it's only the entity type machine name, I think. So that's step two, pretty easy. Copy, paste, change a few lines in there follow the documentation step. Third and final step is to implement the post update hook. So I'll demonstrate the code here because this code is, again, it's actually copy and paste but if it's in on one screen, so I'll show it because it's easy enough. So the post update hooks, they are being put in a separate file. So at the very top of the screen, in the case of shortcut module, you'll see that we've added a shortcut.post underscore update.php. So they don't go into your standard.install file. You put them in a separate file. Code here is indeed very straightforward. We have a helper class here that's needed for many reasons. So it's not a service or anything, it's a helper class. They explicitly need to call. The parameters they need to put in or they need to inject here. There's six of them. So the first one is the machine name of your content entity type. In this case, shortcut. And then the entity type manager, entity definition, update manager and the schema repository, the entity storage schema handler. Or the key value for that. And then finally the database connection. So this is gonna be specific to the storage handler for your entity type. In this example, it's the SQL handler. This is specific to SQL. So if you for some reason have a custom storage for your entity type, then you will need to take a lot of inspiration from this code and implement something that makes sense for your storage. That would also apply to the update hook, I believe, to some extent. So the steps that we're going through here for SQL storage, which is the most common scenario for sure. So that's the, you're instantiating the class. And then at the bottom, we're making a method call where you need to pass in two things, the update sandbox to make the, they can sort of batch up the update, as well as the base fields that we modified previously. So I'll go back to that and just show, because it was the title and the link that we made revisionable. So if we go back to where we changed the base field definition, remember these were the two fields that we set to be revisionable. The name of the shortcut and the link. So if we go back to the post update hook, those are the two fields that we need to inject to the final method call at the very end. Are you still with me? Good. And this is really kind of, this is the crucial point, really, of the update process. This is where the magic happens, more or less. The update hook will take care of a lot of important things, but this is kind of the final execution that makes it all work in the end. Right, so this is, as Andrew just pointed out, this is actually what migrates the data. And Andrew, correct me if I'm wrong here, if I'm going out too deep, but the update hook will kind of set in place a temporary place for your content, because we don't want to risk, there's a data migration happening here, right? And we don't want to risk losing your content in any shape or form. So we're actually putting up a temporary storage that we kind of migrate the data to so that the data is safe and sound. And then we're sort of executing the migration here, ensuring that we never ever lose your data. So if there is an error during the upgrade path for some reason, your data is safe. That we spent a lot of time in sort of ensuring data integrity, because we're updating a lot of Drupal 8 sites here. There's a couple of 100,000s, if not more sites that we know that we're expecting to migrate with this code over some time. So data integrity and the safety of your data was kind of a very primary concern as we've been writing this code. Cool, so those are the three steps. That's the extent of the code changes that you need to do. But then you actually need to execute the update as well. This is straightforward. It's a Drush Update DB. It'll be a standard procedure. You probably go through these updates as a regular part of your development and maintenance workflow. So the way that this would work in production, you probably have some sort of automation in place, so perhaps you're doing it manually. It's folded into the normal update process. And that's really the migration. We have a few more things that we're gonna look at, but I wanna reiterate that we have a lot of documentation in place for this. So do take note of that. I will, or the slides will be made available after the presentation as well, so that you can refer back to that. So let's look at specifically kind of what this will look like in practice. So in the case of shortcut module, this is just an sort of example for illustration or demonstration purposes. So apply this to your scenario. But before, the shortcut module wasn't revisionable. So to the left, you have what it looked like before. You enter in your name and your path for your shortcut. Afterwards, you'll have the revision log message. And in this particular scenario, we've also configured a content moderation workflow for your shortcut. All right, I know a little bit of a silly example, all right. But the theory applies to any type of content entity. So you have the revision log message. As I mentioned, it's part of the interface. You don't have to show this in the user interface, but in this scenario it is there. And the publishing state and the moderation state is because we've enabled a workflow for this particular entity type. So this would be kind of the changes that you can expect when you make your content entities revisionable with the editorial base class. So let's talk about who's currently going through upgrade efforts. So to have a little bit of awareness of what's going on in the community. So as part of the initiative, we've taken on the responsibility to go through this process for most, if not all, content entity types in core. So those are being converted. As part of the 8.5 release in about six months, we're looking to have, obviously, nodes are already revisionable. So that's already covered. But we're looking to have block content, taxonomy terms, comments as well, not with a revision log message. That wouldn't make any sense, right? But there will be revisionable to allow kind of this movement of comments between environments for syncing purposes, et cetera. We're also looking to enable full support for media. They're already a very long way there. So media entity types will also be moderatable and revisionable. And it's a long shot, but we're also looking to convert menu links or custom menu links. Because that might be something that you want to change in one workspace or in one editorial environment that you then want to synchronize over and publish and moderate into another environment. So menu links is also on the roadmap. Tim, Andrea, did I forget any entity types, I think? And we don't stop there, right? There are more entity types in core that we do want to make sure to convert as well. For those of you who already utilize some of the functionality that is already being provided in Contrib, you might know about the Multiversion module. The Multiversion module kind of helps, enables revisions for more entity types. So if there is a Contrib module that doesn't, by default, have revisions on, Multiversion kind of jumps in there and takes over the show and makes it revisionable, automatically. So we're upgrading that to basically make the Multiversion redundant. We wanna upgrade ourselves out of business with the Multiversion module. The idea being that this should really be integrated into core directly and into the Contrib modules directly. So there shouldn't be a case for Multiversion module to exist in the future. So as maintainers of Multiversion module, we're working hard on that. Andrei number two on the front row here is working hard on that. So that's one of the efforts going on. There's also the team behind work bench moderation. There are providing upgrade paths here. This is not exactly the type of upgrade that we've gone through here, but it's a kind of semi-related upgrade effort in relation to this. A little bit of background info here. The content moderation module that made it into core was the idea of it was really born in the work bench moderation module in Contrib. So that's where the code and the ideas and the efforts came from. But before that happened, obviously, many users started using work bench moderation and they will now have to move over from using work bench moderation to using content moderation. So there's an upgrade path there. So that's being worked on as part of the VBM2-CM module. There's actually a couple of initiatives going on there, but this is kind of the main one. The prime example of this effort here would be the Acquia Lightning distribution. They started early with the moderation workflow using work bench moderation. So that's kind of where the upgrade effort is coming from. All right, just want to end here with some credits. So if you want to read more about sort of the background, if you want to understand how this, the conversation, how this upgrade path had evolved, you can go to this issue. It's a long issue. It was a year-long effort. Just kind of, we went through multiple rewrites to ensure, as I said, data integrity, making sure we don't lose any data for all of these hundreds and thousands of sites that are going to upgrade. So it was a long effort. I'm very impressed by Andrea's stamina in carrying through and over 150 comments. As I just mentioned, there were multiple rewrites and it wasn't by any mean only Andrea who did this. Team here on the front row as well as many other in the issue queue did a huge effort to provide this upgrade path. And that's it. Thank you very much. We have a fair amount of time left, so please feel free to step up to the mic. If you have any concerns or questions of how your sites are affected, anything like that, I am more than happy to answer your questions. But perhaps everything is super clear. No one is word at all about any changes. Yeah, there's a question. Please step up to the microphone. Yes, so we talk about revision. I think about the bigger side. We will have more and more revision. Is there an initiative in order to manage a life cycle in the revision to read off the very old revision in order to reduce the data size in the database? Very good question. And this is actually a very common question that is being brought up, so excellent question. It's good that we're addressing this. So there's a couple of things going on here. There's two main concerns when it comes to adding more revisions to your database. The first one perhaps is storage and the size of your database. That is indeed an issue. The size of your database will certainly be bigger. However, these days, storage is very, very cheap. So it doesn't cost you necessarily more. It will maybe take a little bit longer to synchronize your database back and forth. So although there isn't anything in place right now, there is discussions in the issue queues about revision purging. And there's a couple of different techniques that you can go about that. It is important to keep certain metadata of the revisions left in the database. You don't need to keep all of the content because in complicated scenarios, you do want to keep around some metadata so that when there are conflicts, if you move revisions between workspaces from stage to production, there might be conflicts. So we need some revision metadata, kind of the history tree of the metadata at least in place so that we can resolve conflicts. So there's different techniques there, nothing in place right now, but there's definite plans. The second concern about growing number of revisions is also performance. So will my site load slower? Will my end users experience a sluggish site? This issue is entirely mitigated with how the API and how entity caching works in Group 8. So revision queries with the way revisions are stored, there's very little to no impact almost on the way that you query kind of your default published revisions. So if you're in an editorial workflow, there might be a small, very small impact on it, but in the most common use case, there's no impact at all. Couple that with our excellent entity caching mechanism in Group 8, you will almost notice no difference at all. Vijay, question. Yeah, thanks for this great session, actually it's a classic dig session, like a very simple plane. Thank you. Could you talk a little bit about the translation? Two questions, first one is, how the revision and the translation kind of working together. And the second one is, you know that D7, no field revision or something, Alex came up with the module just to avoid field level revision creation. And we spoke more on the entity level, but is there anything that we can say, I need entity level, but for these fields, I really never needed a revision. I need to pull Andre in for detail domain expertise, knowledge here. So let me introduce Andre on the stage. So what was the first question? I know the second one, but... Translation, how translations are affected. Oh, they're not affected at all. I mean, the translation API, part of the entity API is working the same as before. There is really no impact on translation. So fields can be both revisionable and translatable at the same time. And on the second question, in Drupal 8, there was a big change compared to 7, in that entities are always stored as a, are not stored considered a whole as a single unit. So a certain field cannot have different types of tables or not. If the entity is revisionable, then the field, all the fields are revisionable. I mean, if you specified to be revisionable. If the, yeah, so a module like that wouldn't be possible in Drupal 8, basically. You were talking about field SQL, yeah, okay. I forgot the name as well. It was something that didn't create the revision tables if they weren't needed. But Drupal 8 does this automatically for you. If the entity type is not revisionable, then you don't have revision tables, basically. Yeah. Thank you very much, Andrei. Are there any other questions? Yeah, we have a question here, excellent. Clarification on the install process. Clarification on the install process. So once you've upgraded all your entities to be revisionable, you still have to then install the multi-version module, which does another upgrade. Because at the moment, if you were then to install workspaces, it's got dependency on multi-version. So this is, this kind of use case here is more when you don't use multi-version. This is upgrade for your custom entities for contrib modules. So if you use multi-version, let's talk afterwards. That's kind of a little bit of a tricky scenario. So this is kind of the more common scenario. So you could then just install workspaces from after doing this then, is that? Right, so the workspace patch that is going into core will not at all rely on multi-version. And as mentioned, we're working hard on providing the upgrade path for multi-version. That will be, that's a parallel effort to this here. Did I answer that right? I think so. Other questions? Yeah? Are there any content types where you wouldn't want to upgrade these to be revisionable? There's, there's, there are perhaps use cases for that. The kind of question that you need to ask yourself is whether or not it's editorial content and whether or not if you have kind of a moderation workflow or a staging scenario where they in any way relate to other revisionable content entities, then you will want to make them revisionable just so that they can move between workspaces so that they can move between environments. Do you have anything to add there, Andre? No. Yeah, so kind of revisionable, the way we're thinking about it is revisionable by default. That would kind of be the ultimate scenario because more often than not you probably will want to have your content entity types revisionable. And that's kind of the infrastructure if you may for all this functionality. And it doesn't necessarily mean that you need to create new revisions on every change as long as the entity type is revisionable. It's the API needs to be there. You need to have revision IDs that this functionality can kind of hook on to and tie on to. You don't necessarily need to create new revisions for every change as long as it's revisionable that that metadata is kind of hangs around if that makes sense. One way to look at it is if the entity type is somehow user facing either in the front end or in the back end of the website, then it probably needs to be revisionable. If it's just a narrow use case, like, I don't know. Yeah, users is a good example. Users shouldn't be revisionable. So, yeah. There are use cases for entities to not be revisionable. But if it appears in the UI somehow, it probably should be. Very good questions, other more. Yeah, yeah. Yes, so you mentioned the module material version and the way to patch the existing code to make the same, if I understood well. At the current time, what you advise is starting using the module or jump into the patches to foresee the next step? So, if you currently are using multi-version module, we would advise you to stay there right now and continue to rely on that. And if it's not urgent for you to upgrade other things, stay put because we are working on the upgrade path for you. So that would be the recommendation if you use multi-version. If you don't use multi-version, then I would recommend you to wait until we've included the workspace module into Core, which there will be an experimental version in 8.5, so in six months. And by then, we have also gone through the upgrade path for many of these entity types. So if you use multi-version, stay on that. Don't upgrade your entity types because we will be providing the upgrade path for you as part of multi-version. And again, if you do not use multi-version, then wait until 8.5, where we kind of provide a lot of this for you. Does that make sense? Very good question, actually. Sounds like that's it. Thank you, everyone.