 All right, catch us here, which means we can get started here on time So this is our final core conversation of the conference. I hope it's a good one We're gonna be talking about the Drupal 8 release cycle and how we can make it more future friendly So first off, let's talk about what our release cycle is now at least the story we like to tell ourselves So it works like this some version of Drupal comes out and it's wonderful and we all party and things are good and we open up development on the next version of Drupal and You know we try to make everything better and we can improve anything in the skies the limits And we can break API's we need to in order to improve things And this is what core developers sees that we can improve anything and this is what everyone else sees We can just change stuff and that's scary And we have this idea that we allow ourselves to break API's in order to innovate that We make Drupal better by changing things and that's true to an extent but There we eventually hit this API freeze phase Which is actually sometimes more of an API slush and Okay, maybe yogurt. I don't know Kind of sort of frozen it chills eventually And we go into the phase of just fixing bugs we're fixing bugs all along and adding features now we're just fixing bugs and Time passes while we do that and eventually we get down to zero of criticals and we release and everything is happy And we start all over again and this is how core development works Those of you who have actually been involved in core development, how realistic is this? The yogurt is spot-on Here's more what it actually is like speaking as someone who's now lived through three as a major contributor Here's what happens. We release a major version of Drupal and Everyone falls asleep for a while because we're exhausted and burned out and we and then the core team Just kind of ignores that stable because there's a new stable to start working on we've got the next version let's go work on that now and We just keep on doing this stuff we were doing before that now we can do again because we can break API's again Wonderful and we start designing new stuff and add new features and you know improve the UI do all this wonderful stuff and We don't necessarily know that it's right because the new old versions only been out for a day and a half and we're already designing new stuff Did does the old stuff actually work? Well, we could our guesses are sometimes pretty good, but there's still guesses We are not actually operating on data to determine if we actually need to change something And we have no idea how long it's going to be until the next release because Dries doesn't tell us And so we're all in a panic trying to figure out. Okay, can I do something big? Do I do something small if I do something small does it mean I won't be able to justify the big thing? If I do the big thing will I not get it in in time which means that nothing has gotten done I can't plan when I have no idea how long it's going to be so I'm just going to panic and then try and get stuff It's done as much as I can and we try to fix everything we possibly can In as spastic a method as possible because that's all we can do which means that we change everything and break it for everybody as usual and then eventually eventually Dries announces of a feature freeze date and It's usually way closer than we want it to be given wherever we are at that point and so we panic them again and Keep on panicking throw time into it work on weekends and evenings and stay up late at night And we finally hit freeze by which I mean slush by which I mean. I don't know whatever API freeze is such a loose term and At this point a lot of people wander off because We're not adding cool stuff. We're just fixing bugs fixing bugs isn't sexy So we're gonna leave or I've already been working for the past year and a half I'm exhausted so I'm gonna go take a nap and let someone else fix the bugs and this phase of Drupal 7 lasted way too long and I see some people here in the front row who are nodding in pain and And really only this the only the few stubborn gets remain in core at this point to Finally clean things up and get it released because everyone else is just kind of burned out and tired and Finally we stumble into a 1.0 a point zero release of the next version Barely awake so you know carrying each other over our arms because that's all the energy We have left and we get this release out and then we try and do it all over again This is actually how core development works for those of you who haven't been in it Yeah, this is all software projects used to work. So how did that change exactly agile hmm? So why are we panicking? This is a quote from Chris van der water Clips to you see who's the initiative lead for scotch the blocks and layout This is about six months ago that he said this. Yeah, I'm burned out I'm losing sleep, but I'm too stubborn to care about my own feelings because I know I have to live with us for the next three years Because we expect at this point our core releases are going to last a long time Programming is like sex make one mistake and support it for the rest of your life And when you don't know if it's a mistake or not for three years, it's kind of worrisome another problem, you know to be nice to get companies involved in core development and We talk about this at every conference. How do we get more companies involved in core development and supporting Drupal? You know large companies like long-release cycles You know large anything you know likes a long-release cycle large nonprofits large for profits governments anyone who has large investments They like this idea of a long stable release cycle But it also means that why am I going to bother contributing when if I put an engineer on something to work on core I'm not actually going to be able to use it for three years I'm just talking about core say nothing of contrib catching up and So a lot of companies don't Because you know it wouldn't be of any value to them that the turnaround time is too long Sometimes these API breaks we do are necessary to evolve that is completely true if you don't ever break APIs You turned into Windows Millennium Edition And I see some people nodding in pain on that one, too There's actually code in Windows Millennium that special cased SimCity 2000 the game and retained a bug from Windows 95 to support just that one game because they couldn't break that We don't want to do that Believe me. We don't want to do that We we go to the other extreme. However, if the only way you can innovate is breaking APIs I've got news for you. You don't actually have an API You have a pile of code a pile of code does not qualify as an API API's let you evolve and innovate without breaking things by sneezing Drupal 8 breaks a lot of things it was a necessary shift We are catching up with eight years of development in the PHP world with Drupal 8 We are fast-forwarding through eight years of evolution in two years That's a big break It's necessary that should not be the normal case Drupal 8 should be an exceptional release both is in exceptional really cool and exceptional is in not normal I'm seeing a lot of people nodding with that one, especially people who have worked on core So what do we actually need here? You need want the picture? Okay? What we actually need to be doing is We need more stability. We need ability to rely on something both for ourselves and for our clients That is not going to change all the time without having less development. We don't need less You know fewer changes because you have fewer people working. That's not what we need at all We need to be able to develop with better stability as we go forward We need to be able to iterate rapidly. We need to be able to improve in smaller chunks in bits and pieces We need to have a shorter ROI we need to be able to say I'm going to put time into this task into this improvement And I'm going to be able to use it in four months in six months not in three years Because if I can't use it for three years why am I going to bother unless I'm a completely crazy fanatic? Who is going to do this just for fun because these are sick and twisted human being? I? Admit it we need to be able to improve with out breaking things This is something that core has been very bad at contrived actually has been better at than core in many ways So how do we get there? How do we have a system that lets us do that? You need to have loosely coupled components Loosely coupled components let you evolve individual pieces without breaking each other We need to have a clear separation of concerns We need to be able to have this piece of code deals with the database this piece of code deals with display this piece of code deals with HTTP this piece of code deals with validation and They deal with just those one that one piece just the their piece which means changes to the other pieces don't affect them That's how you get stability. We need clear boundaries between our API's when you have a Big gigantic blob of arrays is not an API because you don't know what side of it you're on you don't have that Separation concerns. You don't have that clear boundary behind which you can make changes We need swappable components if you can take a component rip it out and replace it with something else that follows the same interface That means you can improve that that system the thing you rip it out with and replace it with can be just the same thing only iteratively better and As long as you don't change that interface nothing else breaks your code is still stable your API is still stable That's a feature for backward compatibility for forward compatibility for supporting alternate systems For testability. This is just good code in general Basically, we need clear interface interfaces and real honest-to-god API's That's how we get to the point of being able to improve without breaking things by improving Guess what we've been doing for the last two years in Drupal 8 We've been refactoring the system heavily For exactly this reason all of those things are have been the guiding principles for Drupal 8 architecture All of those things we need that are prerequisites for being able to evolve our system Without massive API breaks are exactly why we've been breaking so many APIs in Drupal 8 to get to that point Things like dependency injection get us this ability Interface driven development it gets us this ability Putting you know separating out code into the Drupal component library Gets us this ability because this forces us to have loosely coupled systems that are not dependent on the application layer Drupal 8 makes this possible for the first time in the history of Drupal So let's do it How exactly would that work what how would we be able to evolve our code now? For example, here's a small example. This is the class book manager It's in the book module Mark Sonobon has argued that it's a terribly named class and which should be called a book repository Maybe he's right. Maybe he's wrong. I'm not going to debate that. Let's assume for the moment He's right. How do we make that fix without breaking APIs? You simply extend the class and then anything that type hints on the old class. It's still going to work Core uses the new one it gets the new capabilities of the new object anything using the old one It's not going to use the new capabilities anyway. That's fine It can still, you know, use the old code Even call it by the old name. That's fine. Use the old tools and core can do the new cool stuff and Contrib that comes out after we make that change or updates after we make that change can use it to No API is unbroken. We've added functionality There's some cases where it's not quite that simple you need to actually, you know branch the code all right We have, you know, our old interface for something and we have a new interface and then all right Do whatever the default logic was before but if you're using the new stuff very based on that great Modules can use the old code and just keep on trucking with that or you can use the new code and get the new benefits out of it Whatever makes sense for them if they just wrote it once and are bored and leave it there for two years It's fine. It doesn't break Does this introduce a little bit of cruft a little bit? That's okay When we do it gets to the next point where we can change API's again. This is really easy to clean up It's really easy to say. All right. It's now called book manager It's not called book repository. We're gonna fold all those interfaces the new methods up into the base class again I'm just gonna merge it code. That's already been updated to that within Drupal 8. It's not an API change for that Only for code that hasn't been updated, but you have the opportunity anywhere along the way To keep up with the clear direction things are moving Anything that's a service can be swapped out That's why we have this dependency injection container in core because not just can you easily swap out your database based cash from memcash or easily swap out database-based router for Something uses MongoDB You Can also swap out that database based router for a better database based router that uses a different Algorithm for finding routes you could swap out five different classes that are all composed behind the router interface itself and Completely rearchitect that system as long as the parts that we say are the public part of the routing API doesn't change We can completely refactor the stuff behind there We did a little bit of this in Drupal 7 during development in the database layer where we changed internal data structures Without changing API's I couldn't tell you which direction we changed it because it didn't matter the API never changed This is the next same idea the next level up We can just add functionality just straight out add stuff for example We don't have a UI for the rest module in Drupal 8. We decided it'd be too complicated Let's just get the API working you can edit the files directly It's ugly, but in the clunky, but it works. There's a rest UI module in the product in progress in contrived Once it's stabilized throw it into core. I'm cool with that There's no reason not to it doesn't break any API's but people using core get a clear benefit out of it core gets better Without having to break anything Right now our support for hyperlinks on rest resources is okay. It could get better We can improve that without breaking anything because nothing's actually using those yet We're just exposing new capabilities Right now we primarily support Jason and XML is kind of a second-class citizen beef that up fully support at the XML variant of Hal Our link documentation for defining the links we have Is currently Mediocre at best do we have any actually okay? We don't have any built-in automated link documentation Build in the systems to do that that doesn't break anything just adds more functionality and improves the support for RESTful web services. This is just within the rest module Here's four or five things we can do just in the rest module to make it better without breaking anything Who is here for Eaton's session yesterday on snowman? They're talking about you know put in one Install profile with for core that's fully baked, but there's really these three general concepts fine ship one with Drupal 8 Add some others later. Those don't break anything, but they add better experience for new users Who can say all right? I'm not an organization trying to express my mission I'm just a personal person with my you know profile or whatever So I'm gonna turn on that instead we can add that easily without breaking anything There's changes to the admin this there's an issue I couldn't actually find the issue for it the taxonomy admin page the pager on it when you have and nested taxonomy terms and a very long list the pager on it makes it really hard to use and There's been an open issue for a long time to figure out what to do with that that kind of a you know of interface change We can totally do in ways that don't break apis. It just makes the UI better We can totally make our UI better without breaking apis really We can continue to refine refine the style guide for seven There's been a lot of just tweaks to the style guide improvements tighten things up Standardize better. We can just continue that work because that doesn't change an API That just means that all of our fonts look better and the user experience is more pleasant That totally is doable We can improve our caching There's an open issue right now to add caching to the state system if that didn't happen by the time 8.0 shipped It doesn't break anything to add it later. Cool. Fine do so There's been talked this week about our fooby or Sam here of of course they aren't it's been talked this week of having you know a new way to register Controllers for the routing system using a dedicated class of some kind maybe annotations, you know I don't want to deal with that right now But if we find later on that is a good You know a good improvement we can add that as another option for ways to register routes and it doesn't break any of our current ones There's talk at one point of bridging from hooks to events so that you could use events to listen to a hook call Or use hooks to listen to an event or some kind of stepping stone to get people moved from hooks over to events Probably not happening for 8.0 adding that doesn't break anything So we could totally add that as a guide for people a way to let people step up and get used to new these new tools And this is just what I came up with in about five minutes of thinking about it I'm sure if we you know we could spend an hour here coming up with another You know dozen things that we could do that don't actually break API's but still make Drupal core better That are not simply throwing more modules into core There's way more things we could do to improve Drupal than just throw modules into core in order to do this We need to understand what an API is until recently. I'm not sure who changed it on me before I got a screenshot of it Until recently if you asked Drupal con in IRC what an API was it responded with this unfortunately This was actually how we thought about things until recently it is fundamentally wrong and this is why we can't have nice things We need to get away from this idea of an API in order to be able to change things about breaking API's because as long as this is How we define API we can't do jack Now I'm sure some of you are gonna ask, you know, didn't we do that for seven? We did add features in seven kind of We back ported smaller things. There were no real big changes in seven. We added some hooks here and there made some things Yeah, post release it's to seven after seven point zero, but none of them are really large Who actually had saw a big change in the seven X series? I didn't I mean really did anyone notice maybe not okay one person here module developers who needed one specific hook Probably noticed that's a very very small use case Because the attention for most developers was always on Drupal 8 because we have no idea how long it's gonna be so panic and get stuff done So that's where all the attention was seven was abandoned as soon as it was released Okay, not for support. We've got you know plenty of bug fix support and so forth But in terms of actual feature development, it was abandoned the day it was released and that's been our de facto policy for several releases and Even then Drupal 8 a year later after seven was released wasn't that different Okay, we moved everything under the slash core directory if we hadn't done that Drupal 8 as of you know early 2012 was what? 95% API compatible with Drupal 7 it wasn't that big. What's that more than that? Yeah, it still took a long time for us to really get around to breaking things We could have done that Straight in seven. Yeah, we were too tired to really break things. We needed to take Yeah, we're no strength left to break things take a nap first So here's what I'm suggesting here's the proposal Once Drupal 8 is released do not open developments on Drupal 9 for at least a year I don't even need the rest of my bullet points then great We can then go ahead add significant non API breaking features to Drupal 8 He's gonna be my easiest sales pitch ever Force ourselves to actually think about backward compatibility We have trained ourselves to think that innovation only happens by breaking APIs We need to unlearn that force ourselves to unlearn that When we come up with things you want to do that are gonna have to break backward compatibility Keep track of them. Don't just forget about it. Have a branch somewhere have an issue somewhere You know, don't forget about these things and once we decide, okay We're finally ready to open Drupal 9 developments, which is at least a year away Focus just on those Don't have a big free-for-all do anything just have a here's the API changes. We need we know we need to do Here's the large refactoring. We know we need to do there's been talk of You know replacing form API with a symphony form system totally not happening in Drupal 8 any version Drupal 9, let's talk. There's talk of you know removing hooks entirely and just using events totally not happening in Drupal 8 Drupal 9, let's talk You know if we need to you know, really break the I know the caching system in some way, you know We want to add if functionality completely changes how cache tags work and is incompatible with the old way Save that for Drupal 9 do some skunkworks on it But that doesn't actually get attention doesn't get committed until we get to 9 Which means at least a year away making sure we can do things on 8 and refining 8 making 8 better This would be hard We are very used to the idea of I can improve things by breaking it We're used to taking things apart and putting them back together again and having pieces left over because that's what we do It will be hard to adjust. I'm not gonna sugarcoat that it'll be hard for the core team to get used to this model But this is how we can have our cake and eat it to or at least part of our cake This is how we can move forward as a project So let's assume for a moment. I'm gonna be an optimist that 8.0 comes out in January Told you I was a stand-up comedian So in February we release 801 Yes, some semantic versioning because that's what semantic versioning means is exactly this model This is the versioning system used by pretty much every project in the universe except Drupal and web browsers And they don't break things between even their large numbers March 802 fine April okay 8.1 with whatever new features we've come up with in the meantime What's that? The upgrade path so at 8.1. That's where we get our Migrate path from Drupal 6 In May 8.1 1 keep on going with this pattern bug fixes or security patches You know if we find security holes we fix them the same way we do now Keep on going just work our way through for at least a year in this pattern now here I have every three months for a feature release Four months is good to it's not a huge deal to me what the number is three or four months though means it's Within the time period of most mid-to-large client projects That means we could get clients funding core work that would actually be available in a stable core release by the time the project launches Oh my god, that's never happened before Microphone microphone. Are you assuming as soon as you come out with 801? That's not good either 801 that people are going to immediately upgrade from 802 My assumption would be that we might have some people on 801 and some people on 80 which means we're now going to be maintaining security patches and critical bug fixes to 8081 and That's two slides later. I'll be there in a moment So there are potential problems here and I tried to come up with what's where the likely pushbacks So first of all You know, how do we innovate when we can't change things the same way that everyone else does we learn how to Advance it without breaking things. You can do it many projects do we can too? You know, do we know that all these new APIs we've built are actually up to this task to be honest I can't guarantee that they are I can say they are going to be more up to the task than triple seven was and Let's find out we will probably find some of our APIs are really easy to extend without breaking things and others we screwed up And are not as extensible as we thought and we can then know that with data rather than guess work And we know what we then need to change in Drupal 9 You know, this is push Drupal 9 even further away all these changes we want to do for 9 pushes it further away Does it or does it just mean we've got two years on Drupal 8 and then one year total for Drupal 9 and We've still got a three-year release cycle, but it's a much smoother path rather than this big her key jerky break and freeze nonsense Or if you mean Drupal 9 is four years away instead of three years away, and we've still got all these new features in Drupal 8 I don't think people are going to complain that much if we can still get our new toys Drupal.org probably can't handle this at this point It's infrastructure Drupal Association is responsible for that. I'm sorry This is the easiest one to solve DA put some money into making the system our packaging system supports semantic versioning. That's your job do it That's not an actual objection Translations in the past we've had an idea that you know we cannot touch strings Because that would break all of our translations and then people with a non-English site suddenly have English texts showing up That used to be true Localize that Drupal.org is now offers translations straight over the web We're downloading translations as part of the installer dynamically I'm pretty sure we can figure out a way to release new translations for core at the same time All right Drupal 7 you can already do this although it's quirky So we make it less quirky and we're good to go We just say all right string freeze is a month before whatever the next stable feature release is going to be so we can update our translations great But I like the playground I like the sandbox You know that that's a fair criticism, but you know what? This is a multi-million dollar industry We're not a sandbox project anymore. We're supporting the websites of Fortune 500 companies world governments banks major social movements It's not our playground anymore Contribute can be a playground core is not a free-for-all playground. We can't afford to be This is growing up Security releases as you mentioned. This is one concern. You know, what do we do with support for? 8.0 8.1 after 8.2 and 8.3 come out So, you know, do we provide security patches for older? older stable releases You know and do we have an idea of a long-term support release? Do we have to pick one of these and say all right? We're going to keep supporting bug fixes on this one and security on this one, but not some of the others There's an open issue where we've been discussing this for like three years and hasn't come to a conclusion yet You know, do we expect people to upgrade? You know when it's 8.2 comes out We expect all those people on 812 to upgrade immediately What do we support them? They're probably not going to Upgrade immediately and the security team is really really worried that they're already overworked and we're now telling them They've got to support five versions of Drupal We know what do we do with that? This really is the only difficult problem and frankly, I don't think it's as difficult as we make it out to be Since there's so few changes a breaking changes between releases Backporting a security lease from 8.2 to 8.1 is a get cherry pick. It's not hard You want to you know if we decide we do need more people on the security team, which the security team insists we do all right Security team is not directing Drupal development. Hey DA get some security sponsors get companies that will just sponsor the security team You know to pay for a few hours a week of a dozen people's time to keep track of this stuff There are options here, you know either one of these is a valid idea. There's probably more You know, we can talk about this one, but this is a solvable problem This is not going to block this if we decide to go forward What we get out of this I should talk faster We get faster We get faster improvements we get the ability to iterate faster and improve Drupal faster than we do now We get lower stress because we know when our next release is we know that if I don't get this new feature in this Improvement to the plug-in system in by 8.2 8.3 is three months away. I can deal I can live it's okay It's easier for companies to get involved You know because they can put people on and know they're going to get a return on in three four six months rather than three four six years And I'm not just talking about big companies. This isn't just IBM Cap Gemini You know the White House. What's that? Who are you with you know publishing companies? NBC it's not just them. It's also the small shops if you can have a developer work You know five hours a week for three weeks as part of a client project to improve something for core and By the time the project launches core has that feature in a stable release You've now had your client fund core development during your normal business hours. How awesome is that? And it's a learning experience, you know Improving without breaking API's as a different skill set this makes us better developers by forcing ourselves to do this by giving Ourselves a constraint we can learn to become more creative and more Capable developers and frankly it's good PR Drupal does get knocked for the fact that we break API's all the time If we can say you know what you're right. We're fixing that That's good. That makes customers more interested that makes other developers more interested that raises our profile That's good for the project and when we get to developing Drupal 9 we actually know what's needed because we have data to say This doesn't work. This doesn't work rather than I don't like this I think this is gonna not work and so forth and I have to start now because I don't know when I'm gonna be able to fix it otherwise What do we do to get ready for this? Decide what qualifies as an API is An API is not a function just because something is a function does not make it an API That's state statement one. So is it just interfaces? It's that the only thing we qualify as an API is The structure of a FAPI array an API. I'm gonna go ahead and say no a specific admin forms FAPI structure is not part of its API Because if we say that then we can never touch that form and we lose the entire point of being able to iterate You know don't change the structure of FAPI itself. Don't change what the elements are But a specific form if we have to change it in order to fix a usability problem do it Clearly communicate what qualifies as an API In a I don't think we're doing this anywhere But some projects use an API tag in their documentation, which means this is stable. We guarantee this thing will not break on you Let's start marking those things that we can say that about which may not be the entire system in 8-1 We can say all right. We're sure about this part now and expand the list of things that are so marked Symphony does this Yeah, no JS has a mechanism for doing this symphony does this this is not something that would be a Drupalism This is something other projects do we're already marking things as deprecated if we know that book manager is going away and being replaced by book Repository mark book manager is deprecated that signals to developers. Hey, by the way, don't use this use this other thing Instead, you'll be happier later. We're already doing this in core. We can keep doing it Just make sure that all the stuff we've been doing for clean interfaces and swap ability actually works In many cases that's try it and see what breaks and if it breaks, let's fix it Same thing we've been talking about with contrib modules at this point And then make of course make sure the Drupal that or it can handle this kind of packaging solvable problems Dries likes to talk about Drupal growing up and You know the stages of life and you know, we're any young adult getting our first job This is what adulthood looks like as a project and that's okay discuss microphones up here and These are some other issues this December issue. I'm going to put these slides online This is the magic versioning issue where we've been arguing this forever Web chick recently suggested actually using private variables and private methods, which currently we don't touch. I don't like privates If we can actually I'm talking about the military I'm not a fan of private variables But you know if doing that lets us more clearly to find the API I'm open to discussing it if we can actually you know use that as a tool to help communicate this better thoughts so when you open up Development for for nine. What happens this are we still doing these minor releases with eight all through that time? And do we have enough manpower person power? Sorry to make that happen? Maybe or maybe we say you know by the time we get to eight point five All right, stop on that just do security releases on that while we work on nine. I'm flexible I think there's a good argument both ways we could probably depends on what we have lined up for D nine We can decide it then Hi, I'm sorry. I was late to the beginning. So I hope I didn't miss this but um, I'm kind of wondering when you're saying like eight point three with the feature release Text next to it. Are you talking about having eight evolve or preparing to evolve nine for a release? This is eight evolving and having new features having new functionality improving the API's Changing them in ways that are backward compatible and so forth within the eight cycle before there before a Drupal nine even exists okay, so you're basically saying to have a Release process for eight that is fixed, but a feature process for eight that is flexible Is that am I understanding that right? I'm not sure I understand so feature development would happen on a schedule in eight and When we get to nine we can probably follow the same process more or less there So which means Drupal nine itself nine point zero it becomes a less earth-shattering release Then Drupal eight was and a lot of the changes in it People have already been queued up on so they already know what's coming and can prepare for them Cool. I like that. Also. I would love to see this applied to Some of the stuff that we've already done like can you give us maybe later after the con? An example of one small Project that was within the scope of eight that we've been working on that maybe made us run late that could have been Taken to this type of example and shoved out into a feature. That's a planned release during the eight process That was Exactly So here here's an example Here's an example. It's near and dear to me. I'd love to get for a database layer to not be dependent on Drupal Right now. It's in the the core the Drupal core namespace and there's a few calls in it to Drupal functionality like you know an altar hook If within Drupal eight we replace that altar hook with a Rapper that's part of the database layer that can be anything else to extend and alter that That that query Push the whole thing over to Drupal component Have stub classes in place that are just you know bare extensions of the previous classes. They're empty classes and One implementation of that little bridge that does the hook We have now refactored the code to be Drupal independent and can use the database layer outside of Drupal and nothing in Drupal notices That's something we could do The plug-in system was been designed in two parts the same way for exactly that reason I think I heard someone mentioned profile module. There's probably stuff we could do to make profile module better I'm not sure exactly what they said Okay Adding Here's why I forgot Adding fields to files file entity in core in the medias Session yesterday. It sounded like pretty much everyone agreed files need fields and that never quite happened in core We can still do that because it doesn't break anything and so forth. Those are the kind of things. I think we can pull off In your examples about the versioning you may you give examples with a fixed timing per month Do you think that this means we should go to tower the scheduled? Releases really here meaning every quarter quarter. There will be a new feature version Or is it will it would still be driven by what really happens? I? Think we should go ahead and just schedule them. We already have monthly releases for core If nothing got in all right, maybe we skip it that month Yeah, but I'd be surprised if there's an absolutely no features that come out in a new release I mean we like to write code. Let's be honest Do you think that this process should have been used for the entire Drupal 8 life cycle because it's a complete mind Well, not complete my shift, but a significant mind shift. Do you think it's too late to change to this mind shift now? And you should have had it at the start In other words that this this release for this method Would be much more appropriate to be nine like a rather than the eight For the development of for the shift from seven to eight this would not have worked Because eight needed to change so many things to catch up with eight years of development Within the eight cycle once we actually launch a point zero. That's exactly what a good time to start doing this now That we actually can We could have done the wonder because I Thinking about a lot got to face some of the places that was presentation and the thing that drove me to it Was I didn't like the Drupal 8 open when it did and I would have liked for it to open nine months later Because in practice hardly anything got in but it made it twice as much work to fix the Drupal 7 stuff We could have done that that's something we could have done that we didn't do I actually put on Twitter. I actually said that and bit like a month before Drupal 8 opened up So I was not happy So that one thing That we would do now like we could have done that all the rest of it. No, like no, it was not possible Yeah, we didn't use get choking and and you didn't have to commit access to people eight they would like a little The process issues but that they did make it harder in practice. It was hard. It should not have been harder It was You asked about what is an API? Anything that's documented on Drupal code or in the code a good content module. It is can use that So if you break that you're potentially breaking Contra modules in a features upgrade So I would say You define as much as possible as unbreakable and then if you find that's causing problems in development Then we look at certain edge cases, but if you start strict It's a lot easier to loosen up that policy than the other way round and you could get You know when 8.1 comes along and you've told everyone it's gonna be an easy upgrade And that's something they find that half the contra modules broken that could lead to a lot of bad PR So are you suggesting we start off with a big list of things that are a qualifies an API or a small list a big list And then if you're finding that you really need to break something then you think carefully and possibly say yes This this particular case we can break Drupal 8 release maintainer gets his own mic, so if if you say something's an API and 8.0.0 and in 8.3.0 you say oh, actually no, no, no, it's not an API I Say That's fine. If you've got documentation that says this is an API contra modules can use it So if we can define that up front, that's great But I've got the impression that's gonna be a difficult task to do But so when the other wouldn't think you would say like if it's if it's marked API We only mark the things API that we don't think you're gonna change if you use anything else There's gonna be my break So and if we break it then sorry Maybe we even hold it back if it's if it's not been released yet or something That is I that is the place where I think we've been doing it wrong I think that's a stability thing. Yeah, you mark. This is stable and this is like Or and if it's not marked it we don't guarantee it's stable And then we add things that stable like but there's gonna be a lot of work to define What isn't isn't stable? I'm gonna sit down So what's I'll give you an example what symphony did with this and they did follow this Only some methods and some classes are tagged API and those do not break Except for security reasons and if they can break can fix a security issue without doing so they do Anything that's a public method. That's not tagged API. They try not to but it's not a promise But each release 2.0 2.1 2.2 2.3 the number of things tagged API has increased So like the forms system in symphony did have API breaks from 2.0 to 2.1 There were changes in the routing system from 2 1 to 2 2 Some of them security related some not Eventually they got marked tagged API and now those do not break. I think that's a better model so we can have things that are part of the Public API for the recording. I'm doing air quotes here That are not tagged with API which means We're gonna try not to break this but we're not gonna promise it stuff with an API tag We're gonna promise not to break unless we have to for security I think that's a good way to do it And then we can just grow that list of things that are tagged API as we Stabilize that we settled in I think that works well for the maintainer of the content module I think you need to be careful about someone that their site build a role that they need to know whether their content Module is going to support the newer feature release of Drupal. Yeah, and we do need to communicate when we do break those others We need to communicate it well We can build tools for that Wait, what do you think we are software developers? Larry thanks for this presentation I think the topic has been looming above the project a bit and I think you're providing a clear path forward towards Figuring this out and have this right of passage into adulthood. So thanks. It will enable the right discussions What I'm thinking about is what will we put in place to not have time boxed panic? So I'll have time box panic for months of undirected scattered efforts that will still have 10 10 yeah for the for the point one release have I mean we could I mean initiatives for Drupal 8 has been those clusters that might become sequences For point releases, how do we make use of the effectiveness we can gain from having time boxed releases? By not scattering but maybe focusing so to some extent as long as we agree that we're not going to break an API in the process We don't have to make it tightly controlled. We can still have you know crowd sourced Hey, this is a cool feature and hey, it doesn't break API. So hey, let's do it We may want to have Like for example, we may not want to have a micro initiative for hook event Maybe that's just one patch. Maybe it's you know gonna be 12 patches. We have you know a very small initiative for it I'm not sure yet. It'll probably vary by the the task But you know one of the advantages is if we don't get hook if hook event in by 8.1 8.2 is not that far away So, you know to some extent, I don't think we need to drastically change That process just let it happen organically once we have that additional control in place of don't break an API and see what happens And adjust as needed There's definite benefit, but there's definite risk and I think we're gonna have to have a mindset change to Site maintainers, you know a new point release comes out and I almost upgraded without testing. I mean I trust it, you know I mean it goes through a lot Smaller rigorous testing process than everything else because usually the security upgrades are pretty darn good You know, this is going to force You know Contrib module maintainers are is my contrived module now an 8.0 or 8.1 or 8.2 and a security update comes out for 8.2 That forces me to upgrade, you know my contrived to 8.2, but now I have to upgrade It does complicate things and it's going to force a change shift and and an exponential version control Difficulties for everybody. I I think there's a lot of benefit to it But we're gonna have to we're gonna break some major site doing this, you know, you know NBC is gonna upgrade accidentally. I pulled them out of the air. I don't You know some offense to ABC some major company is going to accidentally upgrade and it's gonna break their site We might get a bad eye on it. So there's gonna have to be a training here in doing it. Sure True, but this makes core no less stable than the tier one contrives which have been doing this for a long time They've had feature development that doesn't break APIs and people upgrade and occasionally it goes wrong You know, it's no worse than views Okay, probably better than views Yet there's been the expectation that we know that happens so we do more testing for those contrived modules when we upgrade Yeah, so we definitely communicate this change. Yeah The issue of API tagging which you mentioned repeatedly seems to be implicitly about PHP here How about possible raised API's we could have or evolve during the later dot-x layer lifecycle Or maybe even JavaScript and possibly markup so that the streamers have things which are stable during evolution That's a good question. I don't know For rest, I think we can definitely add stuff very easily. There's still plenty to add But yeah, like There's still an open issue to move the path that we use for entities to the correct entity path That would be an API change that we can't do after 8.0, but it's already been approved to happen before that Adding more links to it. We can do JavaScript We probably want to do something similar in terms of marking what is and isn't stable there And just to treat it the same way. I'm not a JavaScript expert. So I can't say what the conventions are there Markup, that's a very good question. I don't have an answer. I I don't know if markup should count as an API change or not That's a Morton question Someone in the back of the room is saying yes, it should So in the same way that you can't pull the rug from underneath the feet of a PHP developer No more can you pull the rug from underneath a front end developer? So in other words, you have to the same problem you have in defining an API for PHP You have to define an API for markup CSS and JavaScript to find what is not going to break and what might break No different harder job, but same job. So I'll ask you the question then I as I said earlier We can probably say the structure of a particular form in FAPI is not part of the API If we have to change it in order to improve the UI there, we will Are there parts of markup? We could say that about or not. I don't know Okay, that's a good question for us to keep the pondering here The point is you just need in the same way that you need to know what's going to break That's the key part. Yeah So first of all, I don't want to say hallelujah So so I've been involved in Drupal for like kind of three years where You know, we're a small enterprise company, but like 30 websites. We built our own distro on Drupal 7 So we've got a little bit experience. I'll be on a smaller scale of this problem So we have 30 websites sharing a distro and we learned after about six months It's a lot easier to break stuff little bits regularly than it is to break stuff every six months and it feels like that's gonna be true of Drupal 8 9 And plus like as I mentioned in the other session, you'll get less developer fear as well Because if you're not changing a whole bunch of stuff, it's easier to get your head around lots of small changes Whether it's for a year or whatever So but I think to answer the question about you know one thing's breaking if things break, but they're regular and but small It's a lot easier to keep up And obviously everyone's gonna have BDD tests, right? Yeah, I mean if we break something and we have to break something for some reason It should take contrib modules no more than an hour or two to account for it if we can get to that point then any Accidental API breaks or security-based breaks are probably something we can swallow Still don't want them, but if we can keep it to that level of scale will be okay For security So that that's catch saying yes We do still occasionally break APIs in core for security reasons we try not to but it does sometimes happen and Yeah, same thing still applies Yeah, just working to add about the preceding question about So the standardization of markup we still have some something going on like Componentization based on the smacks Guidelines that there's already a start for this. So it's not completely out of Do you see so contribute right now releases are different there's a Point one branch and a point two branch and stuff and of course You're not gonna have the same rush of people wanting to get features and crazy and in all these Contrib modules. Do you see the contrived modules using the same kind of? Version naming scheme or a different approach? Why not? We probably still need to mark them somehow for what contribute version they're compatible with on the expectation that D9 will have Non-trivial API breaks and that's okay, but yeah, I see no if core is going to use You know three-point semantic versioning I see no reason. We can't expose that to modules as well thoughts on the initiatives You should be very important You haven't my opinions How do you think that going forward in the group line? Are you thinking maybe it should be? Do you feel like it's been a success in Drupal a to divide the work like this? Continued at the Drupal line. I Think initiatives were absolutely necessary for the scale of what we needed to do in Drupal 8 We could have organized them better a lot of the initiative leads have been talking about how we could have been organized and better most of Which comes down to more people clear resourcing better planning and so forth For Drupal 9 we may or may not find that we need something at that scale I don't know yet something the scale of replace form API with symphonies forms We probably need an initiative on that with a core team of four or five people and Schedules and all that kind of stuff like we had for these initiatives for Hey, we've got this list of five API changes. We want to make to the plug-in system to make it better Give Chris Vanderwater two weeks to go do it and say it's okay, and we're done We don't need an initiative for that. What is going to need an initiative for Drupal 9? We'll find out then My hope is that fewer things need it that we can do smaller bite-size changes Now that we have this framework that will support it now that we've done the work to get there we can get the benefit from it When I was already started whenever someone asked a feature for Duba 7 then someone said yeah I have to develop for Duba 8 first and I said, okay I cannot install Duba 8 on a machine. It doesn't work. I give up so When we have 2.9 started will we also cherry pick things up like develop on Duba 8 first and then cherry pick it up to I don't know. That's a good question So if we if we do this and we have say However, however often the lease is out say three four six months at a time So you get to 18 months and you have between four five leases on Drupal 8 and then you open up Drupal 9 But what you say is the technical debt that we accrued over the past 18 months of backwards completely lay as they get ripped and then Things that we could not commit to Drupal 8 like a complete form API rewrite that Is probably already being worked on anyway Against Drupal like against the current state of Drupal 8 but when you open it then you get start trying to get quite quickly to change it around You should have a much shorter Drupal 9 release process And you're not gonna be making like all of the other changes that you were actually making to Drupal 8 Drupal 8 might get like I think the I've lost what I was gonna say now that it's not it's Drupal 9 isn't gonna be like Drupal 7 or Drupal 8 was it's gonna be like Drupal 8 was from Like whenever it releases It's yes, exactly. It seems to back. What yeah, so Someone who wants to develop for Drupal 8 actually has to install Drupal 9 and get it running Yeah, I mean I If we can get to the point where Drupal 9 only takes us nine months because all we're doing are the saved up API changes It doesn't matter if we backport stuff because it's only nine months away One idea I have is maybe that for some phase in Drupal 9 like the beginning phase It's just a number of feature branches or of Experimental branches and they get rebased from time to time maybe and then get makes this way easier And some point we switch and say now Drupal 9 is the official thing But before that we can still rebase and develop on Drupal 8 I would say at the moment forward porting is impossible even for security patches It's really hard if we need to see how it goes And we also need to see how we're gonna deal with like inter Drupal 8 branch critical and security fixes And we're after the experience of working on that this This this is easy to answer. Well, I don't think it's easy to answer now I wouldn't want to say yeah, we're gonna start forward-pointing and I wouldn't want to completely roll it out either So one in a really thing I want to say is I love it because it will also allow Development driven by real needs and real work needs and feedback Okay, last comment then So I'm the last one shoot. That's okay So I'm very interested in and what does happen to the backport policy with this And I think I the thing that I found the most compelling was this sort of like at least maybe even just for this cycle The scenario where we we do the three or four or maybe six month releases Probably three or four month releases feature releases Until we open 9x and maybe for the first six months after we open 9x But then stop doing that and focus development on dribble line that would make that would make the backport policy still manageable um I Think That's just off the top of my head And then the other thing that I'm really interested in is is the question about initiatives because when we for for for reason for Angie the An initiative is a is a feature package thing like it's not just do a bunch of work in core Which is although certain initiative leads have used their shiny features and excuse to do a bunch of API work Well, you're talking about I can't imagine but so the idea is like we could if we are making if core initiatives are building Things that require a lot of work, but don't require breaking API's like we like I think that's half point right, but I'm I'm still I've been trying all week to wrap my mind around what happens what we do about the major API Architecture that means doing how we how we do resources for that and so it may make sense to have for example a You know really make rest awesome initiative that has two or three people that are focused on Non-API breaking changes that really make our rest API way more robust that may make sense to do within the Drupal 8 cycle You know something like you know changing our form API for Drupal 9 definitely be initiative scale The things that are initiative scale, but not the like oh, I need a configuration system Now I'm going to put lots hundreds of thousands of dollars into this not the Greg dot that much himself Yeah, so that's Well, I think that's the kind of thing we'll figure out as we go and say all right this particular thing is big enough We need to just You know put some larger structure around it So on that are we saying that Drupal 9? Will not have any features that are in Drupal 8.6. I'm not going to say no features But necessary API changes will be the focus rather than features because then you because then you can add features one yeah So, oh, that's that's a good point. I mean I feel like something there are some features that are going to require significant contrib obliterating changes I feel like I Feel like a like I'm not positive about this and Chris might think I'm horribly wrong But I feel like something the scale of like what blocks and mails really wanted to do is they're shiny Would break way too many price paid structures to ever backport that feature to to coral like I mean contrib can do awesome things Very likely so I feel like that that as a feature initiative would have to be a Drupal 9 initiative feature feature wise so I Think this sounds really good, and then I wonder like is it gonna happen who's gonna decide it what the trees think So I'll answer it this way. Is there anyone here in the room who is against this? All right Then that's let's all go crowd around Dries and make sure he does it What's that Yeah, I think just from you know casual conversation, I think Dries would be open to this And there's enough people That Dries trusts who certainly sound like they're open to it that yeah, I think we can make this happen So let's make it happen. So obligatory messages. Please review the session. You should all know where to go for that by now I'm gonna see all of you at the sprints tomorrow, right? good and I think we have a coffee break and then closing ceremonies. So let's all go there and see where Drupal kind is next year Yeah I'm not the first one to suggest it I'm just a loudmouth I've been Five six