 I just sent a really angry tweet which is a great way to get this started. I'm Greg Dunlap. I'm the initiative lead for the configuration management initiative for Drupal 8. You can find me on the interwebs all over the place as Hay rocker. I want to talk about how we can make our core development practices more sustainable and I'm probably going to focus on one thing but if we have other things to talk about feel free. Two years ago I gave a core conversation at Drupal con Chicago. It was the first ever court. Was that only two years ago? Man, that's a long two years. And it was the first time we had core conversations and it was much less guy standing up in the front as it was a bunch of people arguing with me and it was really fun and really exciting. And so while I'm supposed to encourage people to run up to this mic to ask questions so that they can be recorded for posterity and I understand the importance of that but I also only have about 10 or 15 minutes of presentation and then would like to have the rest be discourse and yelling and throwing things. I think that would be really great. So feel free to walk up to the mic and feel free to just yell stuff out too and I'll just make sure that everybody's opinions get recorded for the video later unless they're really terrible. So later blackmail. Right. So here we go. Core is big. Core is really big. You won't believe how vastly hugely and mind boggling big core has become. Not just in terms of code size but in terms of the number of people working on it. The processes and methods that we need to make core work when there are 50 core contributors sitting around a room and Antwerp is a lot different than when there are 1500 core developers in a room in Munich Germany or as there will be this weekend I believe 3,500 core developers and not even all of them are here and they're not all core developers. I'm just talking about people. Maybe we can get them later but the numbers are large and they're much larger and they're much larger than they were five years ago. That's sort of the point. Things have become so large that even Larry can't code a whole subsystem by himself anymore. That's really something. So we started doing some things to address this scaling problem for the Drupal 8 cycle and one of the things is that we named initiative leads to lead big projects for Drupal and so there's myself and there's Larry who's leading the web services initiative and even that was too big for him and so he spawned some of that off and suckered Chris Vanderwater into taking on that part as the scotch initiative and then Gabor has been running the multilingual initiative and John Albon has been running the mobile initiative and this was sort of a way to insert for lack of a better term middle management into Drupal to take people to become essentially project managers for big stuff that's important to Drupal's future. But even by the time we got to the point of doing that it was already not enough to insert that one small layer and by the time we started getting into really digging into some of these things we already needed more people than just that. This photo is from a code sprint that we held in Boston about a year and three months ago it was February of last year and already to start getting into decisions about how we wanted Whiskey alone to look. We have three initiative leads in that photo plus a gathering of about a half a dozen other core developers and the project leads of two major open source projects. Because to the right of Dries is Fabien Potentier who's the leader of the symphony project who flew from France just to take part in this code sprint. That's unbelievable. So one of the things that happened later in the Drupal release cycle is that when we wanted to put views in core they didn't have an initiative lead they had an initiative team. And that was actually really successful. I wish we had all established initiative teams from the beginning. So we have Damian Cloep and Tim Plunkett and Daniel Vayner and XJM Shoe. So I think that was really great. And I think that was a valuable lesson from the views team that even one person can't run a project anymore. They had a cross functional team of people with mutual respect who could take on responsibilities as people's time or interest or wanting to choke each other. And so that was one of I think one of the big learns of the Drupal eight cycle which is that teams work and that we can't have one person leading projects anymore that we're already too big for that. We grew too big for that before we even tried it. And so I know that Shannon Vettis who's been helping us a lot with PMing for the initiatives throughout the process is giving a talk in this room tomorrow at 3.45 and she's going to talk more about building teams about projects and I urge everyone to go to that because it's super important and it's something we're really going to have to think about in the next release cycle. So there's that. And then another thing that there was a lot of focus on around is that as core grows and people grow, those people need a way to work on core for more than just their spare time will allow because as I said before, even Larry doesn't have enough spare time to code an entire subsystem anymore. And so that means funding development and so there's been a lot of innovation around funding models in this release cycle and it's interesting because there wasn't really a focus on it. It's not like there was a big thing where everyone said we have to think about funding for the Drupal 8 release cycle. It just sort of happened out of necessity. So we still had a lot of people who were sort of funded part-time by their jobs to work on core or could work on core in between projects and stuff like that. But we also had a lot of stuff like Apria started their large scale Drupal project to fund development. I made this big funding drive to fund development which was really successful and I hope can serve as a model for other people looking for stuff in the future. There's been a lot more instances of a company sponsoring developers to go places, to do things. There's been more people getting hired full-time or part-full-time with the intention of working on core and things like that and I urge everyone to think more about funding because funding contribution is absolutely necessary for core to continue to grow. I've said it, people are probably sick of hearing me say this but right now Drupal has all of the business pressures of a mature software project and none of the dedicated resources because if you want to build a configuration management system and you've got a software project you hire a dev team and they build it and we don't work that way. But we need to figure out a way to work more that way if we want to be able to get more big things done in future versions of Drupal because in order to plan them and work them out you need dedicated resources with predictable availability and it's impossible to do anything in a timeline or milestones or anything like that if you don't have that. So funding contribution is absolutely necessary for Drupal to grow. One of the things I heard a lot when I was fundraising was that people were reluctant to give funding for something that they weren't going to be able to see for two to three years and that's a real problem because you know while you can make all sorts of ROI cases to people about why the configuration management system is good for their business and why it's going to save them money and make them more competitive if they are working in a business which has you know somewhat tight returns and their margins are really thin they don't have a lot of money to be contributing to something they're not going to get back on very quickly and that's that's and I adjusted by focusing on the kinds of people who had longer development models on people who were more product based in more competitive industries who can see longer-term investments pay off better but that's actually a big problem because a lot of the people places where the real money is in Drupal right now especially at the big media companies and the services agencies are they're not really able to invest that much in stuff that they're going to see so far out and so that had me started thinking a little bit about our release cycles and that's sort of where I wanted to talk about some today but in addition to having it be a problem with funding there's there's another really really big problem with our release cycle right now and I think it's very well I heard I heard a quote from somebody on a call that we had that really shoved this point home for me oh that's okay that's the wrong side I already talked about that here's the other quote I think I'm burnt out I'm literally losing sleep over this problem and I'm too stubborn to care about my own feelings because I know I'm going to have to live with this for the next three years that's a cliff gc chris van der water talking on one of our initiative lead calls about a technical problem that he was having a great deal of trouble convincing other people to get into core and one of the reasons that that I think we have a lot of burnout in the core community and we have a lot of sort of bike shedding and stuff hitting walls in issues is because of this it's because people know that they're going to have to live with whatever we build here for three to six years that we're not going to be able to iterate on it that we can't just get something in that's good enough and work with it and and see how it grows in the real world we have to get it in and it has to be right because we're going to be stuck and that's a real big problem it's as a community who needs people to be able to you know keep being refreshed on working with core and feeling like they're making progress and feel like you know their mistakes aren't so final um it's it's it's really contributes a lot to the pressure that we have um but on the other hand the three the three to four year release cycle is super important because it provides the long-term stability that enterprise clients want it's been a reason one of the big reasons that Drupal's been that's has been adopted so widely because they can install and and and build a site and know that it's going to be supported for at least five to six years and so just saying we are going to move to a 12 month release cycle and we can break whatever apis we want to in that 12 month release cycle probably isn't going to fly very well it's really hard for product-based companies it's really hard for companies doing large implementations of Drupal um it's it's difficult in a lot of ways and I think it would hurt our adoption rate significantly if we did that so um I've had some ideas that I've started to think about about about release cycles um and I'm going to talk about one of them here it's not a proposal it's just sort of some stuff that's been floating around in my brain and I encourage any other people to come up with ideas and stuff around this or start talking about it here today because you know this isn't the kind this is the kind of thing that's big enough that I can't that you can't just throw a proposal out and convince people of it it's a thing that sort of consensus has to be arrived at as a community and I'm sure that there's technical stuff that I haven't thought through nearly enough here so uh so I'm going to talk throughout some ideas and then we can start talking and if you want to talk about that or about other stuff about how we can grow and maintain core development well that's great I highly encourage it so um so this is our current like let's look at Drupal 9 so we know that we can't throw out a stable release cycle because we need it to keep companies adopting Drupal and growing with it and getting in place so we should we have to keep that and I think it makes sense too because there are a lot of stuff that takes a really long time to build um for instance you know rewriting the entire routing system is something that is going to break everything in core it takes a long time to do it takes a long time to convert everything and so that's the kind of thing that's well suited for a very long release cycle so it makes sense to keep that in some way shape or form but one thing we can't start doing is layering shorter release cycles on top of it um and starting to be a little more aggressive about the kind of things that we build into interim versions of Drupal um one of the things that we're really reluctant to do is to add major features in the middle of release cycles there are some reasonably good reasons for that one of the reasons is because um for instance let's say that we took um standard tables in a middle of a version of Drupal and made them responsive by default out of the box in the middle of a major version of Drupal that seems like a really good feature to add but one of the things that it does is it is it changes our UI components all over the place and that for the way that Drupal is right now is actually technically an API break because so much of our um ability to add functionality into Drupal is focused on page and form altars and if you change the the format of those or the way that they're arranged then you're going to break um contributed modules um I think it's worth having the discussion about whether or not we care about that um because because the ability to add major features mid version of Drupal is really important and if we start breaking those kinds of APIs then maybe we it'll it'll encourage us to start war uh building a system that isn't so tightly coupled to the way that our layouts work and start building real APIs on top of that instead of pretending that form arrays are APIs so I think that's something that would be really important to do um we could also start building API big breaking features into Drupal mid release if we provide backwards compatibility that would be something obviously that we've trended not to do in the past but uh again I think it's worth considering in certain cases because of all of the benefits that we've discussed here and obviously in some cases that's really difficult because there's always that balance between backwards compatibility and performance and it's impossible to provide backwards compatibility and not impact performance we're already impacting performance pretty heavily in Drupal 8 no matter what happens right now so um and but on the other hand um as we decouple Drupal more and make more stuff pluggable providing backwards compatibility in in an area is going to become more easy and so um I think that's something we should really start thinking about and reconsidering in mid release cycles and then of course in between those release cycles we can release our normal bug fix and security releases as we do already and they just become minor releases rather than major releases if we really wanted to we could also fix our numbering system so that Drupal 8.1 and 8.10 weren't technically the same release I think that would be great but I'm just nitpicking at that point um I think that a release cycle like this has a lot of appeal I mean I was thinking about for instance how we could start working one of the things about building new APIs is that it's really difficult to understand how they're going to work until you start using them we actually re-architected CMI essentially twice during the release cycle because as we did implementations we started learning a lot about core use use cases that we hadn't considered and we started seeing ways that we could provide abstraction that made more sense than others and we started realizing that you know multilingual is really difficult and stuff like that and so um getting something in place even if it's only for contrib to use if core is not using it I think it's something that's worth discussing doing things like for instance we could have had we could have provided CMI that wrote to CMI but also to the variables table so that you have a situation where we're still using the variables table as canonical but if you want to start experimenting with CMI there's some stuff there in core that's using it stuff like that it may be a dumb idea but it's something to think about how we can make a cycle like this work is I think really important to our future and I would like to start talking about that and other topics that I brought up and that's the end of my formal presentation if people want to shoot me down or come up with other ideas and talk about things and then maybe come up with sort of breakout plans that we can start throwing at Driess and the other core committers please feel free the mic is right there go Gabor yes hey so I think I think the mobile ecosystems gave us a very good example recently uh so google just had their huge keynote last week and everybody expected them to release their brand new shiny android os and they didn't release their brand new shiny android os they released a whole set of great updates that run all of the shiny android os is on the world so I think what so I think they I see a parallel there that we that we that Drupal is really suited more for the android model where we have all kinds of distributions and all all the sites that run Drupal are distributions in themselves because they have custom modules and themes and they are forks essentially of Drupal right we could I guess say that in an extreme so we would we would need to figure out a way to put stuff on top and and follow more like the android model but what we do instead is we put more and more features into course so when we want to actually update those features we'll only be able to follow the apple model where we need to release a whole new os to update our maps or update our email software whatever um and I mean that comes to the core of one of the great debates in the Drupal world for the last five years of course it's the small core stuff the small core stuff and it doesn't make sense for Drupal to be useful out of the box or not because of course Drupal 8 is the first release I think ever that is really useful out of the box for anything but of course it comes at a price as you're discussing yeah but but so what android does well is when you get an android phone it has a lot of stuff right especially if you got a Samsung phone or an HTC or whatever it has even more crap stuff and you don't think about it as a distribution of android it's android but you're getting a lot of stuff and then and then google can still update update all the goodies without without even others involved in there right so I think so I think we have very good proof around that this model works very well so I think we might we might be able to use this to argument our way into this somehow I think I think that's I mean you know because there's been a lot of talk about how Drupal itself should be a distribution of the Drupal platform with a bunch of default modules enabled and stuff like that right and people have been saying that for a long time I mean it does start to come at a little bit of a price I mean one of the things that happens in the android world for instance is that there are a lot of phones that call themselves android phones but they have vastly varying differences in functionality because the the cell phone providers do so much customization and bling on top of them that they can't easily be updated when new versions of android get released and so it's really frustrating for people who have android phones and see people with nexuses with all this cool new stuff but their providers haven't updated for the new stuff right and that kind of market fragmentation it's funny because people say that android has such a huge percentage of the market but I would argue there is no android because all of the phones outside of the google phones are so highly customized that they don't even really resemble what android is anymore and I think that's something that we would have to think about in that kind of a model and really consider but on the other hand we could just as easily say if you want all of the new stuff early just use Drupal and then you know just like because that's essentially what google's saying with the nexus devices and and then you know that's your problem there's you know there's a lot to talk about there Larry so plus one the on the release cycle a ton of the changes that have gone into Drupal 8 have been specifically so that we can do this kind of thing that's not been something we're discussing explicitly but you know good oh separations everything mark was talking about in the last session about you know separate you know good domain models good architecture lets you do exactly that without breaking apis I think the important thing there is that we have to understand is big nested arrays are not an api if you want to think about things in this way big nested arrays need to die because those are not an api any you know you sneeze that you break that api and that's why we've been so paranoid about breaking backward compatibility to date so the cultural shift that we have been making needs to continue further and faster for us to pull off something like that although I do fully support it second second points earlier on I think the probably the biggest thing core miss is missing right now development wise is predictable resources whether that's you know you and me as initiative leads whether that's you know developers like you know effigentsia who's just paid you know he has 40 hours a week to work on core or whatever it is just being able to predict we're no we're going to have this many hours of time to throw at problem x in this time frame is the single biggest thing that is missing from being able to actually manage core in some worthwhile fashion right now I know it as an initially myself anytime I sit down to do anything I'm torn between do I try and write something do I try and review something so that someone else can get stuff done or do I try and you know write documentation so that someone else can write something else whichever one of these I'm doing I'm doing it in spare time where it's a little bit time and I'm sure changing all of the others that's just terrible for everyone involved especially me but I've grown else too so that's the problem we do need to solve and I think you're absolutely right I think all the initial leads have said the same thing at this point views is the only initiative that got that right with a fixed core team of multiple people who had guaranteed minimum of time that they could put towards it and we all need to be doing that and it's hard because even even when we have a fixed time a lot of our resources don't so for instance somebody will come into CMI and say oh I really want to help out and then it turns out they only have two weeks to help out and then another project starts and then I've got a real and then I've got a re on board some brand new person into a complex project I think that long term the answer to this is for people to just get jobs at places that rely on Drupal for instance I know that I know that Jared Smith from Blue Host is over here and they're hiring people right now to just work on WordPress for instance because so much of their business relies on a stable WordPress and so it's in their benefit to have WordPress in place and so I mean you know they're a hosting provider let's not pretend that they don't have an incredibly large number of WordPress sites running on there you know and to convince companies that rely on Drupal even the hosting space the training space or the services space and etc to to hire people full-time to work on Drupal is I think an incredibly important thing that we have to try and figure out how to do down the road add on to that something more controversial there are times when fewer people are better something I've said since the beginning of the whisk initiative give me five people 10 hours a week for a year I can do whatever you want give me 50 people one hour a week but not every week for a year I can do jack all yep and that's another problem we need to solve not just funding but also just culturally the idea that person managing the 50 that's and it's not convinced that in some cases yes but not in an awful lot of them I'm happy to talk about anything that impacts our ability to grow sustainably here so I was encouraged to hear your idea about the revision our release cycles and I'm in favor of breaking apis as long as the module maintainers actually keeps track of all that and so maybe an overarching project manager that keeps track of all that stuff and that filters out to the module maintainers and it's a good thing there aren't very many of those and and to combine to combine the idea by the teamwork I'm really excited about that I think it's a great thing so maybe the module maintainers instead of actually writing code they would manage a sub team for that to so apis won't break in their in their module so it's just an idea yeah I mean another thing is to is to have you know we could start using deprecated a lot more for instance and laying new apis alongside existing apis but you know we're still going to have the problem at some point of you know I think we have to start considering the impact of api breaks too because some api breaks just aren't going to have as wide an impact as others and maybe worth doing you know it's not hard for us to grep through the code base to find what people are doing I broke I removed an alter hook in in Drupal 8 hook image styles alter because nobody in all of the 25 000 contrib modules was using it and so I was like well I guess this probably isn't that useful I don't think anyone's going to notice there's probably someone here whose entire website relies on hook image styles alter is now going to cost me after this so Jess yes I guess two things the one thing that I was thinking about when you were talking about I'm in favor also of finding a way to provide backwards compatibility but like you said performance is always a concern and the idea that occurred to me just now is that maybe maybe we need to maintain two branches maybe we need to have the press flow ish kind of branch that's only focused on you know that the stable public api that does not provide the new features and then we have the option to add add new apis that are wrapping that BC layer I had that could be an absolute nightmare to maintain and I I'm sorry but I can't see like I'd be very concerned I mean unless we I mean that or we'd have to be very very sure to do you know performance testing in actual real use cases and then my well I'll wait my turn for the other thing I will say yeah well I will say Larry are you coming to the to cat the core conversation Kathy and I are doing tomorrow morning I'm I'm putting him on the spot on the microphone here on purpose I'm gonna okay we're gonna talk class in terms of I mean forking the Linux kernel that's sort of that although they generally don't break apis to my knowledge but there's what the every three months we add major features if you want to and then there's at least two major versions that are long term supported that's not what I came up with I I wish I had something really clever to say that I wish but I came up here because I feel like what what you Greg and Larry were talking about I mean that's I've been I was helping you out a little bit with a few patches here and there for CMI I've been involved a bit in whiskey and doing a little bit of work but not anything consistent and I see of course that that's I mean potentially it's even it's even negative because I take up all of your time when you could be doing something instead of explaining to me what I'm supposed to do and I mean I would love for my employer to pay me to work half time or full time or whatever on on Drupal that would be great maybe we can work something out but how do I convince my employer to do that is is the question that I'm I think one of the things that I've I've been maintaining for a long time is that because we're we're nerds and we lack self-confidence in in stuff because you know there's there's this thing called the Dunning Kruger syndrome it's a defined psychological thing and basically it says that the more competent you are at something the less competent you think you are at something and so everybody in this room is incredibly competent and we probably all think we suck because everybody else in this room is more competent than we are and one of the things that I've been maintaining for a long time is that employers will provide time to work on Drupal if all of the developers insist on it because we are the limited resource in the supply and demand chain in the Drupal world and so if nobody goes to work for anybody who doesn't provide time then they'll have to provide time or they can't get us to work for them uh you have to insist on it and and I kind of I want to be I every time every time I talk to people about this I want to be like Norm Aray with the big you know with the big banner standing on the table but it's true it's super important for people who are working in the Drupal community I mean we are in so high demand it is ridiculous for any of us to be working in a job where we're not completely happy that's what I had to say about that I'm gonna zero in on your comment about versioning there's a pretty popular mechanism to do versioning and in Java and other communities even comment was made about the Linux kernel where you've got a major minor and sort of like a build number or break fix or security patch plus one I would love to and I think it's really we need to get to that yeah I think it's really important for us to look at other models that other open source projects are using successfully so Java uses Maven a lot for build that's built into their versioning process within the Java community if you're using Maven which I don't know anyone who isn't it PHP version three three positions now they kind of leave it up a little bit to your interpretation of what those positions are which we kind of need that we need to be able to interpret what's a major minor and build right so I'm just gonna zero in on that one comment and leave it there cool thanks two quick comments Greg first of all uh versioned apis I think it might be interesting to look at more more formally saying hey this version of Drupal has version x of this api and version x plus one of this api and modules themselves saying hey I'm expecting to find version x of this api that's what I'm coded against if that version goes away in core I know as a module hey you know right yeah and that's similar to sort of what the what the views what guys have been doing for a long time because they provide that function that provides the api number that views has and then you have to be compatible with the specific number that sort of thing more widely adopted within within core and within I think that would be a really interesting idea to explore yeah second thing is from a from a funding standpoint um you know you talked about one of the things we did was try to go out and help with the CMI initiative I would love it uh you know if we would continue to to push that cause um you know from a from our standpoint from business perspective it was really nice to have these core initiatives to say hey this is what our funding is working on and to be able to go to business decision makers and saying this is going to help us because of x y and z this is what they're working on so kudos for the core initiatives and bringing kind of more you know insight and in marketing for lack of a better term to the to the initiatives that are being worked on I think that will continue to help getting funding and thank you for your help bluehost uh was one of the major funders of the CMI initiative and and then I mentioned that we're hiring and bluehost is hiring so talk to Jared Smith afterwards if you're looking for a job um so with great project we also need great tools and something as somebody who's one of the long tail core people who is passionate about certain issues and not others um it's when I work with other open source projects I get jealous about their manage tools for managing milestoneing etc and while I think we've collectively saved billions of seconds just by adding the subscribe button to d.do it might be a good in in looking for making a development more sustainable it might be good to do some discovery about what additional tools do we need to introduce to manage things like issue queues milestoneing and things like that there's a core conversation by Gabor about about tooling up for running initiatives and improving our tools on Drupal.org um and so I encourage you to talk to him um that's a very controversial topic in the Drupal world now of course because in order because just like Drupal projects in order to improve Drupal.org projects we need funding and people to work on them and things like that and um and you know um traditionally our our history with doing that is not the best so um but I think I but I that's not to say I don't agree with you I completely agree yeah I think the short answer to that is it's a metal level on top of our initiatives so we want to work on core stuff and we suffer because we don't have the tools so we can go one metal level up but then we don't work on our core stuff right so we have deadlines for our core stuff so we want to work on that so it's very hard to find the balance between working on the metal level and and and the level that we work on. Brighton it's really hard because we're constantly putting out fires and we never sort of take a step back to see the trees and the things that cause those fires and sort of put all that aside and work on the stuff that will make us you know we the long-term view it's really hard to find sometimes. I work for a college that has been using Drupal since 2006 now and I came in on the ground floor then and I've been doing all of the incremental upgrades to the various versions and I'm very pleased that you're finally that there's finally some talk about the way that versioning works mostly because this last time around going from D6 to D7 took us I'm estimating three person years to accomplish. We have about 250,000 lines of code and actually doing that that whole process is getting very very difficult to justify to our CIO and it's really starting to get to the point where we may not be able to go to D8 simply because we can't afford to and if there were to be more incremental upgrades or a fork or what have you something that has something has to change it really can't it can't be that we need to spend three years doing this again it's simply not going to work for us. And that's and that's an interesting thing that I hadn't really thought about this which is that you know breaking APIs in the middle of a release cycle sort of spreads that pain out. It does and I think that may actually be be useful because we can say to our CIO well you know they're coming out with a new version in this API changes so it's going to take us you know another three weeks to get this version up and we can we can justify that more easily than we can saying it has to all happen at once. That's a great that's a great comment. Thanks. Going back to the point about resources since Larry's not going to be at my conversation tomorrow. So I will agree. It's definitely very very valuable to have a team of developers who has a committed amount of time every week. That's what made VDC work is that the four of us each of us had like two days every week for eight months to work on views and that was amazing. We could trust each other. We all know what was going on and that's incredible but on the other hand we as a community have a resources problem in terms of in terms of talent working on core and you can't we not you specifically but we can't afford not to invest in those 50 out those 50 people who have one hour because if we don't invest in those 50 people who have one hour they will not become the people who end up getting paid by their companies to work two full days to work under Blake or because it you just you just have to give someone a little and it's a numbers game. We see we see this in the core mentoring program where we help a hundred people and out of those a hundred people a lot of them will never come back. They work on one or two patches and some of them don't even really enjoy it and some come back week after week but then there's like that one percent that makes a leap and become core contributors who are passionate about it who work all the time who go from being just participants to mentoring other people and that's that's the reason we can't afford not to invest in those people. So I agree that an individual initiative lead trying to project manage and architect and all these things can't manage those 50 people in that one hour and on the other hand though you have to be you have to find a way to include them in your process let them give them a chance to make the mistakes. You're still if you still have someone to review their code and say okay no actually this is not a good idea someone who's not you then it's okay and then you have a way to mitigate the time but you need to be able willing to invest just just a little bit of time. Well that's one of the things that having a team makes easier when you're just one person it becomes really difficult to scale that up but when you've got two people who can still focus on building things over here and two people who can focus on helping people out over here and then when these people get burned out switch off and stuff like that it makes all of that stuff so much more easier and that's one of the things that like me and Larry have talked about a lot is that we're constantly having to like weigh our priorities even though we know that helping people in mentoring will help us long term in the short term we've got like five like issue fires over here that we have to put out and pay attention to and that whole thing is it's the same as Gabor's thing right it's like when you when you only have enough people to do exactly what you absolutely have to it's impossible it's impossible to scale out to the point where you can grow sustainably. But on the other hand in the past in the past year actually we've built a community of people whose primary interest in Drupal is is mentoring other people to work specifically on core. A year ago at Drupal Con Denver six of us decided that we were going to do a sprint that was like core office hours in real life and we had three times as many people show up to it as we wanted and there was a huge demand six people at this sprint we have over 40 people signed up to do that so there is a resource you can take advantage of it's not all on you to figure out how to work with those people and that's just what I want to reiterate if you can spare an hour to talk to the person you can spare eight hours that will make a big difference and I'm done now but come to our core conversation tomorrow. And that's totally true the growth of the core mentoring group in the last year has been absolutely stunning it's been really amazing and I think it's a really good thing to see going forward for the Drupal community so yes. So I work for a company that outsources all our Drupal work so all we can really do is give money but we need help I need help selling that to management because management doesn't even understand software development let alone open source. We looked at large-scale Drupal and it was just too big a leap for a first time in. Who can help me present it to management and explain it in better terms than it's it's good karma it's the right thing to do somewhere down the line this will help us. I actually think well first of all I actually even though you may not be able to get into the large-scale Drupal program I would assume that I don't want to speak for Michael here but I would assume that he has plenty of resources he could provide to you on on selling open source to big companies because that's what he does for a living and so he has all sorts of white papers and stuff like that but on a more concrete level what I would find is something that actively costs your company money and then look to support whatever initiative is fixing that and so like for instance when I was looking for support for CMI I would go to service providers and I would say I want you to sit down and estimate how much time it takes you to deal with features and update hooks and all of this stuff and imagine taking all of that off of your bottom line and give me a small piece of it because I mean it would it's in a lot of projects it's like 10 to 15 percent of their estimate is dealing with this stuff and it will make them more competitive and give them more and give them more you know leeway will make their margins greater and stuff like that so every everybody has pain points and and everybody has people who want to fix those pain points and I think the most important thing is to find the ROI arguments to put those people together and that's what I would really do because that's where the most compelling arguments are going to come. Well here's my pain point it's 30 different vendors of very poor quality building bad Drupal sites. What's my what can I use what can I point out in a Drupal initiative to say this will help? Well one of the things you could point at probably is... I'll take all of your cards at this point. Well the thing that the thing that usually I would point out there is finding vendors of higher quality of course they're also of higher cost. Actually I don't mind the cost because you pay the cost anyway. But I don't have a big answer to that other than supporting training programs and documentation programs which will grow more Drupal developers than we have right now because I think that one of the reasons that a lot of people get stuck using low quality Drupal developers is that there are so few high quality Drupal developers right now and so focusing on initiatives that will grow training and documentation and the kind of things that will scale the community of good developers is I think something that you might want to think about. Okay great thank you. Okay I took notes while I was in line. Okay so the problem of poor quality Drupal developers in your subcontractors if you're looking for ways to fix that problem and find funding for that I don't know how you can pitch it but if you require your subcontractors to require that their Drupal developers contribute to core a certain number of hours per week the core community will train them to improve them. We don't let people contribute to issues and make patches without giving them the feedback in a way that improves them so their future work is better. That when we were talking about teams for initiatives and how to build that team I may propose that one of the people in the team be a mentor specifically specializing in that initiative to relieve the other people that are part of that team that find mentoring frustrating. There are people that can do that and not in a general core because the core pool is so big now that we can find people to be part of initiatives specifically to do that for them. Sorry. Yeah building teams that are cross functional is super important because no team can run 30 people with just like four hardcore devs running it. That doesn't work that way. And then we were talking about how to pitch to companies to pay us to work on core and one aspect of that can be that when so I got hired to work on core um 10 hours a week and then I blog about me working on core five hours a week and I get paid to do that and what one of the things that my company gets out of that is they are able I think to hire more Drupal developers and better Drupal developers because Drupal developers want to work for companies that do that so it's like a recursive kind of a thing so if a company wants to have good staff the way they get it is to pay their current staff to work on core so that the person they haven't hired yet wants to come and work for them and so it's you don't have to revolt exactly although I think that's a good idea Alex um by the way I just want to point out I think Dries's keynote is at 1130 and it's 1105 right now so I'm happy to stay here as long as people want but I just wanted I also want to see Dries's keynote so I'll probably have wrap up in like 10 minutes. Okay really great ideas here and I'm thrilled that there's been so much support so far for the kind of the major release cycles a long major release cycles with minor improvements along the way I also agree with what Larry said about a lot of the object orientation that's gone into Drupal 8 kind of sets us up to be able to do that a lot easier than we've been able to in the past the other thing I want to sort of point out though is that the the things that aren't like strict apis like hookform alter or hook node view alter or a lot of our alter hooks are the things that actually make Drupal awesome and that a lot of people who started developing for Drupal and making really cool contrib modules making cool distributions making cool themes making cool custom websites you know that's what they love about Drupal is that you can alter everything yeah and I would I would argue that a Drupal without alter hooks is actually not Drupal anymore so but I think a way that through that is to is to is to think of minor releases as an opportunity to allow allow breaks of those things right so you keep your your nice object oriented apis you know backwards compatible but you allow certain bc breaks in the in those kinds of things and you know they're smaller and they're more manageable and as someone said you know like if they come at certain you know intervals where that are predictable and that people can keep up with them one of the interesting things that that causes though is it means that right there are going to be websites for example that don't update from 8.1 to 8.2 right away or from 8.2 to 8.3 because they you know some of their custom modules will break some of the contrib modules they they're using will need time which means for a period of time we're going to be needing to support security updates for multiple minors maybe we've gotten to a scale of community size here that we can do that but i just wanted to sort of point those things out the security team is always under resourced so yes that's what's actually one of the pain points if we can find a way to get more resources in the security team that might be able to support this kind of plan security team would love that but you're doing something very direct it gives you a very direct return on investments because you're getting more security system out of it here's a nice bit for it yeah i mean the cost the cost of having to save your site when it gets hacked versus the cost of giving somebody a little bit of time to run on the security team is like that there boy that's a strong RRI argument right there several words of magnitude difference oh yeah absolutely thanks Alex hi hi Mike Gifford accessibility maintainer right looking at at the idea of culture change and the importance of culture change for sustainability not necessarily the culture change of the Drupal participants in Drupal shops certainly the Drupal shops who are here most of them get that but it's a culture change with our clients and with the the people who are implementing Drupal but don't really get Drupal the people who are here and realizing that that the only reason they can use the software and it's what you know the Drupal core is rated at 14 million dollars if you were to go off and develop it from scratch today so you're getting 14 million dollars worth of software for free how about you contribute a little bit even you know a couple hundred dollars or a portion of the budget or have something so that there's a regular way to go off and to contribute not just a core but also the contributed modules to have some way to make it easier for the clients to give directly not necessarily to their Drupal shop and as a Drupal shop I I definitely want them to go off and to give to me but I say I want my clients to go off and pay me but I also want them to go off and realize that that there's some stuff that you know I'm not the maintainer of a bunch of modules I don't do a lot of different stuff but but there's if they can find ways to support the functionality that they need and it's useful that is like right now there's no way to to give directly to those initiatives that matter to you as an organization you can go and hire a Drupal shop that believes in these issues and and support it but there's no way to to do that when we were hiring to go off and to bring or we're sorry not hire we were fundraising to try and bring Vincenzo here to to to Drupalcon it was you know all of the money that came in with some minor like 95 96 of the money that came in was from Drupal from the Drupal community who are bringing Vincenzo to come here it wasn't from government it wasn't from university it wasn't from the organizations like the NFB that are using Drupal like there's a lot of people in the accessibility community that are using Drupal they didn't give the Drupal community did the people who are here gave to make sure that Vincenzo could come so we get it but how do we create the culture change so the people outside of here are able to go off and understand I mean beyond Drupal gives beyond that sort of sense of here's that sense of pride and we'll point some links to you as sort of highlight those people but sort of make it clear that part of Drupal part of this wealth of software is also that that is a mindset change that is different from any other software I don't really know I mean it's hard I mean one of the things that I've thought about a lot is the possibility of us setting up like for instance Mozilla has a non-profit that manages all of the money that gives that they give to their developers and and I think and and then we would have a central agency that would go out and you know do marketing and PR around around raising money for core development and I think that's an interesting model but I also know that Dries has been really resistant to it and of course we have the DA now but they're completely banned essentially from funding any core development directly they have to they have to fund infrastructure and cons and stuff like that and you know it's it's really hard because we as a community have grown very distrustful of centralization and and and a lot of other open source projects are funded by commercial entities you know like all the Google projects and Oracle projects and and MongoDB and Ubuntu and all of this stuff you know canonical symphony that's right canonical has 150 developers that they pay to work on Ubuntu you know and so we as a community are are distrustful of centralization which makes the problem that you're talking about much more difficult and and so I don't have a really good answer for that really but obviously it's important to talk about if we can have diversified ways of people giving money either yeah yeah you know a flatterer link on on pages or on issues for people to go off and you know I think we could get a lot more people contributing to Drupal core if they could even get pizza money once a month you know if for the hundreds of hours that they're contributing if they could go to their go to their partner and say hey it's it's beer and pizza tonight on Drupal great stuff like it'd be so much easier to go off and get people but there's nothing right now except for geek cred which is which only goes so far if your boss isn't paying you to do it right sure no no you're right yeah so I do have an answer for that name Ben Melanson of agaric ended like the last four Drupal cons um especially Allie Micah of Advantage Labs and I have put together you know a boff on discussing ways that the community can support community projects and so we have that boff coming up again Thursday at noon in a 104 but what I really wanted to do was and it this is a key part of that I feel is even though this is a core conversation to remind that Drupal more than other projects really lives on a really really really healthy contrib space and that we don't need enlightened self-interest from the shops and companies using Drupal we just need to help them act in their own self-interest and that if we can come up with ways that you know when there's 50 libraries that need something or there's you know 100 development shops that all have the same need to actually coordinate and make that need happen this this is not happening and if that happens then we have sort of a practice for how to do the same thing on the core level so I would really love to see more thought at how we make things bubble up because yes it's hard to make the case work on something that's going to benefit you in three years but we're not even getting people to collaborate on things that would benefit them in three months if they would just do that collaboration so I really feel if we can nail it on the more immediate more local more small level and so that addresses the the fear of centralization and it actually like gives us a chance to build up this ecosystem of oh wow if I contribute help either you know so fund it or have developers do it all these various ways we're talking about but do it on a lot more smaller scale um contribute projects and then it'll bubble up in the core I think that's a great idea and I encourage anyone to join that boff and discuss it uh it's 11 14 I'm gonna have to give Alex the last voice at the mic here um so that we can all get off and and watch Dries show us lots of pretty graphs so I'm gonna be I'm gonna be really quick um okay Greg thank you for bringing me into core depth I had to say that uh that uh you know it's interesting because people talk about mentorship and stuff like that and it's like I I give a court I give a conversation at triple con Chicago um called a shot in the arm and it's about how just just doing little things to support people can push them off into into great directions and um seeing Alex grow from uh from sitting down at my table at Denver to a core maintainer in a year has been really amazing so teams plus one like working with you wow but the the other interesting thing is about conflict of interest as core development becomes more professional more supported by companies we're going to have conflicts of interest and I don't know how we're going to deal with it because right now if I go and get employed I'm not employed I'm employed at home maintainer um if I got a job and I start committing patches that are in their interest who knows it's different well we already do have conflict of interest indirectly because people are scratching their own inches through their companies like that's how a lot and it's not just people getting up where you mean because that's your word man man yeah I just want to say I've only been a developer in jubilee for two years so I've been a php for about 10 and uh the market is in the immediate direction where sulfur gets generally pushed even and you know nothing about it you just get it and you know that everything will continue to work so I think that uh the market is telling us what we have to do is that updates need to be incremental rapid and immediate and uh the slow cool uh thanks everybody for coming please let's continue these conversations not just this week but going forward because I think this stuff is super important