 And before I start, I wanted to clarify something that just came to me. Rob picked this title for the talk, which I realize now might have missed a lot of few people. So this is not about dependency injection, but rather about, well, I mean, the title kind of suggests something like that. It makes a lot of sense to me that you said that. This is more about handling dependencies to external libraries that Kruple is now increasingly using and that you might be using for modules or some project that you're working on. More specifically, this is really just managing these dependencies with Composer. So there are probably other ways of doing that, but I'm specifically going to talk about a tool called Composer, how that tool works, which is a tool for all kinds of PHP projects not specific to Drupal. And then later on, I do want to take the conversation part quite literally and talk about how to use Composer better in Drupal, what kinds of things one could do with Composer in Drupal in the future that isn't quite possible yet, and maybe also about what I think is wrong with how Composer is currently used in Drupal, but we'll get to that part later. I'll just start with the introduction so that we're all on the same page and what Composer is, what you can do with it. To explain the background bit, I'm not actually a Drupal developer at all. I installed Drupal in the previous session. So I'm going to rely on all of you to, on your input on the whole using Composer in Drupal bit. I do work in PHPB as my teacher kind of gives away, and work in various other open source projects. I really enjoy working in these kinds of open communities, which is why this conference is great, just getting to know another open source community and seeing what these people have to say about certain things, how they interact with each other, and just seeing another one of these communities is always really exciting. And, right, that's also how I got into Composer. So about a year ago, a bit more than a year now, we decided to move to Symphony with PHPDB, similar to what Drupal's been doing. And we wanted to come up with some new, properly designed mechanism for installing plugins for the software. And we started talking to some of the Symphony people, and they had come up with this bundle concept. And they also needed a way to handle dependencies between bundles, libraries that they want to use in bundles. And we realized that this is really the same problem. And there's no point to finding another specific solution just for Symphony or for PHPB, but that we should really try and address this problem in a generic way that will help all PHP projects that can be used for various tools and projects out there. All right. So that was how Composer got started. And you might think that there's already something like this in PHP. I'm curious, who here has used Pair? All right, that's a lot of people. Who here used Pair to install PHP unit? That's so quite good. Who's used Pair to do something else? All right, that's actually a surprising result. Typically when I do this, but 95% of the room used Pair to install PHP unit, and they've never used it for anything else. Here it's a larger group, which suggests that people here are building bigger projects, maybe trying more of these tools. But basically the experience that I hear from most people who've used Pair is that it's really tricky to set up. I actually look at a documentation for Drush while compiling the command line tool for Drupal. And it actually, first it starts with the line that you have to execute to install it with Pair. And then come about 10 other lines to force install some package that you can try if it doesn't work. And then maybe you can overwrite this other package, or you can try all depths, and maybe that'll work. So that already suggested to me that there's something wrong with this, and it's just a bit too complicated, more complicated than it should be, at least. So that's the other rationale for behind building a new tool to handle dependencies rather than using the one that already exists. Composer's made up of a couple of different pieces, or the Composer ecosystem. Composer itself is the command line tool that you use to install dependencies for your project. The focus, as I said, is on being easy to use, not having all these little flags that you have to use to install it properly. And a key difference to something like Pair is that we always install dependencies on a project level, so like within the directory that you're in. So more in the natural way that PHP developers like to work, where they maybe have a set of directories with a different project in each, rather than the way that Pair typically is used, even though it can be used in this way as well, where you install some system-wide library that is then available across all PHP code on your machine. It's also designed to be embeddable, so it's really just a very thin CLI wrapper around a library, which can also be used to build, for example, a web UI to do the same thing, which again comes back to our PHP vbackground because we want to use this to let end users install plugins. So we don't need a command line tool to install dependencies, but we need this to be accessible from a web user interface. And a few things we pay attention to is encouraging best practices, PSRs, for example, a naming standard for classes and auto-loading that I'm hoping most of you have heard of at this point. SPDX, which is a standard naming scheme for licenses so that you can automatically filter by certain licenses and make sure that you don't install some dependency that actually has a license that is incompatible with what you're doing, and semantic versioning, which is just a standard description of how you should version releases so that the numbers actually represent in a commonly understandable format what kind of or what level of change is taking place between these versions. And there's packages. So we've had this tool that you install dependencies with, packages to the central repository that all these packages are available on that you can install them from. Composer is not centralized, so there are alternatives to this, but it's like the one big thing where anybody can just push with whatever PHP code they have onto for whichever frame rate they like. And it has a great search so that you can just find things that might do what you're looking to do. And it's extremely easy to use again. So typically you just enter the URL to your version control system and it automatically picks up versions from the tags that you used in your version control system. All the metadata is just read from the version control system. So when you do a release, you don't actually have to do anything on the site. All you do is you tag your release in your version control system, which you would do anyway, and that's really all there is to it. There's a little screen share from that website where you can see the kind of metadata. Twig, which is going to be used in Drupal 8S, well, I think that's certain at this point, right? You hope still? I wasn't sure how far that was. All right, never. So you might be familiar with Twig. And you see the different versions listed with its dependencies. In this case, it's just a PHP version that's the only dependency and some metadata, as well as some information about how often it was installed. And at the top, you see the search box where you can just type things in that you might be looking for. Then I said it's not necessarily centralized. This is just a very simple way to get to open source projects. If you, for example, have code that you don't want to publish to the entire world, but rather have some internal, set of libraries that you use within your company or something that you just want to have local, there's another one of these repositories, which is a lot simpler to use, it's just a command line tool that dumps this kind of metadata, which can then be used by Composer. So for example, you can run this in some machine in your local or internal company network behind some VPN proxy, whatever you want to set it up like. It really just generates a single file that you can use to load the metadata for other dependencies. There's a bit newer one, which is called Composer installers, which is actually a package that you can install with Composer, which makes it easy to then install packages in the way that is required by a particular framework. So the standard way that Composer installs things is just to unpack them into some directory, but typically frameworks have some other directory structure that they follow, well, Drupal itself does. So this is a very simple way to just define certain mappings for which files need to go to which directory inside of these frameworks. And there's actually already one available for Drupal there, and I'm gonna show how that works later. First, I'm gonna just run through how do you actually use Composer. I've used the example of setting up a new Symphony project here, this is actually their standard recommended way of installing Symphony at this point, so it's a good example for how one's supposed to be doing this. I start here in this case by taking their Git repository, and then installing Composer. Composer is actually just a single file, have you heard of file files? All of you know of, right. So the problem with file files is that the file extension in PHP has a lot of really weird issues, and a lot of the time, when you try to run a part file, it'll just die on you without any kind of error message or output some random crap, and you don't know what that actually means, or it won't provide you with an error message, which is why we then ended up writing this installer script, and all this installer script does is check a bunch of PHP configuration settings, and then download the file for you. So you can instead just download the file yourself as well if you're certain that you can execute file files. But that's why we recommend installing it just with this little snippet on the command line. But again, if that's not something you can do, then you can just download the file for yourself. All right, and then all you do is you run PHP Composer for install, and that's really the only thing you do to install all of the dependencies, and it just goes and looks through the metadata of this package that you, or the project that you have, which was the symphony standard edition, and installs all the dependencies, like the templating engine, symphony itself, the entire framework, some extensions for Twig, and all the dependencies, all the dependencies, et cetera. And there's a little note at the end, which is kind of interesting. I'll get back to that. It says something about generating autoload files, because Composer also makes it a lot easier to then use this code without having to think about where it installed this and where all the code is located. I mentioned this, typically it's put into the vendor directory, so this is the result after running Composer install. You have, monologue is some logging library that symphony installs, but if symphony itself, Twig, the Twig extension that you saw. And there's autolo PHP, which contains this autoloading mechanism that you can just include and use right away. And within Composer, there's a bunch of metadata, like the list of the installed packages in which versions they were installed and a few things like that. There's an alternative to the setup that I just showed. With a package that's set up like this, you can actually run the command called create project. And what this does is exactly what I described earlier. It takes a copy of this project. It then runs Composer install within this project. So it installs all the dependencies for this project. So you can imagine doing something like PHP Composer for create project, triple project or something like that. And you would just instantly have the entire Drupal installation with all the dependencies installed without Drupal itself having to contain all of these in the downloadable distribution. All right, so we've seen how you use this, but the question is where does the metadata come from and what format does it present it or do you define these dependencies? And we use a JSON file that is typically located in the root directory of the project in which you have a require key with which you define the packages that your project requires to be installed in order to operate. The first part is the name. We follow a standard of having a vendor prefix and then the package name itself to make sure that names are sufficiently unique. And the second part is some kind of version constraint. So you can require some particular version like 21 dev in this case. You can require anything larger than 100 dev where you can require any one point version as you see with a quick example which will install 1.5, 1.6 depending whatever is the newest currently but won't install a version two because that will probably have API changes that break your code and that you would have to first make changes in order to be compatible with. And the second part required dev is kind of a nice tool for development where you can additionally specify packages that you just need during development. For example, to run tests and kind of utilities that you just need during development but they don't want to actually have you on your deployed system in the end. And anything here in the note below you can specify preferred source when installing and rather than downloading some target zip, zip file, whatever and unpacking this it'll actually do a Git clone SVN check out CVS or whatever you're using so that you can actually go into this vendor directory make changes and commit them right there and done. It's kind of a problem if you just work with pair and you have the system wide installation you can't easily just if you've got a problem with some package in pairing you want to make a small change it may be bug fix you can't just go edit this committed there but you will have to well first debug this make the change and kind of take this diff send it to them hope that they'll apply it install a newer version it's kind of a long loop and this makes it really easy to during development to just commit this kind of change maybe even to your local fork of this package and to then later update to the official fixed version. If you want to publish a package you need to add a bit more metadata like the name of this package it's exactly the same composer JSON file that you're using your project if you want to publish this as a package you can add a couple of keywords which are used for the search on packages for example I mean the rest is really self-explanatory and the last bit is this is how you define auto loading in this case with the PSR zero standard you can also find a class map and it will automatically generate the necessary auto loading code All right there's this part you can install composer packages you can define them you can publish them there are a few neat things to composer that it does differently from other systems that help you avoid chaos this kind of version chaos that starts to exist if you want to work on a project with many people or if you want to distribute a package to a lot of people because something you quite frequently encounter is that someone on your team install builds a package with particular version of a dependency and then you distribute this to a coworker maybe a week later coworker installs the same thing and he also installs the correct pair package let's say and it doesn't work for him and after a couple hours of debugging you realize that he did install the correct pair package however there was a minor bug fix released within this week and that actually broke compatibility with the code that you originally wrote so just a small delay already resulted in broken code and having a really hard time finding out why it broke so instead composer has a composer lock file so when you're in composer install with something like the one point asterisk which will just install any one point version it'll record which version actually picked so for example in this case it could record that it install version 1.0.0 and then you typically commit this file into your repository and share it with your coworkers or with users of your product and when they run composer install it'll install exactly 1.0.0 even if 1.0.1 was released within the time frame from you sharing this code and them installing it and you then explicitly run composer update to update this lock file and this just makes sure that all your again all your coworkers always have exactly the same version if you distribute a product everyone using the product has exactly the versions that you tested your product with and if you have a large set of machines you also make sure that each of them has exactly the same version because that as well might be a problem if you have something like 10 web servers and I don't know if you add a new one and that one suddenly has a different version of some library that will call a lot of problems. All right this is some details on the all loading that I mentioned already you can also have a class map where you just specify some directory of files that'll just look for classes inside it if you're not following PSR0 yet and for especially older code like peer code you might need a include path that's not recommended because adding a lot of these makes including files a lot slower because it'll help check each of these directories every single time it wants to load any kind of file but it is there if you still need it. That's the file it generates that I mentioned and all you do is you require this file you can immediately start using all the classes you find in all of the dependencies that you have. So you never again need to think about where is the code coming from that I want to use here it's just all directly and available to you to use in your application. All right I mentioned that we have different ways of defining repositories this is how you do this in your Composer JSON you basically specify the type like Composer which is the same type that packages itself uses or you can directly load packages from a VCS repository for example like Git repository without actually setting up anything like SATA sort of packages. You can disable packages itself as well if you want to for example only use packages from your reviewed package list on your internal network and don't want to use any of the open source code and there's a neat trick here as well there's also a repository type called package which allows you to define a complete package in line so that if you want to include some code that you just have a zip file for which doesn't contain a Composer JSON or maybe this is something like jQuery that they won't care about Composer because it's some PHP dependency management thing. Then this allows you to define a dependency on this other project without them supporting Composer so you just provide all the metadata that Composer needs within the repository definition and essentially this is a Composer JSON within the Composer JSON so you can also define things like further dependencies of this package and then in the end again you require the package that you just defined. All right, okay that's basically what I wanted to explain how Composer works I think you all have a good idea of an intuition of how to use Composer, what you can do with it. I wanted to say a few things about the state of Composer as I mentioned at the beginning it's only a year old. We've had a huge amount of contributions though. This is actually a bit outdated I think at the screen share but there are a couple of hundred contributors in GitHub at this point I think, so we've gotten a lot of feedback, a lot of help with this which has allowed the project to grow this quickly and to be actually very usable already after such a short time frame. Packages already has over 2,700 PHP packages so that's a really large amount of libraries and whatever kind of problem you come up with there's most likely already a package there that solves exactly this problem. So the next time you're working on something and you think you've got some new problem that you need to write some new code for maybe first check there, there isn't already a library which does exactly that for you. We've just released an alpha five version so again it's still pretty young. At the same time it doesn't matter that much that it's still an alpha release officially because it's a tool that you use during development. So it's not something that could crash your site which is running or deployed. It's solely used during development where basically the code doesn't run once the site is deployed. There are a bunch of frameworks and libraries that have started to officially support Composer which has helped adoption of spreading this. Like I mentioned, Symphony does send framework too. It uses this as the recommended standard way of installing any Zen framework to project. There are a lot of other ones that I don't remember just now. I don't know, libraries, even the Facebook PHP SDK is using this, it's got a lot of adoption at really quick time. And there's also some support by platform as a service providers. I think Pagoda Box, which is one, they actually automatically run Composer for you when you deploy something on there. Some of these PHP hosting services do this automatically now. And I've seen at first a few attempts using Composer as a library to install plugins in an application which is what we originally started with to install the plugins in PHP. Right. There are a few things we still wanna do which are preventing us from having a better at this point. We do wanna improve support for web assets through like JavaScript files, image files a bit because at the moment they all end up in the same vendor directory. But typically you have some other directory where you keep all of these or if you wanna separate your web accessible directory from the directory that all your dependencies are in. This is maybe not something that happens in Drupal all that often. But it's the same thing with PHPB so we don't have this problem either. But it's something that we wanna work on so that you can actually collect all of these files from all the different dependencies that you have and keep them in a single directory. There are just various cleanup things to do, small missing features. But that's why we're grateful that we have so many contributors now because they've helped us identify all of these strange bugs that we ourselves wouldn't have come across with certain proxy configurations and all these kinds of things that you have to consider with a tool like this. So we're hoping to release better in September and we're definitely hoping for some more help. So go to GitHub where all of our code is hosted if you wanna help us out. All right, so that's Composer. Let's talk about how Composer is already being used and Drupal can be used in Drupal and might be used in the future. First off in Drupal 8 which is now using, well we're potentially using twig for example which is now using some symphony components. Composer is used to install these. There's a Composer.json file which unlike what I explained earlier is located in the core directory rather than at the top level of the repository. And these dependencies after insulation are committed into the repository. So for any regular developer this doesn't make a difference because they don't have to run Composer install to get these dependencies because they're already in the repository and you can just start working on this. So it's certainly an upside that people contributing to some other part of Drupal don't have to even think about this. I'll come to what I think is not that great about it later but for now that's kind of how to play. The other thing that already works is a Composer module for Drush. Is it Drush or Drush? Drush I guess, right? Which basically is just a wrapper on Composer for Drush which has a few nice features. For example, if you use Drush to install some module that module can actually contain a Composer.json file for dependencies which will then automatically be installed when you run Drush.gl. So in this example there's a module called less. Less is this language for CSS well defining CSS in a slightly nicer way which you can compile into CSS. And there's a library called lethal less PHP which is the implementation of less in PHP. And the less module simply uses this library to provide this functionality as a Drupal module. To make this a bit nicer, there's another module, the Drupal Composer autoload module which basically makes sure that these generated autoload files within each module then get loaded when you're running Drupal so that again like I explained earlier you don't have to bother with loading the code from the right directories. Or having to instantiate the autoloading for each module yourself when you need it. All right, so we'll get into the discussion after this. So basically what I'm trying to say is you should rather than always writing code yourself maybe you're using libraries but you're at least always building a set of modules for your project that might contain some functionality in these modules which is really something that could be shared with other PHP projects which aren't necessarily Drupal projects. So this kind of sharing across projects becomes a lot easier if you separate your code into these kind of single purpose small libraries which you can easily publish on something like packages and then you can find these from other people as well as make them available to others and get help from people even outside of the Drupal community because these small and reusable libraries which you then wrap in modules for example are just as useful to people who are not using Drupal but PHP. All right, these are adventure links if you want to learn more about this. There's also help in a Google group if you just have questions after or on a pre-note IRC, there are always people around to help if you want to try some of these things. And at this point I really want to start the conversation part of this. There are a few items in here that I put there which are I think points that we should discuss. Like I mentioned the composer JSON of Drupal is currently located in the sub-directory which means that if you for example have a Drupal project and you want to have your own dependencies you will have to separately install them or you will have to modify this composer JSON in the core directory which is really not a great idea. And there's no simple way to just add dependencies to the system that already exists but you're kind of always stuck separately installing your own dependencies from the ones that Drupal core itself has which can actually result in problems if you end up with different versions which is kind of what composer is trying to solve by having a single way of installing these dependencies for the entire project. We've seen that you can install modules and with their dependencies however currently with this Dresch tool that I showed these are installed per module. So you have three different modules that all have the same dependency. You actually get this dependency three times which again it's typically not a problem as long as they're in the same version but it's some amount of overhead because you've got the files lying around and the auto loader would be instantiated three times. It wouldn't actually get loaded multiple times so it's not, it still works, it's just a bit overhead. And then something else, a discussion that actually came up on Drupal.org somewhere at some point is whether or not one should simply use composer.json files instead of the module info files. And I'll actually point out another one that says you can actually have both like in this example and not have too much of a problem. And something that I would like to suggest is that Drupal core itself might actually become a dependency of a Drupal project so that you have a composer.json on your project level which defines the Drupal core as a single dependency which is then installed where if you ship basically what you would ship for a new user is a version of this skeleton project including the already installed Drupal core. But what this allows you to do is have a composer.json on the root level where you for your project define all your dependencies that you yourself wanna have. At the same time, Drupal core is a separate package with its own composer.json where it itself can define all the dependencies it has. And composer then takes care of making sure that the versions match each other and that you can correctly install dependencies that you have on your level as well as that are compatible with what Drupal core supports. All right. Questions, ideas. Anyone wanna get started? I'm right here. There's, yeah, there's a microphone in front. Everyone over at home for this? I mean, I have more stuff if I should just keep going but I figure at this point there were probably some questions or there's, yeah, comments, like really conversation, yes? So you don't have to ask me something, you can just tell each other something that you think about this if you want to. Come up to the front, just line up for the microphone so we can record the sound. So I'm pretty sure the only reason that composer.json was put in the core sub-directory was out of some kind of fear that if it was in the top level directory people might think it's something that they can mess with. And but I mean, I don't think that justifies going against the convention. Is there already an issue to move it? I don't know what the problem currently is but that actually doesn't make sense because you do get into trouble if people start editing this file. That's why I'm suggesting that you actually have Drupal core as a dependency because if you wanna update from one Drupal version to another that typically involves changing the version of these dependencies, right? So in an update of Drupal this file might be modified during the update. So if someone goes and modifies it themselves you then kind of end up with the situation where there might be some merge conflict between the two and it gets really tricky. So I do understand why it was put in the core direct and this actually to a degree makes sense. I mean people might still go and edit it but I think the clean solution to this is to simply separate the two and have a composer.json for the Drupal dependencies and consider like the Drupal core a separate package and to have a composer.json which requires the separate package which is made simply for users to mess with, right? So where they can define their own parts. So I think that the correct way is to simply split this up into two parts. So I see two directions that one is you have a module that is only made for Drupal and it knows about the surrounding and you can make a lot of assumptions that things are there. And the other thing is that you have a package that only works, that works in Composer and that is meant to work everywhere. It just has to be PHP. And so one direction would be that you still make your modules that gets a lot easier because you don't have to be so general. Or the other thing that would be modules become components and then they have to be, they have to work everywhere. It's like what now is a Drupal module would turn into a component and then you can use it everywhere. So you also have higher cost at making it work everywhere. So the first thing is that a composer project doesn't have to work everywhere or a composer package, right? So you can just have a Drupal module be a composer package which only works within Drupal. There's actually something, I think I kind of skipped over this earlier when we looked at the definition here. It's actually type in the second line. So I mentioned these custom installers that automatically install, for example, Drupal modules into the right directory. And that you would define a composer JSON for a Drupal module that only works within Drupal is by selecting a type like Drupal module. And then these installers actually install this in a different way and within such a package you can just make all these assumptions. So for one thing, these two don't end separate. And the other thing is it's extremely useful to separate these, to have a module that wraps a generic purpose PHP library. But at the same time, that doesn't mean you always have to do that. In simple situations, it might just be faster and easier not to do that. I was just hoping to encourage you to think about whether that might make sense, whether there might not be some benefit from doing that. So with this type thing, you could actually have some of the rest for free, like it would be implicit, like the autoload would always use a Drupal standard outloading, whatever that is. The autoload you can really skip over. Just to know where the folder is and everything. Yeah, you can actually, the autoload might be an exception to that. But in general, that should sort of work. There's another thing to mention here is that you actually have an extra key which I haven't shown here, which allows you to just define any kind of data that this custom installer might need. So if you want to add some kind of metadata that is only useful to Drupal modules, then you can just add this to this file and it'll be available to this installer. I'll have to get back to you whether you can skip the autoload part automatically. Okay, the other thing would be if you separated the module and the Composer packages, would you already ship that in the module or would you have to some separate download thing? Basically you can do both. You can basically use Composer to install this and you can ship a version where this is already installed, which just allows users to then update this dependency themselves maybe to replace it or something. But it still doesn't require that users necessarily use Composer to install the dependencies. So I would typically recommend that you have some kind of built process where you run Composer, install once, and then you package this up and ship it. Okay, yeah, so do you know what Drush make? Sorry? Drush make. I do not know, that's why I'm sorry. Sure, no, no, that's fine. So we actually go to the last page. Yeah, sure, that's fine. I'm not even sure why I'm asking that now, but because you had a nice list of, oh no, I thought the very last one, you had this list of things that we could consider. Oh, yeah, yeah, yeah, yeah. I'll figure something. Right, okay, yeah. So, yeah, using Composer to install modules and this notion that you're talking about where we have core as a dependency essentially that's pulled in, that's the same principle that drives Drush make files, which are basically manifests that, and then Drush make runs, it finds everything, it downloads it, it drops it into place, it applies patches if necessary. So it's same principle, way less flexible in terms of the actual number of things that you can do and Drush make typically cannot be rerun in order to update dependency, it doesn't have a lot of concepts. So there are a number of substantive additions in Composer that I would say are improvements over Drush make, but essentially it's the same concept. So the community at large is not at all, the notion of treating core as a dependency is not at all foreign and maybe if people were hearing him say that and scratching their heads, like can we really do that? I just think, we've been doing this for years, this is Drush make, this is not crazy. It's good that you say that, because I don't know, right? No, no, no, for sure, for sure, and because I was trying to process it for a second, I'm like wait, no, this is exactly what we do with Drush make, it's the same idea, where what you start out with is basically a manifest file of some kind, and then it just assembles everything together. I've actually got a, yeah, another slide here, actually let's just start with this one, which is kind of how you could imagine this working. So you can use this create project thing to create the basic composer JSON, which just has core as a dependency, which you see down there, and it would generate something like default directory structure, as well as a company code, is that exactly what you were saying? Yeah, the place where it is actually maybe a little trickier is the versioning, which we barely talked about at all, but we are nowhere close to semantic versioning in Drupal. I realize. Yeah, and there are some actually, I've gone back and forth on this, Pat should get up and make his points about this as well, but given the fact that we include the major version that our software itself, that our module is compatible with in its versioning string, it's not a bad thing. No, the thing is that becomes entirely unnecessary with this, because what you do is you always define Drupal core as a dependency of your individual package, and through that you define exactly for which version you're compatible with, and you're still free to, for example, if you have your listing of modules on Drupal.org, to just display them the same way. The only necessity really is that Composer needs to be able to compare version strings for a larger, less-done, which becomes really tricky with this kind of setup, because there's a lot of logic in having it constructed like that. Yeah, yeah. Potentially, we can actually make that work, although I'm not entirely sure how. Yeah, I was gonna ask is the mechanism for doing version comparison pluggable, would we be able to write a Drupal-specific version comparator that just plugs in? Again, Composer itself is just one big library, which all uses dependency injection, so for example, you could just build your own wrapper around this, just load all the regular code, and just replace one of these parts of it, or maybe even better, you can just modify Composer to support this if it detects that format of a version string, because there's really no downside to supporting this in general, I think. As long as we can separate them from the other kinds of version numbers. Yeah, yeah, there's, I'm still, I haven't been able to wrap my head around the idea that, because I get that it becomes implicit and the major version number could just go away, and maybe it's just that I've been saying like 7.x, 2.x in my head for so many years that I can't suck it out. I think it might be that, yes. It might be, it might be, and I'm trying to get myself there, but. It's a very important piece of information, but as long as you make sure that you always require the Drupal version that you depend on, you at least, like functionality-wise, you don't lose anything. Right, and that's actually, and I'm clear on the technical side of it, the part that I'm sticking on is whether or not there's some important part of the fact that we include the major version string for people who are choosing their modules. Right, so I think to some degree, it's really a matter of displaying this kind of information in the right places. Yeah. So there's a lot, you can basically, I would imagine that Drupal, if they went to do this, would have something like packages, but integrated Drupal.org, which has the same functionality, and it would just focus more on which version of Drupal does this module actually work with, and display it in a very similar way to how it does it now. And I think that would cover most of that. Yeah, the other note to just quickly make is that we do, as part of our packaging process, we inject a license file. So we do have to do slightly more than just say, tag it and push it. We actually have to say, we're gonna roll the release on this because we do inject a license file. You do not have the license file in the repository? No, we do not. It's not directly included in the repository. I mean, that has its own whole long history, but it seems like a really strange thing to do, because you do want to verge on the license information with the kind of like... Well, no, that's the thing. In order to push to get on Drupal.org, you have to click something that says anything I put on here is GPL. So by putting it on there in the first place means it has to be GPL. So if you put your own license file in, then you actually kind of can't do that because by putting the code there in the first place, it was GPL. So what we're doing is when, at packaging time, we are enforcing the contract that we set up when you agreed to get access to version control in the first place. So again, I would imagine that Drupal would not necessarily use packages directly. Yeah, sure, sure. So for example, then framework does something very similar where they have a different packaging process and they just run their own repository. Right. And it's really not very difficult to set this out where you just package, like you would just continue to have some kind of packaging process and all you need to do is in the end generate this kind of metadata that composer can then use to find these packages. Right, yeah, and I don't think that's impossible, although the one piece in here that I've still missed is where is the information kept about what a vendor source actually is? So how does Packager figure out that Drupal should be looking at something like for Drupal slash core. It looks at vendor as Drupal, so it knows that it's looking for something on Drupal.org? So this is really just a string, like it doesn't separate the Drupal and core. So it's just a Drupal core string and it just goes through all the repositories that you've loaded and looks for the string and it'll return all the packages and all the repositories that you've loaded that are named this way. And then it'll match the version constraint that you defined to only get the packages in that version that you wanted to use and then it will install that. All right, I should sit down so that you can ask questions. There's plenty of time. So I agree with you completely about it'd be better to not check the symphony components into components into core directly. The primary reason that we still can't move to that at this point is that since we're chasing dev versions of symphony right now and of Twig and everything else, that means it's quite possible that someone working locally will have a branch of Drupal core that they're working on and another branch of Drupal core that they're working on that require different versions of some library. And that's easy to switch. If you remember that command and if you have a network connection at the time, given how much Drupal code is written on airplanes, that second one might be a challenge. I'm not sure though, but even in that situation with the current one, you still need these different versions of symphony present, right? So I'm thinking of this branch, we have symphony to master these three libraries from this date and this branch, we have the same libraries from three months later. Right, yeah. And if I'm working on this branch what you're saying is you don't actually have these installed at the moment, but on the plane you switch to the other branch and you want to then install these dependencies. Right, so it's other dependencies. And that's not an uncommon workflow for Drupal core dev. So how would we go about solving that problem so that we could do that? So one thing is that composer might start solving that somewhat because there's a proposal for having local caching. So then on the plane, as long as you've at least once checked out this branch before and downloaded this version, it would have these cached locally. So if you just ran composer install in a different branch, it will realize, oh, these are the versions and see, oh, I've got these locally so I can just move them back. So I think that kind of local caching would solve this problem. To a large extent, not completely, but to a large extent. Yeah, but. Okay, do you have any idea how far along that concept is? There's a pull request open for it, I think. So that kind of needs reviewing on a few minor changes, but it's basically there. Post a link on the project or on the presentation note on the site after I can try and harass people to review that. Yeah, okay. Thank you. All right, more questions. In that case, I've got more slides. Yeah, so we talked about this. And just to complete that, that would basically be the update process for if you wanted to update Drupal. You basically just replace the version of Drupal Core that you wanna now work with. You can maybe leave the modules that you're using in the same version and either the composer's gonna complain when you try to update and say, well, this isn't actually compatible and you might have to update one of the modules as well. But it basically takes care of figuring out whether these things are compatible or not. And again, you can wrap this in some kind of Drupal tool so you don't necessarily need to use Composer SS. Another point on that following up on what he was talking about. One of the side effects of our current versioning approach is that some modules will reset their version between Drupal major versions. So as an example, the Organic Groups module got up to a version like eight something for Drupal five and then dropped back to 1.0 for Drupal six, which works totally fine in our current model. I suspect here would blow up horribly. Again, it would just be having the version comparison work on that kind of version numbers, which is not that big a deal. That argument could certainly be made. No, but there's some conversation. If you all haven't read it, Tom Preston Warner is the one who originally authored the semantic versioning proposal. It makes sense. And when I say that Organic Groups did it wrong, I mean they did it in a way that wasn't very semantic and didn't make sense. And really I have yet the only case that I have seen which doesn't entirely fit the semantic versioning proposal, which is a well thought out proposal and really all of us can pretty much follow me. Now the colonel's doing it. Is this case where we want to express some like external dependency information with our leading digit, which is not strictly something that we ought to have to do. That information could be recorded somewhere else and I agree with you on that. It's really just the sort of can we make the shift? Can all of our in for shift and can all of the expectations that everyone has around it shift easily enough? Which ultimately is a question of how centrally controlled can we make the process of managing that shift so that we don't have people in different weird and consistent version states for some unknown period of time? But yeah, the OG did it wrong. Is Moj in here? No, good, so I... Maybe? He agrees that he did it wrong. He didn't realize that that was what the major numbering versions were for and you increment major numbers at certain different types of things. The fact that we reset when we actually change across major version numbers is itself a little bit of an oddity but we have an issue inside of the community right now where it's like, so for example, I can talk about panels two or panels three and those exist across different Drupal major versions and we maintained our major version numbering because those actually represent different applications and have significantly different functionality but so it made sense that we retained that there was never a 7.x, 1.x or 2.x whatever branch for panels because this is panels three and that's what that major version number represents. A lot of the time when we do, when there's sort of this resetting of major version numbering, I think it's because we're not following a very disciplined numbering schema in the first place and switching to the system would actually encourage us to be more proactive in thinking about that which I think would be beneficial. Highlighting that we would have to make that shift. We would definitely, yeah and that is one of the things that would make this shift difficult is there's a lot of retraining involved that people think about it but yeah. Someone besides us get up here and talk a lot. Anyone? Well I can show you a few more cool things you can do with Composer which is like the thing I've always got leftover for the end which is, it goes a bit beyond the simple examples I gave earlier. I kind of touched on this with the example of you have some peer package that has a bug and you want to edit it and that doesn't really work all that well. So in this example where I think basically you're working on some kind of symphony project and you have a dependency on this monologue bundle for symphony which itself has a dependency on the monologue library, right? So you've got this kind of your package, the bundle and the library itself. It's like three items, kind of a chain of dependencies and this library has this, it's missing this essential feature for me. So I'm gonna fork this library, add this feature that I really want to have and then I want to keep using this. Now I've got this other package in the middle, right? Because I wasn't originally using the library directly but I'm using this bundle thing in the middle as a dependency and this bundle itself says it depends on a particular version of my library which I just forked and made a new version of. So aliasing in Composer is a way for you to define that some branch that you created, in this case it's actually Git branch which is automatically mapped to a version that you can use that my feature branch and you can pretend that this is version 1.1.0 of the same package so that the monologue bundle which depends on 1.1.0 can still correctly install because it thinks its dependency is satisfied because you aliased your own new feature branch version as the version it was expecting. So through these aliasing things it's very useful if you're working on a project or on multiple parts libraries of a project so that you have development versions of various parts and you wanna try this out while you're working on individual parts and this allows you to very easily use these branches without having to edit all the dependencies just to try this out, all right? Yeah, that's actually kind of a neat feature I didn't really mention before. As I said, for example, packages automatically extracts all the version information from tags from something like GitHub or your SVN repository and it also automatically creates a depth version for each branch that exists so that you can always specify kind of this running target as a dependency for your project which is useful in development. Mike? This is me thumbing my nose at other people on infra. What mechanism do you use for generating those? Because you have an actual tar ball that can be pulled down, whatever, right? So how do you pull the external repository to determine that a chain has been made on a branch and that you should repackage the branch tip? So the package, well, basically at the moment we're relying on whichever repository there is to provide us with the package version already so this works perfectly if you're using GitHub. I see. Because GitHub automatically provides you, yeah. We're not hosting any code. So if you're using something like an SVN repository at the moment, it'll always install an actual checkout of the SVN repository because there aren't any zip files for those. So it's okay. We are planning on adding that to packages. We can talk about that later. It's just one of these things nobody's done yet. No, for sure. We can talk about the approach to that later because we've talked about it a lot on our infraside and I can tell you what not to do. Right. All right. Yeah, any more questions? Anyone? All right, I think in that case, thank you all for listening and participating. Yeah, and I'm still here for the rest of the day and tomorrow if you want to, if you think of something that you want to know about Composer that you want to talk about, please talk to me. I'm happy to have any conversation whatsoever about this project.