 This is decoupling Drupal modules into PHP libraries. I am Bon Jovanovich, Bonzi, and I am the Commerce Development Lead, which means that I am paced to work full-time on commerce to the techs, our Drupal Commerce rewrite for Drupal 8. And outside of the commerce ecosystem, I'm also responsible for view bulk operations and inline entity form. I work for a company called Commerce Guys. We are the creators of Drupal Commerce and Platform.sh, which is our cloud hosting solution. We have offices in Paris, London, and in Ann Arbor, Michigan, about 40 people growing percentage being remote, including me, I remote from Serbia. So we started Drupal Commerce when Drupal 7 was still in beta. We started it from scratch with the sole purpose of making use of the new technologies that Drupal 7 gave us at the time. And that meant using the entity API for data model, using views for all kinds of listings, including the shopping cart, including the administrative tables, and using rules for all kinds of business logic, the business logic being the kind that the merchant adds that's specific to their sites. And by doing all of this, by starting from scratch, we created a solution that was instantly recognizable. If you knew Drupal 7, then you already knew Drupal Commerce. And we also aimed for it to be minimal and to make as little assumptions as possible so that you can take the parts that you need and use it to build your site. If you don't need a cart, no problem. If you need just the product catalog or you're building a custom checkout page, all of that should be possible. And four years since the first release, the release being at DrupalCon London in 2011, we have now grown to 60,000 reported installs, not all of them alive due to the nature of Drupal's install reporting process, but it's still an impressive number and certainly the high point for e-commerce on Drupal. But in those past four years, a lot happened and the world around us changed completely. PHP 5.3 and 5.4 changed the way we write code. Composer changed the way we share code. Symphony 2 happened, Drupal 8 has almost happened, giving us a new base to build on. And if we want to repeat the same success, then we need to start once again from scratch, re-evaluating all of our ideas but staying true to our initial values. And that gives us a good opportunity to also re-evaluate our problem space. Because when you think about it, there is no such thing as the e-commerce problem, just like there is no such thing as the content management problem. Instead, it's a collection of problems, problems such as payments and addressing and taxes and pricing and catalog management and order workflows and a lot more. And when you think about it, none of these problems are Drupal specific. They're not tied to Drupal. So if they're not tied to Drupal, maybe we can try and solve them outside of Drupal and by doing that, gain additional exposure and additional users. So this is what we decided to do. Instead of starting with the Drupal code first, a year and a half ago, we decided to start with libraries first. So we would build these libraries as completely independent PHP code, not tied to any framework, and then we would build a Drupal code on top that requires those libraries, pulls them in via Composer, and then provides the rest. The first library we built, and I'm going through these libraries quickly just so that you can see an example of what I'm going to talk about, the example of these kinds of use cases. So we built the INTL library first and it was needed to solve our currency problems. So in Commerce 1.x, we hard-coded the currency list and we told the community to provide patches to keep it up to date. But currencies get, that list gets out of date pretty quickly because old currencies disappear and new currencies appear and symbols change and minor units disappear due to inflation. So what we saw is that every few months that list would be out of date. So we decided to look for a source that we could use to automatically refresh that list from time to time. And we found that in the CLDR project. The CLDR project is a Unicode project that's maintained by large companies such as Apple and Microsoft and Google, and it provides a large, completely open repository in JSON format of locale data. So it gives you lists of currencies and countries and languages and a lot more. It's completely amazing. And it even gives us the list of currency names and symbols in all languages. So we decided we wanted to use that and we decided to create a library that would pull in the data set and then provide a data model on top. We also knew that we would need locale specific currency formatting because if you have a euro amount, it does not look the same in France and Germany. So we knew that there's an INTL PHP extension that can be used for that. It has a number of formatter class, but that extension is not present by default on many installs. And even Symphony, which is much more developer oriented, did not manage to get away with actually requiring the INTL extension. People kept complaining until they added a fallback. So we knew that we couldn't do the same for commerce. We couldn't tell people go and install the extension. What we found out was that CLDR actually has all of the data for formatting currencies. So we managed to build a number of formatter in 300 lines of code. And with that last piece, we completed the INTL library. The next one is called addressing. And it's basically the library version of our address field module. So it handles storage and validation and formatting of postal addresses. Now postal addresses are different from country to country. Different countries have different address formats. They have these rules for which fields are used and in which order and how they're labeled and how they're validated. So we take these rules and we list them all in an address format. And we use that address format when formatting an address and validating an address or when building an address form. Now, when we had all that, we realized that the next step is, of course, to have a good data set, to have the data for the countries. And we used to have our own data set back in 777, but we actually managed to find an even better one using Google. And I don't mean we Googled for it, but instead Google created their own data set for Android that has address formats for 200 countries and subdivisions for more than 50. That's things such as administrative areas and municipalities. So we contacted them and got the permission to release that in an MAT licensed library. And that's how Commerce Guy's addressing was born. And it's our most popular library at the moment. The next one we created is Zone, which introduces the concept of geographical zones. So geographical groupings, a list of countries or states with or without postal codes. And that's very frequently needed when you're doing shipping or taxes, because you might say, I have one shipping price for Spain and Portugal and the other for the rest of the European Union. And it's also used for tax because tax does not apply strictly to country lines. For example, German VAT is used in Germany and a few Austrian postal codes. But Austrian VAT is used in Austria without those postal codes. So now that we have this concept of a zone, a zone entity, we can actually describe that and attach it to a tax type. And our crown jewel is the tax library, which represents more than three years of research into tax problems in e-commerce in general. It includes a data set with tax rates for all of the European Union, present and historical. And we are relying on the community to provide more for their own countries. It includes a smart data model that has support for changing tax rates. So it has a start and end date and will choose the correct rate based on the calculation date. It also has a pluggable system of resolvers that can have per country specific logic for determining which tax type and tax rates to use. So that means that we can handle things such as B2B versus B2C, physical and digital products. And we even handled the use case of digital VAT in the European Union that has been a problem this entire year, which says that if you're a Spanish shop selling to a French customer, you need to charge French VAT, which means you suddenly need to know all of the EU tax rates and the logic of how it applies. Well, we did all of this and we implemented that in a library that was never seen before. And all of that got us a lot of attention. We were trending developers for a while, which felt really good. And it allowed us to connect with a bunch of other projects. So the first thing that happened is that we managed to influence Symphony a bit with Symphony INTL. So Symphony INTL was a library where Symphony had English-specific fallbacks and after we did the CLDR data approach, they decided to do the same thing and we even managed to share tricks on how that should be parsed and used. The Symphony version of the library still doesn't have all of the parts of the data set that we do and they don't have the number format that we do, but I'm hoping that in the near future we'll actually work with them to merge the remaining parts and into the Symphony library. We also got in touch with the developers of the Celias project, which is a really promising e-commerce solution for Symphony with a really smart and enthusiastic community. And when we talked about our libraries, they decided that they would like to start using our addressing zone and text libraries for themselves. Unfortunately, that hasn't happened yet, mostly because they don't have any full-time developers, but if any of you are looking to start contributing to a Symphony-based project, I think this would be a good start. And the last part is that we have been adopted by SAS Solution, so both Foxy Cart and Conk have started using our text library to handle their own text needs. And this has been great because they've been using the text library in production for six months, which means that they found bugs such as incorrect percentages in the data set or, for example, you forgot about American companies registered in the EU to collect EU VAT, so they've been finding edge cases. And we have benefited from all of that and have a much more solid solution now, both in the library and for Drupal Commerce itself. So it has been a great ride so far. And we have more to come, so we've been thinking about interaction between taxes and discounts and shipping and payments. There are a lot of problems that could be explored there, but this time around we decided that we want them to first mature in Drupal Commerce itself, and then we will start separating more, simply put, the existing libraries took a lot more time than we originally anticipated, and that's something that I will comment on. But the main question is why would you do all of this? And my first need was to just re-examine our problem space. If you're solving addressing in a library, then you really want to know how to solve that complete problem space. You want to cover the edge cases, you don't want to take shortcuts. If you just need an address form for a large application, you are much more likely to just implement the simplest solution and continue on. So by doing this, we did end up with a much more solid implementation of the various subjects that we attacked. We also got additional exposure because people came to us, they learned about the libraries, and then we could say, you know, there's more where that came from, which is definitely good. We also got a chance to validate our implementation. So Drupal Commerce 2 is still not ready, won't have a beta for a month or two, more at least, but our addressing and text parts have been in use for many months now, which means that many bugs have already been solved. We already had the bug reports of your postal code validation is not precise enough in Taiwan or the various tax issues. All of that is already being solved, allowing us to have a really solid release on day one. And this allows us to also get development maintenance and help because it's really time consuming to maintain an up-to-date list of tax rates for all countries and a list of address formats. But if we do it at the level of the PHP community, then we can actually produce something that's much more solid. And it gives us time to focus on the more fun parts of our e-commerce projects. And finally, this allows for backports because this code is nicely isolated, which means it can be plugged into Drupal 7 and Drupal 8 and Symphony and Laravel. And wherever I go, I know that I can at least bring those parts with me and that makes me motivated to try and take out even more. The not-so-hidden cost of that is time. You know how if you build something for your own site, it's much quicker than building a generic contrib module. Building the contrib module takes a lot more time. Well, the same thing happens when you compare building just the contrib module and building the module and the library. And this time cost has three pieces. The first one is just learning how to write a good library. And when we started, there were almost no good libraries because everyone wrote libraries that were tides, specifically to Symphony or some other projects. We needed to learn a lot, but I think there are more examples now, so that cost is smaller. The second cost is, of course, the cost of evaluating and solving all of the edge cases. And that benefits us, so it's a good cost to have. And so those are some of the costs that we have. The third one is evaluating the differences between the various systems. The code that we write needs to appear native to developers of different systems. It needs to feel native to a Symphony developer and a Drupal developer and a Laravel developer. If code feels weird just because it uses an external library that we've lost already, and that's going to be refactored out at one point. So this means that we needed to evaluate all of these systems and the differences between them to make sure that we are solving the right subset of the problem that makes sense here. And it means taking into account how different these systems can be. For example, Drupal is this dynamic introspective configuration-driven system, which means that users expect a UI and a slightly different flow. But Symphony is this static code-driven system, which means that people expect to write more YAML and just don't care about configuration that much. So all of that needs to be taken into account. So let's look at how a single library looks like. We start with the interface. Let's say we want to represent a currency. So in this case, we start with a currency interface that has only getters. This is the basic contract that the currency needs to fulfill. This is what other services need from a currency. So they need a way to get the currency code, the name, the symbol. So you might wonder why I have no setters. Well, because I don't actually need to make assumptions for how that object is going to be populated. It might be a value object, or it might be an entity because users are creating currencies in the database. Or other than allowing the parent application not to have setters, it also allows them to have different setters with different type hints. For example, if the currency interface is a type hinting against the country interface, but the parent application extends the country interface, it will want to type hint against the extended interface in most cases. So in that case, the parent application can define their own extended interface with the setters and do what they need. We do provide extended entity interfaces that have the setters for projects that need it. If you know that your doctrine or Drupal site will use those exact setters, then you can go ahead and extend the more extended interface or just use it directly, but we don't actually enforce the usage. We also provide a default class, and this class is used in two cases. It's not used by Drupal or Laravel, but it's used when we load data from the data set. So from JSON that ships with the library. For example, we loaded the currency from disk, and it's used by Doctrine, because Doctrine is a system that allows you to specify a mapping separately from the entity, and you can describe how the entity is persisted and loaded, and then just reuse an existing class. So all of our classes are usable by Doctrine directly, which makes bundles a little bit smaller. Sometimes these entities have just bits of logic, so some kinds of helpers. Object-oriented purists will tell you that a totally anemic data model is usually not a good thing, and at the very least, you might have small helpers. So for example, in the address format, you might have methods that tell you which fields are actually used in the address format string or give you some other format of that. So what we did for sharing here is use a trait. So who here knows how a trait works? More than half of the room. When I did that in LA, there were five hands, so that's awesome. So a trait is basically language-assisted copy-paste. We define the trait like a class, and then we use it from the default class and the Drupal class and the Laravel class, and we share those methods, those methods that are common and have some logic. And by doing that, we can also unit test just the trait and not repeat the test on each side. So it's just a nice little trick to share these bits and pieces. And Drupal actually uses traits for the same thing in its own entity API. Now, the next thing is collections. Many entities have relationships. And for example, a blog post has comments. And Drupal will give you just an array of comments. But doctrine has this really neat trick where they actually give you a collection class. This is a class that behaves like an array, but since it's a class, it can actually lazy load the data, which means that the first time you actually want to access the comment by specific ID, it will then load it on demand. And this is not actually a part of doctrine, like the big project. It's a separate library up on GitHub that has just those two small classes, those two small collection classes. And this allows us to actually depend on the collections class and provide the required type hints and behavior in the default class. So this allows us to have the default class that's usable by doctrine directly in a way that doctrine users expect it. But at the same time, we get this really nice way of dealing with our local data as well, because for example, I use it with subdivisions. I'm not going to load the Brazilian municipalities until you request them because you might not need them in this request. So this does not affect Drupal at all, but it does provide a more flexible implementation all around. And once we have entities, we need to know how to load them, where to get them from. In Drupal 7, this would be a controller class. In Drupal 8, this would be a storage class. But the outside world usually refers to them as repositories. So you can see here a very basic interface that has two methods get and get all for loading a single address format or all of them. And this is all a service needs. It says give me an object that can give me a single address format or all of them. And then we can have implementing classes that pull the data from disk or from Drupal entities or from anything else we need. And once we have the data model, we want to do something with it. That's one of the main reasons why we created the library. We need to offer some kind of value, some kind of business logic. So this exists in classes that we can call services. And the service is just a regular PHP object that implements an interface. And it does not assume it has a dependency injection container. So it just asks for the dependencies it needs in the constructor. And then that can be passed manually or it can be wired into a symphony bundle or a Drupal module. In this case, we can see a zone matcher, which is a class that will return the best matching zone for an address. And you can see in the constructor that it needs the repository. It needs the repository because it's going to load the zones. We type in using the interface because we don't care which repository we get as long as it gives us some zones. And then below, we just use the repository, load the zones, match the address against them and return them. This is an example of a service. A service is a small class that does one thing and does it well. Validate an address, format a currency amount or determine which text type to use and so on and so on. We need to have tests. Ideally, we would have anything resembling 100% test coverage. We write tests using PHP Units the same way we do in Drupal 8. And PHP Units is great because it runs really, really fast. We basically aim to have a matching test for every class that we add. And after a while, it just becomes second nature and then we can use Travis on GitHub to run that on every commit. Let's look at some problems and how we can solve them. So the first problem that we have between applications is translatable strings. Now, I don't mean translating entities. We can always assume that entities can be multilingual. For example, a currency can have translations in multiple languages with the name and symbol in German, French, and so on. But we have a problem with a single string. Let's say that administrative area types. An administrative area could be called a state or it could be called a province or something else. And we need to localize these strings before we show them on the address form. Now, Drupal and Symphony do this differently. Symphony and Laravel, for example, will tell you to separate your strings into a separate file and give them an ID. And then in your application, you will say, give me a translated string with this ID. For example, commerce guys addressing administrative area type state. But Drupal has that T function that wants the original string as the argument. So this means that we cannot have unified translation. So what our libraries do is they reference constants. And then it's the job of the parent application to replace that constant with a translated value. And this is possible because we have a fixed set of possible values. And it would be possible to have a lot of them which would pollute the interface that we're using. So we started using this concept of an enum class. This is, other languages have enumeration. So in this case, it means that it's a class that lists the possible values. So for example, this is the list of labels a city field can have. It could be called a city, a district, a post town. And then the class has just a little bit of sugar that allows us to say, give me the default value or give me all values, which is really useful when you're building UIs. For example, the UI for defining where I think address formats. The commerce guys enum library is a hundred lines of code, but it has helped us clean up some of our other libraries. The second problem we need to think about is pluggable implementations. So let's remember how text types are resolved. We call resolvers until one of them gives us the answer. So we call the one for France, one for the EU, one for the US, and we keep calling them until we get something back. Now, the main class does this invoking. It does not care where the resolvers came from. The library does not care about discovery, and it shouldn't. That's the job of the parent system. Now, both Symphony and Drupal 8 have this concept of tagged services, and it's one of the most, one of the new major Drupal 8 extension patterns where basically in your services YAML file, you add all of the resolvers that you have, and you give them a tag, and since it's the same tag, the container can get all of those services, and then you can pass them to the central class. So that's one of the ways that it can be done. Now, Drupal 8 has one additional extension pattern, and that's plugins. A plugin is a class that gets configuration from somewhere else, usually a config entity that's edited through the UI. And here below we have an example of a plugin. This is a constraint plugin, which is used in validation. A plugin is not registered in the container. It has an annotation on top, and in this case we can see that Drupal decided to use a plugin to integrate Symphony validator. So the plugin is extending the Symphony validator library class to make it discoverable and make it work. So this is one additional pattern that might be expected from integrators if there's configuration coming from outside. The important part is knowing when to stop. Know which problems should not be solved. For example, Symphony and Drupal have wildly different form implementations. And Laravel has none, it has no official one as far as I know. So this means that it's highly impractical to provide forms in your library. You just don't want to do it, leave that to the parent application. You might be tempted to say, okay, I will build a generic form and then implement transformers that output the Symphony form or a Drupal form, but that you're once again back to weird code, code that does not feel native to the developers of either system. So just don't do things like that. I also wanted to mention a few details around the Drupal integration that we came across when doing commerce. So of course the first one is that once you need Composer libraries, Drupal modules are only installable using Composer. You cannot download the module manually using a tarball and then download the library manually to some folder and have it work. Libraries and all PHP packages in general must be installed using Composer. And if you've been following along Composer and Drupal, it has been this long battle but a constant set of improvements. And it basically means that you need to edit the root Composer JSON, it's at the root of your Drupal install and add the requirements there or use Composer as a drush make alternative and download the module. So if you do Composer require Drupal commerce since it does recursive dependency checking, it will also download all of our libraries because they are the dependencies of that package. Now I would need a whole talk just for the Composer details so you can look at a blog post that I wrote just recently detailing all of the tricky details around Composer right now. I should mention that we've had a series of talks here at the Drupal con that will make that blog post significantly smaller and less awkward. So we are definitely making progress just in time for RC1. The second thing I wanted to mention is implementing entity types. So let's say that we have a Drupal currency entity. This allows us to have a config entity where users can create a custom currency or edit the details of an existing one. This means we need to have a currency interface and interfaces can extend multiple other interfaces. In this case, we are extending the library one saying this behaves like the currency the library expects and it extends the library for a config entity saying this object behaves like a config entity should and that's all there is to it. But it's good to remember because if you implement only the library interface then you've missed the config entity one. Let's talk about library ideas. Which problem should we actually tackle now that we know how to tackle them? The first one is the set that I call hard problems. These are the problems that require domain knowledge. These are the problems that require experience. So that means that you can actually take your knowledge and turn it into code form and create immediate value. A good example of this is tax. So these are the problems that other systems frequently get wrong and e-commerce is full of such problems, taxes and addressing and so on. The other kind is boring problems and these are the problems that are not hard but they have a lot of edge cases. So an example of that is making sure that you have all of the address format information that you need and you've handled all of the quirks of Monty Negro's VAT system and so on. So sometimes the boring and hard problems overlap a bit. And in this case you really benefit from the additional attention and the additional users that you can get by releasing to the wider PHP community. And when I talk to other e-commerce maintainers, they say, yeah, we love libraries. So let's create a cart interface. The problem with that is that nobody cares about your cart interface. Why? Because someone else can create a cart interface in 10 minutes. Everyone has an opinion about data models. If your library can be reinvented in an hour, it will be reinvented in an hour because you're obviously not providing enough value. The library should be something that saves someone a week or a month of effort. And by doing that, you know that you're providing true value. And an example of a boring problem that you probably don't want to solve is, for example, logging. When was the last time you wrote a logging library? You don't. But if you use monologue, suddenly you have a hundred handlers for all kinds of backends. So this is an example of good value that you probably wouldn't reinvent. Unlike routers, for example. Everyone has their own. And now that we've seen all this, it's time to think about what it is that the troupel can give back. People say that you can find anything on packages, but there's quite a lot you cannot find on packages but things that the troupel has. And this is just my personal opinion, so I'm just throwing ideas out there. It's definitely something we should have a big discussion about. One of the examples they have is migrate. Migrate is one of my favorite projects from troupel six times even. It has a great API for loading and extracting and transforming data and performing data migrations. If we had a generic ETL library, then there would definitely be value in that. And we should look at whether we can move some parts of migrate outside of troupel. The second one is our darling views. We always say that views is something that makes troupel special. And what I've seen is that views is starting to get reinvented in the symphony community. For example, Celius now has a reports bundle and Oro platform, which is a really advanced technical platform that powers Zakenel and Oro CRM. I don't know if you've heard about them. They also have their own views thing that basically allows you to define a view in YAML and then transforms that. But it's all very primitive. It doesn't have the concept of pluggable fields and displays and all of the bells and whistles that we've accumulated over the years. So there would be value in providing some pieces of that, at least as a guide to follow. And there's more. Let's take the incredibly cool Drupal 8 render caching. So if you've seen the talk that Wim Lear's and Fabian have given, you would have seen that Drupal 8 is really smart about knowing what it can cache and for how long. Because each object that's a part of the rendering provides its own cacheability metadata and then that is used to assemble the final knowledge that's then used to cache different versions of the page or decide whether they can be used. So I don't know what can be taken out of that, but there's certainly some knowledge there that could be taken. And there are more and more examples. You can surely think of them. Modules that have been worked on and reworked for many years that have some kind of domain knowledge that can be documented and shared. So that's something to think about. Time for questions and a discussion. Let's see how much time we have. Okay, so we have five to 10 minutes. So there's this shy period every time when no one asks and then I get one question and there's five more. So let's start with that first one. Hello, so first of all, thank you very much for this talk. It's good to see people getting together. It's more of an idea rather than question because twig is something that we use a lot everywhere. And probably on that, like even twig extensions, do you have any twig extensions yourself that you maybe thought that you could share because those are in general really, really simple to share across different programs? Yeah, commerce does not define its own twig extensions. I don't know if core has something that's unshared, but I do know that people behind twig and core have always been in tight contact with the rest of the symphony community. So if there's anything shareable, I think they are probably aware of it, but I haven't looked into that myself. Yeah, it's just very simple for everybody to do. And even like in general, share these kind of simple snippets with the community at Drupal and everywhere. I will be at the commerce guys booth downstairs in the exhibition area for the rest of the day and tomorrow as well. So if you have anything else to discuss, feel free to find me. We also have this huge sprint on Friday. We will be working on Drupal commerce. I will be available to discuss libraries. So feel free to find me, get a small task maybe you want to work on or just work on anything that's correlated. We do need to get RC1 out after all. And make sure to evaluate the talk. Thank you.