 Thanks for coming. I hope you're having a great Drupal con so far Unfortunately due to covert regulations and me feeling under the weather. I can't join you in person Thankfully the organization was able to get me up on screen, which I'm super happy about And also they allow me to attempt virtually to help answer any questions through that with the live Q&A So first things first, let me introduce myself. My name is Ronald. I'm a lead developer at open social Which you might know from our Drupal distribution I'm working with Drupal now for over 10 years And I would love to share some of our newfound knowledge with you all in this case It even has a fancy working title project cable car First I would love to set the stage by giving you some background information some necessary context for the what and why of this all We will obviously dive into the military monoripositories. What are they? What's the differences? Our pain points and what made us reconsider our approach and give you some of the learnings we got along the way For those of you who are not familiar with open social, we build community engagement platforms Basically digital spaces that empower your members to connect share knowledge and spread ideas and Open social is an open core product. It all starts with an open source group of distribution Which comes with all of the essentials you need for your community platform This is available for you on Drupal.org or through github where we do our active development as it is open core We also offer a range of proprietary extensions as we say at open social these turn your platform into a powerful purpose-built tool We also do custom enterprise work on top of that whether that's an integration with the one-off custom tool You're using or change requests tailoring the community to your needs in the end It's all built on top of the open source distribution So basically that means you could see our open core and SAV model as a layered sphere At its core we have the Drupal distribution basically everybody gets this it's always there On top of that we have a default setup as I like to call it It's basically configuration and modules permissions, etc. There's an extra layer that all of our customers get For example, our customers don't really need all of the capabilities that a site builder in our open core version might need Then the next layer is the extensions marketplace It's basically there for our customers to enhance the default product with extensions and tailor the community to To their needs and make sure that we solve their use cases and last but not least We also do customizations and integrations and if you combine all of that you basically have open socials flexible customizable offering So with that context in mind getting back on track Multi-ammono repositories, what are they? Both mono and multi repositories have one thing in common They are a strategy to host and maintain your code base Where a mono repository uses a single repository to host all of the code for multiple projects Multi repository has those projects in entirely separate repositories whether that's BitBucket, GitLab, GitHub Separate so at first glance the difference might not feel like a big deal But the decision you're making will very likely have quite the impact on your development workflow So let's take a little deeper look at these two a Multi-rapo was the standard way of developing applications within open social and probably in a lot of companies You know a repository for each application or project or product logically split into its own project makes sense It also gave a lot of autonomy right teams could make their own decisions on what they needed to solve a specific use case within that repository And it also came natural to us You know put a module or set of modules together solve a certain use case get that in one repository It basically screams what we do on Drupal.org and be mindful It also has a lot of benefits regardless of what we're talking about today that we went from multi to mono There are a lot of benefits to this For example the performance. It's a very small chunk of code that you're working with so it's easier to maintain performance within an IDE much faster It's quicker to iterate You know from product owner or customer request to adding that value into your project You don't have to really worry about the rest or any potential third-party library conflicts You don't know about it's all external contracting that you have to deal with So any conflicts for that matter That are now isolated in your code instead of breaking down or grinding development to a halt Basically, you isolate it into that repository and it only that repository breaks and not everything else Also being able to tag individual versions for a project Made you able to ship value fast right once it's merged your work. It's tagged and released You can basically deliver that value to your customer by that release Instead of needing to wait for the rest of the product. You can just release it at its own fast pace And of course last but not least as you're working per repository. You can also easily revoke or add access To the repositories and to those working on that specific package So that's not really bad. Is it? Now let's take a look at the mono repository on the other side of the spectrum It's basically set up to increase visibility and transparency You're working in one repository together so you can see everything that's going on from other teams everybody working In that repository it basically by nature promotes collaboration You also have the single source of truth Everything in the main branch should work right? Everybody's working from the main branch and it should work And you're releasing from the main branch as well Every project or package in that mono repository all very likely gets the same version once released So you have a single source of truth one commit tagged with the version that you can go and look back to So you can see what's going on if something breaks And if you're working in one repository for your projects, you can just Set up your code quality standard for that repository and it will work and run for all of the projects inside of it So you don't need to worry about having to update quality standard checks within every single project on its own your developers basically gets very Standardized consistent experience and imagine having to refactor a big chunk of code across multiple projects Potentially opening up five pull requests or more over Different projects having to worry about administration and planning and releasing Now every project is in one repository So you don't need five different pull requests You can just fix it in one and make sure that you add the impact for the entire mono ripple Ultimately, it also helps with code sharing and dependency management, right? You have all of the dependencies in your mono ripple So you can ensure that if you update in dependency you update it for all of the projects living inside of mono ripple Or make sure that you can easily share code between projects So I guess you can say that both of these strategies have quite some pros and with that also some cons Now let's take a look at how that plays into our product at OpenSocial In order to maintain these layers of flexibility from a code repository strategy perspective We need to dive a little bit deeper than this image We'll tell you because what this image doesn't really show is what a typical customer from a code dependency management perspective Our development perspective looks like and how our teams can work with that So let's take that towards a typical customer in OpenSocial like I said everybody gets a distribution So we have a we have a version of the distribution installed Default configuration on top and a given set of extensions in this case just five, but it could be any number of extensions All of these extensions live in their own happy little private repository And then using Composer and in our case private packages We can install update downloads the dependencies as necessary for a specific version All good. It works Imagine there's a new customer customer B. They've almost got the same set of extensions However, they also requested for some custom celebration notification based on a user's birthday and who doesn't look Who doesn't like a good birthday party? In our case the case of OpenSocial that might mean that a developer First need to enhance the distribution potentially with a new team component The distribution is a new minor version Then the developer needs to update the birthday extension with a new functionality Potentially it's not backwards compatible So they need to release a new major version and finally there's some customized work for that customer Perfect. So we solve their request and we've added the value and we even enhance the product at the same time So why is this an issue if you look closely to customer a They have the officer 65 integration enabled and unfortunately the work done in the birthday extension Breaks the sink between office and open social the team working on the value for the customer Missed that they weren't aware of the integration in their workings And now basically customer a is left behind and additional work needs to be done before they can update to the same version But who does this where do we plan this? Is it the problem of the initial development or is it a product problem probably the letter, but how do we deal with that? Basically after years of scaling and growing into that situation as we outlined just now We we've noticed quite some pain points coming from this and I've noticed three down for you to try and summarize them First development happened within an extension While that makes the iterations faster and easier it also means that the impact is noticed in a later stage You know, it happens on the deploy And for the range of the entire products any potential interoperability Between the extensions or dependencies that needed to pay it were also only noted at the later stage This basically led to estimations missing out complexity Quality issues because we needed to rush compatibility work in And and with these amount of extensions There's basically no real overview or ownership on Who is going to fix it how to deal with that how to make sure that we know beforehand what's going on? Obviously there are contracts, right? So we tagged it with a major version that will tell that there's something breaking potentially But it's still very hard to make sure that you tell this and then still your own product is suffering from it at a later stage Obviously we grew to over 100 extensions that by itself is already a challenge Not all of these extensions are sales related. Some of them are enablers or helpers or rappers but you can imagine that these extensions themselves have dependencies and You can imagine the pain coming out of that, you know, just look at that image that you see on the right It's a dependency wheel of all dependencies of our extensions and Our distribution so basically all of the dependencies that we use if a customer would enable everything But they wouldn't enable everything likely they would enable a subset So it's really difficult to test that combine that make sure that Everything is taking into account And we grew into this dependency hell so to speak with none of the extensions really knowing or understanding What the other extension does and it wasn't necessary in a multi repository, right? And last but not least Was also good to notice that the level of flexibility that we offer with the customer request the customizations The deadlines that connect to that It kind of demanded sometimes to update individual customers Which led to these version differences on different customer platforms Which ultimately then we're led then led to support unscalability issues, you know, if a ticket came in It was really hard to understand The root cause analysis Or actually perform the root cause analysis because it could be on a different version It could be that it's fixed in a newer version The customer just needed an update but the update couldn't go through because another extension was missing Compatibility so we kind of put ourselves into this or we grew ourselves into this big pain pot So we needed to fix this so this year after Drupal con North America in Portland We were working remote with a couple of colleagues and myself in San Francisco And we were discussing these pain points and we came up with project cable car the working title and named after the famous cable cars riding around in San Francisco and We set ourselves out to fix these pain points So what did we want to achieve? One of the bigger things is we wanted to get ownership We are all working on open social as a product even though it's an extensions or an open source But it felt like we were all siloed and focused on solving a specific need instead of enhancing the product We also needed more productivity and quality focus as I'd like to say and and reduce the amount of production issues coming out of this big pain point So with more structure for releases towards customers Even though that means sometimes they have to wait a little longer as the development It's not necessarily going slower, but it's released in a slower pace towards them. That would give us more control We would gain back control on our predictability there There would also be no reason for us anymore to have our customers out of sync Know with all of these variations and extensions we offered the quick cycles of adding value a lot of the work Came together for us on a deep line, which is quite right on this image that you see And it's we noticed that it's way too late for us. We wanted to shift that left, right? More to the left more to the early stages the earlier that we encountered the impact or the potential problems that Less costly it is for us to solve and a monorepel would help us with this With a monorepel we would at least also know everything and everyone is on the same version from the same source giving us more control not per se in reducing the amount of production issue but control in Either helping out with the root cause analysis because everybody will have the same version So we don't really need that administration layer of checking which version and if it's already fixed Or prioritize the tickets because we know that there's made potentially five people with the same issue or Write tests to make sure that it never happens again because now we have the opportunity where extensions are Running next to each other in the same source so we can have them test testing together basically So ultimately this monorepel will give us the tools to set ourselves up for success And that's something that resonated the most with me personally Is that with the monorepel we can also focus a lot of our efforts on tooling and and do even more than what we're just thinking about You know, we just started this So how did we do this? Actually going from a multi to a monorepel is a matter of moving all of your projects into a single repository Right, that's that's the only thing So we've created the best script based on all of the extensions that we have and the magic Is just native git So I think the most magical is the git retreat and it allows you to merge information into an existing index Giving all files a prefixed file path for the merge Fancy words for it allows you to merge a project into a new monorepository Specify a new path for its files and make sure that the history and the files reflected new path correctly in our case That became html modules extensions also allowed us to do some cleanup You know, we don't need the big bucket pipelines or get the actions per extension It's just now in a monorepel. So we only needed once and we could clean it up And we have our first monorepel release, you know at the time of writing the proposal for triple con Prague We were barely starting on a proof of concept and now we have our first release So we don't really have the results yet of how this is going But at least we want to share with you some of the lessons that we've learned along the way Luckily, we also didn't have to do everything from scratch So I want to highlight two tools that really stood out and helped us First of all made by the creator of php rector. It's a tool called the monorepel builder It helps you in all of the aspects of maintaining a monorepo from releasing to migration to dependency management to Automation It's all symphony based so you can plug and play away with all of the knowledge Your organization might already have with working with Drupal and symphony It's really key for for our work And we didn't really discuss how we install our dependencies at open social on our customer sites It's not necessarily something for the monorepo debates But I do want to bring this to the presentation as a tip for you to work with private packages It's by the creators of composer And it allowed us to easily switch From serving our projects from the multi repo towards the monorepo and even have to have a way to support either or so We could have a customer still on the monorepo while working Still on the multi repository while already releasing others on the monorepo And it also ensured we could roll back easily and make that switch But I think the biggest lesson we've learned and are still learning is the new mental model Because our entry point of development was usually always one extension Either a new or an existing one and from there we could work our way through the product Now our entry point is the monorepo. It's basically the entire product Even though it's a good thing and it helps us with what we want to achieve You know that mental model change is something you shouldn't underestimate And with that in mind you really need to plan this Because the technical implementation is very likely the easiest the change in development pace Migration or switch to the monorepo for your customers The planning the new mental model is likely the bigger impact of this all We don't have any data yet At least that's what we took into consideration and I feel like we did that right So ultimately is a monorepo something for you question mark Looking at facebook google netflix uber Or a little closer to home symphony. I think they all use mono repositories. So it must be the best approach, right? Hopefully by now it's clear. There is no one sites fits all answer It all depends on your requirements The best approach may well change over time even for us And I think key is to keep in mind that these strategies will ultimately change your organization or a reflection of your organization So make sure that your organization is ready for this change and it changes it for the better So before we dive into the questions I want to take a moment to bring your attention to the contribution opportunities I'm helping others greatly appreciate it and I will be super grateful if you could let me know any feedback You might have about this presentation And now it's time for questions