 We celebrated some birthdays last month. I had a big one and square turned 12 Now you might know square as the little white card reader that plugs into cell phones that lets you take payments on the go but we've actually developed a suite of hardware solutions to help merchants take payments and A suite of software to help merchants manage their business and that software is called dashboard and This is what dashboard looks like Merchants use dashboard to keep track of their business They can see their payment activity. They can send invoices and they can run payroll Dashboard is the home base for square sellers And dashboard is an ember monolith Today i'm here to talk to you about modernizing our massive ember monolith dashboard My name is abry newton and here i am at ember conf in 2018 with tomster and zoe I have been working with ember for about five years Two years in our dashboard application and three years in a separate application That was a greenfield app and it's a little more modern because it was newer I have been participating in code upgrades and migrations and dashboard and have recently been driving engine adoption from a product team not an infrastructure team role And i'm here to convince you that you can build ambitious web applications with ember that stand the test of time Today I will convince you that with ember you can build for the long term Ember apps can be used for years almost a decade in our case This is important because time is a finite resource When we at square have done the analysis in the past a rewrite is always more expensive than the proposed upgrades By spending less time on total rewrites, we're able to spend more time Working on features for our customers and there's less chance of regression And finally when we stick with ember, we're able to build knowledge and expertise in our engineers That is long lasting rather than changing frameworks every couple of years Our engineers become fluent in ember which makes them more productive over the long term Today we'll convince you that with ember you can build resilient apps Ember apps can be used by tens of thousands of people every day and have hundreds of developers contributing because we've seen this in our app And today we'll convince you that with ember you can build apps that evolve We built an app in 2011 and now in 2021 we are using 2021 web technology in fundamentally the same app So long as you're on the standard ember path ember makes upgrading easy via documentation and semantic versioning Long-term support or LTS releases, which are versions of ember that receive security updates for an extended time after their release And code mods and lints Code mods are tools that automatically make the tedious syntax changes to your code that you would normally have to do manually Code mods can help ensure that you are using the latest patterns and platform features And lints are static code analysis tools used to flag programming errors bugs stylistic errors and suspicious constructs and they often use code mods to automatically fix violations in code But even in applications with weird wear Which i define as deprecated packages or experiments in code where we've strayed from the standard ember path those times You know when we've decided to roll our own It's possible to stay current with some effort Even in a massive application We've done this over the last 10 years Our app has a dedicated team that runs most of our upgrades which helps a lot But they do it in conjunction with product teams because they don't know how everything in the app works Um, but upgrading is possible even without a dedicated team Though it will be easier if you don't use any weird wear In a much smaller more standard app. I was a dedicated upgrade team in addition to my product feature responsibilities So this is the story of square dashboard To give you some context This is the commit visualization of the last almost eight years that dashboard has been in its own repository And to give you a sense of scale in 2020 We had an average of 140 developers contributing every quarter with an average of 1500 commits per quarter And by my count, we've had over 600 individuals contribute to this code base at one point or another in their careers at square But this isn't 100 percent my story to tell I joined in 2016 and then I switched to a non dashboard ember team for about three years and came back to dashboard in 2020 Several of my co-workers helped me write this talk and I want to thank them for that So let me take you back now to 2010 Wow, do you remember what the web looked like in 2010? I honestly didn't until I saw this screenshot At this point for square our main product was swiping a card on your phone and emailing our texting or receipt instantly In 2011 we have a slightly newer looking website. It's still very simple though You can take a payment you can see your balance and you can see when your balance will be paid out to your bank account And in november of 2011 we introduced sprout core one Which was a precursor to sprout core two, which was later renamed to ember In our ruby mono repo next to our urb templates and the commit message for this says first day of hacking And that's exactly what it was. It was basically 100 lines of script tags in 2012 We had what we considered sort of our logged in web Payment experience. It's so cute, right the buttons have shading and depth Um, this is the view layer of our original square rails app There's a simple trap of transactions and you can click to view your receipt details And you can filter your transactions by date In 2013, uh, we started to break up our web properties into separate apps Dashboard was already getting too big and slow and early code sharing via gems and get sub modules didn't work well And so we moved dashboard into its own repository And it was a rails app and we were using ember rails and we added ember version 1.0.0. rc3.4 How's that for a version number? And in 2013 our app has some tabs across the top adding more functionality as we are developing more features In 2014 we did a total redesign in css rewrite And now this is where it starts looking much more like the modern web to me And as you can see we continue to add features across the top In 2015 in january, we upgraded to ember 1.7 and for reference 1.7 was released in august of 2014 At this point in time upgrading wasn't a big priority for square and we didn't have a team primarily focused on doing it Um, this is also where we decided to start looking at our dashboard navigation project and figuring that out But other than that there is not a lot going on at the moment with the ember framework and dashboard We're busy building features. Uh, for example, we built and launched a square payroll Ember wasn't being super high maintenance and it just kind of faded into the background In 2016 in january ember is still on 1.7 and dashboard And then in june ember 1.13 lands And it was that was launched to the community in june of 2015 for about a year behind But luckily we didn't get stuck on 1.12 So when I joined in august we were on ember 1.13 and I am told I dodged a bullet because I showed up late in math Or maybe I showed up right on time In any case a lot of people had a bad experience upgrading to ember 1.13 because it was really painful And ember maintainers learned from that experience and they try not to do that anymore and they tried to make upgrades a lot easier 2016 was a year of more features Everything at this point was written in coffee script That's because when we started dashboard in 2012 javascript was a very different language You know es5 didn't exist es6 was introduced in 2015 And so coffee script was created to as an attempt to expose the good parts of javascript in a simple way And the code compiled one-to-one the equivalent javascript and there was no interpretation at runtime And so coffee script was the web the way to write web applications easily and that's what we were using But by 2016 We have es6 and so we're starting to add new code and javascript and and at the same time we were encouraging component use in ember Components had been available starting in 1.0, but we sprout core folks held on to the mvc structure a little longer And so for me in 2016 I was learning ember I was learning es6 syntax and I built this cute component that shows you your subscriptions and your monthly charges And in december we upgraded dashboard to ember 2.0, which had been released in august of 2015 That brings us to 2017 And I am dubbing 2017 the year of upgrading This is where I think our dedicated team really shines They were the ones driving this upgrade timeline. So product engineers didn't need to know how to upgrade our app But they still worked with the product teams to verify that the features were still working And so this verification process and Requiring many teams to work together means that our upgrade timeline is a little slower than you might expect So in january, we upgraded to ember 2.1 and 2.2, which had been released in the fall of 2015 In february, we got to 2.5, which was released in april of 2016 In may we got to 2.7 and 2.9, which had been released in july and october of 2016 So at this point we're about six months behind And in september we got to 2.15, which was released that month And so we're all caught up And in december we moved to an ember cli app and we had to rename hundreds of components to match ember cli conventions Which we did with a code mod And we also upgraded to ember 2.16.2, which had been released a couple months earlier Around this time coffee script was deprecated in favor of es6 and java script And so we decided to do a coffee script migration using a tool called decaffeinate Which is a code mod that converts your coffee script to java script And it was written by one of our developers and is an open source tool We had coffee and donut parties and we would be hanging out in a conference room running the code mod all together So it was like a really great bonding experience for dozens of engineers helping out with the migration But now my name is on thousands of lines of code that i didn't actually write At this point we also have a team effort to manually eliminate partials Components we're becoming and now are the preference in the writer frontend community And this was an effort to move our code to match the current ember path to make upgrading easier In august of 2017 we figured out what we were going to do with our navigation structure and we split out this navigation into a separate ember app And so now even to today we have two ember apps running on the same page And one of our developers actually updated the ember inspector to support two ember apps And that was pretty painful, but we got it working and now you can choose which ember app you want to expect inspect But this is actually a must much less active repository and actually Renovate bot which updates dependencies is contributor number two In 2017 we also created something that we called sub apps They weren't in repo add-ons, but for simplicity You can think of them as in repo add-ons and we did later convert them to be in repo add-ons And we used a code mod to move all of our files around and this decomposition led us run fewer tests in ci Which made builds faster and less prone to dealing with flaky tests, which was a good thing for developer experience It also gave us better code organization because each product team Had their own sub app that they owned But it also created a lot of weird where that we had to maintain over the years And we struggled to maintain isolation between our sub apps So when the isolation wasn't perfectly enforced Uh developers could break our main branch By changing code another sub app relied on Without knowing that the sub app was going to break because ci wouldn't run the test for that other sub app And so their their pr would pass and go green, but then the main branch would fail And we used linting to help maintain isolation, but it definitely wasn't perfect Um in any case, I think people tend to underestimate how much experimentation you can do in ember We were experimenting with our dashboard navigation and with sub apps You know at this point we have two ember apps running on the same webpage And I think that ember gives you more flexibility than you might think And you'll see that as I keep explaining the next couple of years in dashboard history But I think you also have to balance that flexibility with your tolerance for weird wear In 2018 in january We created a first version of our css library that was released as an ember add-on And we had extracted common styles and components from dashboard to be used across many of our square ember apps Uh, and then we also upgraded to ember 2 dot 18 which was released that month In april dashboard got its first engine And in august we upgraded to ember 3.1 which had originally been released in april earlier that year On 2018 we also added a package called wire js And this is based on squares open source package called wire which is designed to incorporate protocol buffers and android and java We use protobufs for a lot of our services service communication And it's often easier for developers to create apis for our dashboard front end using protos But protobufs are the worst on the front end And so wire js is what we built to make it a little bit better One of our tools one of our developers built this tool to make it easy to consume and use protobufs apis in dashboard And it also let us do a little bit of like modeling without using, you know Ember model or ember data or some other structure on the front end But this is definitely a weird wear alert for you This package hasn't been open sourced and I think we'd like to find a better solution But it hasn't been top priority But other than that in 2018 We just keep building features and we never really hit sort of a cliff of like we can't keep going because of the framework or whatever else And that momentum is important because it helps our engineers trust ember as a reliable stable framework And they continue building up their fluency with the framework's patterns making them more efficient In 2019 we rewrote the dashboard home page and we added these widgets And you'll notice that the navigation looks different We ended up writing the home page as an ember add-on which is starting to modernize our dashboard even more In march we upgraded to ember 3.2 which was originally released in july of 2018 In april we got to 3.6 which was released in december of the prior year And in august we upgraded to ember 3.12 which was released that month So we were again caught up and that is an lts version Um, we also started doing more with yarn workspaces So yarn was released by facebook in 2016 and we were actually pretty early adopters in dashboard and brought it in And then yarn workspaces were released with yarn 1.0 in 2017 and we added them to dashboards shortly thereafter But we didn't start using them earnestly until 2019 because workspaces were buggy For many of the first releases of yarn Probably until about 1.12 which was released in 2018 at the end of the year And we actually ended up contributing a fix to ember cli so that it could find ember add-ons in a workspace And what we had Experimented with and discovered was that you know in repo add-ons work and you can do that But for us workspaces were a better solution We had learned with our sub apps that when you use in repo add-ons You don't really know where the boundaries are between add-ons and the app code and between individual add-ons And yarn workspaces let us keep our add-ons in the dashboard monorepo, which improves dependency management While also delineating boundaries using standard ember add-on patterns Um, and then at this point my team restaurants started moving our sub app code into a real add-on which is an important move in towards app decomposition with engines because engines wouldn't have access to sub app code or in repo add-on code At this point we also added a react running concurrently inside of ember in dashboard There was a team that needed photo editing capabilities for a feature they were building and that allows merchants to Insert their own images into marketing emails that they send And they decided they wanted to use a package called photo editor sdk which was a react app And so we ended up packaging the react app into an ember add-on So that we could use the ember asset loader to asynchronously load and execute the entire react framework and app code And we did extra work to make sure that react would be lazy loaded Only when you visit that feature's page, so you won't always be loading react And it's important to note here though that we're not actually sharing any merchant context with the react app In 2020 we upgraded to ember 3.16, which is another lts And we opened the pr in may and unfortunately It's still open because we had too many observers in ember model that broke going trying to get to ember 1.13 But even though we're behind we're feeling okay because we're on an lts And for a little background ember model is a lightweight modeling library for ember with a limited feature set And it was released as an open source project in 2013 and we started using in dashboard around that time And its last release was in 2018 and we had continued to maintain a special square branch to patch problems as we continued to encounter them But we had ended up using ember model for several several years and in 2020 it all came to a head and we encountered this crossroads Ember model was weird where and it was breaking dashboard So like we had before almost annually we asked Is this the breaking point? Is this where we decide to toss it all and rewrite it? And if we do that should we rewrite it in react? And the answer was no we decided That though the upgrade path was costly in the short term there were so many benefits and the long term that it was worth it to migrate from ember model to ember data We'd get rid of the ember model weird where and there would be thorough better documentation and more community support for using ember data And we would also get more extensive functionality for the models that we migrated We also realized that we'd be able to upgrade ember more quickly in the future and finally Rewriting in any other framework we knew would be at least an order of magnitude more difficult than the proposed migration Although we didn't know the exact difficulty The established ember path has a lot of value and so we decided to stick with it So we started an ember model to ember data migration and to give you a sense of scale We've converted over 800 models in about 500 files off of ember model And we're well on the way. We're not done yet But i'm really excited about octane features to come that we'll be able to get once we're able to upgrade past 3.12 And to be more on like a standard ember path again and ember infrastructure So we won't have to go through this kind of painful migration again In 2014 square open sourced a package called qunit bdd which stands for business driven development for testing Which we used extensively in dashboard But in 2017 square deprecated this package recommending that folks use mocha instead for bdd style tests However, we continue to use this package into 2020 and we knew that it was weird wear And so one of our developers wrote a code mod to switch from qunit bdd to more regular qunit modules And another developer wrote a code mod to remove some custom assertions that we had built into dashboards qunit in favor of qunit dum And i really like qunit dum i had used it on my other team and was excited to have it back Um And i think it's important to recognize that ember the ember open source community has a lot to offer And i'm excited that we are going to continue incorporating more of it into dashboard In 2020 we also had a push for more engines We created a generator for conveniently setting up add-ons and engines in our dashboard monorepo My first ember i wrote my first ember engine in the monorepo And i ended up writing a lot of documentation about how to do it And since then i've taught A class on how to build engines internally and we've seen an uptick in teams interested in building their own engines And we've also started a typescript conversion Um, so you may be asking why engines? We have decided that engines are the future for square dashboard because they improve development for hundreds of our engineers They give us better code organization like sub apps did And better understanding of what product teams own which parts of code Engines are a new and for us a better isolation primitive because it makes it less likely that we're going to collide with other teams Because our engines own their namespaces Uh engines also let us share code between ember apps So we can have it running in multiple places with a single source of truth Which saves time for our developers while giving our customers a better product experience Um, ultimately we hope That we with enough lazy loading of engines that the customer experience will be faster when loading dashboard because they'll only have to Load the pieces that they need when they need them And engines and their dummy apps make iteration much much quicker for developers Running the dashboard app or test locally can take anywhere from 30 to 60 seconds to do a full rebuild after making changes Ember dummy apps and tests can build rebuild in three to five seconds and this makes developers feel more productive and it just feels better Um, and in the future we're looking forward to hopefully be able to deploy end in engines individually Instead of requiring a complete deploy of the entire model with when changes are ready to go out And this will make it easier for teams to own their production code and not be blocked by other teams And here we are in 2021 and we have come a long way So what made this possible? Think number one is all of the tooling. Um, we used code mods and lints extensively And lints were especially important because they let us Um, make sure that migrate during migrations. We aren't regressing and we aren't introducing a now bad pattern Another thing that was was a lot of time Um upgrading requires attention. We had a dedicated team that was anywhere from, you know, three to eight engineers So it wasn't too big uh to maintain our app and it has gotten smaller and more focused over time And we also have an internal community of developers that are encouraged to work on these sort of platform features inside of dashboard in addition to their product work Another thing that made this possible is ember stability, you know Upgrading and doing this over the last 10 years is part of ember's philosophy And that like this is a good idea and a good path ember doesn't say start over Which is good because we've considered a square and we can't afford to start over And upgrade paths are helpful and lts versions are, you know, the backstop when we can't upgrade quickly enough And if you stick within the standard path and patterns you can leverage ember for a long time and still have room to explore And finally the open source community Um, so ember is opinionated and the patterns are widely used which make it easy to google And easy to transfer knowledge between ember apps I left after three years and ramped up to dashboard Uh at the beginning of last year faster than my manager was expecting because I was fluent in ember Um, but also I think it's important to note that like Weirdware is part of the ember ethos and it's a sanctioned enough path that so long as you periodically revisit your decisions To see if you should change them It's okay, you know ember doesn't want stepping off the path to be a full eject And but I will say that it's the weird way that makes updating updating and maintenance difficult But upgrading and reversing weirdware decisions is possible and we are proving that even today So today, I hope I have convinced you that with ember you can build for the long term With ember you can build resilient apps and with ember you can build apps that evolve We've used ember for 10 years for our main front end project at square and we have chosen to stick with it Looking ahead We want to continue decomposing our monolith into engines and this is going to require that we finish the ember model upgrade And that we move those models then into add-ons that the engines can access And also that we move global values into services for the engines We're also looking forward to upgrading to octane and good news. You don't have to get rid of jQuery to upgrade to octane Um, but we are actually already using some octane features in dashboard despite still being on 3.12 Things like angle bracket components native class native classes and decorators the on and function modifiers and splat attributes Now currently we are not able to support glimmer components and the at-track property We're going to have to continue removing deprecations to get to 4.0 We've essentially silenced a lot of deprecations instead of doing the work to remove them so to fix them. So we'll have to do that We're going to continue upgrading our tests and tags And we're going to write more typescript And yes, we're hiring