 Hello everyone! I'm super excited to be here at EmberConf this year. I've attended this conference multiple times, but this is my first time being on this side of the microphone and I'm excited to share some of my knowledge with the community that has taught me so much over the years. First, a little about myself. My name is Joshua Lawrence and I've spent the last six years at LinkedIn as a member of our developer experience team, building internal tools to help our engineers work more efficiently. For most of that time, I had been leading a team that owns a tool called CRT, or the Centralized Release Tool, which is an Ember-based web app that our engineers use to track builds and release code at LinkedIn. When I started building CRT over five years ago, we were on Ember 1.13. This app has been through just about every change and new feature the Ember core has been through. Engines check, pods check, native classes check, and the list goes on. This app has seen it all and continued to steadily grow in size and complexity. After being around this long, I thought we could handle anything. That is, until recently, when we attempted to upgrade to Octane and realized one technical decision we had made years ago may have cost us big. I know a lot of you out there have been working with Ember for some time now and also have long-lived apps. But even if you just finished running Ember new for the first time, one thing we all have in common is we're always peering into our crystal balls trying to best predict what the future holds for our apps, for Ember, for the JavaScript ecosystem, and the web as a whole. But the truth is nobody knows where we're heading with certainty, and the decisions we make that work well today may not work so well in the future. This talk is about how to get back on track when we find ourselves coded into a corner due to unforeseeable changes in an ever-changing ecosystem. In the case of my team, the seed of our discontent was sowed years ago from an unlikely source when we decided we wanted argument type checking for our components. We were frustrated by our inability to tell which properties were arguments and what types the arguments were. Take this example. Someone on your team tells you, oh, there's a component for that already called Ember mascot. Go use that. So off you go to look at how it's being used in your app. So here's our Ember mascot component and let's see. Okay, the name argument is probably a string and the likes argument is, I don't know what. It could be a number, but perhaps it's a Boolean. It could also be an array, I suppose. And I wonder if there are any more arguments that it accepts. I guess we'll have to look at the template file. Okay, so here's the name arg we saw before, but what is like count? It uses this instead of an at, but that could just mean that it's still an argument just with a default value. I wonder if it's a number or a string or a Boolean maybe. And likes is an array that we can see here, but an array of what? Is it an object, a Boolean? I guess we'll look at the mascot like component and down the rabbit hole we go. The first solution we implemented was to explicitly define every argument in the component class and add typed information using JS doc. And that gave us a good starting point. But over time, we found that these comments would become outdated and inaccurate, which is even worse than having nothing at all. So we found a way to enforce types via an add on called ember prop types. Now this worked for us for a time, but we weren't entirely happy for various reasons, and eventually settled on a library called ember decorators slash argument, which I'll refer to simply as argument decorators. As you can see, with argument decorators implemented, all arguments are clearly defined in the components JS file and identified with the argument decorator. We also now have easy to find type information that is enforced during development. We were convinced this was the most complete solution for argument type checking and we happily implemented it in every component in our app, and all was well until ember three dot 13. Long story short, the argument decorators library was not compatible with changes made in ember three dot 13 related to glimmer component support. Even worse, there seemed to be no plans to fix the issue with the library. And that being the case, we can update our app past ember three dot 12. Our ability to upgrade ember had been jeopardized by our adoption of a pattern in library, which we implemented in the name of craftsmanship and reliability. We couldn't believe that such a well intentioned and seemingly benign decision made years ago had morphed over time to get us completely stuck. But like I said, nobody can see the future, right? So what should we do? Remove all the decorators and give up years of meticulous type definitions just so we can upgrade? Should we get everyone on the team to brute force update a huge amount of components back to the previous approach? Can we do some crazy reg X find and replace? Or what if we stay on ember three dot 12 forever? We decided to seek the wisdom of some ember masters and set up a meeting with none other than Robert Jackson and Chris Garrett of Ember fame. In the meeting, Chris introduced us to a new library called ember argument types, which would solve our problem if we can migrate to it. The only problem was that the migration would require moving all of our type definitions from our component class to the template. So why the template you ask? Well, in octane introduced the concept of template only components, which don't need to be backed by a class. So the question for argument type checkers like us becomes, so where do we type check the arguments? Chris explained that ember argument types provides a way to type check component arguments in the template via helpers and is thus the most future proof solution available. By this time we had over 400 components using argument decorators and the thought of moving them to the template was daunting to say the least. Robert Jackson suggested we use code mods to do the migration and that's where the real story begins. Now before we dive into the world of writing code mods, it is worth noting that if your app needs to migrate to newer patterns related to changes in ember itself, there are likely already code mods to handle your case in the ember code mods repo. Since our patterns were based on a third party add on, we didn't have that luxury. The good news is that when we implemented argument decorators, we treated its usage as if it were an extension of the ember core and were consistent throughout our app. This consistency throughout our code base made code mods a much more viable and useful tool. So pro tip, you can learn a lot by watching how the ember core deals with migrating code through an ever-changing ecosystem and apply similar principles to how you manage migrations with your app code. Now back to the story. At this time, I knew fairly little about code mods other than they were magic. They were written by some wizards who lived on github and the ones I had used were written to support migrating ember core features, not patterns implemented by third party libraries. So the idea of writing one was not something I had seriously considered. I did at least have a basic understanding of what code mods were, scripts that refactor your code. I also knew that js code shift was the library of choice for performing code mods. Let's take a quick look at it now. When you get started with js code shift, which I hope you do after watching this presentation, the first thing you need to be aware of is the docs are lacking. Which is to say, there aren't really docs outside of the source, so be thankful github has go to definition support. There's a great opportunity here for anyone who enjoys writing documentation to help out a broad community of front end developers get started with js code shift by contributing guides back to the project. Now regardless, you'll want to at least look at the repos readme to get your bearings and for us ember fans, you'll also want to, if you'll find it useful to browse through some of the ember code mods repos. There are also many blog posts on js code shift, most of which give basic tutorials for how to get started. Now before you start opening a bunch of tabs and dig into all this, let me give you a high level overview of how js code shift works. The basic idea is pretty straightforward. You start with a source code file as an input to a js code shift transform, which is really just a node script. The transform then uses a parser to convert the code into an abstract syntax tree or AST that allows us to select individual pieces of code and perform some sort of mutation on it. Finally, we convert the updated AST back into source code and save it to the input file. Now a good note on ASTs, if when you hear this term, you immediately think of JavaScript compilers and start to tune out, stay with me. For our purposes, all you really need to know is an AST is a tree structure, an object that describes your code. You've seen something very similar to this before, the DOM. The DOM is a representation of your HTML document that allows you to interact with it, and the AST produced by js code shift is a representation of your JavaScript code that allows you to interact with it. I'm sure you've used jQuery to select and mutate parts of the DOM before, right? Well, using js code shift, we can select and mutate parts of our AST. Makes sense? Cool. So to recap, you parse some source code into AST, make some changes to it, convert it back to source, and save it. Easy enough. But every code mod I had seen or used previously modified code inline, it didn't move it between files. We needed to take the argument type checking logic from our components js files and move them over to the template files. So here's the big question. How do we write a code mod to move logic between files? Thankfully, Robert is no stranger to code modding and had some ideas. First, rather than attempting to do everything in a single pass, we could break up the work into three separate transforms. The first transform would parse our component classes, get the type information from the argument decorators and store it in a JSON object, which we could reference in the next transform. The second transform would use the JSON from the first transform to create and add ember argument types helpers into each corresponding template file. And the third would go back to the component JavaScript files and remove all the argument decorators and their imports, which would no longer be needed. It seems so obvious in hindsight, but at the time my mind was blown, everything started to seem much less daunting and I could see a realistic path forward. Robert also introduced me to a tool called Code Mod CLI that would allow me to easily create, test and run multiple js code shift transforms from a single project. More on this later, but for now I had a plan and it was time to execute it. First thing first, because I intended to open source the code mods, should I be successful in creating them, I had to examine the argument decorators library and determine every possible usage of it. I then recorded all these possibilities in input js files, which I would use as inputs while writing the transform to ensure I covered every edge case. I also spent some time determining how I wanted to structure the JSON output and made corresponding map files that defined the expected output from running the transform on the input files. This is a pared down example of what those files look like. Next, I examined the ember argument types library and did the same exercise. Only in this case, the input was a basic handlebars template file and the output included prepended arg type helpers that perform the same logic that the argument decorators did previously. Finally, I wrote the input and output fixtures for the third transform, which simply removed any trace of argument decorators from the input js file. I now had expected inputs and expected outputs for our component js and template files, which I could use to start writing the transform logic. With this in mind, I started to write the first of the three transforms. The idea for this transform is to look at each property in the component class and determine if it has one or more decorators provided by the ember decorators library. If it does, it parses the arguments of that decorator and stores the data in our JSON object. The JSON object stores the name of the file we're working with, the list of properties that have argument decorators, the decorator names, and the arguments that are passed to them. If you're wondering how or where do you actually write the logic, so was I. Unfortunately, there are a couple of tools that are absolutely great for getting started, the most important being AST Explorer. AST Explorer gives you an environment to quickly experiment with js code shift, among other things. Think ember twiddle for js code shift. Here's how it works. The top left pane accepts source code. This is where you paste the code you wish to change. The top right pane shows an AST representation of the source code provided in the input pane. This code will change depending on which parser you use. js code shift uses a parser called recast, so make sure you have the correct settings applied. The bottom left pane is where you can write your actual transform logic that will be run by js code shift. And the bottom right pane is where you will see the result of running the transform on the input from the first pane. This is super helpful when writing traditional inline transforms because you can see the output update in real time. For our purposes, we're not actually transforming anything in our first transform, but instead generating a json object as a side effect, so the bottom right hand pane isn't really useful in this case. Also, and unfortunately, AST Explorer currently doesn't support transforms dealing with decorators, so our usage is limited to the top two panes for this case. Still, this tool is useful for inspecting the parsed AST and determining how to select the nodes you need to use. Since I couldn't write the transform logic in AST Explorer, I needed to do it locally. For this, I used the codemod CLI to generate a new codemod project. The CLI provides bootstrapping, creation of text fixtures, out-of-the-box testing, and documentation creation. It's really a nice tool and takes care of many of the concerns you'll have while writing codemods. After creating the project, I used the CLI to generate the first transform, which creates a placeholder file for your logic and a couple example test fixtures, which I replaced with the input-output files I made before I started. Then, using the parsed AST pane from AST Explorer and referencing whatever docs I could find, I wrote the transform logic. I don't have time to go too deep into the details, but we can take a quick look at what the transform looks like. So, let me explain what we're looking at here. The code in the top left of the slide represents the input pane from AST Explorer. By clicking on any part of this code, the parsed AST pane in AST Explorer would update to show me what I had selected. And the bottom left pane is the parsed AST pane from AST Explorer. By looking at this pane, I could determine how to select the relevant nodes using JS code shift methods. And finally, the right-hand pane is the JS code shift script I wrote in the project generated by codemod CLI. So, let's walk through how I used these three resources to write the first transform. You might need to watch this section a couple times if this is all new to you. The first step was to select the nodes I wanted to work with. To start, I grabbed all the classes in the file, which is usually just the default export for Ember components. But other classes could exist, so I ran the logic on all of them. Next, I got a reference to the array of nodes in the body of the class. So, I could filter them down to just the ones that were relevant for my purposes. I only wanted properties that had type information provided by the argument decorators library. So, I started by filtering out any nodes that did not have decorators of any kind. And for the properties that did have decorators, I further filtered them down to ones that were call expressions and had a name of either type or argument. This ensured I was only working with properties with decorators provided by the argument decorators library. Now it was time to strip the relevant values from the AST and store them in a more portable format. For each node, I called a function shown here as parse decorator args that pulled out the relevant type values and stored them in a plain old JavaScript object assigned to a variable named parse decorators. And last but not least, I stringified the parse decorators object and wrote the result to a file that I could reference in the second transform. Once I had some logic written, my dev workflow was to run the transform using a test fixture as the input and manually verify the correct structure was being generated in the resulting JSON file that was saved to the root of the project. If it didn't work, I made some tweaks and tried again. Once I was satisfied that the transform worked against all my input fixtures, the next step was to run it on a single component from our app and verify the results. Then I did a directory that had three components. And finally, I ran it on the entire app. Of course, I did this many times as I tweaked the transform to cover edge cases and whatnot, but eventually I was left with a working transform that built a JSON object that described all the argument decorators for each component in my app. I followed roughly the same process for the second transform. Only in this case, the transform would run on template files and use the JSON file from the first transform to determine how to update the template AST. We can use AST Explorer to work with handlebars files as well by changing the language select to handlebars. Set the transform setting to ember template recast and you'll be able to experiment with writing your transform directly in AST Explorer. In my case, the goal wasn't to read or modify the existing code. It was to generate completely new handlebars helpers and prepend them to the input file. This is a little tricky at first because while I knew what the helpers should look like, I didn't know how to write the JS code shift code that would build them. Thankfully, there is a great tool called AST Builder that neatly solves this problem. AST Builder follows the now familiar for paint approach that AST Explorer uses. The top left input pane is where you enter the source code that you would like to generate AST for. In my case, this was the argument type helpers with various arguments. And the bottom left pane shows the JS code shift code needed to generate the AST that, when converted back to source, will generate the code provided in the input pane. The other panes show the input's AST representation and the expected output, but in this case, the left two panes are what we needed. Once I had the JS code shift code to construct the helpers, I could swap back over to AST Explorer and paste it into my transform code. The code here is super simplified, but hopefully it illustrates the idea. From here, it was a simple task of filling in blanks of the builder code with data pulled from the JSON file back in AST Explorer. After all the arg type helper AST had been created, I then prepended it to the template files AST and returned the updated source. I verified that the transform worked as expected by looking at the output pane of AST Explorer. We can see here that the arg type helper has been added to the beginning of the file. Next, I used code mod CLI to generate another transform in my project and pasted the code from AST Explorer to the transform file. Since for this transform, I was actually mutating the input file, I was able to take advantage of the built-in testing provided by code mod CLI. The provided tests take an input fixture, run your transform on it, and then compare the result to an output fixture. This made debugging any issues easy and I didn't have to spend any time at all writing tests, which was awesome. The tests also verify that running the code mod multiple times won't cause issues, which is important. I made sure to write the transform in such a way that the files that had already been processed basically just no opt when the transform was run on them a second time. This made it easy for me to gradually roll out the transform in my app, starting with one file, then a few, and eventually all of them, just as I had done before with the first step. Now, with the second transform completed, the final step was to go back to the JS files and remove all the argument decorators that we no longer needed. This is definitely the easiest transform to write since I already had the logic written to select the decorators from the first transform. Once I had selected the decorators I wanted to remove, I simply called a JS code shift method to remove the node. I also did some additional work to find the import statements and remove them as well. And finally, the updated ASD was converted back to source and saved to the input file. Now that all three transforms are written and working, the next step was to actually run all of them in order on the components in our app to see if all the hard work had paid off. For this, I pushed my code mod CLI project to GitHub and used NPX commands to run each one sequentially, though I could have run them locally as node scripts, but I wanted to make sure they worked when run in this manner since that's what other users would do. The result after running the transforms was that my app was about 98% of the way there. I had to clean up a couple components that were doing some weird things and were incompatible, but that took about 10 minutes. I could have spent a bunch of time trying to capture this edge case, but I felt I was at the point of diminishing returns. The code mod got me the vast majority of the way and it was a more efficient use of time to do the remaining manual work manually. Once that was done, I built the app and somewhat to my surprise, everything worked. In a single commit, I was able to update a huge amount of components from using an upgrade blocking pattern in library to a new and preferred solution that was octane compatible. Almost immediately thereafter, we followed up with a commit to bump to Ember 3.20 with octane enabled. We could start to write glimmer components and even template only components and still have dynamic argument type checking. When I started this journey, I was pretty worried. My team was really happy with argument type checking and we didn't want to give it up. We simply couldn't live with a blocked upgrade path for Ember though. I started to feel like my past technical decisions may have led us to a dead end despite being made for all the right reasons. In hindsight though, I see it was because of this attention to craftsmanship that we were able to use code mods to get unblocked. Sure, there may not have been an off the shelf solution waiting for us, but with a reasonable amount of effort, I was able to get our app unblocked and moving forward, while maintaining all the benefits we had been enjoying for years. Looking back, I've learned a few things from this experience. First, your code doesn't have to be perfect to be useful. Keep in mind, code mods are meant to be run one time, so they don't need to be particularly performant, elegant or even well tested. The code I wrote isn't great. It's not optimized, it's not bulletproof, but it did solve a disruptive and costly issue for my team, and if it worked for us, then it could work for someone else, so why not share it? Of course, don't set out to write cringy code, but you don't have to obsess over it and make it perfect. Don't be afraid to carve your own path, especially if it provides significant benefits to productivity. Even if you get stuck at some point and have to dig your own way out, the benefits are likely worth it in the long run. Keep in mind though that sometimes, rather than relying on the community to solve a problem for you, you need to take the reins and solve it for the community. You won't be alone, there are brilliant members of the Emmer community who are willing to help should you ask for it, and there are excellent tools available to us. JS Code Shift is a little alien at first, but it's really not that difficult to pick up and get a basic transform working. Go try it. Keep the momentum going. Following this experience, I immediately created a code mod CLI project to hold code mods for my team and wrote one to standardize some data attributes that had become inconsistent in our app. I'm now always on the lookout for opportunities to clean up tech debt and keep us moving forward. And finally, there are a lot of opportunities here to contribute to improving the tools around code mods. JS Code Shift needs guides and clear documentation, AST Explorer needs decorator support, AST Builder has some display and possible parsing issues, and maybe this cross file approach could be better supported by code mods CLI. These tools are indispensable for working with code mods, but I think that due to the nature of code modding, get in, fix the issue, get out, and don't look back, they don't get the love they deserve. If you are looking for opportunities to give back to a broad community of users, look no further than these tools. I hope the strategies and techniques I've shared today help some of you keep your apps in good shape and upgradeable, so they can continue to take advantage of the great things coming down the pipe from Ember. Thanks everyone and stay safe.