 Good morning, EmberConf, or good afternoon or good evening, depending on where you're joining us from. I honestly didn't think I'd be giving this talk virtually, but it has amazed me how the Ember community, and especially the EmberConf team, have come together to make this all happen, and it really is truly incredible to me. We may be stuck, but at least we're stuck together. Anyways, my name is Chris, and I'm a software engineer at LinkedIn, and I'm also an Ember Core team member. You might know me better by my internet handle, Pazurak. This is me on Twitter, GitHub, Discord, basically everywhere else. And today I'm here to talk about reactivity. First off, what is it, and why do we care? Well, in a nutshell, reactivity is how apps update when things change, and this is why we care, because we're developing ambitious applications. We want to show changes to the user as things occur, we, as users, click on buttons, or as we return API responses from the server, or what have you. So we care a lot about this question, and modern frameworks solve this with reactivity. Auto-tracking is a form of reactivity, but there are many forms of reactivity, and today we'll take a look at a few of them and see how they compare and contrast. But before we do that, it's important to understand that reactivity is really a subset of updating. There are many ways you can update that we wouldn't call reactive. So you might be wondering, okay, why does it get a cool buzzword then? What makes it special? What sets it apart? And why do I want reactivity in my app? Why do I care about this? Why not just use any old way to update? Well, I think that's best described with an example that's shown. So let's build an application. This is TodoMBC, and it's a commonly used example app for frameworks to demonstrate how to build with them. And today we're gonna imagine that we're building a framework from scratch, and we're using that to build TodoMBC. We're not actually gonna build it from scratch, but let's just imagine that we are, and that we are now trying to figure out how users update things in our framework. And so we're gonna iterate through a few solutions and see how they work and compare and contrast them. But to keep things simple, we've already decided that we want this to be a template-oriented framework, and we like templates, and handlebars is a great templating language. So we're gonna use handlebars templates for our framework. So here's the template for this application, and let's take a look at it. So first up, we can see that there's an input at the top, which is where users can add new to-dos. And then we have the currently displaying to-do list. So whatever is currently displaying, we have their checkboxes and their titles. And then we have the footer, where we can see the items left and the filter buttons, where we can select all to-dos or active to-dos or completed to-dos. And here's the JavaScript that is backing that template. It's a lot to take in, so let's take a moment to read it. So starting from the top, we have the list of to-dos, all to-dos. So completed, not completed, they all go in here, and we have the displaying property, which controls which to-do list is currently displaying. Then we have these getters, which is how we define things like active to-dos, completed to-dos, and which to-do list is currently displaying, and to-dos left. Finally, we have our actions, which is where we actually perform the updates. This is what will happen when a user actually wants to add a to-do or toggle it. And we can see here that we're just using plain JavaScript as is. We're just pushing into an array or setting a property, nothing special here. This presents a problem though, because the framework doesn't know that we've made these changes. It doesn't know that it needs to update the template and to show the user these changes. So the question is, how do we let it know? Well, we could do it directly. We could call something like this.rerender whenever something changed, but that would be pretty difficult to remember for us. It would be hard to remember in a lot of complicated code paths that every time you update some state, you have to call this method. And it would also be not very performant because our application doesn't know exactly what changed. It just knows that it needs to re-render because something changed. So we could try to get a bit more specific to fix that problem, but this only gets more complicated. Now we need to tell it, okay, re-render this list or re-render the items left, re-render this particular to-do. And that's a lot to remember. That's a lot to think through every time you wanna update some state, which parts of the view it's gonna affect and which parts need to re-render. That's a lot to put on the developer. We can call this burden annotation overhead. It's the extra code or thought that has to go into code in order to do it in the way that the framework wants you to do it, as opposed to how you would do it without the framework. And that annotation overhead here is not only constant whenever we wanna update state, it's also combinatorial. As our application grows, there will be more and more parts of it that could be updated at any given time and more and more ways to update it. So it grows combinatorially with the size of our application, which is not a good place to be. This is really the state of the art in the era of Backbone and JQuery. Everything was pretty ad hoc on the front end at that point. So you would have one-off plugins and this was fine or small applications. It was fine for these because things weren't complicated enough for this to really become an issue. But as application size and scale began to grow, it started to become more of an issue and eventually became too much. So this is where reactivity comes in. Reactivity was a way to solve this, to make sure that your application complexity, that your updates and complexity grows linearly instead of exponentially with the size of your application. And that way it doesn't get out of hand. So let's define reactivity. Let's see what sets it apart from plain updates. But before we do that, I do wanna say real quick, this isn't the only definition of reactivity you might hear. It's one of those words that's kind of fuzzy and it's a buzzword that's used a lot in context right now and people understand what it means, but it doesn't really have an agreed upon, a super agreed upon definition. So this definition attempts to look at the goals of reactivity, the end result that it seeks to achieve. And I think it actually ends up being a very useful definition because of that. We, it shows us the similarities between a lot of different frameworks such as Vue and Ember and React and Angular and Svelte. And it shows us how different those frameworks actually are from ones like Backbone and libraries like jQuery and really shows us that core difference. So reactivity is a declarative programming model for updating based on changes to state. Okay, seems simple enough. But what does that mean? Let's dig in a little bit deeper there to understand exactly what we mean. So first off, what is declarative? Well, declarative is about describing what you want to happen without describing how you want it to happen exactly. So for instance, HTML is a 100% declarative programming language because you aren't telling the browser exactly how to render this HTML. You're not telling it, okay, make a header, add some text to it, apply some styles to it and paint it in the DOM and then make the form. No, you are handing it the structure that you want to see and the browser handles the details of that. So that's really what declarative means. It's a way to describe our intent and allow the system to figure out how to handle that on its own without us needing to figure out the details. And then what about the second part of our definition, what is state? Well, state is effectively anything that can change in your application, things like variables, properties, user inputs. And really there's two kinds of state. There's root state and drive state. So root state is state that has an actual value that is the actual data that underlies your system. So in this example, the first name and last name properties are the root state because they are real properties with real string values. By contrast, derive state derives its value, thus the name, from other values, either other root state or other derived state. So in this example, the full name getter is derived state because its value is based on first name and last name. And when we step back, we can really see that most modern web frameworks are just a way to turn root state, some API, some data into HTML, which makes that HTML derived state. So putting it all together, reactivity is a declarative programming model, meaning one that we declare our intent in and it figures out the details for us with. For updating derived state based on changes to root state and other derived state. Cool, okay, so now we understand what it means, but what does it actually look like? Well, let's take a look at a few reactive solutions and see. First up, we have observables. Now, the observables were made popular by RxJS and they're used in AngularJS. And the idea behind them is that they are a primitive that is an event emitter. They emit events which then travel through what are called streams, streams of events. And the streams can have transformations applied to them which change them into different types of events or collects them or what have you. They can also do things like debounce them. They can split streams into multiple streams or join them back into a single stream. And so you end up with this network of events and transforms and streams. And on the other end of the network, you have subscribers who are listening for events and reacting to them as they come through the system. This is known as push-based reactivity because we're pushing changes through the system as they occur. So let's see what this does to our code. The answer is a lot. So stepping through it, here's our root state, the list of deduces and the displaying property. But it's changed from being a plain array and property to being a observable because now we need everything to be in terms of this new reactive system. Everything needs to be made in terms of events. So we create our observables with their initial event which is the state of the system. Next up, we have our derived state, the getters originally. But now they are transforms on those observables because again, they have to be defined in terms of the system. So they transform the events, our state, coming through the system and split it off into multiple streams. Finally, we have our updates. And our updates once again need to be re-rationalized in terms of the system. We're no longer just updating objects, we are pushing new events, representing the new state through the system. And I do this with an immutable approach here where I clone the previous state and add a new state to it. So I'm never actually mutating anything because you don't necessarily want references to objects that are being mutated floating around the system. But there are many ways you could possibly do this in RXJS, and this is just one method for it. The key point is that our annotation overhead has grown significantly here in some ways. We now have to wrap every part of our application, the root state, the derived state, and the actions. So that is definitely a lot. And it may seem like, oh wow, that's not ideal, right? But this is actually not that different from what we had in Ember Classic as it turns out. In Ember Classic, you had to wrap root state, things like Ember Array and Proxies and whatnot would be used to wrap root state quite frequently. And you wouldn't have to do it for properties, but you still had to do it. For derived state, we had to tell the system about every computed property and all of its dependencies. So that was quite a lot of annotation that was required. And then for adding or actions, we had to tell the system with Ember Set or Push Object. So those were other forms of annotations that were required. So overall, Ember Classic was the same amount of annotation overhead as observables. And that actually isn't too surprising because Ember Classic was push based. We were pushing events through the system and it was very similar to how observables work under the hood. So overall, this solution is a massive win for performance by default. We are only updating the parts of the system that need to be updated. For ergonomics, it arguably is a win because it's no longer growing exponentially with the size of our application in terms of complexity. But we have lost a lot in the process as well. Now you need to know about the system and think in terms of it from the very get go. So to become even a little bit productive, you need to learn a whole new layer on top of normal JavaScript. So that's a lot. That's a lot to take in. Is there a way that we could have a system where the annotation overhead is lower and we can still be reactive? Let's look at a solution on the other side of the aisle, virtual DOM. So virtual DOM was made popular by React and the idea with virtual DOM is that rather than trying to understand how state flows through the system in specific detail, it just asks one question. Where did the state change occur? What component, which part of the program changed? And then it reruns that entire component and all of its children and re-renders them. But it does so virtually with a virtual representation. And then in order to actually update the UI, it diffs that virtual representation and only applies the parts that have changed. This is much cheaper than actually fully re-rendering. So that's a massive win. And this is a form of what is known as pull-based reactivity because we are pulling on the state changes. As they occur naturally, we're not immediately propagating them when they occur, but we are using them in the next render and that allows them to naturally update as they are used. So let's see how this affects our code. Looking first at our root state, we can see things are kind of back to normal. We have a plain JavaScript array, we have this displaying property, it's all on this state field now. That's actually not very necessary for virtual DOM though. It's more of a reactism than something that is actually required for the strategy. So back to pure unannotated root state. And then we have our derived state. And our derived state is also back to normal. So that's pretty great. It's when we get to our updates where things are a little bit different. And part of this is I'm still using the immutable pattern here because that's what React prefers itself. But really the key difference between this and our original solution is we now need to call set state in order to update state. And if we don't call this, it won't actually update because this is what tells the framework that this part of the program, this component needs to re-render. This is a very small amount of annotation overhead to have. And it's naturally tied to updating states. So that really helps. This is something that is honestly one of the main reasons why React was so successful in all probability. It made the developer ergonomics so much better because it gave developers a huge amount of flexibility in how they solve their problem and allowed them to write code in a way that made sense to them. You may have noticed I'm actually not using the latest and greatest from React. I'm not using hooks. That actually was an intentional choice because hooks kind of make it seem like they're adding more annotation overhead. You have to learn about these use state things and the various other types of hooks and how they have to be run in a particular order every time in order to write code in the system. And that's true. There is more annotation overhead in general, but that annotation overhead doesn't have anything to do with reactivity as a whole. Because from the reactivity model standpoint, all it needs is something to tell it what has changed at what point. And that's what happens when we call set to do from a hook. It tells the reactivity model, hey, this part of the program has changed. We run everything below it. So, yeah, all the extra annotation overhead is on the programming model side of things. And this really demonstrates how flexible React is or at virtual DOM is as a strategy. It can handle both of these very different models, hooks and class-based components without needing to know the details of them at all. So that's pretty great. The downside is performance because this model unfortunately doesn't scale at a certain point. Rewrunning the program entirely, even subsections of it, and even only doing virtual DOM is still very expensive. And this is why things like use memo and should component update and reacts concurrent mode exist because they are allowing users to manually try to optimize and make React faster and make this virtual DOM strategy faster. And this is where a lot of complexity can begin to enter React apps because now we're doing the math in our head again. Now we're trying to manually figure out what are our patterns and how do we update things and prevent updating things. So we're doing some more work of the reactive system. So let's step back. So far we've looked at two different reactivity models. Observables were very performant but they required a lot of annotation and they had us constantly thinking about the reactivity model. Everywhere we wrote code, we had to think about it. On the other side, we had virtual DOM and virtual DOM was much more minimal in terms of the annotation that was required. You only had to put it on updates and it was very flexible because of this but it wasn't very performant by default. It required some manual optimization at a certain point, at a certain scale. And so, yeah, there seems to be this trade off between the amount of annotation that the system requires and how much information it knows about what's going on and the amount of annotation overhead. So the question is, can we bend the curve? Can we have a reactivity solution that is both performant by default and that has minimal annotation overhead? Enter auto-tracking. So auto-tracking approaches this from a different angle. It tries to focus entirely on root state. In JavaScript, we already have a state model. We have a way for updating state and that's through things like pushing into an array and setting properties on an object. And auto-tracking tries to capture that so that it can interpret those in a way that allows it to tell us when things have changed, when needed. So let's see how that affects our application. The first thing we'll notice is that there's this new to-do class because in auto-tracking, we need to wrap all of our annotate, all of our root state so that the system knows about it. So we have the tracked decorator to track our various properties, title, completed, the displaying property on our to-do list component. And we also have this tracked array, which is just a reactive array. It's not provided by Ember, it's provided by the tracked built-ins out on that I maintain. But you can just think of it as an array that you can treat like a normal array, but when you mutate it, it will let the system know that changes have occurred. And once we wrap our root state this way, we can see that everything else kind of falls into place. We, our derived state is just like normal and so are our updates. They are just like the original example without any details needed to be changed. So this is a massive win for ergonomics overall because it requires minimal annotation overhead only on one part of the system, just like virtual DOM. But it's also a massive win for performance because it is just as performant as observables, if not more performant. The secret here is that auto-tracking associates root state with output. And it does so without caring about the details of how that output is generated to it, all of those details are just a black box. How does it do this? With a technique known as memoization. Memoization is a technique where we return a previously computed value if nothing has changed. So for instance, in this example, we have a memoized render function which has a call to a real render function within it. It stores the last return value of that render function and it stores the last arguments. So the next time it's called, if the arguments it's called with are the same as it was last time, it will return the last result and skip calling render altogether. Memoization is a way that we can really speed up applications because we can skip unnecessary work this way. But in auto-tracking apps, we wanna do something slightly different. We don't want to memoize based on the arguments that are passed to us. We want to memoize based on the values we access while running the function. So for instance, if this was our track state and this was the function that we wanted to memoize, we would want to memoize based on these values that are accessed on the track state during the call of the function, such that if any one of them changes, the next time we call this function, we rerun it. And if none of them change, we just return the previous value and that's it. So how do we do this? Well, it all starts with a single number, the global revision counter. And whenever any state changes in the application, we increment this number. I like to think of this as a clock, only instead of tracking time, it tracks changes to state. As we change state, we are creating new versions of state in the app. And the clock always keeps track of the most recent version. So you can see how this would be a good rudimentary way to understand if something has changed in the app. We look at the clock, we memorize the time, we go back to whatever we were doing and the next time we want to check if something has changed, we look back at the clock. Is it later than it was? Is it a higher value than it was? Yes? Cool, something has changed. Now we know that. But really we also want to tell if like a specific piece of state has changed, usually, not just like any state. So for that we need tags. Every piece of state in the application has a tag and every tag has a value. And that value is a version of the clock. Whenever we want to change this piece of state, we first increment the clock because something has changed and then we assign that value to the tag. So in this way, tags always contain the most recent version of state that they were updated in. Okay, so now going back to our original memorized function, as we run through and execute this function, we, as we encounter a track state, we store the tags and save them for later. We store them and we also save the value that is highest out of all of them, the maximum value. And the next time we come back to this function, we check that maximum value again. We iterate through the tags again and find the maximum value again. If that maximum value is the same, then we know for sure that nothing has changed within this function and we can return the previous value. And if it's higher than it was previously, then we know something must have changed and we need to rerun the function. How can we know that? For sure, well, let's step through it. Let's say we change something within the function. We increment the clock. We update the tag to match the new value of the clock. And the next time we come back to this function, we check the maximum value and it's higher than it was previously. So we know for sure that something has changed. We rerun the function. And let's look at the opposite. Let's say we changed some state somewhere else in the app. We increment the clock again, we match the value like before, but because that tag was not used in our function, it's not part of the set we check. So it doesn't affect the maximum value of the tags of our state. And so the maximum value stays the same. So we know for sure nothing has changed and it's safe to use the previous value. Our memoization strategy works. Okay, let's take a look at what operations that required. To dirty state, all we had to do was increment a number. To validate it, to check whether or not it was still valid, all we have to do is map through an array of tags and take the maximum value out of all of them. This is all done lazily. So you can update 1,000 pieces of state and all that's doing is incrementing 1,000 numbers. And then the next time you go to render, you are checking validation only if needed. If you skip an entire section of the app, if you deleted it or got rid of it, you don't ever need to do that validation. You get all of this and all you need to do is annotate root state. Every other part of the system handles itself. Root state and the portions that you want to memoize. This is what bending the curve looks like. Incredible developer ergonomics, performant by default. This is hands down the most exciting feature to me in modern number and one of the most amazing things about the framework as a whole to me. So what's next? Now that we have this amazing new reactivity model, what are we gonna build on top of it? Well, first up, I would like to see more libraries and patterns and common abstractions built on top of auto-tracking. We had a lot of these for computed properties. One of my favorites was Ember macro helpers, which was a way to define various macros for computes. Stuff like that is gonna be very, very helpful for auto-tracking. And I've already started with the tracked built-ins add-on, which creates tracked versions of JavaScript's track arrays and maps and sets, allowing people to use them as root state using their standard APIs. And I think we could see more of this. We could see tracked local storage and tracked index DBE, tracked versions of Apollo and Redux. I think that making more libraries like this will be very helpful and I'm very excited to see what the community does with that. Beyond that, I think that tooling is very important and I would like to see us invest a lot more in that space. I think that actually auto-tracking kind of gives us an unprecedented ability and chance to really make some amazing tooling improvements. We can do things like when you're paused in a debugger, tell you what you've auto-tracked so far, how many things are memorized above you, what changed to cause this code to rerun again, questions that would be very difficult to answer otherwise. And I even can see some potential improvements to the Ember Inspector. I imagine a state timeline that shows the exact order of operations, the exact state changes that have happened in your application to bring you to where you're at in that moment. And because of the way auto-tracking works, we can show you exactly which components those state changes are related to. We can show you which components dirtied and updated and were created and were destroyed. And we can show that all to you and show your entanglements and everything. Finally, I wanna say auto-tracking in a lot of ways is to me larger than Ember. And what I mean by that is that it is, it's really a general reactivity model. It's something that can be used outside of the context of a rendering engine in a view layer. It can be used anywhere. And so, and it also is a reactivity model that doesn't really exist in the wider JavaScript ecosystem yet. There's nothing really quite like it yet. MobX and views reactivity are similar, but there are some pretty fundamental differences. So I'd really like to see us extract it and make it usable not only in Ember apps, but everywhere so that we can share all of the utilities built on top of it, that we can share all of the value generated by it in general. And yeah, I think it would be incredibly valuable to do that. I've even thought as time has gone on that it really would make a good system, a good solution for adding reactivity on the language level itself, either to JavaScript or to another language. Unlike previous attempts to do this, it's not something that requires synchronous code to run and it's not super invasive. So it could be kind of ideal. And beyond that, it's paradigmless. You don't need to write object-oriented code for it to work. You don't need to write functional code for it to work. You don't need to write imperative code for it to work. You can write any kind of code and it works with it because all it cares about is that you have properly annotated your root state. And for a language level feature, a language that is you're able to do all three of those in quite a lot, that would be ideal, I think. So anyways, that's all I've got. If you're interested in this topic, I've been logging in more depth on auto-tracking, so check that out. And thank you.