 I'm Trent. I go by Trent and Willis on Twitter and GitHub and I've done a lot of work on Ember engines and so that's what I'm going to be talking about tonight. So a couple weeks ago I tweeted this so after a lot of hard work we at LinkedIn finally have lazy engines in production. So this has been a long time coming in the Ember community. People have been talking about engines for close to two years now with the biggest promise being that you can lazy load chunks of your application. So that promise is finally being delivered and so tonight we're going to go through an example of how to do this pretty quickly. Before we dive in though, this is the app we're going to be working with tonight. It is a new documentation website for Ember engines. You can find it at Ember engines.com. It has fully updated guides as of the latest release of Ember engines and can walk you through the basics and concepts of engines all the way to actually like getting a lazy engine working in development. And so click through. Simple, pretty simple. Have some animations and some different pages to walk you through. And so this is what we're going to be working with tonight. All these codes available on GitHub under the Ember engines org. Currently this is the only thing in that organization. We're hoping to move the rest of Ember engines related code there so that way eventually everyone has a single source of truth to go look at whenever they have questions. And if there's anything missing from the guides, feel free to open PRs. And this is the first time we're announcing this. So obligatory tweet about. Okay, so let's dive in. I have the app up and running locally. As we can see everything works. Click around. Really simple. Tests are passing. Everything is great. So we're going to switch over to our command line and color server. And like a good dev, we're going to check out a new branch. I use lots of aliases for my command line. So I will try to make sure I comment on what exactly I'm doing as well. So we check out a new branch. And the first thing we're going to do is install Ember engines. So by default a lot of the behavior for engines is upstreamed into Ember core. So if you're on the latest version of Ember, specifically Ember 2.10, which was just released yesterday, you'll have most of what you need to use engines. There are a few things though, specifically like building lazy engines that are not currently supported out of the box. And so for those, we need to install Ember install the Ember engines out on. Currently, this is that version 04-beta4. But as soon as we verify that all the build is stable because we just merged the PR today, we plan to release the stable 0.4 version of this, at which point you can just do Ember install Ember engines. It'll work. So this is the nervous part of the talk. And it worked. Okay. So at this point, we have our application. What we can do is just Ember G or Ember generate an in repo engine. This is very similar to an in repo add-on. Hopefully you've worked with them before. But if not, they're basically just like normal add-ons. The only real difference is that they live within a single repository instead of an external repository. And they don't have their own isolated testing. That's something that's been in the Ember CLI RFCs though and is hopefully coming in the future. So we'll generate an in repo engine. In this case, we're just going to call it died. So maybe you go to the documentation website and you just want to read the first page, the overview and like how to get started. But you don't necessarily want to dive into like the full meaty depth of the guides that are going up. So as you can see, we have a bunch of stuff created. And at this point, we'll hop over to a text editor. Hopefully that's big enough for you all. And we can look inside lib, which is where in repo add-ons go as well as in repo engines. And we can see that we have our guide engine. And so to give you a quick overview of the anatomy of this, we have an engine.js file. And as you can see, this should look pretty familiar to anyone that's worked on an Ember CLI app before. It's very similar to the setup for a normal Ember application. Engines by and large are very similar to applications. The primary differences is that they have to have a host that actually instantiates them, whereas applications boot themselves. And they don't have a router. So the router, the global routing comes from the host that actually instantiates them. Pretty straightforward. So after that, since this is a routable engine, I can talk about routable and ratless engines at the end. But by far the most common need is for routable engines. These are engines that represent chunks of routes in your application. So here we have a simple route map that exports a function wrapped inside this build routes thing. And this is literally just like your router's route map. And so what we can do is we can jump into our application here and take a look at the router map. So right now it's really simple. We have a guide route and then it has a dynamic page route underneath. And so what we can think of is we can think of this function, the callback for this guide route as what is defined in our routes.js file. So what I'm going to do is just cut this out and then jump over here and then over here. That's it. Back over here, though, in the actual router, we want to change this from saying it's a route to saying it's a mount. And so this is basically saying this is where we are mounting this engine. This is how we are going to use it. This is the route point at which we expect it to be. So moving back over to the engine, we can see that inside templates it has its own application.hbs. And so like a normal Ember application, the top level template that renders for an engine is called application. And so this is very analogous to what you would find in a normal Ember application. So if we go and look at our templates in the actual app right now, we have our guide template. But since the guide route no longer exists, we want to actually move what lives at the guide route into the application route. So I can copy all this, move it over here into our application template. Then now back up here, I'll go ahead and delete this. Okay, so we also have the page route and an actual index route for the guides that we have. And so currently these are defined in a subdirectory within our application. However, like I said, because everything now is relative to where the engine is mounted, aka the guide route, these actually become top level templates in the engine itself. We can just drag and drop these down here. And that's about it. So we can go ahead and delete that folder as well. Cool. So moving on, we also want to make sure that we move over the corresponding with JavaScript for all this stuff. So similarly, if we have our guide routes, JavaScript files, these are all pretty straightforward. Nothing really specific to engines in any way. They have some imports. And they like return some static data that we use to generate the guide content. So in our engine, we'll create a new folder, just called routes. And we can drag and drop these down here into the routes. I'll get rid of this just to clean things up. Okay. And so at this point, we've moved over the routes and the templates. There are also some utilities that we're using in our engine. And thankfully, these aren't actually used by the main page. So we can go ahead and just like drag this whole folder into here. So this is the majority of what is actually involved in moving code from an application into an engine. It is literally just like moving it from the application proper into the engine itself and removing any namespaces that have already been applied. So one thing that we'll want to look at here is, for instance, our route JavaScript files, since they used to be nested within a guide namespace, they're actually nested too deeply, or these imports are assumed to be nested too deeply. So we should cut one of them out. Do this over here as well. Okay. Let's close out some of these tabs. So at this point, we've pretty much moved everything over. So let's fire our server back up and see if it works. Great. The index is loading properly. But none of the guide pages are working. So let's see what we get in the console. So we get an ember error. Let's blow this up. So it says assertion failed, you attempted to define link to guide.guide. That doesn't sound right. Who would nest a route named the same thing within another one? And so let's go look at the templates that we have for our guide. So if we look up here, we'll see that we have a link to a guide route, but it's not guide.guide. And so this error is a bit confusing. It's a to do task to make this more readable and like, debuggable for people. But basically what's happening is, our engine scopes everything to the mount point that it's at. And so because we mounted our engine at guide, and we're trying to link to a route guide, it prepends the mount point to it. And so we wind up with guide.guide. So in other words, if I had mounted this at a location, say like through, and then tried to go to guide, it really go to through.guide. And so this is obviously not desirable. So what we need to do is our guide route, which was previously our mount point is now our application map. So we just change that to application. And that should fix it. So let's see what happens. Great, but we got another error. Guide to application. And so this is, excuse me. So we got to make sure that we don't have any other potential problems here. We want to go through and make sure that like all of our other guide names based ones are being cleaned up as well. Okay, that and take care of that. This is actually supposed to be index. Similar to the way that like your normal application works, you usually don't want to link to application directly if you actually want to link to your index route. And so for the guides, that is actually a table of contents. So at this point, it's rendering, but there's nothing on the page. We don't get any errors. So what's going on? Well, so earlier, I mentioned that we have some animations going on. And in here, we're using a liquid outlet. So unfortunately, we need to improve the errors in this case as well. But this just means that like, it's trying to render something into the liquid outlet. But it doesn't know about liquid outlet because engines have their own namespaces. And so even though our global application is using liquid fire, our engine itself doesn't know about it. So this is thankfully easy to fix. Once you get past the unintuitive, like it's not doing anything. All we have to do is add it to our package.json file. So we can specify, oops, liquid fire. And then we peg it to a star version because we want it to just use whatever version is available in the host application. This would be different if it was a standalone engine and not an in repo engine. And in that case, you would actually want to peg it to a specific version. But for simplicity's sake, everything is in repo in this talk. So we also have several other dependencies that we have. So ember remarkable is one. That's how we actually display the guides and all the content on the site because it's written as markdown files because no one likes to maintain raw HTML. And then we have ember fr markdown file, which is another add on. And all that does is make it so that way you can have your markdown files live in separate directories and they still get pulled in properly rather than having to like, inline the text into your templates. Save that out, kill our server, and we start jump back over here, and ta-da, everything is working. Kind of. Doesn't look like our animations are working properly. And it also looks like this isn't correct. The intro tab, we would expect to take us back to our homepage. So let's dive in. Let's talk about the link issue. So the first one is so if we look at the intro link here, you see that it's also pointing to our index. But since everything is scoped to our engine, that means it's actually pointing to the guides index route. And that's not what we want. We actually want it to point to the host applications index route. So unfortunately, because all of our links are scoped to the mount point, that's not really possible with the normal link to semantics. And this was a explicit decision made for engines to have everything scoped to your engine. And that is to prevent you from accidentally introducing too much coupling to the route structure of your host application. So there's an alternative construct that is called link to external. The idea is that this represents a link to a place that is external to your engine. So let's update this. And let's give this a more descriptive game, say maybe home. And so what this is saying is it says we want to go to this external place that we're calling home. And so this is a dependency that we ourselves don't really declare. Like this route lives outside of our application. So over in our engine, we want to specify this as a dependency. And so we define these as external routes. And external routes is just an array. You can name them various things, just except strings. So like home. And then maybe say we want to go to a blog and then to settings or whatever. You can specify as many of these as you want. The only drawback to specifying a lot of them is your host application then has to fulfill all of them. So we're just going to stick with home for now. It's important to know I'm not going to talk about this too much here, but the other dependencies that engines are allowed to have are services. And so host applications can pass in their services into engines. So that way you can share certain states such as things like maybe Ember data store. So at this point, we've defined our dependency in our engine. But now we also want to make sure that our application is doing the right thing to fulfill that dependency. So we specify a property called engines. And then within this object, all the keys are the names of whatever engines you are mounted. So in our case it's guide. If you have a dasherized engine name, say for instance like Ember-blog or foo-bar, then you would convert it to the proper camel case form of that. So we have engines guide. Then we want to specify its dependencies, lots of nesting. And we want to specify the external routes that we're fulfilling. Now the external routes from the host side is a bit different. So we specify first the key that the engine is expecting. So in our case it's home. But then the value that we give it is where we expect home to be. So in this case we want to say home is at index. And this is relative to the application, right? And so this is the true global like index route. We can alternatively say it's at some other route, say maybe like home or yeah home.index or something like that. This accepts any like actual route path that is in your router map. But we just want index. So at this point if we save all that, we've got a rebuild and then looks like intro is working correctly. Great. And so I want to pause for one more second to just talk a bit more about these external links. So one of the things that makes trips people up a lot is when thinking about linking to stuff outside of their engine, they want to use the same type route paths. So for instance if our app was fulfilling this as home.index, it would be very tempting for me to want my engine to be home.index as well. But this is actually not a very good pattern because then you're like relying on these specific paths that don't really make sense for what you're trying to accomplish. So the thing that I always tell developers when introducing this concept is to think of external routes as from the engine side as specifying what you want to go to. And then it's the application shop to tell you where that thing is. So like in our case our guide is saying I want to go to the home route and then the application is saying well the home route is located here. And that prevents you from coupling yourself to your host application. Great. So that's working. So the last thing that we have to figure out is why the animations aren't working. So none of them are working even from like the top level home page to the guide page. So let's go take a look at our transitions.js file. If you haven't worked with liquid fire, I apologize this might be a little bit confusing. Oh it encouraged you to use it. It's a really cool library. And so let's just go through these one by one. So our first transition is supposed to be from our index to our guide route. But this doesn't seem to be working right. So let's throw this.debug in there and figure out why. So it says rule zero. No, here. Not important yet. Now if we transition. It was rejected because new route was application. So we're trying to match index to guide route. But what we're getting is actually index to the application route. And so this is a bug that I actually discovered today as I was prepping for this talk. And we'll have to be fixing it in liquid fire. But so apparently the scoping semantics that come into play with engines are properly translated into how liquid fire does its route matching. So what that means is that we actually need to specify application route. Which is not great. But if we refresh it looks like it's working now. So if anyone has time please submit a PR. Great. So now let's talk about these other ones though. So these transitions are between the guide index and the guide page. So like between our table of contents and between actual pages. And then this other one is within the different pages. So within the page content there's a liquid bind that's used. These are scoped to the engine and only happen within the engine. So really we should be trying to make sure that these transitions also live with our engines. So what we're going to do here is we'll just go ahead and copy over this entire transitions file. Go into our engine again. And then add a transitions.js file. And then paste. Now this global one we don't really need here though. Because that doesn't apply. And then we want to make sure that these are properly scoped. And let's save that. And then while we're at it we should probably remove them from the global one. And then save that. Okay rebuild done. Uh-oh. So unknown transitions smart fade. So smart fade is a custom transition written within this application as another end repo add-on part of the scroll tracker thing. It was a simple animation or transition I added so that way it doesn't liquid fire doesn't jump to the top before transitions. Just makes for a nicer user experience in my opinion. But so what that means is that our engine actually needs to depend on the scroll tracker add-on as well. So we can do is we can come over here. We can specify ember add-on. This is the field that Ember CLI uses for kind of all the various metadata that you could possibly associate with an application or an add-on. And then we can specify paths. And so this is the field that tells Ember CLI hey these are the other end repo add-ons that I am depending on. Look at these paths. So in our case it's simply just like go up a directory and then look for scroll tracker. Pretty simple. So let's save that and since we changed our package to a son file restart our server. See if it works. Great everything seems to be working now. Let's go check our tests to see if that's if those are still working as well. Wonderful. Looks like everything is passing. So at this point we have a fully functioning engine. And this is the route that I encourage most teams to take when they're moving to engines is to do it first in this fashion because everything is still eagerly loaded which means that the risk for things going long is a bit less. If you were to jump straight to lazy loading you have to make sure that you do the entire migration at once. Make sure everything is completely isolated and overall it's much more likely to lead to just landing one giant like PR bomb into your code base which is not fun to review or to really work on. So at this point though everything is working which is great. So let's talk about how we actually make it lazy. So you'll notice over here in my terminal that we get this deprecation every time. It says guide engine must specify the lazy loading property to true or false as to whether the engine should be lazily loaded. And so what this means is in our index.js file you actually need to specify a lazy loading flag. So by default this defaults to false because we don't want to opt people into this like relatively new and kind of wonky behavior. And so we keep the like simple tried and true path as the default. However we still want people to set this even if they're expecting it to be eagerly loaded because that makes the choice explicit and so if we decided to change the default behavior in the future say to make lazy loading true and make it a lot easier for people to just automatically dive into it we don't want them to like all of a sudden be impacted by that decision. So this is what it is implicitly right now let's just change it to true. Then if we kill our server and serve again let's see what happens. If we go to our index route the index loads fine as is to be expected if we're doing lazy loading of something. But let's keep an eye on all the requests that are happening in the browser. So if we switch to the guide tab we have all of our lazy engine assets loading. So you can see that they have a size to them even if it's non-trivial or relatively small which is awesome. So in our app well so I guess before moving to that so this is all you have to do. Once you have an engine literally all you need to do is flip a flag from false to true and lazy loading should work. Assuming that your application is actually isolated and self-contained it will work. This is something that we spent a lot of time on in effort to make sure that it's an easy path to success. We don't want people to have to configure a bunch of things in order to get their build working a certain way. It should be that like you know if your if your engine works eagerly it should be relatively straightforward to make it work lazily. And so this is this was a huge deal for us at LinkedIn. So going back to the tweet that I had up at the beginning we moved our application which currently has on the order of like 20 25 engines or something in it which is a lot of code. It took us less than a week to move those 25 engines from normal eagerly loading engines to lazy loading and this in turn resulted in us shrinking our JavaScript bundle size down from about I think it was seven, eight megabytes when we started down to about 3.5 in that time frame. So we still have a lot of large way to go in terms of like bringing it down even further but it was a tremendous win that we were able to do in a very short amount of time. Obviously there was some ramp up leading up to it and like actually moving to engines but the investment pays off in the end and that when you can have these RBs that are literally like three or four lines and you can say hey this entire chunk of our application is now lazy loading and our users only have to pay for it when they're actually using it is pretty awesome. And so to have this talk up let's go GFR tests are still working there and so this is a byproduct of the fact that this application only has acceptance tests and so they're running like they normally do on the browser and so like if lazy loading is working in their browser then the test should be working as well. There are still some open questions if we go and look at the guides actually the last section here is testing and PRs are welcome we haven't quite figured out quite nailed down what our testing story is for lazy engines yet. So if you have ideas reach out to us because one of the weird things about it is that Ember CLI and the way the default testing setup works is we expect all your code to be loaded eagerly so that way we can like load all your unit tests and integration tests and import everything all it like boot up before you actually run the tests but this causes problems if your code is being lazily loaded. We have thoughts on how to approach this but we haven't figured out like 100% bulletproof solution yet. One thing a lot of people are doing is just preloading all the assets up front and then running it normally and then just hoping that like we did our due diligence and the lazy loading part actually works properly. This is obviously not a great solution so community involvement is definitely very welcome and we want to hear from you guys especially once you start using it on how you plan to solve it how you want to approach it and working with those of us that have been a core part of working on Ember engines and how to solve it. So yes the last thing to do would just be let's add all of this. Hey, move guides, totally easy. I don't know what to push it up. Are there any questions? So fast boot, good question. So there has been some work to integrate with fast boot. Currently at this moment it does not work. One of the big outstanding questions is with these lazy assets like okay let me back up because I don't I imagine a lot of people are not super familiar with how fast boot works. So at fast boot when you visit your app when someone visits your application in production what happens is the application is actually running on the server and you try to navigate through a route hierarchy. So if someone is deep linking into an engine so in this case if someone were to visit like emberengines.com slash guide slash introduction it would try to run that route in fast boot on your server. But if your code is lazy loading then it's not going to be able to do that because your server is probably not going to be like making these requests to load assets at runtime. So we need to preload those assets in the fast boot server the exact idea of how to configure that and like set it up is a bit up for discussion. I know it's something that we have been working actively to kind of repatriate back to the community from LinkedIn and it's something that the Ember CLI team is currently working on and has some has had some lengthy discussions on. At LinkedIn we are actually currently running a fast boot like not exactly fast boot environment for our application. So we have done this but our way is very artisanal it works for our environment but not for like the general open source community and so we want to help push a lot of that back into the community. A lot of the stuff is already in place like as far as defining like the assets that you need to load and making sure that everything works properly when it is actually being run in the node environment. Those pieces are in place but there are still just kind of a few outstanding possible pieces. So question was around the asset URL. So right now everything gets built into engines-dist slash engine name and then the name of the asset file and so this is the current pattern. It's subject to change. There wasn't like any super rigorous process around it. Okay so the way fast boot works currently is everything basically goes into package JSON and then you can install stuff into the server side that way, correct? Okay so about specific server side dependencies I'm not sure how it is going to work in fast boot. I do know from conversations that I've had with folks involved in trying to make this work with fast boot is that the goal is to put everything on the top level. There shouldn't have to be any of this like walking through all the built engine files to like figure out what all needs to be included. We sort of do this today. At build time we generate an asset manifest and this specifies all the assets that should be needed to load the engine on the client. The goal is to eventually generate something that is the same but for the server and actually goes into the package JSON file that fast boot references so that way there's a single source of truth for everything that fast boot needs to do in order to get itself set up. So the question was around if we've had any issues with different engines having different version dependencies and if there's been any issues with compatibility. So this was a problem we actually thought long and hard about before moving our application to engines and the solution we came up with is actually it works but it's a bit iffy. With the we're hoping that the community starts moving to yarn in which case that may help a bit and like it has an option to basically flatten stuff out and make sure that you're using one version of dependencies across the board but the way we do it in our application currently is all of our engines still live as in repo engines because of that they're pegged to a single source of truth in our pack in our applications package JSON. So this can this has the downside of making it a bit harder to upgrade because you have to make sure everyone upgrades at the same exact time but does prevent us from running into like weird production issues where maybe our fee our news feed is expecting to load like one version of this add-on for like our drop down but then our messaging is expecting a different version of that add-on. So they're they're trade-offs for sure. So the question is basically how do we identify all the edges of the engines as we were moving to them and then if we ran into any production issues like how do we like you know mitigate that risk especially as you move to eagerly loaded engines before you move to lazy engines. So there's definitely some risk involved here. We did have some issues with like not actually fully isolating engines before trying to roll them out to be lazy engines and unfortunately there's no super great way of doing this statically at this point in time. One of the like cool but then kind of frustrating things about Ember at some points in time is the fact that we have a resolver and we can do things like dependency injection. So all these are great and allow us to build dynamic applications and can also make identifying these edges hard because you may be injecting something say like Ember data store gets injected into routes but you may not necessarily think of that like as an explicit dependency of your engine. So thankfully these errors are really easy to catch most of the time and that like if you run your app with a lazy loading engine they usually just like blows up and so you should be able to catch this before you get it to production. Enough chance that you don't though our advice would be to just roll back in quickly fix it or if you know exactly where I went wrong fix forward. We want to invest more in static analysis and we've been looking into like developing add-ons to help enforce constraints on the way you build Ember applications to make them more statically analyzable so that way we can easily tell you like at build time hey you're using this thing over here but it doesn't seem to be a dependency of this it's like you might have problems when you run your application. So the question is around basically how stable are like non in rebuild engines and the ability to share code between different applications. So we haven't done a whole lot of this here on the project that I work on but from everything that we've experienced and a lot of the feedback we've gotten from the community is they seem to have more or less the same stability as in repo engines do at this point in time. The one weird bit which is more of a general kind of Ember CLI issue is that when you're working on an engine by itself as a standalone add-on the dependency story is a bit wonky because you have some dev dependencies as well as like your normal dependencies that you want to ship with your engine but when you do the testing the dummy application that works to test add-ons uses both those like development dependencies as well as the normal dependencies so this can lead to some like weird dependency graph problems that we're trying to get a better story around but for the most part I would say go for it if you have any questions jump in the Ember community slack now that things are kind of slowing down a little bit at LinkedIn we will hopefully have more time to help support you guys but there have been several other folks in the community that really jumped on board and have been helpful at answering other people's questions and then just like providing support for common issues and things like that and you can find an issue that you firmly believe is actually a problem with Ember engines itself has always opened an issue on GitHub or submit a PR thank you