 Let's get started. I want you to imagine that you're at your regular rails team retrospective. Oh, wait. Imagine that you actually have regular retrospectives and you're there discussing technical debt, the pain points you have in your development process, performance issues, and the like. And someone on your team chimes in and says, I think we should switch to Webpack. I've read a few tutorials and I think it should be pretty easy. Now imagine your response. This happened to me. And a year later, I've put together a talk called Look Before You Import, a Webpack Survival Guide for Rails Developers. My name is Ross Kaffenberger. I'm at Rosta on Twitter and GitHub. And I want to say congrats to you all. You made it to nearly the end of Rails Comp, second to last talk. So give yourselves a round of applause. When I found out I was speaking before Aaron Patterson, I was both excited and also a little relieved. See, I'm excited because he does some amazing work with Ruby. And generally, it's better to speak before Aaron than after him. And when I see this profile pic with the flowing locks, I recall that the original title for my talk was How to Webpack Without Losing Your Hair. One of my colleagues reminded me that I might not be the best person to give that talk. It's been a while since my hair is migrated to my face. And my imagination does get carried away from time to time. So when I'm doing amazing things in Ruby, this is what I think that I look like. Like I said, my name is Ross. And although this will be a very JavaScript focused talk, I do love Ruby, I like to think that I've worked hard to make a name for myself in Ruby open source software. And when I say make a name for myself in Ruby open source software, I mean literally the name for myself is in Ruby open source software. And speaking of great names, there's a Ruby conference coming up next month named Ross Comp. In my opinion, this is the best named Ruby conference out there. I'm very honored that they named the conference after me. That's not actually true, but it's in Amsterdam next month. So if you're there, let me know how it goes. I work for a company called LearnZillion. We are building curriculum as a service for K through 12 education. We're built on Rails and more recently Webpack. We're looking for an engineer to join our remote team. So check us out at learnzillion.com slash careers. I'm also a dad to a toddler and to a terrier. And I'm often asked by my colleagues, what's it like to be a dad? What's it like to have pets? And I can pretty much sum up my experience in one word that every day, each and every day of mine is filled with tons and tons of joy. So much joy I can imagine carrying around that joy in my hands. And if you all are inclined to have children yourself or pets, my hope for you is that your days will be filled with joy as well. So speaking of things that bring me joy, let's talk about Webpack. So about this time last year, my team has had built, we had built over several years our Rails app on the Rails asset pipeline. And so you can imagine for an application that's been around for several years, there's a lot of legacy code that's built up. So we had a heavy use of global variables in our JavaScript, JavaScript snippets in our Ruby templates, tons of Rails asset gems. And we weren't quite sure how this would all fit together as we were moving forward, our development experience had started to decline somewhat. It would take a long time for the page to refresh. We were frustrated that our use of source maps was somewhat limited. And so we made this decision. Let's switch over to Webpack. We had some vague idea that things would be better. Turns out for us, the task was not so easy. See, we had all these assumptions that we had made and part of the challenge was how do we get Webpack to work with our legacy decisions that we had made over time? But we had a lot to learn about Webpack itself. And so we had a lot of assumptions coming from our experience with the Rails asset pipeline that we assumed would just work in Webpack. And the problem with having a sprockets or Rails asset pipeline mindset when using Webpack, it's like a little bit like trying to fit a square peg into a round hole. So we made our job harder in ourself. And now that we've made the switch, we're better informed about how Webpack works, about what we do differently if we were to do it again. And so this is the talk now that I would have liked to have seen before I started. And hopefully, regardless of whether you're considering upgrading a legacy app or starting a Greenfield project on Webpack, I hope that we can pull out some lessons for everybody. So the main structure of this talk, we're gonna talk about three general scenarios that fall under the umbrella of packaging assets for the browser. We'll talk about six problems that our team encountered in making this switch to Webpack. And we'll come away with seven general lessons, survival tips that I think will be applicable to anyone considering this change. But first let's make sure that we're all on the same page about sprockets and Webpack. They're meant generally to solve the same problem which is packaging assets for the browser. And by assets, I mean JavaScript, style sheets, Web fonts, images. And it's important to understand that they go about trying to solve this problem in fundamentally different ways. And one of those key differences is how they group files together into one or fewer that are served to the browser. And so in sprockets, when it puts your JavaScript files together, it does file concatenation. And when that JavaScript is evaluated in the browser, it's evaluated in the global scope. Webpack is a static module bundler. That basically means each one of your files is converted to a module that has its own scope in the browser. And just this key difference has a lot of implications for the kinds of decisions you can make. And it took us a while to actually let this difference sink in. We had a vague sense when we made the switch that there'd be a lot of improvements. And now a year later, I'm really happy to say that that bet has paid off. We have better development tools I feel in the webpack environment alongside Rails. I feel that webpack is a better strategy for managing dependencies. There's a huge ecosystem bolstered by the large JavaScript community that we can more easily take advantage of. We can write our code in ES6, although I should say sprockets four, which is still in beta would allow you to do this as well. And we can take advantage of a lot of new features that are not present in sprockets or the Rails asset pipeline, one of which is dynamic code splitting, which I'll get to a little later. So despite all these great ideas and features and improvements, spend a little time researching webpack on the web and you'll come across quite a bit of emotion. Some people hate webpack. Some folks passionately hate webpack. Sometimes getting things to work is a little bit like trial and error, often because we don't necessarily understand everything that's going on. Some folks feel that webpack is over engineered and that all this work it might take to get this configuration right is really not of benefit to your team. There's even a website devoted to those WTF moments that folks have when trying to get webpack set up. And to be honest, I've had a few of those moments myself, some of which you'll see today. Although one of the key decision points for us to encourage us to make the switch was that Rails announced official webpack support through the webpacker gem last year. And so what webpacker basically does is try to bring Rails conventions, this Rails as Omicase approach convention over configuration, these ideas to webpack which generally doesn't make any decisions for you. It's fairly unopinionated and that goes slightly against what we typically think of as Rails philosophy. So this won't be a webpack tutorial. I will though recommend these three resources which really helped me understand and get up and running with webpack. This survived JS webpack book by Juho, the learning academy from Sean Larkin who really is the tireless spokesperson for webpack itself and webpack from nothing by David Copeland. So let's get into these survival scenarios, things that we encountered as we made this switch. So we had to address some challenges with organizing our code, with taming the dependencies that we were moving over to webpack, and predictable caching. So jumping into this first category, one of the first questions we have once we get webpack up and running in Rails environment is how do we organize our directory structure? And taking a step back to the Rails asset pipeline, the default layout typically for your JavaScript would be to place everything in your app assets, JavaScripts directory. Some special files in there would be configured to be pre-compiled to be output into the public directory when you wanna serve these assets in production. You might need to use some Rails configuration to say hey, in addition to application JS, I want these other files to be bundled up as well. Let's take a look at how the directory structure should look like when you integrate with webpacker. So taking this source code that we had in the Rails asset pipeline, you can move it over to the app JavaScript directory when you get webpack set up. A really nice feature of this separate layout is that you can run sprockets and webpack side by side. So what we actually did was move these files over one by one to get things working. And you'll notice that there's a special directory in under JavaScript called packs. It's just a term that webpacker created. But the generic term that it really refers to when webpack is called entry. And these would sort of correspond to those bundles that will then be output into the corresponding public directory. And in webpack parlance, this is called output. And really there's no extra configuration that needs to happen. Anything that goes into the packs directory, directory specifically JavaScript files will be pre-compiled, so to speak, into the public directory. Unfortunately, a lot of folks get this wrong. I pay quite a close attention, a lot of attention to the webpacker issues. And a lot of folks essentially report problems, not with webpacker, but that ultimately go down to misunderstanding how to organize their directory structure. See, instead of laying out their files this way, they make it look a little bit more like what we had in sprockets. They put everything in the packs directory, or this tends to be a common problem. That's kind of what we used to do in the Rails Asset Pipeline, so it must work in webpacker, right? Well, this actually is a problem. If you were to run your webpack build and pay close attention to the output, this will basically tell you a lot about what's happening when you build webpack. And you'll see every individual module in that packs directory get output or pre-compiled into the public directory. And webpack will say it's emitted. It's created a file for each one of those that should be served. That's probably not what you want. So really, this is just a quick fix. Just keep in mind this convention. Only put JavaScript files that you want to be bundled as entry points in that packs directory. And really, you should see only a limited number of files in that output. So survival tip number one, really, the main point here is don't assume just because things work a certain way in sprockets in the Rails Asset Pipeline that they'll work the same way in webpack and webpacker. All right, another issue that came up was we wanted to do something like how do you require tree in webpack? So we get this really nice directive in sprockets where we can include all the files in some directory automatically with this one line. So this actually doesn't work by default. If you were to import some directory in your ES6 code through webpack, you'd actually have to do a little bit of work to make this functionally the same. So an API that's very specific to a webpack build is called require context. And it acts a little bit like directory glob in Ruby. You would use this to build up a list of files within that directory. If you had an index directory, essentially an index file essentially aliases to that directory name. And you'd have to recurse, or excuse me, iterate over these files and require them one by one. What if you wanted to actually grab a component from that directory? Well, you have to do a little more work. You'd have to use the require context which builds up again that glob of files. You'd grab the module, you'd build up a list of modules. You'd have to give them some name by default. Maybe it's based off the file name. Then you'd export that out from the index file so that's available to other modules in your system. Okay, so it's all right if you didn't get all that, but the main point here is that you can do a lot of the things that you get in sprockets, but you have to give up a little more of that friendliness that we are used to, maybe more on the Ruby side. Webpack tends to give you a lot of flexibility and control and maybe the APIs aren't quite as nice as we'd like them. Just something to keep in mind. It can be easy to get frustrated when you're playing with these more low level concepts. Okay, so moving on to taming dependencies. And one strategy we used in our Rails Asset Pipeline that we wanted to bring over to Webpack, something called code splitting. So this is a simplified version of how our sprockets build looked a year ago. So we had an application.js and a vendor.js that we included on the page at the same time. Really the strategy here is to improve our long term caching. We would put the big libraries like jQuery and Lodash into our vendor.js module or a bundle and the rest of our source code was in application.js. And that might include some vendor libraries like some of these jQuery plugins. And I'm gonna highlight those two because we ran into a problem when we moved this concept of manual code splitting over to Webpack. So you can imagine, we tried to create an application in a vendor.js pack on our Webpacker side. And instead of include tags, we have pack tags now and we started moving jQuery over with an import from our node modules. And we could actually do the same with these jQuery plugins as well. And we were adding these to our bundles and we noticed that there was a problem. I would like to point out that one thing that we had to do to get this all to work, and this is specific to our needs, is that we told Webpack, which doesn't put anything in the global scope by default, is that we need jQuery to be available there because remember that we have a lot of this code in our Ruby templates that assumes jQuery is available in the global scope. That's how things worked in sprockets. And we can actually tell Webpack, do this just for jQuery, put it in the global scope. Turns out that that had some serious consequences for what we saw. Okay, so we're gonna step through the process of moving these dependencies over to Webpack. So we added jQuery and it becomes available in the global scope because of our configuration. And then we added this chosen js plugin. It turns out this is a fairly mature plugin. It hasn't been updated in a few years. You might be able to see where this is going. We could test to see that it worked on the DevTools and the browser. The chosen function was added to the jQuery object that's available globally. And then we pulled in slick carousel, it's another jQuery plugin that's actually fairly active. It's more, I would say, up to date. And we could see that that worked once we imported it into our application bundle. But then something happened that shook me to my core. All of a sudden, chosen was no longer defined and the functionality that we intended to provide was broken. I had no idea when I saw this how trying to do the same thing in Webpack that I'd done in Sprockets would just not work. Turns out that there's a great tool that you can add to your development environments to help you see what's going on inside your Webpack build. If you already use Webpack, I really encourage you to use the Webpack bundle analyzer. Webpacker gives you some configuration that makes it a bit easier to add in Webpack configuration. So you can add this plugin with some code that might look like this in your development configuration. And what Webpack bundle analyzer will do is give you a visualization of the relative size and contents of all the modules that you have in all the bundles that you're building. And you can actually click around and zoom in and get some stats. And so this is what our bundles looked like after we had added these plugins in jQuery. You might be able to see a small problem. It's actually a pretty big problem. On the right is our vendor bundle where we explicitly imported jQuery. And happily I can see that it was imported and included in that bundle. What I was not so happy about was that jQuery is also included in our application bundle. That's not what I wanted. So this is actually a big problem, especially when we were doing what we did, which was trying to put jQuery into the global scope. Well, guess what? The last version wins because there can only be one jQuery in the global scope. That meant any plugins we had added at some point got clobbered. We could prove this by looking at the source code of these plugins. So Chosen.js actually looks like this. I took away most of the functionality but there's this one key line. It assumes jQuery is in the global scope. So when we included it, it worked because that's how we configured our Webpack jQuery import. With SlickHair cells very different. It uses a more, I would say, modern pattern. It doesn't assume by default that jQuery is in the global scope. It has a snippet of code at the top of its library definition that looks like this. This is typically called the universal module definition. It actually tests for the presence of some common module loaders like CommonJS and AMD. And turns out that Webpack understands these snippets here. And so when it gets to this line, it says, oh, SlickHair cell, looks like you require jQuery. I'm in the application bundle right now and no one else has required jQuery so I'm gonna go ahead and add it as a dependency. And that's why we were getting two versions. What we needed to do is tell Webpack, we want to include both of these bundles on the same page so you need to treat some of these common imports as one. So rather than manual code split like we had been doing, we needed to let Webpack do the work. Now there are a number of ways that you could approach this and it turns out unfortunately this code that I'm showing you here won't work in Webpack 4 of course. So we're using Webpack 3. It's maybe important to get the main concept which is that this plugin that we've added, it's called the Common's Chunk plugin, will extract all common modules into the vendor bundle. So if we require jQuery in both, it's only gonna end up in vendor. Just keep in mind we won't be able to do this in Webpack 4. So we could see that this worked when we looked back at this bundle analyzer and we could see that jQuery is now included and the vendor bundle's on the left. And our application bundle's actually gotten a little bit smaller because these common modules have all moved into the vendor bundle from application. And there's only one instance of jQuery now and our plugin started working again. This all brings me to my third survival tip. You really need to know your dependencies in Webpack because they might be bringing in dependencies that affect some assumptions that you've made in your configuration. All right, another issue that has come up that we've sort of alluded to indirectly. I call module shimming. This is basically applying transformations to your dependencies. We do this in Sprockets already. We have things called preprocessors that change say our copy script or our ES6 code into normal JavaScript. And it turns out there are a lot of preprocessors in Webpack that are much more powerful, I would say, than what we typically have in Sprockets. It turns out they're not called preprocessors though. They're called loaders. And I have alluded to this before already but we were putting jQuery into the global scope using something called an exposed loader which says expose something to the global scope. Again, we're looking at now some Webpacker configuration and we can add in a loader that's targeted only to jQuery dependencies. And we can say, hey, jQuery, when you're imported, expose the dollar variable to the global scope. And as you could have seen already, that caused some problems and we found a better way to address this issue while it did get our chosen JS plugin to work because it could find that jQuery dependency in the global scope. We could actually flip the problem around and address it from the other way. Instead of exposing jQuery to the global scope, we ultimately decided to change how chosen JS works. So we can use something called the imports loader to actually import the dependencies that it needs where it otherwise expected them to be available in the global scope. This is sort of like applying the universal module definition to an old dependency. And here again is some Webpacker configuration which says every time we try to import chosen JS, apply the imports loader and replace the jQuery and the dollar reference with an actual import to jQuery instead. That way we can inject the dependencies rather than assume that they're available in the global scope. That way we could start moving away from that typical sprocket's way of looking for things globally. So survival tip number four, if you have legacy dependencies, I highly recommend you fix them through these loaders to get them to work with Webpack or just get rid of them altogether if you have the time. Webpack really wants to force you into a certain way of doing things. All right, finally, predictable caching. Let's talk about fingerprints. So when we ran our build and we have these two bundles, just like we have in the Rails asset pipeline, we get fingerprints added to the file names. So for those who are unfamiliar, this represents a digest of the contents of these bundles. And what we want is for those digest to stay consistent if the contents don't change, so we can cache them for long term. And if the contents do change, we want that digest to update. So let's just imagine in our application pack we're doing something with some function we're importing and let's add a new dependency. And that dependency might actually just be very simple. It doesn't bring in new dependencies. It's just a plain module. And if we ran that build again, we could actually take a look and see if the fingerprints changed the way we would expect. Now, if I only changed the application bundle, I would expect that only the application fingerprint should change. You might be able to see where this is going. So this is before and this is after the change. This is the fingerprint for the application bundle before and after the change. And you can see that they're different. And unfortunately, if you look closely at the fingerprints of the vendor bundles before and after the change, they are also different. And at this point, I wanted to throw my computer out the window. I really had to search deep inside my soul to figure out what was happening. I must have had some real misunderstanding about how Webpack works. Like how could it be that caching is just broken? Turns out it actually kind of is. And to illustrate why I had to learn how Webpack works and I'm going to illustrate now. Let's imagine when you build your Webpack bundle, it's going to start at the entry file. It's a JavaScript file. And what Webpack will do is parse that file for all your import statements and pull in the dependencies that they refer to. And recursively, it will go through each of those dependencies, parse those files, and pull in the dependencies of them. So what you get is a directed graph of dependencies when you build Webpack. And Webpack will link all of these dependencies together. It will translate these imports into function calls in the browser that the browser can understand when these modules are then put into your bundle. It also will, Webpack will insert some code which I'll call the runtime that will make this all work. The runtime needs to keep track of these modules somehow. It will actually assign each module an ID which you can think of as indexes into an array. Literally that first entry file gets the ID zero and so on and so forth. Well, as you've seen earlier, we were actually using two bundles, vendor and application.js. And because we were pulling common modules into vendor.js, that meant the runtime also ended up in vendor.js. And when we made some kind of change to application.js, those IDs would have to change as well. We might insert a module, it needs to get an ID itself, other IDs might change as a result. And it turns out because the runtime keeps track of all those IDs, like a manifest file, any change that we made to the application bundle would cause the runtime to change. And since the runtime is in the vendor bundle, the vendor bundle would change as well, which is why those fingerprints were updated. Well, it turns out there's a quick fix to this. More configuration. So if you recall, this is what our common module configuration looks like and it turns out what we wanted to do, or one way to fix this, is to pull that runtime out of the vendor bundle. So we had to add yet another instance of this common chunks plugin to pull the runtime out. And we also added some other plugin here, it's called the hashed module IDs plugin, so that what basically it does is says, don't assign numeric IDs, actually have the IDs of each module represent the contents so they won't change if we shift things around. So the end result of that change meant the runtime was pulled into a separate bundle. So if we changed application.js, runtime would change as well, but vendor would stay the same and that was a compromise that we were willing to make to at least get vendor caching to be, I guess, stronger. Okay, so survival tip number five, you may need to understand a bit about how this runtime works. This is a completely new concept that we don't get in the Rails asset pipeline. And if you're looking at all this and you're starting to wonder, oh, dear God, why would I ever want to learn about all this stuff? I'd have to say that I want to leave you with, on a positive note here, so I want to come to one last issue that actually I feel is a real big win for bringing Webpack into your project called asynchronous bundles. So let's imagine some code that looks something like this where you're using a pretty large package to do something only some of the time. So for example, at LearnZillion, we use PDF.js to render PDF documents into Canvas elements, but we really only need this on one particular page of the site. And it wouldn't really make sense to put this in our vendor bundle and force users to download this package on every single page load. So what we can do is take this code, which would end up, which would place PDF.js in the vendor bundle with our configuration and use this one little trick here. So changing that import from statement to an import function call is actually very different behavior from Webpack's perspective. So to read this, what we're saying is only when this code is evaluated in the browser will Webpack make a request for this package, which is not in your vendor bundle anymore. When Webpack runs its build, it will actually separate this package out into its own bundle that it can then request on demand. When this function call actually returns a promise, which won't resolve until that package is downloaded only when this line is evaluated. And then we can then continue with whatever it was trying to do, in this case, render a PDF. Really, this is Webpack's sweet spot. In fact, it's the reason why the creator of Webpack wrote it was to get this asynchronous on demand loading. One little tip, if you're going to use this type of import, you would have to use a magic comment in the function definite or the function call here to ensure that this chunk or which chunk meaning bundle in Webpack terms is actually gets a consistent name. Again, Webpack doesn't always assume what you wanna do. So you have to tell it in this case, I wanna call it a certain name. And when you run this build, you can see now that we actually have a fourth bundle that's only used some of the time. And we're able to limit the size of these initial bundles that are included on the page load. Survival tip six, if you're gonna use Webpack, I really encourage you to prioritize using these asynchronous bundles. Otherwise, why to go to all the trouble of setting it up if you're not going to use what Webpack was really designed to do. All right, so we've covered all the issues that I've felt relevant to share today. And although you may not run into the same issues that we ran into, I really hope that you can see the bigger picture here about how some of the mindset we had coming from Sprockets really needed to be adjusted when we came to Webpack. That said, I have to agree with the author of this blog post that Rails with Webpack isn't for everyone. Don't switch just because you heard that it's cool, okay? I would encourage you to really consider the trade-offs. If you have an application that has a lot of JavaScript, if you want to take advantage of these new tools, that you want to use things like asynchronous bundle loading, yeah, maybe it's the right choice for you. You'll still probably have to invest some time to learn how it works. For other folks, maybe something like Sprockets 4 is good enough. And Webpacker really needs more support and attention, like DHH referenced in his opening keynote. It's a leaky abstraction, even though it hides from you a lot of the complexity that you might get, otherwise if you had to set up Webpack yourself, ultimately, when I used it at least, I still needed to understand how it all worked anyways. But you know what, that's actually a good thing. I feel like it's the start of something that can improve over time. Just keep in mind, you will have to stay up to date because this project moves very fast because Webpack itself is changing constantly. So survival tip, my final one, understand that Webpack and Sprockets are fundamentally different. They are set up in very different ways. They have different strategies for bundling your JavaScript code together. They have very different philosophies about what is considered best practice. As we saw, Webpack really doesn't try to solve caching for you. That's something that's left up to you. It tries to push you towards smaller bundles, not big ones. It wants you not to use global code, and maybe even not jQuery at all. And these are things that you may need to reconsider if you're going to make this switch. But most of all, I will probably be frustrating if this is your first time using Webpack. Keep a positive attitude. I know that this can be frustrating at times as I've experienced myself. If you remember these tips that I've shared with you, maybe if they didn't sink in fully, come back to them later, and I promise you will survive your Webpack journey. The slides for this talk will be up on my website, rosta.net, slash talks. And I really like answering questions about Webpack or other Ruby and Rails types of topics. Reach out to me, my email address. I'm Rosta on Twitter and GitHub. And thank you all for coming to RailsConf. Thank you for coming to my talk. Get home safe after Tender loves talk tonight and hope to talk to you all soon. Thank you.