 Hi, everyone. I'm Joe, and I'm the author of Broccoli. Broccoli is a build tool that is backend agnostic, and it's used for compiling JavaScript applications. This is my Twitter. But before I start talking about Broccoli, let me talk to you about what motivated writing Broccoli. So let's say you're writing a web application, like an Ember application, that has a lot of JavaScript to compile. And so if you're running on Rails, then you're probably using the Rails asset pipeline. But if you're not on Rails, then chances are you're going to be using Grunt. So you configure Grunt to compile all these, to compile your coffee script files and your templates, and to concatenate everything into one big blob. And then you minify, and you probably also compile SAS, and maybe a few more build steps. And you also, when you edit something in the editor, you want it to automatically rebuild. So you configure Grunt Watch. So you edit something, and then you reload in the browser, and it's automatically rebuilt. And everything is great. And then you work with your team on your application, like just a small team, couple people, for a few months. And the application grows, and it gets slower and slower. And a few months in, you edit your files in the browser, in the editor, and then you wait 10 seconds. And then you reload in the browser. And clearly, that is not good, right? That is your core feedback loop as a developer. This added, rebuilt, reload cycle. And we want it to be as fast as possible. So just a quick show of hands. Who in here has used Grunt? Awesome. Who in here has had performance problems? So great. I'm preaching to the choir. Let's try that again. Awesome. OK. And so we've been having this problem, right? That we really need a build tool that scales to complex applications. Also to complex libraries, like Ember Core, it was built with rake pipeline, and it was really, really slow, like 60 seconds. And now it's we, Robert Jackson, rewrote the build definition for Ember Core in Broccoli. And now the initial build still takes a while. It still takes like 20 seconds or something. But when you edit a file and reload, it takes about one second. So that's much better. So this talk has three parts. First, I want to show you, give you a demo of how to use Broccoli in practice. And the second part is about the plug-in API, how to write a simple plug-in. And the third part is about how it fits into the larger ecosystem and Ember in particular. So let me switch to the editor here. Am I speaking loud enough, by the way? Can you hear me in the back? Is that OK? Very good. So here we have an application. And let's take a look what's in here. We have a public directory with an index HTML and a cat picture, which we're going to use later. We have a styles directory with an app.sas. We have a vendor directory with jQuery and a lib directory. And this application doesn't do anything. But to sort of simulate realistic build times, I just dropped in 200 placeholder files with coffee script code. So what I'm going to do now is say, Broccoli serve. And it's doing the initial build. It's still building. OK, so it took about seven seconds to build. And let's open it up. There we go. So it loads in the browser. Now we're going to try just picking a random file in here. I'm going to just take file 123, one of these 200 files. And we're going to add some stuff in here. So I'm just going to do like body append. Let's put a cat picture in here. I hope I got this right. Awesome. And I'm going to enable, oh no, it's cut off. I don't know. I'm going to enable live reload in here. And now if I save, without smart rebuilding, it would take seven seconds again to rebuild everything. So let's see how long it takes. One, two, three, go. Awesome. OK. So this is the build workflow that I want you to have, right? Edit something in a large application, and it shows up immediately. Also, something where tools and the make lineage of build tools fall on their faces a lot is when you delete a file. If you have something like make, you delete one of the input files, make is not going to be smart enough to delete the output file. So let's see what happens if I remove this file, this 123 file. OK. Broccoli picks up the deletion as well. And this is not just some kind of weird edge case, but this is super important whenever you switch branches. Probably some file is going to be deleted or moved around. And with make or tools like rake pipeline, I've had really big problems in the past where whenever I switched a branch, I had to restart the server because it wouldn't automatically delete the file. So let's take a look at what the build definition for this is. I'm just going to take the code out there for now and write this thing from scratch. So this is the brocfile.js. And it's like a grunt file. It contains the definition for your build. So we're going to start by just exporting. This is a node code, by the way. We're going to start by just exporting the public directory. So now if you run broccoli serve, it's just going to serve all the files in the public directory. And let's use the SAS compiler to compile the app.sas file. So I'm going to define a variable app.css here, which is going to hold the output of the SAS compiler. So let's take the app.sas file and compile it into assets slash app.css. And then for this to show up in the output, what we're going to do is we're going to merge this app.css tree and the public tree. So app.css is a tree object. It represents a tree that is going to be rebuilt continuously. And it just contains one directory assets, which in turn contains one file, app.css. And so here we merge this tree and just copy the public directory on top of it. And so both of these are going to live in the same file hierarchy. Similarly, we are going to generate the app.js tree. And we use the concat plugin. We're going to run against the lib directory. We're going to take all the JavaScript files. And we're going to generate assets slash app.js. And again, we're going to merge this into the output. Now you may remember we had a jQuery.js file in the vendor directory. So what we're going to do is create an artificial tree called JavaScripts, where we merge the lib tree and the vendor tree into one tree. And we're going to use this JavaScripts tree and pass that into the concatenator. And then we want the jQuery file first, and then everything else. Thank you. Awesome. So finally, we have coffee script files. So we don't want to use the lib directory directly, because it doesn't have JavaScript files. It has coffee script files. So what we're going to do instead is we're going to create a lib-compiled tree and run the lib directory through the coffee script filter. And that's going to give us a new virtual tree object called lib-compiled. And we're going to pass that in here. Let's see if that comes out right. I'm missing an array here. The treacheries of life coding. All right. Now it should work. Awesome. Very good. So let's visualize what we just did. So the primitive object in Broccoli is not a file. Other build tools use files. And that's the approach that I took initially. But it turns out to cause problems for compilers like SAS, which have imports. So you don't just operate on one file. You operate on an entire tree of files. So let's see what's going on here. We take the tree of files in the lib directory. We run it through the coffee script filter. Have a virtual tree called lib-compiled. Note that we do not need to have a temporary directory here. We do not need to manage all these intermediate build outputs. Broccoli manages that for us behind the scenes. So we have this virtual tree called lib-compiled. We take this, as well as the vendor directory, merge it into a JavaScript tree. We run the concatenator plugin on this to get an app.js tree. We do the same thing with the styles tree. Run it through the SAS compiler and get an app.css tree. And then we merge app.js, app.css, and the public directory to get the output. So should you use Broccoli right now, it's pre-1.0 beta software. So I'm pretty confident in the architecture, in the tree-based architecture that I showed you. I'm not as confident in some of the API signatures that we expose. And I think there may be some changes coming up before 1.0. So if you're going to use Broccoli right now, be prepared for some upgrade paints occasionally. But I think in general, it's quite usable already. The 1.0, the stable version, should be out, I hope, in a few months. So in the second part, let me talk about the plugin API that we expose. Something we learned from the grant developers was that if you expose a big API in your build tool, you're going to be in trouble later on, because you can never, ever change that API. There's going to be an entire ecosystem of plugins built on top of it. So every mistake you make in your original API is going to be set in stone forever, basically. So I took that advice to heart and tried to stick with Broccoli to a really minimal API. And then push as much of the code, as much of the smart rebuild code that gives us the sub-second rebuilds that I showed you. All of that code lives in external plugins. So the Broccoli API is just a few methods. And then all the complexity lives in external NPM packages. If you write a plugin, you're going to at least use the Broccoli writer base class for your plugin. You're probably not going to write against the Broccoli API directly. But more likely, you're going to use the Broccoli filter base class, which is for a large category of plugins that have one-to-one file relationships. So let's say one input file goes into one output file. An example of this is the CoffeeScript compiler, which maps every .coffee file, gets mapped into a .js file. The Broccoli filter plugin exposes basically one method that you have to implement if you write a filter-based plugin. And that's process string. It takes a string argument. And so, for example, the CoffeeScript source code. And you return the compiled output. So let's see what this looks like if we implement the CoffeeScript plugin. This is the CoffeeFilter plugin. So we tell it to run on all the .coffee files. And the output is going to be .js files. And then this is all the code we need. The process string method is going to take the CoffeeScript source code. We pass it into the CoffeeScript compiler and return it from this method. And this is all you need to really implement a Broccoli plugin. Importantly, BroccoliFilter has a cache. So if it didn't have a cache, then whenever we rebuild, whenever anything changes, it would rebuild all the files. Like all the CoffeeScript files would be reprocessed. And that would make the rebuild very slow. So BroccoliFilter, because it imposes the one-to-one relationship of input files to output files, it knows exactly how to cache files. So when you rebuild, if a .coffee file hasn't changed, it's just going to pull the .js file from its cache. It's going to know that it doesn't need to rebuild. And it's only going to call process string on the files that have changed. And so finally, where BroccoliFilter doesn't work is compilers like SAS, which take more than one input file. So an app.sas file might depend on a mix-inst.sas file. And that file, in turn, might import another file. So we need to rebuild the output file, the output app.css, whenever any of these input files change. So it doesn't match this one-to-one input output file constraint that BroccoliFilter imposes. So for compilers that have an import, that have some notion of file imports that take multiple files and turn them into one output file, BroccoliFilter is not suitable. But a surprisingly large class of plugins are expressible as filters. So how does BroccoliFilter fit into the larger ecosystem of JavaScript tools? So as you saw in the demo, we have a Broccoli command line tool. So you can type BroccoliServe to start up a server. And you can type BroccoliBuild to build your project into an output directory. But I don't really foresee Broccoli being used through this command line interface all that much. What I really want to happen is for Broccoli to plug into task runners like Grunt. So to clarify this, Broccoli doesn't try to replace Grunt. Grunt is a perfectly good task runner. And we've just been abusing it as a build tool. And I think a task runner is probably not really suitable for building applications. So Broccoli wants to provide the core build pipeline inside your tooling. And then Broccoli should plug into Grunt. And there is, in fact, a Grunt Broccoli plugin that you can use to kick off Broccoli from Grunt. And it's very minimalistic right now. It just exposes the serving and the building. But in the future, I hope that we are going to have hooks into the rebuilt lifecycle so that, let's say, you can automatically re-run the test suite whenever you rebuild. I think it would also be good to integrate with backends more because oftentimes people want to serve an API and the static assets from the same host. And so people tend to resort to things like proxying and stuff like that. And I really like how the Rails asset pipeline did it. And I would hope that a few months from now, we're going to see Broccoli plugging into Node, for example, but also Rails. Like, I would really like to replace the Rails asset pipeline and have a Broccoli Rails gem so that you can use Broccoli with Rails. And finally, I don't really think we want to, whenever we start a new application, we don't want to rewrite a complete build definition from scratch. The demo I showed you was like seven lines, but realistic sized build definitions are going to be much larger than that. They're going to be like 50 lines of code or something. And I don't think we want to be writing that from scratch every time. What I really like with Rails is that you can type Rails new, and you get a scaffolded Rails app that you can start implementing immediately. And you don't have to set up the build definition for that. And I really, really like this having a default stack that gets generated for you. And people have been working on the same thing with Ember. There is a new tool called Ember CLI. It's kind of the spiritual successor of Ember AppKit. And it uses a similar app layout. And Ember CLI is Broccoli-based. So if you use Ember CLI and you type Ember new, my app, it's going to generate an Ember app like a skeleton. And it's also going to generate a Broc file so that it automatically gives you a skeleton build definition so that you can get started immediately. So that's how I would really like to see adoption being driven through tools like that. So if you want to read more about the Broccoli architecture at this URL, I posted a blog post where I talk about some of the kind of use cases and considerations that motivated some of the architecture, some of the decisions in Broccoli. And if you have questions, how much time do we have? Three minutes. So let's have a quick Q&A. I got three minutes for questions. Yes? For example, you said that in a few months it's going to be ready to discuss 1.0 version. What's missing right now? Just so many cases that have to be published? Yeah, so what's missing for 1.0 is I think I'm unhappy with some of the API surface that we expose. And I've talked to Robert Jackson about it. And we both think we can probably do a little better. So we would like to get that fixed before 1.0. There's also some API that we expose for continuous rebuilding for the watching, where we would like test runners to plug into that. So we would like to have an API that allows you to kick off your test suite whenever there is a rebuild, that kind of stuff. So that is basically those other parts that are missing. There's nothing terribly hard. So if we start using it right now, the major changes we're going to do, it's like some mappings maybe, and stop using maybe some methods. There will be some incompatible changes, but nothing terrible, nothing major. Yeah? In terms of the plug-in landscape, I mean, is there any big gaps that you wanted to fill? Something that's in terms of the plug-in landscape, I think the biggest gap is JavaScript concatenation, maybe, because there's just no standard for that. I hope to get a sprocket type, like a Rails Asset Pipeline type compiler out soon with slash slash equals require. It's kind of terrible, but it works. And what we would really like to see is, of course, tools like RJS, BrowserFi, or even better, ES6 modules plugging into that. We have an ES6 module compiler, but that one is too much alpha. I don't recommend using it right now. Yes? Could you see broccoli being used as deployment tool? So as for deployment, I think what's going to generally happen is that you have a deployment tool like Capistrano, and it's going to call broccoli to get the static files. So the same way with Rails, you have Rails Asset Precompile being part of the deployment process. You would have something similar with broccoli. Assets, digests, like when you compile it, you really usually get a digest again in the file. So asset digests like fingerprints, so let's say application-longhexcode.js. That's what we're talking about, right? The tricky part about getting these digests into file names is not renaming the files. I mean, that's straightforward. The tricky part is rewriting URLs. And I have not seen a sort of coherent proposal for how to get the URL rewriting into all the preprocessors. I think the URL rewriting support has to live in like each compiler, like SAS and something like EJS, like Embedded JavaScript. So those are the parts that are kind of missing for that. In the meantime, there's some talk about maybe using a there are some tools that just will run over your project and use some reg access to rewrite the URL. And there's some talk about turning that into a broccoli plug-in as a stopgap measure. I think we're out of time. So thank you, everyone.