 Thank you. So, I'm going to present to you how this project works, but first I have to introduce myself. My name is Rafael, I'm from Brazil. You can find me on GitHub with Rafael Franca and Twitter too. I'm a member of the Rails core team and I work at Shopify. And one of the things that I like about the Rails core team is that you can actually choose what you want to do in that team. And I like to say that I'm the person that usually do the things that nobody wants to do. So, I'm the Rails maintainer. That means that I usually release the gents in the blog post and do issues to reage so we can have nice people doing really awesome things and not doing these boring stuff. And I also have a secret review here. I'm also the Rails bot. Yeah, that's actually true. I run this bot sometimes. And that's it. So, Rails is present in a lot of different ways. It's not just Rails james but there are a lot of different components like action view, springs, gcarry.js, turbo links, sprockets. So, you can see that the Rails project is present in almost all the layers of your computer, right? You have things on the process level like springs and also things running in the browser like gcarry.js and turbo links. So, this talk is to present something that is not well known that is how the access pipeline of Rails works right now. So, I'm going to present it to you how, why do you need the access pipeline? Which are the gents responsible for it and how its work is in a Rails application and I will get some time to show you how to extend the access pipeline. So, why do we need access pipeline? Before we have access pipeline in Rails, I think it was introduced in Rails 3.1. We had a question, where should I put my assets, right? We did not have any kind of convention how to handle client-side coding in Rails applications. So, we had to put our assets in the public folder. So, we had no convention, no code organization and usually you end up with a lot of files that you don't even know if they are being used or not. So, Rails is about convention and configuration. So, that was not something that we should have done with our client-side code and also we have another problem in that days that we had to do some trade-offs between core organization performance because browsers had some limitations like the internet where is low. So, we have to do some trade-offs like, shouldn't we create small self-contained files or do fewer assets request in our applications? Should we write a legible code or should we transmit few bytes to the clients, right? And also, there were some technologies that were being existing that time that we are not using easily in Rails applications like CoffeeScript, SAS and recently AcumenScript 6. So, to solve all these problems, we created the assets pipeline. But how these assets pipelines really works in Rails, right? So, right now, how it works? We have some conventions for our client-side codes. So, our assets live in the assets fold and there are also living assets in the assets fold. Assets are compiled on the flying in development and need to be pre-compiled in production. And we also had, by default, assets that were generated with digest to do casting batches. So, the assets pipeline is made by a bunch of Gens. These Gens are these ones, Sprockets, Sprockets Rails, SAS Rails, ExactGS and Coffee Rails. And I'm going to each one and explain how they works. So, the first thing that I'm going to talk about is Sprockets. It's the Gens responsible to compiles and serve our assets. And it defines a process pipeline so you can actually extend and build different pipelines that you want. Sprockets has some key components that are processors, transformers, compressors, directives. They sprockets the environment, the manifest and the pipelines itself. The processors are the most important components in Sprockets. They are any callable object that accepts input hash and returns a hash as metadata. So, this RubyCode is actually a valid Sprockets processor. It's doing something that is easy to understand, that is just remove the semicolons from the end of your JavaScript files because we don't need semicolons in JavaScript code. So, it's taking an input that has some special keys that I'm going to talk later. And it has to return another hash that has the data as the result of the processor running. The input hash has these keys by default. The data, the environment, the cache, the URI, source path, load path. And the metadata are the most used keys on this hash. And the return hash can be anything but there are some keys that are specials. The data key, the required key, the established key, the links and the dependency. So, the required is, I'm going to show later, but when you actually have dependencies in your assets, you declare them and they are stored in this required key. So, we have some built-in processors, but I'm going to talk especially about three of them. The table processor that is actually taking any Acrimus script 6 or whatever it's called right now. And just piling to JavaScript code so your browser can run it. The JavaScript processor does the same thing with scripting language. And that's how processors do the same thing with styles sheets that use the size language. Another kind of processor we have is the banner processor. And this processor is what is used to run and coordinate all the assets in the video file. So, to register a processor in sprockets, we use this syntax. We are telling that for any application JavaScript, MIME type file, we are using the banner processor to take care of these files and concatenate in the same file. So, the banner processor takes a single file asset and pre-pins out the required library in the content of it. And a special kind of, another special kind of processor are the transformers. And a transform is a processor that converts a file from one format to another format. So, one of the examples is the JavaScript assortment that gets JavaScript file and returns a JavaScript file. The implementation of these processors are really simple. I have the implementation here. So, it's a callable object that takes input. And it actually goes through the script compiler and returns the result of this operation as the data of the return hash. We also have the compressors. And compressors are special kinds of banner processors because it runs on the concatenated file. And you register a compressor with this syntax. And the main difference between the compressor and the banner processor is compressors are used differently. And you can have only one compressor by MIME type. So, Sprockets has a special syntax to enable compressors. And you can, for instance, compress any JavaScript file using these syntax. Sprockets has also directives. And I'm sure you all seen these directives before because they are just special commands that declares your bundlers and their dependencies. This, for instance, is an example of the application in GS that's, right now, is generated by a new application. So, it's telling us that to generate this application GS file, we have to require these three files and also all the files that inside the same directory of the application GS. So, another special kind of directive that we have in Sprockets 3, where the pre-compile list that you are telling Sprockets to actually pre-compile these two files in production. And a part of that, Sprockets had a special support to Procs on the pre-compile list show. So, you had, before in Sprockets 3, we had this code that is telling us to pre-compile all the known JavaScript and styles sheet files in the app directory. So, as you can see, this code is not easy to understand. So, in Sprockets 4, we have a new syntax for that. It's called the directive. So, it's easy to understand what's going on there. So, you can actually see that all the images in the image directory is going to be pre-compiled as JavaScript and the styles sheets too. And I can actually use this directive to compose new libraries. So, I have that link to my engine that's also defining its own manifest file. So, it's now easy to understand and to compose too. Not that we are going to remove the pre-compile list, but these new directives are there to help to extend the pre-compile list. So, we have all these directives by default in Sprockets. And I will show later how you can extend that directive to create your own directives. Another component of Sprockets is the environment. And that is exactly where your code actually runs. The environment has methods to retrieve and serve the assets, change the load path, and register in processes. So, when you're doing a web request to your asset file, what is going on is that this Sprockets environment is running and is trying to find that specific file and serve it back to you. So, the environment is also where you call all those methods that I showed before that you can register to some processes, compressors, and things like that. And the part of the environment we have the manifest that is just a log of the contents of all your pre-compiled assets in a directory. And it's used to do fast lookups without having to actually compile your assets code. This object is really simple. It actually only points the assets path to the fingerprinted version that is generated by Sprockets. So, when you're having your code, something like JavaScript includes the tag application, what is going to know is that to generate that source attribute of the script tag, Sprockets is going to the manifest object that has a hash like this inside that only maps the name without the digits to the name with the digits. And it also contains the opposite way where you have the digit name and you can find either the m type and the logical path of that file. This, another hash is used to expire caches of Sprockets so you can actually, you can actually use the same directory and use the assets that you pre-compiled in the previous deploys and you can later use this information to expiry out all the assets that you do not want to be in that folder anymore. There are more things about Sprockets that are not going to talk about here but you can find information in this Sprockets documentation and also source code. There are MIME types, Dependence Resolvers, Transformers, Sufix, Bundle Metadata Reducer. So, a part of Sprockets, the assets pipeline is made by the Sprockets Rayer Gens and as you can guess, all this Gens does is to integrate Sprockets to our Rayers application. So, it's defined the helpers that we're using in our application like JavaScript include the tag and style sheet link tag. It configures the Sprockets environment with all the configurations we have in the config initialize bar, use slash assets and it also checks the pre-compile list. This is something not new but it's since the Sprockets tree. We can actually know when we do mistakes in development not include some assets pre-compile list and these Gens is responsible to raise the exception that is telling you that we need to include that full.js file in the manifest before actually using that in development. Another Gens that we have is the size Rayer Gens. So, like I said before, the SAS preprocessor is built in the Sprockets itself but there are some particularities of integrated SAS with Rayers that need to be done in these Gens. And we have for instance each Gens defines the generators that we have when we are running Rayers scaffold something. It generates new SAS files. It also creates an imported that knows about how to handle globs, paths and ERB. And that means that if you have something like this in your SAS files like using log imports or trying to import some kind of ERB file, you need Gens without Gens. You cannot actually make this work. And it also configured the SAS processor with all the information we have in our Rayer's application. The third Gens is the exact Gens. It allows you to run JavaScript code inside the Ruby environment. And it uses the JavaScript environment that is available to you in the machine. We have some options that is working by default in the Gens, like the Node.js environment and the V8 Google interpreter. And to use the Gens is really simple. You actually run JavaScript code inside the Ruby VM. So here we are actually getting the JavaScript source code from the JavaScript website and compiling JavaScript code using Ruby. So as you can see, the Gens is used by the JavaScript Gens to compile JavaScript code to JavaScript. What gives us the next Gens, that is the CoffeeRailGens. And all the Gens does is configures the Generator. So if you don't use Generator, you actually don't need the Gens. And it also defines how to play the Handlers. So you can handle CoffeeScript files from your controllers. So after that, I'm going to explain briefly how the assets are generated in development. So in development, when you have this code, the JavaScript included tag, it's going to generate this HTML code that points to the digest version of that file. Notice that after the application name, there is a dot debug that is telling Sprockets that the bug pipeline is going to be used. So when the browser does the request to that file, it Sprockets Rails understands that that file is going to use the bug pipeline of Sprockets. And the bug pipeline of Sprockets is defined like this. It's actually a pipeline that is going to generate your asset, but put a suspect comment in the end of the file. So after the entire JavaScript code you are going to see something like this. And this is telling your browser to actually get all the information about the source code in the source map file. So to build the source code of this file, Sprockets is going to use the full pipeline. This is inside the source map comment code. So that I'm not going to show here. But the full pipeline is defined like this. It's just a small functional call inside the Sprockets environment. And what this functional call does is check if you have any kind of bundled processor for that type that you are going to handle and use that bundle processor to build the asset. For JavaScript, it uses the full bundle processor inside the Sprockets. And the bundle processor will compile all the required files and measure them. And to compile each individual required file, Sprockets is going to use the self pipeline. And the self pipeline is defined like this. The same thing of the full pipeline, but calling a different function. And what that function does is build a stack of processors. First it gets all the post processors of that type. Later the transformers of that type. In the end, it gets all the preprocesses of that type. And to actually read the file from the file system, it adds a new processor that is the file reader that actually goes to the file system and gets the source code. So it's built a stack like this where each component uses the input of the previous component. So first it reads from the file system and the coffee script processor actually compiles the coffee script code and returns the JavaScript code and later the directive processors get all the required and all the directives. So in the end, the bundle processor measures off there and the result is sent back to the browser. So this is how the assets completion works in development. And the difference, the key difference between development and production is that in production all of this happens in the pre-copyote task and only static asset is returned to the browser. So nothing of that is going to happen in runtime. So how to... We can use all this knowledge to extend these pockets. We can, for instance, create new directive. So this code is actually real. It's from the Shopify application. We have an NPM directive that goes to your node models path and try to get the dependencies from the NPM installation. So we create a new directive processor that inherits from the Spockets directive processor and Spockets use a convention that every single method that starts with process ends with directive is going to be used to the directive processor. So if you have the NPM directive, the method is processed in NPM directive. And after that, we just registered that prep processor for all the JavaScript files and we instantiated the directive processor. And we can use this kind of thing in our JavaScript components now. We can actually load the load dash library from the NPM model installation. Another example that we have in the Shopify application is we actually have a lot of images that are SVG but we have to actually support EA8, I think. So we have to cover today from SVG to PNG. So that's how it happens automatically in the precompiling. And all we need to do is to register a transformer from SVG to PNG. So we can use something like this, like we can actually ask to generate the full PNG file from the full SVG file that we only have the SVG version in our file system. Or we can also ask to generate our PNG file from all the SVG files that are inside the images folder. And the code to do that is really simple. This is the real code. It's just a code method that actually gets the input that is the SVG source code. And ask the our mesh kitchen to generate a PNG file and we return that PNG file in the data. So my effort in this talk is that I know that Sprockets is used in many Rails applications right now. But many users don't know how, don't even know how it exists. Many users don't know how it works. I did not know how it works two years ago. So it's important to you to try to understand your tools. Documenting or the same doing talks or writing documentation for these tools and share with the community. So we are running out right now in the effort to save Sprockets. So you can see more about that tomorrow in the future talk. That's called saving Sprockets of course. And that's it. So we are hiring in Shopify so if you want to work with me, we have a lot of different open positions right now for all the different offices. You can talk with our team in the exhibition room. So there is a Shopify buffer there. And also we had two talks before mine from Shopify people and we are going to have more tools. So we have today how we test Rails at scale at Shopify. I think after this talk. And we are going out to see the Rails 5 features that you heard about with sharing briefing. So that's it. Thank you.