 Hello, can you all hear me? So this is the core conversation on front end performance improvements. If you're looking for general site building stuff, 345 tomorrow is one on general site building stuff. But this is where we get to talk about core. For those of you who don't know me, my name is Matt. You might have heard of me as MFR or MFR, depending on how you all decide to pronounce. The original pronunciation was MFR, draw your own conclusions. I've been droopling for a few years now. It seems like a very long time. And I currently work for HP Cloud, though I've worked for Palantir and Treehouse before. And I worked with a fantastic team on Drupal 7 module development, if you're looking for a good book. So if you want the slides, you can get them right now that I'm going to go through if you want to follow along or if you want to ditch this presentation and just know what I'm talking about. I put Slide Share first up here, rather than Speaker Deck. I'm a fan of Speaker Deck, but this conference is about Drupal on every device. And Slide Share does mobile. So you can pull out your phone, and it works for touchscreen. You can swipe back and forth. So while I like Speaker Deck more, Slide Share seemed more appropriate. But I could not put it on both. So if you want to follow along, you can grab the slides. So first I want to talk about why front end performance is important. I think it's good to understand the reason and the logic before we dive in and do anything technical or work on it or alter Drupal core for it. We've already shown we care about performance. If you go back to old Drupal cons, there's lots of sessions on it using APC. You find all kinds of material on it. We use Memcache. There's the Boost module for shared hosting. You want to deal with caching. There's all kinds of stuff for it. Drupal 7, we made it work well with reverse proxies because, well, we needed that with the size and scale that Drupal's going to. And we've done lots of internal optimizations. I don't know if catch is in here, but I've watched him profile Drupal over and over and find little things that we can do better. And there's a lot of other people who've done profiling. We look at different tools to try and figure out how can we make our Drupal sites better, faster, more efficient? How can we make them scale? How can we make them perform? But there's one thing we really haven't talked about. So the HTTP archive, have you guys all heard of the internet archive? Has anybody not heard of the internet archive? It's an archive of the internet. Well, recently started something called the HTTP archive, which is now part of the internet archive, but it looks at the technologies we use to build and deliver our sites. And it keeps a history of that and looks at it over time to see where the changes are, to see what's going on. And the HTTP archive, a recent run from February this year found that 87% of the time it takes to load a site happens in the front end. This isn't what your server is compiling. So your server, you can turn on the develop module. And you can see that maybe it took 400 milliseconds to generate that HTML page. You think, man, this is fast. That's the 13%. The 87% is what's happening in the front end. Once it leaves your server, what's happening to the network? What's happening in the browser? What's going on? And you see that that's a huge chunk, right? And we haven't talked about it very much. So that's what I wanna talk about today. But being that this is a, we're talking about mobile and on every device, it's not evenly distributed. So when you look at the breakdown of where this performance timing is happening, a company called Strange Loop took a look at it a while ago and they said, what's the difference between mobile and desktop? And how is that breakdown? And on mobile, 97% of the time is happening in the front end. Where on the desktop, it's 85% of the time. These are huge, significant chunks of time. So if we're looking to speed up our sites, this is a great place to look. So when you look at a lot of the statistics out there that tell us what's going on with, does this make a difference? Does it matter? And you can find loads and loads of statistics. There's lists of them. There's infographics. There's all kinds of details from the companies that work in this stuff and have measured it. But this one I really like. A 400 millisecond difference caused a 9% drop in traffic for Yahoo. 400 milliseconds. On the front end, it's easy to make up that type of time. But this is a real difference. And Google and Amazon and lots and lots of others have found that this matters. And of course, this conference is all about the post-PC era. We're not competing with other sites. We're competing with apps and we're working with apps. So I was on vacation a year ago. And I was on vacation and I decided, do you know what, I'm gonna use my phone to look up the details about the local museums when we're gonna travel there. And I'm gonna use the sites and see how they load. You know, I got a 3G network. I've got a pretty good phone. It was pitiful. I gave up. I gave up on the first day. I got in trouble from my wife because I was staring at my phone just waiting for it. She's like, just call them. You'll get it quicker. It's important. Everybody else was working on front end performance through all parts of the stack. Browser manufacturers, they're working on JavaScript. DNS prefetching because DNS is an issue in performance on a lot of other technologies. Mobile networks, we're going to 4G, right? Everybody wants 4G, you know? It's great that the new iPad has 4G. But we're coming up with ways to improve our networks. There's new protocols. There's Speedy. Or as we were talking earlier today, we call it Spidey, that sounds better. But Speedy, right? It's a new protocol that is similar to HTTP. Kind of works in the same way along the same technologies. But it's faster. And then of course the Linux kernel. I don't know if anybody follows Linux kernel, but they recently made a change to something called the TCP slow start that improves performance. It's the change they made is something that Microsoft and Google have been doing for a while to speed up their performance. But they made it part of the 3.3 kernel. So everybody across the stack is trying to do things to speed up stuff that impacts this front end performance. So we as Drupal need to do that too. And of course, you know, what do users think? 85% of mobile users expect sites to load just as fast on a mobile device as it does on the desktop. That's huge. And I know that I'm one of them, right? I expect my mobile sites to load ridiculously fast. In fact, I think I'm more impatient when I'm looking at my phone than I am on my desktop. Because on my desktop, I'm working. It's all right if it takes a couple of seconds. I'm used to that on my mobile device. I'm just impatient. I don't know if anybody else is like that. So if we're gonna take performance seriously, we've gotta deal with the front end performance issues, some of the stuff going on in Drupal, we have to deal with it. Or we can't really say we're taking performance seriously. So through this talk, by the way, if you have questions, if you have comments, I'm okay with a little free-for-all on this. Core conversations have tended to be a little bit like that in the past. So if you see something when I'm talking through some of this stuff, go ahead and speak up. I'm quite all right with that. So we're already doing some stuff while when it comes to front end performance. Lossless compression of our images. If you go look, a lot of our images that are in the miss directory, MISD directory, they're already compressed down because you can, a lot of what's in an image, you don't need to view it and ship it to devices. And even what's exported from Photoshop you don't need. And we've already losslessly compressed this stuff to make it really, really small. And we aggregate our CSS and JavaScript files. That's a really good thing because we include a lot of them. And each one's an HTTP request and that can incur performance hits. Now granted, we can improve on this and how we do it. And there's a lot of thought on that. But we do some of this stuff. We saw that it was a performance issue and we take it into account. And of course, our image handling is extendable. You can do things like cut out metadata inside your JPEG images using our image handling. The image magic module even plugs in and takes advantage of some of this because some of this stuff can happen. So we've thought about some of this stuff already but there is more we can do. So part of this I wanna talk about what's technically happening with page loads and things like that. Because I can point out a few things in Drupal Core but when we understand what's going on it can help us better see other things that we can do and look at how we can do this well. And just an understanding of the technicalities makes a difference. So 4G's gonna solve all our mobile problems, right? Right? No? 4G'll be awesome, right? So there's an interesting statistic I didn't put up here. Google did an analysis. They said you've got five megabits per second downstream and so let's take the difference between that and loading a page and 10 megabits per seconds downloading. And let's take a look at what's the performance improving you get from doubling that bandwidth window. Five percent. Five percent. That's nothing because our limiting factors aren't bandwidth now. So by me going up from 3G to 4G, the increased bandwidth that I'll get, yay if I'm downloading a movie. If I'm viewing a site in small files, that's not really gonna help that much because that's not where the bottlenecks are in the technology. And of course, right, Drupal is worldwide. So the UN Agency for Information and Communications points out that in 2011 45% of people who have mobile networks had 3G or better. That's it, not even half the world had 3G or better networks. So when we look at Drupal, it's easy to think you're up. It's easy to think first world countries, Australia, the United States, you've got all these great networks. Lots of people don't. And lots of people who are using Drupal that we're trying to reach so we can do things that may seem small to us that can have a huge impact on them. So let's talk a little bit more about phone networks. Phone networks, right? You have a desktop latency. You have a round trip request. When you make a round trip request to ping a server, that's a simple round trip request or to get a DNS. We're used to it being really fast on our wired desktop connections. Mobile, it's not that fast. Mobile, there's an additional latency in there for every round trip. That depending on your connection can be two to 10 times for every single round trip request. And that can add up a lot. So when we can do things to cut these out, that can be a nice savings. And TCP connections aren't just so simple as we go out and we ask for a file. We don't just go out and say, give me this file and then it sends the whole thing back. We go out and we ask for a file and we get part of it. And then we say, okay, give us more of that file. And we get more of that file. And we ask for it to say, give me even more of it and keep going. So right now, if you were to say, go get a 143K file, you're gonna make, what is that? Eight round trip requests to do that. Just to go get that file. And in that eight round trip request, if you're doing it, say on a mobile network, it's gonna take two to 10 times as long for each round trip request, each of those, and that's just for this. So it's not so simple. So when we talked about TCP slow start being increased in the Linux kernel, it jumped from, right now it's three, that's the spec, and they jumped it to 10. So that'll be a nice improvement, but it's not gonna be this killer thing that's gonna end all. So we gotta realize the technology, for all these assets we're going out to get, and all the size that's there, we make a lot of round trip requests. And of course, we only make, browsers make six parallel connections to a domain for, at a time. So you've only got six connections in parallel. So if it's outfetching six files, it's not gonna fetch the rest. So if you've got 80 assets in a page that have to be downloaded, because your page isn't just your HTML page, it's your JavaScript, it's your CSS, it's your images, it's all that stuff. And you've gotta fetch all of it on your page load. And it only downloads six at a time. And some of the mobile browsers don't even do six at a time. Android 2.2, still ridiculously popular, does four at a time. So it's downloading even less. So here's an image from Drupal.org. And I've pulled this up in the inspector and I've opened up the network tab. And what we have here is we have the homepage. And I've shown the details here of where the time is being spent and how much time is being spent in different parts of a request. This is to get Drupal's homepage. And you see the DNS lookup was two milliseconds connecting with seven milliseconds. The biggest chunk of time is waiting. This is where they've sent something to the server and they said, all right, now I'm waiting for a response. It's in that round trip time. So, and then there's receiving, right? So 171 milliseconds receiving this file. So when you look at how much time those parts were, if you shrink down the size of a file, you make less round trips, you have less waiting time. So looking at how the technology works can give us an idea of where we can do some great improvements. Let's talk about mobile devices. Because, you know, looking at desktop devices is great. But, you know, as Reeves pointed out, we're gonna be switching to lots and lots and lots of mobile devices. So what can we do about that? And one of the problems with things like cell phones and tablets is JavaScript execution takes about 10 times as long as it does on mobile as it does on the desktop. So if you've got JavaScript that takes 300 milliseconds to execute, on a tablet you're probably looking at three seconds. Maybe not for the latest iPad, but lots of people have iPad twos and that's what it's gonna take. So looking at the types of performance and the processing power and what that does to our pages, you know, when we add inline editing and we add all these features, we have to be aware of where they're gonna use these features and the performance impact that it has. Especially when we do so much development on the desktop. And then of course there's how much RAM you have in these systems, right? So the iPad two and iPhone four us have 512 megs of RAM. Anybody got anything like this in a development system? I've got 12 gigs in my main development system. You know, my other development system has eight gigs. It's, we're not even talking a gig here. The latest iPad has a gig. And so when you talk things like tablets and phones, a gig is the high end of the RAM that they have. And so when we send JavaScript and CSS and images down and they're loaded up into RAM, you've only got limited RAM space. And so we have to understand these devices are really low power compared to the desktop stuff that we're used to. So let's talk about what we can do because that's where it gets fun, right? First thing, let's minify all core JavaScript. Why not? We send all our core JavaScript unminified down to the browser. Now, minification, this is where you take out all that stuff that's unnecessary from your JavaScript or your CSS file or whatever that you don't need for it to execute. You know, we have comments in Drupal.js. Go open it up. You're gonna see these great detailed comments. You don't need it for the JavaScript to work. Your standard user who's using the JavaScript doesn't really care about all these comments. Why send them down? We don't need them. We can remove them. And that's what you do with minification. You can remove those things. In fact, really advanced minification can even do things like rewrite variables. Why have a long variable name? If inside, you know, it really could be a single letter. Minify it. So my example here is Drupal.js, right? The original minified 24% the size. 24% the size. Why are we sending the extra 76% to all of our users? We don't need to. They don't need it for it to work. Why don't minify it? Yes. Okay, so great point. GZIP, right? We're GZIP-ing it. So what are we GZIP-ing? We're GZIP-ing all of those comments and sending them down. So that's additional bandwidth. Great, it's GZIP. But we're sending down GZIP comments. We don't need it. I mean, we don't need to GZIP those comments. Yeah, you get great compression. Our indentation is double spaces. Yeah, the GZIP algorithm is great at dealing with spaces, right? But it GZIPs the spaces and it GZIPs the comments and sends them down. That's extra bytes. Especially, you know, if somebody's paying for bandwidth, they're paying for 76% extra to be sent down. Why send it? And it slows down. It takes more, you know, memory. All these things. GZIP-ing doesn't, GZIP-ing isn't enough. GZIP-ing isn't, I'll cover that in a minute though. I'll talk more in a minute. Yes. Yeah, it's less RAM space. It's less processing to unzip it on your lower powered mobile devices. And when Drupal 8 comes out, right? We're still gonna have things, people will still be using an iPhone 4S, the technology that we have now. And just think, you know, I talked about the iPhone 4S and the new iPad. What about all the people who've got a mid-grade or a low-grade Android device? Cause there's a lot of those out there. And they're not as powerful as the high-end devices we talk about so much and we look at. So think about it. So GZIP-ing, it really isn't up. But I'll talk more on that just a sec because who's minifying? Everybody's minifying, right? JQuery, Dojo, all these guys. They don't just say GZIP. JQuery is 200 and some K in plain text, right? Minified, it comes down to something like 50K. Now you GZIP on top of that, it's like 31K. They don't just say GZIP it because that's not gonna get it small enough. They minify it. Dojo, Mood Tools, YUI, you know, everybody. And even so, highlighted in right here, or yellow here, we have SharePoint and WordPress. So we talk about these guys as being like competition or friendly competition or SharePoint. These guys are already dealing with minification of the stuff because they know that it has an impact. Our competition knows and they're doing it. So here's my question for you. Do we minify on the fly or do we ship with minified files? I put this up here because we've been discussing minification for a little while and this has been one of the debates that we've had. One of the questions that's come up. How is it best to do this? And my answer is ship with minified files. So let me talk about the reasons for that. I looked at jsmin-php. jsmin is a minifier that it's been ported to PHP. It's simple when you try and get into some of this stuff that's really, really complicated. It takes so long we could never ship it because you could never run it on shared hosting. It would just take too long, too many resources to do. But jsmin-php is something we could do. And then there's uglifyjs. It's the new wonder kid of the minification world. jQuery uses it, lots and lots and lots of projects use it. It ended up producing slightly smaller files than Google's Closure Compiler. So that's what jQuery switched to it. But uglify produces 7% smaller file for drupal.js. It produces smaller files. It's able to do better transforms. But it's not a PHP thing. jsmin-php isn't maintained anymore. If you go there, it's not maintained. So if we were to incorporate it and do on the fly, we've now got to take on the baggage of maintaining the project. And I found bugs when I was trying to minify some of our files. I found some bugs in trying to do some minification of legitimate JavaScript. Yes? I haven't. Is it PHP? It is PHP. All right, I'll have to take a look at that. I'll take a look. So I took jsmin-php, right? And I ran it through on my i7 SSD hard drive, lots of RAM system. And I said, how long is it going to take to minify some of this stuff? jQuery took 2 and 1 half seconds. I don't want something like that on a server. And this is a decently powered machine, right? What happens if I throw this on shared hosting? You know, it's easy to talk about our great enterprise stuff, but Drupal's everywhere. And lots of people are trying to run it on shared hosting. We want to make sure that the time and footprint for that isn't killer. And then of course, here's my biggest problem because you can overcome. So all the stuff that I've talked about so far, you can really, really overcome fairly easily in the technical or come up with some way around it. But one of the problems comes down to is you don't want to take out licensing comments. You don't want to take out licensing comments. That's not a good thing. If something's licensed, we need to send those comments down. And some files, like a certain jQuery plugin that's in core has a second license file halfway through from an addition. And it's not the first one to do this and it's not the only guy to do this. So how can you automate keeping those when you do it on the fly? And the answer is you can't because people don't do this in a predictable, common way where you can find that. It just, it doesn't work. They've tried to, you know, come up with ways where you do slash, asterisk, exclamation point. But not everybody does that. Not everybody holds it. So it becomes a very hard thing when you start dealing with licenses and the legal side of things. You know, one of the things that I didn't put up here is right Drupal is GPL. Drupal.js is GPL. You write custom code in Drupal. You know, using the Drupal JavaScript in it. Now you've written GPL code. Now you've shipped that to the browser. Now you've distributed GPL code. Now it's not posted anywhere on the web, but it's in your stuff. So now you minify it. So now you're shipping GPL code that you don't distribute in full source anywhere. So what are the legal implications of that? And this is one of the issues. I've talked to some people at companies who've had trouble where their lawyers won't let them minify some of their JavaScript because of GPL issues. Now you can overcome that, but it's one of those issues that different lawyers might say different things. So it's not always a cut in dry situation. And so that's the other reason I look at, let's ship minified files with Drupal Core. You don't have to deal with the runtime stuff. You don't have to deal with how does it perform on shared hosting. You don't have to deal with any of that. So what we would do then, right, is you say on the performance page, have a toggle. Is it production or development? And in production, you send minified files in development. You send the full source of everything. It's simple. This is a toggle screenshot from JQuery's website because they call it production and development. That's a common naming, minified production. So that's all we really need to do, right? Simple, right? But then the question comes up of when do you minify the files? Because, right, somebody's gotta do it. At some point it's gotta happen. If you don't do it on the fly, do you do it when each file is changed in each patch or pull request that you handle for core? You also have to update the minified files that you alter. Do you do it at release time? So you say, release manager, you have to go ahead and minify everything. It's not all that hard. You can use a script that just walks through everything and does it. Or you have an automated script. You push it up and it just creates those for you. You know, this is one of those things that I would let work out in the issue queue and the people who are on the hook to deal with it would be the ones to do it. You know, if the release managers are okay, you know, creating the minified files right at release time, if there's somebody who really wants to write an automation script, all right, I don't know that I have a preference. For my personal projects, I do it when I'm going to create a release. That's the easiest time and I just work in development otherwise. That's what I personally do. But we can work out the details in the issue queues. And of course, I have a script right now, a module right now, a speedy module that provides all of this for all the point versions of Drupal 7. And it uses just a simple Drush command to generate it. It uses a Glify. One of the nice things that I discovered when I did this, right, is if you look at like Drupal 7.0 and 7.1, the locale JavaScript is broken. It didn't work. It had bugs in it. It would break in browsers. And Uglify threw those errors. So if you use something like Uglify and it's not valid JavaScript, it's actually gonna fail to minify. So it's a great check to say, is it valid? Because we shipped. I think I found three versions of Drupal 7 with broken JavaScript files at point releases. None of the recent ones, but early on ones. And I had no idea until I ran it through Uglify. So the next one, use the JavaScript module pattern. Anybody familiar with this? The module pattern? All right. So a very, very, very basic module pattern is similar to the jQuery pattern. So you've got a function and then left, this is what we currently do, right? So you have a closure anonymous function. What you do is you pass jQuery and you see at the bottom and the dollar sign is an alias. And that's how we do most of our Drupal JavaScript, right? This is what we currently do. This is, they call it the jQuery plugin pattern. It's really a light, one of the lightest versions of the module pattern. And on the right, this is a simple switch, right? We pass Drupal in at execution time and do Drupal up here. And our JavaScript goes in the middle. It's a really simple change to the wrapper. But what it's doing is it's doing dependency injection, right? Instead of counting on globals, we inject dependencies in when this function's called, which is right when it's included. And then we use it internally. It seems like a simple change and it is. But we get a savings, right? So the original is a certain size. It's 24% size, right? The minified version is 22.5% the size. And the reason for this is, is Uglify said, all right, Drupal was passed in. I can shrink that down to a single letter. Say the letter D. And every place Drupal is used within that JavaScript in between, it can shrink down to a single letter. So you cut out all of those characters. And you did it because you did simple dependency injection. And Uglify is smart enough to know it's safe to do that. Just like we alias jQuery to the dollar symbol. It's the same type thing, but Uglify can do this on its own. And we don't care, because we're not gonna read that compiled JavaScript or that minified JavaScript. So it's plain English readable for us. But at the same time, it produces smaller files. Bigger percentage savings and smaller overall files. And this difference between these two minified files is smaller than the difference between jQuery under Google Closure Compiler and Uglify when they made the switch. Because when you do it on really large scales for anybody who does a large scale, that little difference can add up. So if you wanna learn more about the module pattern because it seems a lot of people in here didn't know about it, there's a lot of variations on the module pattern. Ways you can do all kinds of crazy stuff with it. And this is maybe the best article I've read on it. It's a fantastic article. So if you wanna learn more about it, there you go. So let's talk about making JavaScript and CSS handling pluggable. We've been talking about this for a while. The idea behind this is, right now we have one system. It does aggregation. If you want to alter that aggregation, you've got to leach into this theme system. Do ugly hacks. Stuff that's not nice or fun to do, right? It's not part of native Drupal. And let's change that. Let's make this pluggable so we can swap it out with anything else we wanna do. So let's talk about some examples of what we would do if we could swap this out. If we could swap out. So let's take a look at what Google.com mobile does. They do this really interesting thing, right? So Google's mobile site takes their JavaScript and CSS and sticks it into local storage. On mobile devices. So one of the things that's not talked about. We always talk about it's great. Use expires headers, cache stuff, Drupal core cache stuff for two weeks. Let's cache stuff, right? Caching stuff on mobile devices. You might have two, four, five megs of cache space total on your mobile device. So your iPhone, right? You visit three, four, five sites. You've blown away your cache. You go back to a first site. Your cache is gone. You need to redownload everything because your cache is so small. The first browser to really knock this is Chrome mobile, which uses a variable rate depending on how much free space you have on your system. But you know, iOS, all these guys, it's a really, really small cache. And on iOS, it's not persistent. So if you power down your device, you blow away your cache. So how can we keep these files down there? Well, Google and Bing have decided, hey, let's stick it in local storage, right? And they use a cookie to say which things are in there and they send it back. And so the page they send down, if you've got all the latest versions of the JavaScript and CSS in local storage, they don't send any of that down. Shrink's the request way, way, way, way down. And then they just include the stuff out of local storage at page time so they use less bandwidth. It's faster in your devices because you're not dealing with all the network issues we talked about. It's this great way that they've sped things up. But not a lot of people are doing it because it takes time, right? Well, if we could come up with a pluggable system, somebody made it work for Drupal. Next thing you know, every Drupal site can do this. That would be a really nice performance win. So one of the things that I didn't even know about for a long time is, or I didn't think about for a long time, is JavaScript isn't just executed, right? So maybe it's in your cache, but you have to parse it and evaluate it on every page load. So even if you pull it out of cache, you parse it, you evaluate it before you do any execution whatsoever. And that takes time for it to be included in the page. So we send all this JavaScript down. So jQuery, on something like an iPhone 4, takes 320 milliseconds to parse and evaluate before you ever think about executing anything. Now the newest devices are getting a lot faster. But a low grade Android device that you get now, that'll still be available when Drupal 8 comes out, how do you deal with that? How can we do things to speed up their experience in this whole thing? And so there's this idea of lazy evaluation. And don't parse and evaluate it until you need it. So you can download it, there's ways of storing it in comments, in strings, and then at the right time, you convert it to JavaScript. So your modal, instead of including your modal stuff on every page, when somebody clicks the button that launches a modal, parse it, evaluate it, launch the modal. So they'll experience a little lag there, but they don't experience it upfront. And if they're not using the modal all the time, which is, you know, Drupal's overlay, we don't use it all the time when it's included in the page, why have it? Why execute it? Why do this on every page load? Why take the time? And so lazy evaluation is a method for doing that. If we had a pluggable system, we could look at how we could bring lazy evaluation. Now of course, right, Drupal 8 will be out for a really long time, years. So eventually, all the mobile devices will, fingers crossed, not have this problem anymore. So it's not something you'd wanna bake right into core because at some point, you don't need it anymore. Why do it? The mobile devices are fine. Turn it off and switch back to core or to another way of doing it. So if we can have the option of doing different things and make it pluggable, we can get some improvements. And then one of the things somebody brought up yesterday at the mixer out here was there's this thing called bundle cache. It's not used by a lot of sites. So the idea behind bundle cache is, hey, right now we aggregate our stuff into things, you know, is it on every page? Is it themed? Well, what if we aggregated our stuff into stuff that made sense for certain situations? So we're not generating all of these different aggregated files for all these different random cases. We're doing it for very specific things that can always be sent down intelligently, be cached really well for browsers that have a good cache, things like that. And so there's the bundle cache. It hasn't really been worked on in a while. But this idea could, if you made it a pluggable system, you could just take out what's there and make it pluggable, yes. Sweet, fantastic, definitely. So if we had a pluggable system, we could make this much easier to put into core. Yes, yeah. How would that play in with this? Well, so for your SAS files, right? Your SAS files can all be compiled down into like a single file, but there's all the Drupal CSS files that get sent down, right? There's all the core stuff, everything that's shipped with the modules. And modules, you know, so we've gotten really granular. Go take a look at the system module, right? You've got system.menu.css, system.theme.css. You've got all of these different CSS files and they all, you know, there's run for requests. There's all this stuff going on. So we aggregate them and modules do the same thing. So is there a better way of grouping these things so they can be better cached? And we could make this a pluggable system and it could be easily used by lots of people and rather than trying to bolt it on through the theme system or the other ways you have to do ugly things. So these are just three examples. So let's make it pluggable. There's an issue. It's been around for a while. Let's make it pluggable. I know some code was written in one of the code sprints here for it too. So I can't not talk about this library. Rob Lowe showed me. Rob, I see you there. Yes. You pointed out this library. It's a PHP 5.3 asset management system. In fact, the developer has offered to help us integrate it into Drupal. If you know about Composer, it works with Composer. It's a great asset management system and we've been talking about bringing this into Drupal's pluggable system using this because right now we maintain all of our own code across CSS, across JavaScript. We maintain our own system. So if other people are doing this and using it in other ways, could we possibly bring this in? So I took a look at it. It's got some good, some bad. You can tie in with external minification. This is great. So if you have Uglify on your system or Google's Closure Compiler on your, even on your production system, you can run your stuff right through it right there. It's pluggable. You just tell it where Uglify is. You tell it where the Java files are or the Java based ones and it'll do the work and it's already prepared to do this. It can even tie in with image lossless compression if you've got utilities on. So you could tie it somehow in maybe with image styles and do lossless compression if you have those utilities on your servers. So one of the things, we talked a little bit earlier about smaller images cutting out all the junk that you don't actually have to send down. What if you could do that on fly with image styles because we generate a lot of images and so there's the potential for that. Of course it's got filters, extensions. It's designed to be extended, to be added to. And of course this brings in, you can do SAS and stylus and less and coffee scripts which could all be nice. It's just easy wins right there. If you like those, if you like those. It needs a little work. It doesn't, I didn't see aggregation in it which we use right now. A lot of sites that use this aren't gonna be sending 50 CSS files down like some of our sites do. So coming up with that. And of course it needs work. It does SAS and stylus and less and coffee. It's easy to get caught up in rabbit holes of going down these extra technologies and things like that. All right so let's talk about excluding CSS files. So I was talking to a Drupal developer at a big shop and this is terribly paraphrased from my IRC but he pointed out the CSS in his aggregated files he's sending to people. Like 80% of it wasn't being used. And that's a problem. Why are we sending out all of this stuff that's not being used? We have an override culture in Drupal. That's override CSS. Instead of yanking it out and styling the item ourselves. Let's leave that original in. We're sending all of this additional styling down to all these users who have to have additional RAM, additional drawing in the browsers, especially mobile device. All this extra work to do that. Additional bandwidth that's being used. Additional processing. There's all this stuff going on. And we send so much CSS down that we don't need to. Why not just say, you know what, I'm gonna take system.menu.css, exclude it all together and I'll style those myself. So you don't have to deal with the overriding, the extra stuff there. I'll take system.theme. I'll take, you know, fields.css. I'm gonna style this myself. We'll just throw that all away. Don't just override it, throw it out. Start from scratch. A lot of themers will override everything all together but they leave that original too. So now you're sending both cases down. Why? So there's an undocumented hack in Drupal. You can exclude them. It's an undocumented hack. This is an easy way to exclude those files, exclude any of the files you want. Morton has a nice name with some expletives in it for the way this works. But it works well enough. But really, it's an undocumented hack. People don't know to use this. It's not like we say, do this, do this, exclude these files if you're not gonna use them. So let's add a documented feature. It's simple, right? But then you can just document it. You can tell people use this. You can say, hey, it's here. It's not a hack. You're not, you know, taking advantage of a system, using it out of contact. We can, you know, people can write in books and they're not writing in an undocumented hack. They're saying, hey, if you're theming, do this. What was that? Mothership from Morton. Yeah, yes. In the course of the HTML5, fantastic. I like that. You guys didn't have nearly as many questions as I expected in this presentation. So here's what I'll say is I'm doing a session tomorrow. So all this stuff applied to Core that I just talked about. But if you wanna do a lot of stuff in your own sites now, there's a lot of stuff that we can do right now in our sites to speed things up. I'm doing this session tomorrow, faster mobile sites. It's gonna be specifically targeted at mobile, but all the mobile stuff will trickle down to desktop too. So if you wanna come, you can come to this and learn what you can do right now to speed up your sites. And there's a lot of easy ones. Yes. No, I didn't mention script loaders. So we come to this, I expected more. So I'm preparing for this session, right? And I said, it's a core conversation. How many slides should I do? How much should I talk about? And I'm talking to a bunch of people, asking their questions. They say, well, this looks like a good number of slides because you're gonna get a bunch of questions. And people are gonna talk about it because that's what we do in core conversations. And you guys didn't talk nearly as much as I expected. So if you guys have questions, but script loaders. Script loaders are, so what a script loader does, right? Is right now we include our JavaScript and we can talk about header or footer. The idea is, is JavaScript because it doesn't know what's gonna be done. So you include all this JavaScript, say, in your head, right? And because it doesn't know how it's gonna alter the page, it waits for JavaScript to be downloaded and executed before it starts rendering your page. So the convention wisdom, right, is throw it in the footer and it'll get painted later. But what you can get with things like script loaders, like Lab's JS, right? So if we were able to put script loaders in, it could do something like, okay, you've got your onload, your onready event, your onload, and it can even load it after that. So there's certain things that happen. Image is, other stuff that happens into a browser and you can load JavaScript after this stuff and it can speed up that initial page rendering, page loading, your ability to click on things, scroll and do stuff like that. So script loaders can do great and if you go back to something like a pluggable system, right? So if you go back a few years ago, there were certain ways you wanted to do script loaders because you're dealing with old IE. And now there's certain ways that when it comes to mobile devices and network management, you might wanna do things differently to get better performance. What will it be in a few years? So if you took a script loader and you had a pluggable system, you could use different script loaders. You could switch that out over time. That's the nice thing about a pluggable system is Drupal 8, you know, five years from now, Drupal 8 might still be supported. Or four years from now, Drupal 8 could easily still be supported. So what's the technology? What's the browser landscape gonna look like then if we make that pluggable? But yes, script loaders are a great way to speed things up. Yes? A local storage is in your browser. So one of the HTML5 things, not really HTML5, it's got its own spec. But it provides, there's two types of storage. There's a database storage and there's two ways handle that, one of them is SQL, one of them is an object oriented. And then the other one's a very simple object storage. It looks a lot like the cookie API because it's designed to be kind of a cookie replacement. And you can stick data in that. Now per site you have so much you can do. Different mobile devices will have certain limitations on that. But you can stick, you know, like a meg in there without a problem. A lot of apps, so you go to use something like Gmail on a browser, mobile. It'll stick, you know, your most recent messages in local store. So it doesn't go to the server to get it. It reads it from local store. So it's an in browser cache. On iOS you can reset it. The local store doesn't go away. Yes? There are limits to it. I'm not sure what, it's not a standard limit, right? So the limit is, it's variable per device. So it depends on how you use it. There are limits to it. Mine has gone over five megs on the last phone that I had. So I'm not exactly sure what it is. But you're right, there are limits to it. So you wanna use it wisely. So if you do a bunch of minified JavaScript and CSS, maybe it's 200K. That's not gonna kill your local storage. You know, that's something like what Google does now. So I was really, when I saw the first slide, and when you showed the element inspector and it said it was the home.html and that was the time it took. And that was exactly what I saw when I was trying to optimize my site. And I fought with my client because I said, it's the Apache server that your people have set up, which is a problem. And that's why I thought you'd be talking more about Varnish. So I'm really concerned about the home.html or whatever the first page of the index. That is actually what takes the most time. And so apart from Varnish, what are your suggestions? Because if you see the time it takes to load the JS and the CSS, which is overtime, it's really small. The time to do that, it's gonna depend right on where you are. So do a test from, have a server in the United States, do a test from Europe on what that is. And you're gonna find different things. So you wanna use things like edge side caching, right? You wanna put Varnish in front of it. But Drupal's already designed to work with all those things. So I wanna talk about here about what things we can change for Drupal 8 to do improvements. Drupal already works with Varnish, so you can put Varnish in front of it. You can do edge side caching. In fact, you can do edge side caching of your HTML pages and have that work. There are big sites that already do that. And so it loads really fast everywhere because they've done that. And Drupal works great with those technologies already. I'll talk a little bit about some of those in tomorrow's session. But a lot of this stuff we've had sessions on. So yeah, but no, you're right. You're definitely right about doing things like Varnish and things like that to speed things up. No, it's just that I saw the screenshot and I said, you know, if you're talking about that index.php time, you're not CSS and JS, that's different, you know? That's loading still, that's still loading fast. That index is still because of those issues. Yeah, and when you look at some of the JavaScript files, some of the sites that are out there, we're sending, you know, three, 400K JavaScript files. What if those were 100K JavaScript files? So you run into those long loading times. Drupal.org is actually pretty good. But I didn't want to call out a single company or website and say, hey, look at this crappy file you have. I didn't want to, I want to be nice to everyone who's here. Yes. What about image aggregation? I noticed that my themeers usually use very common graphics all over the place. And from Drupal 5.0, we started creating a matrix kind of and using CSS background images. Yeah, totally use CSS sprites. They're fantastic, right? So, but I'm not sure about Drupal core shipping with that functionality. So I'm going to talk a little bit about that stuff tomorrow. But CSS sprites, use them. They're fantastic. But, you know, there's a core conversation. So the idea is, should core generate CSS sprites for you? Should core, should PHP start splicing these things together? Should we start doing this stuff in GD? Which isn't always the most efficient at doing image operations. It can take a lot of time and RAM. And should we add this in when we're talking about, you know, not just hosting this on these great enterprise environments, but shared hosting as well. I got a buddy who does shared hosting and just pings me in the ear all the time. And so when it comes to Drupal core, I don't know that I want that there. But I could be wrong. It turns it into data yours. You can also use something like sass or glue and some of these other things that you can generate it beforehand. I was just wondering what the overall take is on that. And then one comment on my own side on that, and I think it goes more into his account too. I definitely think generalizing, saying like ship out your module with like JS minified is a little tricky because then you get like modules which have that only in there and you can't basically debug them anymore because minifying is also almost like encrypting it in kind of ways so you can't really find a bug anymore. So I kind of like, if it's your module, I really love it. I really like the approach to actually have cached versions of the minified file. So they only generate the ones that cached and then they're pretty much like the same route like you do it, like you ship it out minified. But you have the control over it, you know. Yeah, and I'm looking at core and all the situations that core would be in for this. So when you can do some of this stuff on the fly, but how do you do it well on your crappiest hosting that Drupal's gonna run on? So for those situations, I see why. And when you're bugging, right? So if you have that toggle of production and development, that's an internal flag. Modules can plug into that and do the same thing and take advantage of it just like core could. Okay, cool. Yeah, we'll get together. My question was touched on by the last exchange. I'm wondering if you can compare sprites to encoding images as data inside of the CSS files themselves for background images. So you gotta think about it. CSS, if you're sending it down to all these pages, will the data that you're sending in there be used a lot? And if it's not, is it wasted? Because sure, you gotta deal with HTTP requests, but you're dealing with file size. So you take a data and a data, taking an image and turning it into a string isn't always the most performant way. So a lot of times you wanna use it for smaller files or stuff that's gonna be on every single page because you cut down an HTTP request. But if it's not gonna be on every page, say, it's a certain way to do bullets and you only do bullets on certain pages, it might not be a good idea to do that there and instead do it in a sprite. It's kind of a give and take on how you would do that. Again, I'm not sure where the best place is to do that for CORE, but it might be a good analysis. I think it's great for themes and for specific sites. But when it comes to CORE, I'm not sure where the best places to use something like that would be. Okay, thanks, Matt. Yeah. Yeah, I'm wondering what your thoughts are on how we can handle JavaScript dependencies management in CORE, because I feel like we talk about script loaders and there's a lot we can do with them now. Like most stuff we can put in the footer, we can do Drupal.js, put it in the footer. Most of that we could probably load asynchronously, but CORE has no knowledge of what scripts depend on each other. And so is that something we should or should handle in CORE? How could we? And does that sort of approach the point where maybe we're just redoing something like require.js? Yeah, I don't know the best situation. So I avoided that whole requirement thing, because I didn't look at it as being a high-performance thing we could get in, because I was looking at low-barried entry. And so the debate on how do we handle a script loader and how do we handle required dependencies when I've had conversations over the last few months on it, there hasn't been like this consistent, this is the direction we should go for, and I haven't had one that I've loved. Now, Drupal.CORE, using hook library, has a lot of knowledge about dependencies of JavaScript files, right? And so there's a lot of knowledge there. Now, not everybody takes advantage of it, but it wouldn't be hard to file bugs or to go in and patch that stuff so that we do because the base system's there. So a lot of knowledge about knowing how this stuff works. And so using something like require.js could be used to do that, but I don't know if there's a real agreed upon approach we can get done in, how many months do we have till Drupal.8 is feature frozen with everything else we gotta do? So I don't know if there's an approach we'll agree on before that, but it's definitely something worth looking into. Yeah? You're my hero. Thanks. Oh. First a shout out to the HTML5 folks. Totally. Who is actively separating the base, theme layer and administrative layer CSS and JavaScript so that on a page load you load less if you want to. But my question is really about from like a language perspective, you mentioned a couple of great things like the module, a focused way of handling JavaScript. I was just wondering, and has anyone done like a really in-depth review of all of our JavaScript, all of our CSS to see if we're applying best practices on performance and such? I know there's an issue about the difference between like namespace JavaScript using .org versus .find. But I was wondering if you could tell us any other resources of what we could like reference in order to research our stuff to see if it's good. Yeah, you got a comment back there? So I could search all the .druple.org issues with JavaScript cleanup tag. Look, I think is it just under the JavaScript tag? Or does it have a special tag? JavaScript cleanup, okay. Yeah, yeah, and I'm aware. So, what do you say? So for the recording, for anybody in here, there is a, you can look for JavaScript issues about cleaning up our JavaScript. Search for JavaScript cleanup in the issue tags. And I'm very familiar with the .druple.js one. But we are looking at ways to clean it up to have better practices. We're not all in agreement exactly. There is no standard document on which better practices we should use, which is one of the things we got to decide. But there is some work around that and definitely people who are interested. So if you guys are interested in learning about JavaScript and what you can do better, I'd love to talk to you. And we can point you in the right directions. I don't necessarily have the answers. Use this practice or this one that we've agreed upon. But I'll start sending, hopefully, trying to send you in the right direction. This is probably a crazy idea, but. Awesome. I like crazy. Time for it. What about serving JavaScript or core JavaScript and CSS from d.o.to for all Drupal websites? Much like Google code does jQuery and other libraries. That way you benefit from the. Well, I'll say two things. First, I don't want to speak for the drupal.org folks and exactly how we should do it. Because serving all of this JavaScript from drupal.org, yay, we don't have to suffer the bandwidth. But there's a lot of sites. There's a lot of bandwidth and that costs a lot of money. It's awesome that Google does this stuff, but there's a huge cost behind it. And then there's the whole fact that it's at the open source development labs, right? You guys know how often that gets like d.o.s. because some site is being nailed. So do we want all drupal sites from around the internet being taken down because it got d.o.s. So I would say it sounds really cool, but my guess is is that it's too expensive and too many sites would go down from some other thing being hosted there being taken down or by d.o.s. or something like that, that it's probably not practical, but it's a neat idea. And CDNs are fantastic. Drupal works great with them. There's a hook you can tie in. In fact, the hook, it's hook file URL alter or something like that. The example code in there is about using a CDN. So Drupal is already CDN friendly to do this stuff. I think it's a Drupal seven thing because people wanted to do that. So it's already there for that stuff. So yeah.