 Okay, okay, I guess we'll get started. So, I'm here today to talk about something that I've been diving into over the past 12 months through my work at Pantheon in terms of exploring all the performance characteristics of deploying HTTPS and CDNs with modern 2017 technologies in terms of browsers, in terms of CDNs, in terms of web servers, in terms of the configurations of Drupal. And we've learned an enormous amount through our work and I'd like to convey that so that people, whether they're using our platform or not, can deploy effective acceleration technologies to their sites and understand how big the impact is of getting these things right. So, I think the first thing that is important to make, especially for a lot of the people here who are with agencies or working on projects or have to answer to someone else prioritizing things is what is the importance of these things? How does this impact the behavior of search engine ranking, site visitors, and the experiences of people on mobile devices? And how does that actually impact the bottom line? The first thing, and this is probably no surprise to the people here, is that HTTPS is absolutely essential in 2017. The, right now, Google is already using it as a ranking signal. They've started doing that a couple years ago, but it's now being accelerated in terms of its implementation in terms of its effect in other areas. Where, as you can see on here, with the example for Chrome, it's gone from HTTP being sort of a neutral thing for sites to HTTPS being the basic assumption, especially if there are any data forms on a page. And Google's planning on ratcheting up this effort. They've already started this process by putting NotSecure on pages that appear to have logins or take credit card data. They're planning on rolling that out with NotSecure brand on sites for all forms in the future and eventually ratcheting it up to from a neutral info thing next to the URL to something like this red warning sign that you're seeing there, where it actually looks like what would have been broken HTTPS before. Because honestly, no HTTPS is worse than even just a broken configuration with a bad certificate or mixed content. And browsers like Firefox are also starting to gate keep features around your location, notifications, streaming media, device motion and orientation around the idea of delivering the page with HTTPS. This is partly for privacy reasons and partly so that they can give it as a carrot for people to secure their pages. The reason why I'm talking about this though in as a coordinated question with CDNs is this can have impacts on performance for sites and we'll be talking about the interaction between HTTPS and site performance in a later section. But it definitely has an effect on the time to first bite for pages and the ability for pages to be cached and hitting browsers quickly and search engine indexing bots quickly is also essential. The study done by Moz in 2013 of how Google was using page performance as a search engine ranking parameter for their page rank algorithm determined through extensive analysis that time to first bite seems to be the indicator that Google is using and they penalize pages starting at about 500 milliseconds for that time to first bite. They tested other theories in terms of the overall size of the page in terms of megabytes, the time to first paint on the page. I'm forgetting all the other things that they tested as hypotheses but the number one thing that actually showed a strong correlation with Google's ranking behavior was time to first bite. That's not everything for performance but since it impacts your page ranking and it's also sort of where you get to start the race to loading the page, it's still a very important measure of how the baseline performance is for a site. And changes in search engine ranking position have a massive impact on click through rates for searches. So it's not simply a matter of, oh, that might dump you down by two or three and that doesn't have much of an impact. It means a whole lot less people will actually enter the front door of the site. We're gonna be talking about other drop off as well but the first thing you need to do to actually give someone a good experience on your site is to actually get them to your site and performance matters for doing that these days. And even more dramatic effect happens in terms of user behavior when it comes to the actual loading time of the page for things like time to first paint or one of the things that Google is now pushing for as a metric time to first meaningful paint which is when stuff stops jittering around the screen when the actual main content is there when you can actually start scrolling through the page it doesn't mean it's fully interactive yet it means that you've actually given the user something to do other than stare at a white screen. And users are quite impatient about this. The study done by HubSpot aggregated 12 case studies on how different companies have seen their page painting time impact the conversion rates on the sites and they found that as you add every second or so you lose about 25 to 30% of your conversion rate. So it goes from an average conversion rate of about 1.9% if you're under 2.4 seconds down to 0.6% at 5.7 seconds it actually drops by more than half. So we have two things that are getting compounded here and in terms of the funnel of people getting to the site performance is essential to them clicking through to the site and actually waiting to actually do something on the site and doing something that you want them to do on the site. And conversion isn't just about selling people things it's whatever you want people to actually do on the site it could be signing up for volunteer shifts or something it could be looking at conference sessions or submitting one. Regardless of that people are impatient when it comes to loading these pages. So we need to look at both of these metrics and hold ourselves to meeting these. At the same time we also know from these metrics when something is good enough. Performance can be quite a rabbit hole you can almost always make it better and the efforts that you put into it should be prioritized versus a whole bunch of other things on the project. So if you are actually beating this time to first bite and you are actually beating this time to first paint then there are pretty diminishing returns for getting much more performance than that. However we've seen from our own testing at Pantheon that a lot of sites are not meeting these at least in terms of on a global basis or in terms of the time to first paint with how the assets were getting delivered especially to mobile devices. But so I've talked about HTTPS as a ranking signal performance as a ranking signal and the behavior of users when it comes to time to first paint. But HTTPS and performance can kind of get in each other's way. It's not nearly as bad as it used to be. The vast majority of problems that are inherent to HTTPS have been solved in terms of the actual experience of users, the computation on the devices, the negotiations of connections but it ultimately comes down to this simple multiplier which is how many round trips are you taking and what is the actual time of those round trips. When you look at something like crossing an ocean you're just gonna start getting terrible round trips. Even cross continental connections can get terrible round trips. The latency from the east to the west coast of the US is about 75 milliseconds even under great conditions. The latency across the Atlantic Ocean whether whatever direction you're going is often more than 100 milliseconds and the latency from APAC to Europe or the US is often 200 milliseconds and this is not just one round trip that happens when you load a page. There are many, many round trips when you load a page. And HTTPS can actually add quite a few of them especially when it's deployed under bad conditions. The, at the same time, a lot of these things have been improved with technologies that reduce the number of round trips. I'm gonna go through all the different types of models of how things negotiate HTTPS and what the cost is to page load times but we've solved a lot of problems. So I'm confident in saying that HTTPS can be delivered very effectively if you do it right. CPUs are so fast now that you're not gonna really see any overhead from that. We've solved many of the round trips by using something called TLS Full Start which drops one round trip out of the connection. Things like HTTP2 drop a trip out of the connection. New technologies like Quick are coming down the line to drop more round trips but fundamentally you always have to have at least one round trip for negotiating your TLS for HTTPS. And actually fundamentally for loading a page you're gonna have to have at least three round trips until things like Quick are available as broad technologies because you have to have TCP negotiation or the establishment of that connection. You have to have the encryption established and you have to make the actual web request over HTTP. But there's a pretty dramatic difference between a modern stack and an old stack. The old traditional way of doing this with older versions of TLS, older browsers, older versions of HTTP and TCP based connections resulted in approximately eight round trips at a minimum in series for starting to really load a page. That's not counting anything of downloading the content of the page. That's not counting queuing behavior in the browser where if it actually fills up these six connections and needs to add other things to download on the page those will get queued up and those requests will get sent after the other ones are finished. So in reality you're often gonna be way worse than this in this old case. You're often going to be stacking probably 10 or 20 round trips. So that's a pretty big cost when your latency is across something like an ocean. The modern stack where you have TLS 1.2 a false start where that drops out a round trip. HTTP too, which allows one connection to be used for everything and make all the requests all at once. But no quick in the sense of how we're seeing what's available on web servers as of right now. When I say modern stack, I mean if you deploy a basic CDN or a well tuned version of a current release of something like Nginx, this is what you're gonna get. There's also a sort of future stack where it's going to improve a bit more where quick moves HTTP to function over a UDP connection that doesn't require initially establishing one round trip. So it knocks even one more round trip out of that. But there's a much bigger difference here between eight and three than there is between three and two. So even just doing what is decently installable and configurable on a modern release of something like Nginx or a CDN is still gonna get you a very, very good place. But the effects become even more dramatic once we start talking about the geography of how things connect. When it comes to things running over fiber connections, they don't quite run at the speed of light. They run at something about 70% of the speed of light. Give or take, it's a pretty consistent number, but there's additional latency added in by things like routers. There's congestion over a lot of those connections because they're very heavily used by many, many parties. And so you definitely wanna avoid those. But just to kind of illustrate what a CDN really brings the table in this sort of setup is about reducing the distance of these round trips where you may have to do two or three round trips to actually start getting that page loaded, but the actual multiplier on those, the other half of that multiplier is the actual round trip time. And if you have no CDN and you're crossing a pretty good distance, say an ocean or across a large body of land, you're gonna have a round trip time of starting to approach 100 milliseconds. So three times that is actually adding quite a bit of time to your page load and you're getting dangerously close to hitting some of the things where Google starts penalizing you because this is not, Google's measurement of time to first byte is not just based on them negotiating the connection and submitting the request. It's when the data starts coming back. So that includes things like hitting page caches. If you miss a page cache, you have to hit something like actually Drupal and PHP. So it's about buying yourself a lot more time. If you have something like a pop on a CDN, often a pop is gonna be accessible to a typical user within 10 milliseconds of a round trip. And what's really neat about how these connections get negotiated is pops tend to maintain persistent connections back to their origin. So they do not actually have to go through all of the round trip effort every time that they need to send something back to origin. So even when you're missing a cache, it still dramatically accelerates the page because the end user device is doing all of these round trips to what is effectively a corner store. And then that one has a big fat supply line that is set up in a persistent way all the way back to the origin. So you end up with something where even under a cache miss scenario, you're multiplying say three times versus the latency to the pop. So about 30 milliseconds plus a one round trip to the origin, which is going to be something like 50 to 250 milliseconds. Now I'm gonna make a huge assumption on a lot of these, which is that your Drupal instance can spit at a page in 200 milliseconds. That's probably optimistic for a lot of sites, but I also know that it's achievable, especially on Drupal 8 with some of the lighter weight ways of delivering pages. So you'll have to adapt the numbers that I'm gonna show accordingly in terms of what is the cost of missing a cache for a page. But I still think 200 milliseconds is reasonably representative. So as we were talking about earlier, the time to first bite is sort of when you get to kick off the race for loading a page. The faster you can actually start the race, the faster you can actually start doing the work of really loading and pumping out the page and rendering it on the user's device. The longer this takes, the later you're starting that race. There's no way to make up for lost time for the time to first bite once you've squandered it. The story is already actually pretty bad if you don't have a CDN and use this old style stack on the same continent, where I'm assuming about a 45 millisecond latency here where this is about half the way across the US or approximately across Europe. If you do this old style system, even if you have say varnish deployed at your origin and you have people connecting with HTTPS to the site, you're gonna get a time to first bite of just under 400 milliseconds. That's not buying yourself a lot of time. And in fact, if you miss the cache, you're actually going to already be breaking what Google wants in terms of your page performance. Now, if you deploy a modern stack with a CDN, which I mean, there is a middle ground here where you deploy a modern stack without a CDN, but just to kind of avoid taking into account way too many numbers, I'll talk about the kind of best case and worst case in terms of doing this best practices versus something you might have deployed in 2012. So a modern stack with a CDN has much, much lower round trip times. The typical times we're seeing on systems like Cloudflare and Fastly is about two milliseconds or less to most of the pops. And this means that you get a time to first bite of under 10 milliseconds under a cache hit scenario. That's actually pretty achievable. And even if you take 200 milliseconds to render your page and it has to go all the way back to origin, which is that 45 millisecond round trip, you still comfortably are within what Google is wanting in terms of loading the page from their own metrics. In fact, you have enough head room to actually spend about 400 milliseconds rendering the page on a cache miss and still making the threshold that you want to make for Google. Now, this becomes considerably more dramatic as you start ramping up the distance between the user and the origin. As soon as you start talking about Europe's North America or vice versa, you start ending up with a time to first bite of almost a second on a page miss. This is before anything else is downloaded for the page, before the browsers even parsed the HTML. In terms of a modern stack with a CDN, where it has to do one round trip across an ocean on a cache miss, you're still very comfortably within the thresholds that Google wants for page ranking and still giving yourselves a lot of head room for loading the page in an acceptable amount of time for users because this is, of course, getting added as the baseline to the page rendering. Did you have a question? I have not found good data on that, but it does matter, but that is part of why I'm also showing the same content result where even if Google is indexing under an ideal circumstance where they are on the same continent as your origin, you're still gonna pay a high price for time to first bite if you're missing pages and not using a CDN. You're already breaking the contract, basically, with Google missing on a cache miss that way. And the way that bots actually access these pages is pretty likely to cause cache misses because sometimes they're asking for different parameters than a normal browser would, so the Google bot may not ask for pages in the same way that you might hit your cache for normal users, depending on how your cache is configured. So it's reasonably common to miss the page cache when you're delivering something to a search engine bot. Okay, so in the case of if you're deploying to, this is not advancing. So in that case, you're looking at numbers closer to this if Google's indexing it from the US and you're running your site in Europe. In fact, as you can see here, where even if you deploy varnish near your origin servers and you hit the cache, every time the Google bot is indexing you, you're still breaking the contract already just by going across the ocean and having an old style stack when it comes to HTTP and TLS. And you're still comfortably beating it with a CDN. It gets yet more dramatic as soon as you start having this kind of 175 millisecond latency of crossing the Pacific. So this would be something like Australia to the US or Australia to Europe or South Korea to Europe. You name it, it's hard to get anything less than about 150 milliseconds from any of those locations to Europe or North America. In this case, you're badly busting what Google wants out of your time to first byte and you're actually getting dangerously close to busting through the 2.4 second barrier for page load before you've even started really loading the page. As you can see here, a cache hit in a CDN. The time to first byte is still pretty consistent where most CDNs are gonna be delivering that because they're just hitting the local pop. So the distance for cache hits doesn't really increase as the distance to the origin increases but the cache miss time does. So you still have a little bit of headroom actually with a cache miss if you're say, hosting a site in Australia and Google's indexing it from the US. You can actually then have up to about 300 milliseconds of a budget for Drupal to deliver a page on a cache miss while still making that 500 millisecond barrier. So just to compare all of these numbers, I wanted to plot them out. You basically cannot see the new plus CDN page hit because it's so negligible. It's about six milliseconds across the board but you can see how dramatically the actual rendering time increases for the lack of a CDN and an old style stack when it comes to things like TCP and HTTP 1.1. And you can actually see that really the only way to consistently hit the goal of under 500 milliseconds is by deploying a CDN in front of the site. But this isn't the end of the story for your users. There's a lot of performance still that impacts them from that time to first byte because that is just kicking off the race to actually download the content for the page, render it and provide initially a paint of the page and then eventually an interactive experience for users. This is where we're starting to try and look at hitting that 2.4 second goal. And the clock to 2.4 seconds begins when they start making the requests. So the time to first byte is already counting against us here. A lot of this comes down to size and bandwidth, CPU time. Google is starting to emphasize this extensively with their testing tools, which you can use either the kind of lightweight version of I think test with Google, something like that. But the one that I tend to use which has a much richer set of results is webpagetest.org. And that runs on a simulated browser. And what it does is it actually will confine things like the bandwidth to be a simulated 3G connection because, oh, you can at least configure it to do that because according to Google's own statistics on user access over mobile devices, at least in Europe, about 70% of users accessing pages are going to be accessing it over 3G bandwidth connections until about 2020. And that's going to be kind of the tipping point for the 4G class connections to really start taking over. And when I say class connections, I mean, even if the phone says LTE on the screen, there are all sorts of matters of congestion and interference and distance and congestion on networks that can affect the actual bandwidth that's available to the device. So what Google is saying here is that devices are effectively behaving as if they have 3G connections even if they technically are connecting over a 4G protocol. This starts to really matter when you're delivering the setup of the CDN when it comes to the big question of, are you using a CDN for the assets on pages or are you using a CDN for everything? This turns up to make a huge difference because the old style approach of deliver the actual webpage from your origin server and then just put all of your assets on something like CloudFront and S3 doesn't actually buy you that much compared to doing the full measure. The reason why is that a lot of the technologies for connecting to the server now are much better optimized for getting everything from one shop in the sense that when a browser connects with the HTTP2, it's able to put in a request for all of the assets and the page pretty much simultaneously depending on how much pushing you're doing on things. And that means that it has to negotiate fewer connections. It has to do fewer DNS lookups. And it's able to multiplex and more effectively use its bandwidth without having to wait on a bunch of round trip times. So this is kind of comparing this case where if you have no CDN and you actually have in this case, I'm assuming CMS processing time of closer to I think seven or 800 milliseconds. So a little bit more pessimistic here. But if you had CMS processing time on a cache miss of 700 milliseconds and kind of no cache going on, this is assuming like a no varnish scenario here where you just have a CDN and you just have something like Drupal. You can buy a lot of time by moving the assets over because all of these CSS and JavaScript and image files are gonna get downloaded from a CDN. But until you're actually caching the page close to the user as well and allowing them to get all of the assets from that same location, you can't really hit that magic number consistently of hitting the 2.4 seconds or less. And this is because of the way that the waterfall works for loading pages in a browser. This is kind of illustrated here. So it's a little bit approximated because I didn't wanna just have the really noisy diagram and something like Chrome's developer tools but that's essentially what this is based on. So I spent a lot of time taking pages, loading them in Chrome and looking at the waterfall diagrams and getting ballpark numbers of the effect of different configurations, CDN setups, caching setups, et cetera. And you can see it's a pretty awful case here if you have no CDN, no proxy page cache. Honestly, I've been making the case for proxy page caches since 2008 so hopefully you have at least that but that still only buys you about 700 milliseconds on this. And even buying 700 milliseconds out of this doesn't actually fix your situation in terms of getting to the actual goal of the 2.4 millisecond page load time. And here I'm showing both the time to first byte on here as well as the fact that it downloads the HTML, it makes additional TCP connections to download additional assets. This is assuming an HTTP 1.1 server. A browser like Chrome will make up to six connections to a given host to download additional assets. Everything else has to get kind of cued behind that for that host. And then it will download the critical resources and then it gets the time to first paint. But assuming you're following the advice I've been giving since 2008 and you have an origin page cache and you even have better than that, you actually have a resource CDN, something like CloudFront with S3 for your image files. CSS and JavaScript. What you're gonna get here is you're actually gonna get pretty comfortably down to below 2.4 seconds but this is still assuming that that you actually have a pretty decent response time from the origin for hitting caches, for delivering those pages. And this means that you're at about 1.5 seconds. This is actually pretty decent but it's not as good as you can get. And this is about buying yourself headroom at this point because once you're down below about two seconds or so of page load time, users are gonna be pretty tolerant. You're not gonna have much attrition based on impatience. So this comes down to what happens when you miss caches in Drupal, what happens when connections are congested, what happens when someone is clicking on a page like a view that might take a while to render and run its queries. This buys you a lot more headroom for bad circumstances. And this is a case where you're actually not caching the page just at the origin but you're actually caching the page all the way out to the pops that are distributed around the globe. And, let me just check that in. I wanted to also go over what we're seeing at Pantheon in terms of statistics that we're looking at real world sites as we convert them over to this sort of CDN setup. What we're seeing empirically is that these are the page load times for the median time to first paint for sites that were running where they were running on our legacy setup which actually had varnish but no other CDN aspects to it and still it would have HTTPS as well. If you, and that was based out of Chicago but if you're loading it from the west coast of the US that was giving you a pretty decent experience. In fact, in this case EU here is referring to London that was actually giving still a pretty decent experience in the sense of about 1.5 seconds if you were hitting the page cache and downloading all the assets over that. There's actually a pretty nice pipe between London and places like Chicago though. So this is still pretty optimistic for Europe. The situation would be considerably worse in Eastern or Southern Europe. But then it really busts the budget of performance when you start hitting Asia. What you end up having happen here is these latencies just come to bite you. Going across these Trans-Pacific Connections just empirically was not resulting in great response times and then we compared these same sites as they were deployed to the CDN setup that we integrate with the platform which is built on top of Fastly but you can get these same effects from any major CDN that does a good job. In this case we were able to get sub second page loads actually across the globe simply by moving a well performing high cache rate hit site to a CDN setup that handles the HTTPS negotiation, handles HTTP2, provides IPv6 for mobile devices which by the way has its own performance impact. And that was getting us great results. And this is sort of a graph where we basically just sorted all of the sites into a sort of line so you could see the kind of distribution of where the sites were in terms of loading times. This is about 300 sites that we randomly selected to move over as part of the initial migration. And what you can see here is the darker colors are with the CDN and the later colors are without it. And you can see that there's a dramatic reduction and in fact this actually brings the median page load time which is what I was showing on the previous graph down to under a second in all three regions. By the way Asia was I believe from Singapore. So that actually went, oh I forgot I added the section. So I also wanted to update people on best practices because we've done not just a lot of work on what is effective in terms of the CDN infrastructure but how the complete coupling of Drupal plus CDN plus modern browsers works as a combined unit for getting page load times down. One thing that is totally worth cutting out of your practices is just stop using separate CDN domains if you have HTTP2 on your stuff because what's happening is you're actually just creating more work for the browser. It does allow it to parallelize the downloads more but that's only worthwhile when you don't have HTTP2. So worldwide about 86% of browsers support HTTP2 across all devices, at least in the U.S. the number is closer to like 95% and it's pretty similar for Europe as well. So there's not really, if you wanna optimize for what your users have optimized for HTTP2 not for this setup where you do a bunch of different connections to different hosts with HTTP1.1. That causes additional DNS lookups. It causes additional connections to have to get negotiated. It just actually creates a higher multiple on that latency situation for your setup. The also don't use separate CDN domains because when you have something like HTTP2, browsers just work better using it as a one-stop shop. So this is actually the biggest reason to combine your assets and your page onto the same CDN not just for the waterfall results that I was showing earlier where you just improve page load times in terms of number of, in terms of the caching of the resource being closer to users but it also just reduces the number of connections that have to get negotiated because what happens is that a user downloads a page like HTML, if they're getting that from one origin over here they're gonna look at the, the browser's gonna download that, parse the assets and then see, oh, I have these new domains to look up. It's gonna have to do the DNS lookup on those. It's gonna have to establish the connection on those. You've actually added about even a couple hundred milliseconds to your page load time depending on the part of the world by doing those things. And actually on net, HTTPS now improves performance on browsers. This is another one of the carrots that browsers have done to encourage people to deploy it. Most browsers only support HTTP2 if the site is loading over HTTPS. So getting it onto that will actually create a drop in page load times even though it's making one more round trip to negotiate TLS. Technically the HTTP2 spec does not require HTTPS but I'm not aware of any major browser that supports it otherwise. So one of the other kind of mantras that I've been trying to push is all performance is front end mobile performance. And this is because it's the hardest problem to solve. If you have poor back end performance you're gonna have poor front end performance. If you have poor desktop performance you're gonna have poor mobile performance. And so if you can get a great user experience on mobile then you're almost certainly gonna have a great user experience on desktop and you're almost certainly gonna have a great user experience from a kind of responsiveness of the site from a back end perspective. So I'm not saying that front end encompasses all the things you need to care about but it is a comprehensive measure of how well you're doing across the board. So this is why testing tools like Google, Google's site test tool and the webpagetest.org when you put it on the mobile connection emphasize doing the mobile oriented results for the site because it's just kind of the hardest problem to solve and if you solve it you solve the other ones as a side effect. The biggest things that I'm also seeing in terms of configuring Drupal is compressing images effectively. I haven't looked into all of the options in core for the configuration around it but we probably need to make a pretty big push toward webp as an image format. It shows pretty dramatic reductions for many of the types of images that people have on sites. Google will handily tell you how much data size you will save by moving to it where appropriate. In fact it's usually the number one thing that you can improve based on sites I've tested that is not related to deployment on a CDN and a modern infrastructure. Using appropriate resolutions also matters as well. Delivering multiple megabytes to a user is just a drain on them and when it comes if you're trying to actually have accessibility to users in developing countries the cost of loading your pages can actually be prohibitive if your images are too large and your site is too heavy to load. There's a neat thing on webpagetest.org. There's a little thing that you'll see just to the right of your results it'll have dollar signs and a link and it's just basically kind of dollar signs implemented almost like for a restaurant review listing for how much it costs and when you click on that it will take you to a page that will actually show you what the cost to load that one page is in many major countries and then it even has an option to switch it to purchasing power parity to basically see what the real cost is versus what your eye might be making to someone in say India loading that page on their mobile phone and it can be pretty staggering. Some of these pages that I've tested would literally cost someone the equivalent of a dollar or two to load a single page. The other things in terms of infrastructure are getting HTTP2 out there. I can't harp on that enough in terms of how much it actually allows pages to download much more concurrently and saturate connections. This matters even more on mobile connections because the mobile network itself is often adding up to 200 milliseconds of latency on its own so every round trip you can save is a big bonus to mobile users loading things. A lot of how Google makes technologies like AMP so effective is by reducing round trips to from the phone to the servers and distributing things cash to a CDN and using appropriate image formats. One thing that I haven't personally benchmarked but have seen really promising results from in other places is if you are deploying to a data center and say not using a CDN yourself using good TCP congestion control seems to have a major impact on page loads. There's a new technology called BBR. I believe that it's deployed to Google Cloud at this point. It might also be on AWS, I haven't checked but it basically means that when it's pumping the data down from the server the data center connection is almost certainly way better in terms of bandwidth than your user's connection and as it's pumping down the data it's usually going to run into congestion in the sense that data's gonna get backed up and it's going to have to respond and older protocols do have problems like excessively backing off where they'll see congestion and then they'll just go whoa hold up and they'll basically stop sending data for a little bit and it just introduces additional latency to downloading it and it causes the user's connection to not actually use its full bandwidth when you don't have good congestion control. A good CDN ought to be managing this problem for you if you deployed it one. There are a couple things that I'm disappointed I can't say are today even though they're kind of on the horizon. One of the things I really wanted to say at this presentation was you could turn off aggregation for CSS and JavaScript because of how it downloads things in parallel but it turns out this is not good enough yet. This actually adds a substantial amount of latency to page loads on average it seemed to be adding about 300 milliseconds when you had say a default Drupal install which by the way has an excessive number of CSS files like 100 if you don't aggregate it versus when you turn aggregation on with the Drupal 8 basic install and then you have about three. This still makes a huge difference even with HTTP2 and all these other technologies that I'm saying are available today because even though it can request all 100 files at once they're compressed a lot less effectively and this is because when it comes to file compression in these traditional methods like GZIP they were operated on a file by file basis so the compression dictionary which is sort of the index of things in the file that it's shortening and finds repeated only exists on a file by file basis with things like GZIP and that means that when you display your CSS across 100 files that the compression dictionary has to be sent for each one and they're each independently compressed and you don't get to take advantage of the redundancies between the files which are pretty massive. So you end up adding even a few 100 kilobytes to page loads of Drupal 8 just out of the box if you don't aggregate the CSS that matters and it seems like even though it's requesting all the files concurrently things like some of the side effects of using TCP instead of UDP seem to still have an effect in terms of things like congestion control and head of line blocking where like when it sends a byte of data that byte of data has to reach the device before later bytes of data. There's a sequence to TCP and anything that violates that basically causes it to have to rejigger things re-send things doesn't have a lot of effective a lot of ability to juggle all of those requests in terms of pushing them down effectively at least that seems to manifest in the empirical results. So I'm hoping that with these last mile improvements I'm talking about here that eventually I will be able to provide that device quick. Not only has that advantage of knocking out that one round trip to establish a TCP connection but it also provides much more flexibility for the ordering of the data in terms of getting that data down to the device reducing that head of line blocking problem reducing some of the congestion problems. And there's a new compression format that is fairly standardized not on everything yet. It's I actually haven't seen it too broadly deployed but it seems to be on the path to getting there. It's called a Brootly. And, or is it brutal? I'm hoping I didn't type of that. I haven't had the opportunity to work with it personally because it's not implemented on very many web servers yet but the biggest advantage of this is it actually has a shared dictionary for everything from that connection so that it can take advantage of the redundancies between all the CSS files even though they're separate requests. And then I also did some experimentation with HTTP H2 push. There is a module to do that with Drupal. The way that the module is written right now it sends a separate link header for push for everything. So if you turn off aggregation it actually breaks a lot of caching and H2 servers because it sends a header for every single file and that ends up with over 100 headers for no aggregation but if you aggregate the data it works fine. I also have a patch for it that actually fixes that issue for the other case but it still was not fast enough for me to actually recommend not aggregating CSS and JavaScript. H2 push is a neat thing because it basically has the server say I know that the client's going to need these things I'm going to push them down even before they're requested. The client can then refuse things that it knows it already has cached so it's not terrible if you send things that it already has but I'm looking forward to implementations that involve things like cache manifests where basically the browser can inform the server here's what I have with the initial request so that it only pushes things down that it needs because you could still end up sending users a lot of unnecessary data especially mobile users where it kind of matters from a cost perspective to those users if you just push things promiscuously down to the client that it probably already has. So I do think push will be pretty amazing once it has a bit of a better ability to negotiate it. There are some ways to put that in place manually with something like cookies where basically you can do a thing where you send a cookie that you expect that it already has the file and when it sends the HTTP request you can actually read the cookie and see what you had sent it before and make some decisions about what to push but there's no real automated way of doing this right now and I'm not aware of any CDN that supports it out of the box. So it's very, it's promising but I think these things need a little bit more refinement before I would recommend using that for sites. I did not see a huge drop off in page load times from adding H2 push with the aggregation in place. It probably has a bigger effect on mobile users. So with that I will open up with the floor to questions. So the question is what is the impact of IPv6? The impact is mixed. It goes both directions and here's why. So when you're delivering something over IPv6 most browsers use an algorithm called happy eyeballs to simultaneously try and pull the content using IPv4 and IPv6 and whatever wins in terms of connectivity is what they tend to use. The reason why this has rolled out to a lot of browsers is because there are a lot of poorly implemented IPv6 networks and there are also a lot of cases where the IPv6 is tunneled over IPv4 which causes fragmentation and causes it to work considerably slower than v4 over the same connection. These are not inherent problems with IPv6 so much as how they're deployed in a lot of cases. On the other hand for mobile users IPv6 is a benefit almost across the board and the reason why is that a lot of carrier networks have long since run out of IPv4 space and they run every v4 connection over carrier grade NAT even sometimes more than one layer. So they're having to send it through a lot more processing to convert it to their public address space and that is a cost to the page load time. So in many cases on mobile networks the performance of v6 exceeds v4 especially in developing countries like India because they are the first places that have implemented carrier grade NAT and are not handing out truly public v4 addresses to mobile phones. Another impact that you might want to consider is if you're publishing any apps to the Apple App Store there has been a requirement for a while now that the resources for that app have to be available over IPv6 for the app to be eligible for publishing. So it may just be a requirement that you have to do it depending on what you have connecting. So let's say you're running REST on Drupal as an API and you have a mobile app that's reading that data Apple's going to require that you make that available over IPv6. Most CDN support IPv6 now and it's your choice what records you want to put in place in terms of what you list for users. I mean at Pantheon we recommend all users add their IPv6 addresses to things because when it comes to a possible little downside on the desktop with a likely upside on mobile devices mobile devices are already the more performance constrained case and are rapidly gaining ground as the more common use case of people accessing a lot of things. So I tend to want to optimize for mobile and I tend to want the internet to get better. So I encourage you to deploy IPv6 even though there are a couple caveats. Yes? So the question is if you're using an asset only CDN but you can base, I'm hoping I'm phrasing this well. If you're using an asset only CDN is that an effective deployment if most of your traffic is coming from the same continent as your origin server? So I would still strongly recommend deploying a CDN to it. That's not the right slide. So this is exaggerating the difference a little bit in the sense of you could do the modern stack with no CDN which would sort of help this out but you actually add in, it actually also has slightly worse than this as well because if you use an asset only CDN you're adding an additional DNS lookup for the asset only CDN you will never have the ability really to do HTTP to push because the webpage that knows what assets are needed is disconnected from the thing that's delivering the assets. So you will not be able to enjoy those gains. The also, I find that a lot of CDNs are just actually deployed at much better exchange points on the internet so the negotiation times for connecting are still dramatically lower where we consistently see about two milliseconds to some of these major CDNs whereas even if people are coming from the same continent you're still gonna see latencies of anywhere from 10 to 45 milliseconds from the same continent and then that gets multiplied by the TCP and TLS and HTTP things and then you add in the penalty for the CDN but just to put on my security hat for a second you also need your page to be behind a CDN if you want to actually have protection from denial of service attacks because if only your assets are on a CDN then you can't protect the pages themselves and if the pages don't load then your site is effectively down. Yes, it depends. I think I had a slide in here at one point that was explaining this but there's a concept called tiered caching. I don't actually have that slide in the stack. There's a concept called tiered caching that does make it unnecessary. So on something like Fastly that's gonna be called something like Origin Shield on Cloudflare I think they call it tiered caching and it's one of their paid add-ons for more effective routing of the traffic and the reason why this matters is the more pops you have, the more your cache is kind of getting diluted around the world especially if you have something like say the worst case would be like if you deployed Cloudflare with no Origin cache and no tiered caching because what you would have is about a hundred pops around the world and every one of those would have its own cache and if it misses the only thing it can do is go back to Origin and actually have Drupal render the page and when you have that many pops the chance of keeping them well populated with cache content at least for the pages themselves is much lower. So what you want is something where you have a pop that you hit under ideal circumstances and then as the traffic converges on the Origin it either hits another layer of the CDN that hits a cache or you add a cache in front of your server. I recommend using the tiered caching approach within a CDN because that means you can consistently configure both your pops and your Origin rather than having to manage varnish plus a CDN but you get basically the same impact. It comes down to if you miss the local cache can you actually hit something else because it's still a lot faster for users to hit that than also wake up Drupal. There's definitely a diagram on Fastly's site about this though. So basically the way it works, where was it? So here it is. Basically they hit the edge cache and then if it misses the edge cache at the pop then it hits an Origin server which also has an opportunity to hit. And then when you're loading a site you can actually see this in the results where, whoops, it's just getting a 304 one second. So you can actually see this in the results of these caches. It's probably hard to see but it basically just says X cache hit comma hit. That means that it populated the cache here from the Origin cache that also hit when this local cache missed. So it means that, and it actually shows the route that it took here, that it went through HHN which I don't think that's Frankfurt. I'd have to look it up. I thought that Frankfurt was the closest to here but it basically has Chicago as the Origin cache and then this is the local one that it's hitting the second thing on here. And this is something that you can get from plenty of CDNs. This is not a specific thing to Pantheon or Fastly. Correct. That partly makes it much easier to handle things like revalidation and the aging of the content and the rules around the caching of the content because you get consistent rules deployed to both the Origin cache and the pops that are around the world. And that makes it easier to change your cache rules to say, do something different for particular pages. When we actually deployed the CDN for Pantheon, we, oh, I just realized that people were looking at my speaker notes. The, I was wondering why people were chuckling. That's from an earlier presentation. Anyway, we used to have a varnish layer that we ran ourselves on our legacy edge. And then when we implemented Fastly, we implemented Origin Shield and we no longer run a varnish layer within Pantheon, it coordinates it over the CDN network at Fastly. And so that way we can hit things even if people are coming from different parts of the world. Any further questions? Yes? Yes, I can show that off. It's actually pretty neat the way that it works. And this is not like a Pantheon specific thing necessarily, although we have it deeply integrated into the platform. I don't want to get too like salesy, but the, so we were calling it granular caching and we've made a module available that could probably be easily adapted to working with just Fastly directly or other things that support this. I'm aware of Cloudflare, key CDN, and Akamai supporting this sort of hash tagging, although Cloudflare only supports it at the enterprise level. So we call it, oh, that's the WordPress plugin. There's the Drupal one. So it's pretty early stage, so we're kind of testing it out and showing it off to people, but the basic way that it works, and this is kind of, I don't know, I never get really tired of showing this off. So, oh, actually I wanted to pull these up in Cognito. Because I'm going to log into the site. Let me actually pull it over to HTTPS. And if I go on here, I will, I'm logged in. So what you can see here on these pages, why am I not actually getting the listing here? Oh, that would do it. The text block, oh, I pasted that in there. Okay, so let's load these. You will see that they're hitting caches. So what you can see here is this thing, Sergey key raw, the reason why we output it as Sergey key raw rather than Sergey key is because it's already getting consumed in the Fastly layer and we just provide this for debugging and we don't want a case where someone stacks Fastly in front of Fastly and starts indexing their stuff by keys that they're never going to use in that layer. So this is just for kind of insight. This is all based on harvesting the caching data out of Drupal because it has quite a rich cache key system. So it sets all these keys. Here we have the front page of the site. You'll notice that it's actually annotated that it has content from node one, node two, and node three, as well as a whole bunch of other things related to the configuration of the site. And the fact that I'm an author on these pages, like that's why user one is on here. So if I changed my username, it would know that this page gets affected. Essentially this tracks the ingredients going into the page. Same thing is true on these other ones, although this one you will see only node one and this one you will see only node two. Those are hidden caches as we can see by the fact that the age on each of these is considerably longer than zero. But if I go in here, I'm on node one, I can edit this content, oops, save it. And what this actually does is it puts in a request on the back end through the container system on the platform to invalidate anything related to node one on the site. So basically anything that has content from node one touching it, doesn't matter whether it's in a view, on a category listing, in search results, in a block on the view page, on the front page of the site. If it has ingredients from node one, it knows that it needs to change. So if I go over here, of course, the node one page here is gonna be invalidated. So we now have an age of zero, we've missed the cache. The node two page, though, has actually not been touched because it doesn't actually have anything using node one. The front page of the site, likewise, versus the node one thing, also gets flushed because it has content from node one. Now, if node one falls off the front page of the site, then it will no longer have that tag and it will no longer get cleared if I edit node one. So this effectively allows recalling anything from the cache that has been affected by alterations in the CMS. Yes. Pantheon Advanced Page Cache. I need to make sure that I'm not missing anything. So, yeah, and so if you just look at the source for it, it's pretty clear how you could adapt it for doing it with anything. And then we also have this available for WordPress in case you dabble in that. Yes? Do which one? It's a mixed answer. So for assets for the page, if the CDN is configured properly, like, I mean, the way that we configure our stuff and you can configure yours if you watch the rule set, you should be able to hit the CDN for all these assets, even for authenticated traffic. Second, you will still have gains even if you're missing the page cache because the negotiation for the page will still be this rapid bounce, back and forth bounce between the pop and user, and then the pop maintains a persistent connection all the way back, so it's able to reduce the number of round trips to actually load that. So you can still knock hundreds and hundreds of milliseconds off the page load, even though you're missing a page. And then the final advantage is, as things like the Brootly and Compression start getting rolled out and more things like H2 push with things like Cache Manifest get developed, the authenticated page will still be able to use the CDN to hint what stuff to push to the client aggressively so that as soon as it starts getting the HTML, it's able to render the page really quickly. So there's still a lot of advantages, and then also there's that attendant security advantage I was talking about in terms of protecting the site from denial of service attacks, which the site is actually much more vulnerable to if it's using a lot of authenticated traffic and public users can get accounts. It's a lot easier to create situations where you're putting a lot of load on the origin. Okay, oh, I'm gonna close up here, but thank you. Thank you.