 All right, so we're recording Welcome everyone, I'd like you to I'd like to welcome you to the Drupal Association webcast in this particular webcast today, we're gonna be chatting with some folks from Fastly and We're gonna talk a little bit about how Fastly has been taking Drupal.org and making it more secure Helping us keep our costs under control and generally way more performant A few housekeeping notes before I go into introductions here We do have Q&A open and I would encourage folks anytime you have a question to go ahead and jump in and ask a question I'll give a few reminders throughout the webcast about that And just to kind of kick things off at this point. I think I'll do a brief round of Introductions here on the call with us today. We have Lee Chen Ed Bender and doc from from Fastly and no doc. I did not try to butcher your name again. Sorry and then also I'm Josh Mitchell I'm the CTO with the Drupal Association and Rudy Grigar also from the Drupal Association is with us. So Welcome everyone. Hello. Good morning Or good afternoon or evening National good morning everybody Hello everyone Right Well, let's go ahead and kick into this so when we when we started The story really started with hey Josh you should you should talk to Ed from Fastly They've got this really cool open source program and they might be able to help you out with your CDN and the conversation, you know started really as a opportunity for an organization that Drupal Association as a nonprofit were Pretty reliant upon the goodwill of tech companies that can help us make our infrastructure more robust more performant and So we had this conversation that kicked off and we kind of went back and forth for I think the better part of six months To a year just talking about oh, what would it look like for Drupal? Dot org and our various services to move over to Fastly Started with really correct me on this, but we started with the file server, I believe is the first thing we tested out file downloads so all the packaged Drupal projects then we did updates traffic and Then right before the release of Drupal 8 We moved Drupal dot of Drupal dot org's web traffic over to Fastly as well So at this point we we push about 1.5 billion requests per month about 20 terabytes Which is impressive when you consider that those 20 terabytes are pretty much all text files. I mean it's it's zip code and webpages that represent that 20 terabytes. It's not like we're Pushing a lot of video or anything like that But that's been the kind of general structure of what we have there The open-source program at Fastly, I just I want to thank you guys for it. I think it's it's an awesome program I believe you guys standard kind of Offering is up to a thousand a month for an open-source project to use and credit services We've worked out a great deal with with Fastly where they've given us even deeper discounts on our traffic above and beyond that kind of base limit and It's been awesome. So thank you again, Ed Thank you Lee for the work and pulling all that together. It's been huge our pleasure on that It's definitely in large part due to our great marketing team. So It's our roots are in open source. And so that's a big part of where we come from so glad we're able to continue that awesome Well topics today for today's webcast We're gonna go through quite a bit actually We've got about a half hour and I want this to be kind of conversational again if you have questions and you're you're offline and you Want to ask those to the Q&A? I'm happy to pass that those on to doc and Rudy as we're getting into some of the technical stuff But the topics for today's webcast we're gonna go a little bit over varnish convig. What is it? Why would you use varnish with Drupal? We're gonna talk a little bit about TLS and SSL security benefits that we get through our fastly configuration a little bit about how we handle logs with fastly and and how we can have data streams that allow us to present Some great data back to the community about How projects are being used and things like that? purge speed origin shield We're gonna talk about how we removed our load balancers and use fastly basically to do our load balancing now a little bit about latency improvements and then I'm also going to give an opportunity for the fastly folks to talk about some of Things that are coming next in their product because I know we're really excited about some of the things that are coming down the pipe that are going to give us some opportunities for more performance and a better experience for our end users, so Without further ado, let's let's get into this All right and they can VCL Rudy. Why would you use varnish with Drupal? Yeah, so Way way way back when varnish 2.0 was released back. I think it was around 2008 maybe early 2009 Drupal.org was not using any sort of reverse proxy and So like why would use a reverse proxy? In this instance varnish is a caching reverse proxy. So What we were trying to do with varnish was reduce the number of requests that we're making it back to our Apache server and hitting PHP and kind of saturating that Kind of limited pool of connections. We had at the time so varnish was set up to basically Cache all of our static assets to avoid requests for things like JavaScript and CSS and the you know few images that we were hosting at the time and avoid those requests so we kind of started varnish 2.0 and We're using it and over time starting to optimize it more and more to do things like anonymous page caching and stuff like that And it worked great. It helped us really kind of like build Out the infrastructure and help us scale as triple seven was released and the Drupal.org community grew and we were handling more and more requests and When we had the opportunity to start using fastly what it let us do is basically move that proxy server off of our Internal network and move it to the internet. So fastly is a you know giant distributed varnish And we were already using varnish and since fastly is based on varnish. We were able to kind of take our same configuration that we were using Internally and push it out to the world and truly reduce all of the kind of network traffic hitting our origins and move it closer to the point of presence Fastly caches that were closer to the global community that is Drupal.org. So it was very easy for us to you know quickly Deploy our existing configuration up to fastly when we started using it and then also to Load new configuration and revert changes and test things through fastly's interface interface Very cool So in addition to the the benefits we get out of varnish. Let's talk a little bit about security and in Doc Ed Lee if you guys want to kind of weigh in with some other examples that you've seen in the other sites So you work with feel free to kind of jump in and talk about those as well, but Obviously Rudy we care a lot about security around here a little bit about like what the advantage is With the TLS and SSL benefits that we get from them Yeah Is it another kind of historical thing for a very long time Drupal.org did not have Any sort of HHPS support and when we did roll that out at the time Our servers were relatively handicapped and had trouble with the number of requests. We were hitting with SSL and HTTPS and TLS So kind of knowing already like having a CDN they can handle a lot of that raw traffic For things like the JavaScript and the CSS and all these other requests that were somewhat taxing to our infrastructure was a nice sort of way to offload a lot of that kind of management of SSL requests TLS requests To fastly but then also Like knowing that fastly is caching a lot of these requests and not even hitting our origins at all so There's the performance benefit there of having that and also With having our wealth card certificate and all of our services routing through fastly you could turn on the HSTS support which Is like a client-side browser forcing of HTTPS for anything that matches start out Drupal.org as a domain name So there's a lot of benefits there that you know We now we don't have to manage that SSL certificate Ourselves on the fastly side. We know that they're kind of following best practices around Cypher suites and all of the sort of security around Having the HTTPS on Drupal.org So there's two benefits there and a lot of it is just that we don't have to worry about HTTPS support anymore on the Drupal.org infrastructure. It just it just works Actually, I have a quick question for you now now that you have HSTS for all of Drupal.org Are you planning to include the domain in the preload lists? It is already included in the preload list. Awesome Yeah, and and I don't know just just a quick thing like why would you want? HSTS and why would you want HTTPS because? You know, I know a lot of people who run websites are like, yeah, I don't I don't need encryption like I have nothing interesting on my site Yeah, I mean for us, you know as a there's a lot of reasons, you know, the big one is You know, we want all the traffic to Drupal.org to be secured in a way that can't be Can't be eavesdropped by other people, you know if you're in a coffee shop or something in your Accessing over kind of an insecure connection, you don't want people to be able to intercept things like your Password that you're logging in with or your session cookie or something like that could that could potentially compromise your session on Drupal.org and having HSTS Kind of forces that HTTPS across the entire domain So if you're you know, if that session cookie is tied to start on Drupal.org and you hit something like Over HTTP that session might be attached to it. So having it encrypted kind of ensures your security across the Across the transaction to Drupal.org Doctor, do you have something to add? Yeah, actually, there's there's also the fact that some nation states let's say, you know, for instance, France They might have a giant firewall and if the people inside that country then visit Your website that might be hosted in I don't know the Netherlands or the US If it's not encrypted that giant firewall can then change the traffic as it's going from your server to the end user and Interject malicious JavaScript that are inject. I mean not interject Malicious JavaScript that then does things for them using like this giant distributed network of users within their country So that that's another reason why I personally love encryption everywhere the other thing that I would add to that too is that That's not a nice to have anymore I think in the digital publishing world in particular When you look at somebody like how Google phone not somebody Google is search, right? And their search rankings are now prepping Sites that are encrypted higher than unencrypted sites. So While there's a there's a tremendous amount of security benefit to adjust to the end user I think from a digital publishing perspective as a publisher who is probably using Drupal That's also important just to the core business, right? Like your page rank determines how much how many views you get in a lot of cases or helps drive traffic to your site and having a higher rank on it And search results is also going to impact it. I think the other thing I would call out when it comes to terminating HTTPS at edge is that That is quite a few round trips from a performance perspective Which I think you're going to get into later Josh and Rudy that is happening at the fastly edge versus going back to origin, right? So by terminating SSL or TLS at the that the fastly edge and having us host your certs You are saving a potentially tremendous amount of time particularly with global traffic or you know If your data center is on the east coast or something like that Getting a lower round trip time on those HTTP requests means that not only is the page going to show up higher in a search result It's also going to be better performing when somebody actually does click through Absolutely, and yeah, there's there's so many benefits now to having HTTPS support and really making it easy Configuring with fastly is all wonderful All right. Well, let's let's move on to the next topic logs and data streams And it's worth noting that our updates traffic So every single Drupal site in the world has the potential to turn on the update status Module and this this module basically calls home to Drupal.org and says hey, I've got Rules installed or I've got views installed Is this module up-to-date or is my core up-to-date for Drupal? And we respond back with some XML that gives them information says whether or not there's a security release available Or whether there's a new recommended release available and that's that's a ton of traffic in Our case it's really important for us to be able to aggregate that traffic and turn it into a use to statistics So whenever we talk about the the 1.3 million websites in the world that are using Drupal What we're actually talking about is how many sites have called home using that update status capability to find out what's going on so a Big part of this is is the logging and the the data streams that we have there and and Rudy Could you describe a little bit kind of like what the infrastructure that we've set up is around that and how how we're using those logs? Yeah, so I was really excited when I found this feature in Fastly because our our previous CDN did not have kind of late Real-time log streaming so to basically to calculate our update statistics We have to have the like raw web log traffic to determine usage so the sites that are calling back or requesting like Drupal core and a version and they have a site key and we use that information to Calculate, you know the the site key the version and how many people are basically how many people are using it? so when once we got the the Our syslog endpoint configured. We had streaming log data from Fastly hitting our log host for the processing For this data was happening. So it allowed us to actually speed up and and really improve on how we were calculating the statistics around usage And stabilize those statistics as well. So we were having prior to Fastly a lot of issues with updates like the the logs that we were processing not coming through or Taking, you know three days to get the log data off of the existing CDN and being able to process them More quickly now really meant like we could we could stop worrying about this problem That was another thing where like there was a lot of time being spent getting these logs and and fixing them constantly and and dealing with all the breakage that was happening and so now we have the logs streaming correctly to our our log host over in our syslog syslog endpoint and We can calculate these statistics more more quickly and more accurately Which is really a great feature to give us and it's also Fantastic for kind of debugging issues because not only can we stream back the updates logs We're also streaming back all of the logs for like double double dot triple dot org and all of the subsites and all of the other things around if you fastly so Whereas before there might be some issue Happening on our origin server But we don't know for sure if it's making its way all the way back up to the CDN now we can kind of inspect from the CDN level all the way down through Like the PHP fpm workers running on our our origin servers, which is great for kind of debugging and fixing problems more quickly Absolutely, I love this graph too. They got thrown in there because fastly really deals with our spikiness quite well The the way that traffic comes in for updates is most most sites are on a cron and so we get just hammered on the hour and On any regular cycle. So you we tend to see these pretty regular spikes throughout our usage I've got to say that's that's been one of the things that I love seeing is just how well it responds to it Yeah, and this kind of goes back to just talking about updates a little bit and we'll walk through kind of like all of the Features that fastly has that have helped us with our kind of our update system and reducing the load on our origin because it was before fastly actually Maxing out our RV lands traffic limit for upload on these hourly spikes So that giant spike there with the 2000 requests. We were At one point actually dropping packets, which would slow down all of our websites all of our updates Things things would slow down. We were having trouble there. This helped allow us to solve that problem. So Excellent purging Yeah, let me talk about purging a little bit Astley does near instant purging, which is great for kind of the initial updates and FTP Services that we moved over to fastly We were able to build a sort of a packaging service that would actually On packaging once a release was packaged and once the update data for that release is packaged We we now dynamically purge Those individual releases from the CDN. So if there's an update for Let's say see tools That update gets pushed out and written to the file system package gets written to the file system and then we send a API request to fastly to purge those releases from fastly and we do that with a soft purge which Softly sort of clears it from Fastly while it's doing a background fetch to the origin to grab that new release data and As that works through fastly's caching layers It pushes out to the to the end and what that lets us do is kind of avoid a lot of the Request to the origin that we would otherwise have and also helped us kind of update our of the biggest traffic area of our site which was updates and And downloads but mostly the updates traffic It let us modernize that in a way that we're actually like clearing on release instead of clearing every 30 minutes Which is kind of our previous TTL that we had set because we didn't have the same capabilities in our previous CDN to kind of dynamically purge like this and purge instantly and be able to now push out security updates and Release data much more quickly and much more consistently to the world I was gonna say this is huge from a security standpoint because instead of it being a 30 minute wait We get something packaged up. It's a security release. It's available almost instantaneously to the community Both from a downloads perspective, but also from an updates perspective So if their site pings home during that 30 minute Window in the past there was it was a chance that you know, maybe they only Maybe they only ran updates once an hour and that could really Impact how long it took them to get some of these important updates And now that you're using an invalidation based strategy, what is the TTL set to oh? Yeah, that's a good point our TTL sets at 365 days nice So the big instrument from the 30 minute TTL we had in the past Nice, so are you Rudy? I think we had talked about this a while ago But are you leveraging that same strategy across the standard digital publishing or the standard publishing workflow for? generic content like say on the homepage of triple now org or is that Is that still forthcoming? This is that's a forthcoming thing Definitely the biggest benefit was through updates since we have so many more requests And kind of out of the box fastly was so like so much faster just with like the initial kind of connection latency and All that that we haven't pushed that yet, but it is definitely in the works and some of the stuff We'll talk about or you guys will talk about for the kind of what's next with Drupal I'd like to start leveraging on Drupal.org as well Awesome. Yeah, that's definitely something that we see in a lot of the digital publishing sites that we work with is that That's the primary use case for them around purging. It's really interesting to hear about this in the context of how it's helping updates Awesome, so requests collapsing Talk to me a little about this and doc. I think you were going to kind of kind of explain this Conceptually a little bit for us. Yeah, so a Problem with with with certain CDNs and old caches is that Once a piece of content falls out of the cache either because it's purged or because it expires What that what sometimes happens is that when requests, you know from from thousands of users Simultaneously come in is that all of them get forwarded to the origin server Until the piece of content is actually in cache and that is what we call the stampeding herd So, you know, it's it's like the the floodgates are opened and all of a sudden you get thousands of requests for the same or Hundreds with a CDN for it for the same content with request collapsing What happens is all of the requests wait For the one request that was relayed to origin to complete and once that is complete Then all of the other requests are handled and that is in the case where there is no stale available So you mentioned soft purging earlier With if there is a stale available then that of course is served instead and So what happens is if you have say, you know, you mentioned earlier you had like a 30-minute TTL previously with that case With with us and if you use the the origin shield Which means that all of our data centers go to one of our data centers before going to origin I think that's coming up in the next slide Yep, that's the one So with that every like you would only have one request for one object in 30 minutes And of course if you have you know a year then, you know, you only have one request whenever you purge it within that year That's basically it And that's been huge for us And we'll actually we kind of talk about what this means from how many hits to origin We actually see and this is interesting too because these statistics if you look at March 6 2015 with our old CDN 5.7 million requests to origin and then same date a year later Using Fastly and it's it's 1% of the requests. I mean, this is this is huge from our total cost of ownership standpoint Yeah, I I can't say how happy I am about this It was pretty would you say you're still unreal? I would say I'm stoked. Yeah, okay It actually helped us free up some relatively powerful servers to do, you know more interesting things for the droplet org infrastructure, so I mean it's night and day kind of looking at the request logs that are hitting the origin updates and for FTP downloads and the other services that we're using this for and Yeah, I'm 1% of the requests are now hitting our our origin and we're able to do that sort of invalidation style Purging which is really, you know, just kind of changed a lot about How we do things, you know security updates before like if there was a critical security update We'd have to go through and manually purge everything from the cache Now it just works like it dynamically purges whatever the release is and there's there's a lot more Sort of automation around this which is great Which equates to time savings for the team as well, which means we can be focused on you know new features and improving the site and Taking care of some of the technical debt from a 15 year old website, so That is awesome. That's an amazing stat just a five least stoked. I would say that's really cool guys And just out of curiosity Because you mentioned earlier like how many requests you had in a month, but I forgot what that was So how how many requests do you get in a day? that get reduced down to 60,000 Ooh, that's a good question. So it's 1.5 billion a month I Would have to do the math to figure that out and I'm afraid I'm not that quick It's a lot of it. We'll put it that way. I'll get that number for the end of the talk. How's that? Oh 50 million Yeah, that that sounds about right because we get basically about 15 million page views and again as we talked about before like our biggest chunk of traffic is actually our updates and our downloads So that that sounds about right from an updates to the sixth standpoint So that that's a ninety nine point nine percent hit hit ratio That is amazing It's pretty impressive I've actually watched that little gauge that you guys have in the dashboard and it gives me great joy To watch the hit ratio stay where it stays It takes a pretty big event on our part for it to to drop at all. So Yeah, and those you know five minutes 30 minutes hour depending on kind of cron like the hour Crohn's are you know pretty crazy like we'll get up to the 4,000 requests in a minute Or in that graph Which is per second. I believe yeah per second. Yeah, so 4,000 requests in a second when cron hits and that's Handled no problem. There's really no noticeable like increase in requests to our origins It's definitely huge No, this is kind of interesting because I We just did this right Rudy was less than a month ago that we we moved off of our load balancers Which is obviously a cost savings because now we've we've got We had two pretty beefy load balancers sitting in front of everything and and now we don't need them anymore You can pass that light right along so really if you wanted to kind of describe that that structure a little bit And how that reduced our complexity Yeah, so, you know and in the past we had used varnish as a load balancer since it has you know pretty sophisticated health-checking and monitoring and and sort of like it can take web nodes in and out of rotation based on the status checks and We had these load balancers that had you know decent HAProxy health-checking and and we're set up and they're working fine But we really didn't need them anymore So fastly is now configured with the web nodes is direct back-ends in the fastly configuration and that routes directly So fastly is routing directly to those web nodes and performing the health checks through the varnish back-end health-checking and It's easier for Our staff to take web nodes in and out of rotation because there's a web interface for it You can quickly revert and deploy different configurations and it's also improved the latency of the requests because we had sort of a sort of a weird issue with our internal network being cross-rack and some latency variations that were happening there and having these kind of direct route through fastly Which is now actually Peered with us through our ISP and our data center at the Seattle IX Actually reduced our latency pretty significantly for those first requests So kind of going on into that so fastly is now appearing with Nero Which is like the network for educational research in Oregon or education and research I think which does like all of the public universities and schools and OPB and some other Kind of the it's basically like the government state ISP Which is where the open source lab is at Oregon State University and being connected to That at the Seattle IX and also using the origin shield and routing all of our origin shield traffic through Seattle Helped us win pretty significantly with you know performance and Keeping that like the time to first bite for the requests even lower than it was before This is such a great open source win too because it was one of those things that it came up because we had the relationship and because we had the relationship with OSL, but now, you know, we have the Python project benefiting from this and How what are a few of the other ones that are sitting down there in OSL right now Rudy? Well, even so outside of just like the you know Linux foundation Python software project stuff like that We have like things like github so like this makes Hub faster for anyone that's on that Oregon State or University of Oregon or at a high school in Oregon And improves the kind of the latency there for those people too. It's pretty awesome Okay, so we're gonna jump into some some headier topics some of these aren't actually implemented on Drupal dot org yet, but we want to we want to kind of talk about Some things that are capable in the facility platform and things that we're looking forward to using so Surrogate keys. Why would you use these? When do you use these? No, yeah, well like it says here and doc feel free to jump in Like you you can tag responses the key and then purge those objects associated with the keen one command So there's a lot of ways this can be useful Drupal 8 takes good advantage of this Yeah, so a good use case or example use case I use for this is Say you have two templates you have one for desktop and one for Mobile and then you have different pieces of content if you tag the responses with both a template ID and content ID then Because you're gonna have two versions of each piece of content. You can then purge the a piece of content in one request by just purging its tag and then both the mobile and the desktop version are going to be purged even if they have different URLs and then if you make a change to a template and You make a purge for that template key then everything using that template is going to be purged in one command and This is for instance What that might look like? So you can see the circuit key header the first response has template 3 and article 1938 and the second response is template 5 Article 1938, so they're the same article IDs different templates Yep, so how does that relate to Drupal in Drupal 7? There is a very basic caching support and fastly did make a plugin for that which is based off of the varnish plugin however that caching support was pretty basic and that also means that our You know our plug-in works, but it's even more basic So Your fearless leader Dries Boutart made a blog post about this and We saw that and I met up with a couple of guys among with Wim Lears at Drupal con in LA last year and They showed me that you know, there's this thing called cash tags, which looks a lot like Circa keys and we're just like sweet They're basically the same thing So we had some discussion. I mean there is a size limit in in fastly and it used to be 1k It's now 16k and that If you have a very complicated Drupal 8 site that cash tags header can be Immense it can be it can get really really big. So what we did is As is a little bit of a shortcut We hash all of the keys and truncate them down to three characters Which means that there can be overlap between some of the things If you know, but we only do that if it goes over that 16k limit Which then means that we purge both Why it doesn't matter that much is because when the when you do purge a Key that has overlap so you might accidentally purge two or three keys at the same time It's just a purge. It just means that we grab it from origin at the again All right, so me and Wim talked about this how and and he went back and just you know spent like two or three hours and Came up with a Drupal 8 version of our plug-in which he then submitted to Drupal.org and We've then since taken that and we've added we've gone through his to-do list which was add tests And do a little polish Which means that for Drupal 8 we now have incredibly good caching support like seriously proud of it and And oh, sorry Drupal 8 has very good caching support and the fastly plug-in now really Synergizes with that and makes it like something that just works very smoothly so me and Wim and Fabian I forget his last name Do you guys Rudy do you remember his last name? I Mean France yes, thank you. Yes My audio so I could say it, but didn't want to say it wrong. Yeah, so me Wim and Fabian had One or two very long Skype calls about you know, what else can we do for the future with Fastly? And one of the things can you go back a couple of slides? Thank you It's always is Supporting cash contexts so currently with with Drupal 8 not only are there cash tax But also cash contacts which allow you to serve different contexts based on things like accept language headers or geo targeting and still have them catched and The face of the moon is my favorite because that is something that is absolutely Useless to I think 99.99 percent of the websites out there But it is something that the the Drupal core developers like to use as a as a use case for something Weird like something that nobody uses, but you know They still want to be able to support so that they can say like if we can support this we can support anything I'll try to limit updates to Drupal.org during full notes. That's that's actually Very wise So that that is that is something that we are discussing and working on Implementing for for Fastly as well All right Couple of quick links here. Well our website, of course Boutard.net is the making triple a fly blog post that I showed Vim Lears has a very good talk at I think it was Barcelona Mm-hmm Where he talked about caching at the edge with CDNs and the last link there is the Fastly plug-in for triple seven and triple eight And I put a little plug in here. It hasn't been accepted yet, but I know that you and Fabian put in a talk for Drupal con New Orleans That's also going to be kind of talking about some of these these future forward things. Yep, correct More detail there. I'm really hoping it makes it through the Community acceptance process because I think it's going to be a great one Yeah, I'm looking forward to that too. All right. Well, I love that you dropped in the convestitions That was that was nice as we were putting together this deck that just gave me a little bit of a smile Are there any questions in the Q&A? Is anybody have anything that they want to drop in and ask the participants of the webcast? Not seeing any any any other got kind of closing thoughts from those on the panel like What is something you want to add here at this last bit? So I'll throw one out there The use cases here that you guys are doing on updates is awesome, right? And I think for the majority of folks that are doing using triple and production It's you know, it's a lot of digital publishing and a lot of digital media, right? I guess the the application of What you're seeing from fastly or just actually the sort of the invalidation strategy like how do you feel like that would impact a digital publishing site? The actual end user on triple. Oh, yeah Well combined with kind of what doc was talking about with the being able to tag and and purge that way But even just in general from like a general perspective being able to like publish a a news article or you know big important post Leaving your cash intact and purging that particular post Without having to worry about clearing the rest of the cash and and all the kind of like Slowdown that happens with that Being able to just invalidate one particular piece of content or even one kind of subset of that content Was something like a edge site include or something like that Really helps speed up those sites You know that that slashed out effect is really no longer a problem if you're You know right No, man, you take a deep That that takes me back it does take me back nice Somebody's dropped a slash dot article in the slack at least you know in the last month Awesome awesome. Yeah, it's definitely a thing I mean like for publishing you need to know that it's gonna deliver your content quickly that's gonna be able to refresh it If something changes The news cycle anymore is near instantaneous When people are expecting things so being able to kind of respond to that in a near instantaneous way is is critical I mean, there's definitely like huge overlap there. I mean updates and And packaging and that sort of thing it's not really like publishing like from a media perspective, but we're you know, we're publishing releases and able to invalidate that particular release that gets published and have it your instant available and Not have any like additional load on our origins when that's happening is is really a kind of an amazing thing and I think the use case to that Drupal has for not only the publishing side of things but for extra nets and for building Essentially building that services layer for for new systems I mean we've had a lot of conversation in the community recently around headless Drupal and when you use it and that sort of thing All of this caching still applies to that all this delivery Still has to happen at near real-time in order for these applications to be perform it So I think it extends beyond just the the immediate thing that you see on the web to that kind of this is this is deep infrastructural foundation to build upon and if you if you have to deliver things quickly and with a high degree of Of that that that caching that Prevents your your site or your application from being overwhelmed. It's it's critical Yeah, I mean, I'm obviously biased coming from Fastly but having spent a lot of time in the digital publishing world It seems to me that if you're not using a CDN for any of the use cases That we've talked about here in the webinar. I think you should seriously be looking at it, right? And to me Rudy to your point I think the update service that Drupal runs the Drupal Association runs is exactly the same as any digital publishing New cycle, right? I think the timelines might be a little bit shorter in some cases and others They might not be but the stampeding herd problem is definitely one Feeding news in a real-time manner is definitely one right and when updates need to go out They need to go out and there's kind of no bones about that. So this has been really useful and really instructive to me I'd appreciate I know a bit about what you guys are doing, but it was really great to see it in this context They're kind of all rolled together. I actually have a story from a different customer of Fastly's who Had an origin platform that was getting overwhelmed and they were like, oh my god We have to do something so they they built out a whole new platform with way bigger beefier servers Completely green-fielded it and at the same time decided to implement Fastly as well At first they were just thinking To use this for the static stuff and then they realized, oh, hey, we can we can do all of our pages with this as well and Everything went down really smoothly and then a week later they looked at their their monitoring and went hmm The load on this new platform is so low That we could have skipped the migration to the new hardware and just implemented Fastly and we would have been just fine Absolutely. Yeah, we've seen things like that as well I mean if you look at how bored we have some of the beefiest database servers imaginable and If you look at how bored they became after our transition to Fastly it's it's it's it's pretty significant You know being able to prevent those those requests coming all the way through and Being able to cache appropriately is huge Yeah, I mean, I'm I'm definitely excited about being able to repurpose some of our existing infrastructure and help, you know improve the parts of of our infrastructures that need improvement now and kind of really Really helped us there and I think More to the point about you know media content sites That talk that was given in Barcelona that you're talking about like even in more detail kind of goes over like how small that kind of like origin server Actually can be at this point using the the invalidation and the you know dynamic sort of like invalidation technique for purging content Yeah, a couple of the AWS micros maybe One or two I don't know if we'll be able to get away with that on Drupal.org just yet, but But definitely I think I think a lot of publishers can't I mean they can they can really shrink it down to the minimum possible Minimum possible server and let all the performance come from the edge. So it's huge That's a very cool thread We've seen that theme repeated Not just with digital publishing, but in a lot of other vectors as well I think this is awesome case study up on our site about hotel tonight where they reduced their origin load by 80% Or something like that, right? Wow Yeah, not only that that was on an API Yes, or getting the the hotels available near you that had the Latitude and longitude in the URL Oh Wow, and that long is actually processed at our edge, right? Yes on DCL Yeah, yeah, and we use the varnish configuration language at the edge to chop the Long down to two decimals, which means that you know, I think that's about half a mile Maybe a quarter mile apart So, you know, everybody on the same city block gets the same results and and that made their hit ratio go from zero to I believe like 60 to 70% on just that API end point That's awesome Some pretty sick stuff. I mean, that's one of the great things about varnish and Drupal having such a tight integration with varnish in D8 Is is pretty awesome, right? Like there's a lot that you can do at the edge to kind of further the The the theme of offloading stuff from your origin so that your origin can actually just focus You know, you can take all that CPU and disk and apply it to actual smart intelligent hard problems versus working on Just serving content, which is I mean, that's what the CDN is supposed to do, right? So it's pretty interesting That is awesome. Great use case Yeah, like that the pull henna gump, which is the the the main architect and Developer on on the open source varnish He likes to liken varnish to a printing press. So, you know, Drupal would be the the writer the author You know the the layout maker, etc. The graphic designer and then the printing press. That's varnish that just takes the original and just Makes a hundred thousand copies in a couple of seconds like done nice All right, well, you know, I love the use cases and I want to throw out Like one more thing that I think is a kind of a good thing to close on which is a Huge thank you to to fastly for being such an awesome Partner in the open-source world and particularly a great partner to Drupal It's it's been huge having people who understand open-source help provide our technology stack and It is really extended our reach and certainly made us better stewards of the The the resources that we've gotten from the community. So I really appreciate it. I want to say thanks Yeah, now our pleasure Josh really is Drupal is obviously a huge part of the open-source community Provides an enormous and incredibly valuable platform for a lot of folks out there And just beyond that, you know, we we came from open-source roots and we'd like to stay true to that So I appreciate you less letting us be a part of it, too awesome, well, thanks for the time guys and We're gonna go ahead and close out For those of you who came on a little bit late the entire Presentation will be posted up on YouTube shortly And we'll be sure to publish the link to that and connect it to a couple of the articles that we put out there about Drupal.org and fastly and how they're working together. So again, thanks everyone for the time and we'll see you all on the flip side Thanks, everybody