 Thank you. Thank you for the warm welcome. My name is Gibran Ijaz and I'm a Drupal core developer. I'm a Drupal developer previous next. I'm also a shortcut module maintainer and contact module maintainer in core and I have almost more almost 500 core commits mentioned in Drupal 8. So first of all, let's start with what is cache? Anyone in the room enthusiastic enough to answer that? Like a store. Yeah, for what? A store for what? Data. And why do we need a store for data? Where or why? Why do we need a store? Yes. So in typical software terminology, a component to store the data for the future use which is either source heavy or too complex to calculate. And now like it's a back and talk, I'm going to go very technical in very technical details. So let's start with, you know, like how many types of caches are there? First of all, we have static cache. Static cache refers to the concept of doing stuff in memory when the request is happening. So everything will be in the memory and after the request is done, your data is lost. And similarly, we have persistent cache. And as name suggests, persistent cache can keep the data around across the multiple request. With that, we get added complexity of expiring the data or invalidating the data. So here are some examples how to use static cache. This is Drupal 7 function, a legacy function we have from Drupal 7 called Drupal underscore static. And then we have Drupal 8 object-oriented way to do that, which is defining a non-static class property and using that. So here's a code example. Everyone can see that? Yeah. Here's a code example for using the Drupal static function. It's just, you know, an array of static, a static array in this function. And you can create key value store using your keys and retrieve them whenever you want to. So in this case, we are using the function name as a key or the reference to the data you are going to store in the static cache. So this is an example of class property where data is a class property. It's a protective property. No one can access that other than the children of this class. And when you don't have data, you compute that, else you keep that during the request and you keep on using that. So now I'm going to talk about some examples of persistent cache and how does it work. Normally, persistent cache is used in conjunction with static cache. We have the static cache going if there is no static value, we access the persistent cache and get the value from there. And the persistent cache needs storage and that storage is normally the cache backends. Here's an example. This is the modified version of the object-oriented example of static cache where you have the same property, but this time if you don't have any data in the property before calculating, you carry the back end and you access the data from there. And if you still don't have the data, you gather the new data, set the back end, it will populate your static cache as well, and then you can resume your normal service. So now I'm going to talk about what is cache back end. It's a storage to save the cache items. And in core, we have a bunch of different type of cache backends like in memory where names suggest that you are storing stuff in memory and it's fancy name of static cache because memory only can keep stuff until the request is terminated. Then we have default storage. When you store Drupal using a database, this is the normal cache you get, database cache, and this cache back end is, this cache back end is by default available when you store Drupal. Then we have APCU cache, which is typically for PHP processes, and they cannot share like CLI and, you know, your process cannot share their memory, so there's some limitization around that, but it's for the PHP processes to have, you know, their cache available readily to them. And when you are developing locally or when you don't want to use the cache, you can use null cache back end and it will disable the cache and you will always have to calculate the data and it will slow your site a lot. In Contrib, there are a couple of, there are a bunch of solutions. I'm aware of Redis and Memcache and I have only used Redis in Drupal 8, though I have used Memcache in Drupal 7, and in Drupal 8, setting up Memcache is not, you know, that trivial, but Redis is just, you know, one change in the configuration file and you are good to go. And we'll discuss that in the future slide. There's another very interesting back end, which is called Chain Fast back end and the concept behind this back end is that you can combine multiple cache back ends and if you don't have the information available in first back end, you can access that information from the other back end. I will go into the detail of this in upcoming slides, but let's discuss cache bins first. This is a cache bin. It's a container to store the cache entries and it is always a configurable, it always has a configurable back end. You can change and swap the back end whenever you want to. And if you are changing, you know, your storage from database to Redis, for example. So this is how, you know, like in the first client, this is how you can use the bin, any bin. If you don't pass the bin, it will use the default bin to access the, you know, some information from the bin, some data from the cache bin using Drupal Static method. And then with the container get function, you can use that using the dependency injection in your object-oriented code. There are multiple bins in core, but by default we have bootstrap. By default we have, you know, a default cache bin, as name suggested. It's a small bin with few key variations to store, you know, a small amount of data. And then we have data bin, which is a bigger bin you can store as much as data you want. And then we have discovery, which is frequently used for the plugin discovery, which Drupal heavily rely on, and it speeds up Drupal a lot. Then we have a small cache, like Bootstrap. It helps, you know, bootstrap the Drupal, and it's a small cache. It used on every request, but it's very limited outside Drupal core. The usage of it is very limited outside Drupal core. Then the bigger beast render cache, which has everything, you know, a huge amount of data, because, you know, whatever you are rendering, it gets stored in that bin. Configuration cache to store the site configuration so that they are readily available if we don't have to read from the disk all the time. They are stored in the config bin. And static cache, or which are using memory back-end, which we already discussed, is another way to store or leverage static methods or static cache. This is the default structure of our database table. When you use a cache bin, where CID is the cache ID and data is the data you store in it. Expire is the time when your data is going to expire. It's not the interval. It's the time. That's the most important thing here. Similarly, some trivial things are created when the cache entry is created. Serialized whether the data is serialized or not. Tags, we will go into the more detail of tags, but it's just for the sake of definition. It's the dependency of the data. And then check some is to invalidate the data whenever we want to. This is how you define your custom bin, cache bin, in your code. It's just Drupal provides you everything. You just have to change your argument name and have a proper service name. And you can call the factory service, and it will provide you the storage and all the getter and setter methods. So we talked about overriding briefly. This is how you can override your bin properly in settings.php normally. And the first example shows you that we are going to use Redis for the default storage instead of database. So I have done that on multiple client sites, and I have seen a huge performance boost, but Redis comes with additive cost of configuration and maintenance. It's a standalone service. It can be a standalone service. It can encompassing your web hosting solution as well. So this module has quite a few stable releases, and it's doing really great with the community. And then the example below is for the developers when you want to build your template or your theme, and you don't want to render cache so that you don't have to clear cache again and again while you are changing your theme and updating your settings. So here's an example from Core for the usage of cache fast backend. In the JSON API module, first we declare a memory cache backend dedicated to the JSON API usage, which is a private service because it's not going to be used as a public service, so we may as well declare it as a private. And then, as you can see, there are two calls to append backend, and it shows that we have like two cascading backends working together, and the data first, when we will request the data from this bin, we will get the first JSON API memory cache bin request and if there is nothing in the memory on that request, we will go to their database, or in this case, Revolbin, and we will get information from there, or if not, then we will rebuild that information. So this is really useful in speeding up JSON API discovery for the resources whenever you access a JSON endpoint because you have, during the request, you are storing the cache in the memory, and once your cache request is terminated, you still have the response in the database, and you can still populate your memory from that and then keep on doing that. And this slide change, this patch has increased the JSON API speed three X times in core, which Dries also mentioned in his key note that JSON API is faster than ever in Ruby 8.8. Here's a simple example of deletion. You can, if you have like simple key available to you, you can just go ahead and delete that, but we usually don't do that because there are other ways. We have cache tags which we will discuss in the later part of the presentation, but there's an API available if you want to directly delete certain things from the cache bin. If you know the key, just use that or just delete everything. It has the benefit of using that because we can, it's a little overhead because with tags, you have all computation going in the background whereas delete is a direct call to delete stuff. Similar to deletion, there's invalidation. Invalidation doesn't delete the data. It keeps the data, but on the next request, if the data is required, it will regenerate the data and populate the cache. So it's the hasty way of, yeah, this data is obsolete in the next request or whenever we need this data, let's rebuild after a while. And there are like a very complex API system in place for the invalidation of multiple cache packets for this. Here's another example of expiration. You can set the time limit as we discussed in the table structure. It's a timestamp, so 86, 400, being that it's a whole day, the number of seconds in the day, and request time is the current time. So it's caching for the one day, and after that, it will discard the data. And similarly, if you want to keep the data till midnight, you can just use PHP functions to give you the timestamp for that. The next thing is cache tag. Cache tag is the dependency of the cache data like entities and configuration. They can also be custom tags as well. Normally, Drupal is responsible of invalidating or clearing the tags for invalidating the tags for entity and config. But if you are using custom tags, you have to do it by yourself. So how does all of this come into play? Why are we going into this detail of learning Drupal cache and cache base? Because we use that rapidly with the rendered array. So before stepping into the render arrays, do you need some water? Do you have some water? Do you need some water? That's all right. Yeah, so let's discuss render arrays quickly. In Drupal, a render array is the way to represent your render page in an array format. This is how a node render array look like when you call an view builder to display your full mode. As you can see, it's a complex structure. It has a view mode configuration. It has a node object, and then a cache key. We will go into detail of this cache key in a bit, and it will start making all of this. So render caching is the most important thing in core. The render process has the ability to cache render output at any level in a render array or in the hierarchy, and the render API use cacheability metadata and measure it in the render array to perform the caching. So let's quickly discuss... In detail, let's discuss cacheability metadata. So first we have cache key. This is used to identify the cacheable portion of the render array. Then we have cache tags. This is for the dependencies on the data managed by Drupal, like, you know, like entities or configurations. Then we have cache context, the variation in the output based on the different context of the request. And then we have max age, which is for the time sensitive, you know, information. When is data going to expire, or when we have to regenerate the data. Cache key, as you can see, the biggest question for us in the render array when we are creating a render array is that whether my data is cacheable or not, is it complex enough to cache that data or not? And then we identify the relevant part which can, you know, identify as cache keys. In this example, these keys are explaining you that what and which type of thing we are, you know, talking about in caching. So we are going to view an entity which is a node and it's an ID is one and the view mode we are going to use is full. So we have all the information and we can go back and regenerate the render array using that information as well. Cache context, does my output has variation like based on user location, based on, you know, their authentication, like whether they are logged in or logged out, based on the translation I'm viewing, like it's a different language or not, we can have a different rendered output for everything based on all these contexts. By default we have theme context because every site has a theme and then we have language interface to make sure the default language is enabled on the site and if you are anonymous user, you will have user permission and when you are authenticated user, you can swap that with the user context as well, but normally it's user permission by default. Here's an example of the context available for this node and as you can see there are a lot of them, but it's true that any of these can change and change the render output based on that as well. It can be a multilingual site. You might be viewing it at a different location where time is different or you are viewing it in admin theme or on the front end. So based on that render output can change a lot. Cache tags, these are the dependencies as we discussed before, but more likely we can say that the things we need to regenerate, the things identify when to regenerate our data. In this case, if file has changed because we are viewing any file or taxonomies are showing on the node, if anything gets updated, we need to refresh the cache of this because a user name might get changed or taxonomy term gets updated and that will render our output invalidated and we have to get the fresh rendered output so that we can use the new values which has been updated in the back end. Next stage, which is a very interesting and complex idea that it is to make sure that we are expiring our data properly, co-reparately use permanent cache which is minus one, make sure everything is cached all the time. We don't need to expire things based on time and then we have uncacheable stuff so most of the time code is using minus one or zero in the most cases, either we cache things or if we don't cache things but in real life development you can set it to the specific seconds and time like you can set it to 6 hours, something like that so this is the duration and seconds, this is not a timestamp so the Drupal will do the conversion when it will convert the data from Maxsage to the values we store in the database table. This is how it all looks like when we store our data. As you can see that cache, CRD is comprised of cache key which we have seen earlier and then all the user, all the context available to that renderer and they are all serialized and they can form a cache ID and then markup is stored or in this case markup is stored in the data and as I said the cache, Drupal uses cache permanent or uncacheable everywhere so in this case expiry is minus one and tags are just coming through as a tag and then we have checked some to make sure we can invalidate the data when we can. Let's talk about bubbling up the cache metadata. In Drupal, bubbles mean that all the cacheability metadata along the render elements, let's start from the top. It's a really complex concept and in Drupal 8, Drupal 8 technically bubbles all the cacheability metadata up along the render tree. When it reaches the tree is root, it bubbles to the response level as well. So the idea is that each and every element on the page is responsible for its own cache metadata but during the process of the rendering we will bubble at each and everything to the top and then once that's done we will send it via response to the reverse proxy or the user browser. Once this is all done because that's all the development side, once that's done what's the benefit of that? The benefit of that is page caching. We can cache the pages. So there are two solutions in core. One is internal page cache which was the default cache in Drupal 7. This is the cache for the anonymous users. It has the URL key as a cache ID key and cache key and cache ID as well so that we can relate the response back to whatever the URL is and it does support the cache tags. We can invalidate the response to the cache tag as well. Also, internal page does not support cache context because we are using just straight URL with this parameter and we are not relying on any other information. It does not support MacSage as well and there is an open issue. We will discuss that at the end of the presentation why there is an open issue and we are not able to solve that but even though you cannot use MacSage but you can send expire headers and the page cache will get expired. So there is added complexity on top of the design but we will see why it is there but for now you can send expire header and it will expire your internal page cache because you are storing against the URL so the cache is just storing the response object as it is, it does not care and it can be just the markup as a string. It can be whatever response you are showing on the page. So this is the reason the page cache is so fast that you carry the URL, Drupal goes to the page cache table, looks for the URL and just returns whatever is in the data column. Then we have dynamic page cache that we built in Drupal 8 and it is a very fascinating concept and it is really hard to grasp at the beginning but once you realize it is really easy to understand. It does not cache full pages unlike page cache. Cache is part of the fragments of the page and then if something is uncacheable or varies with the time you can cache that separately and just put a placeholder there and Drupal will be responsible for replacing the placeholder during the render processing process. And these all cache metadata which we went into very detail, Cache Tag, Cache Context and MaxSage they are all supported by dynamic page cache and in fact cache context was introduced so that we can use that in dynamic page cache. As you have seen that internal page cache was only for anonymous user whereas dynamic page cache because we have the context information we can vary the output based on the context and it will work for every user on the website. And because we are not saving the response to the object as such we are storing just a fragment of the response that's why it's relatively slower than the internal page cache but we are doing more stuff like setting up the cache tag changing the context, setting up the MaxSage so that in the long run not just anonymous user all the users of the website will benefit. There's a complexity in setting all these things up there is a complexity in sending the response out but once these are all done the end product is so good that you cannot ignore this. Here's a simple table to compare internal page cache with the dynamic page cache in internal page cache our cache ID is URL where the dynamic page cache we have URL and the context as we have seen in the example and then we have the data in internal page cache the data is the final response we want to show to end user and for dynamic page cache it's partial response with the placeholder which will get replaced during the rendering process and they both support cache tags and we can invalidate these with the cache tags. So let's talk about reverse proxies what's a reverse proxy, anyone? So a proxy server that retrieves a resource on the behalf of the client from one or more servers so if it's one or more servers then it's a CDM and you can have your own proxy server like in case of example in case of varnish we do that. So let's discuss quickly about Drupal and reverse proxies we have page cache which is called a poor man's varnish because it is just storing the whole response for a given URL and varnish essentially doing that at the same level with an added complexity of tags and whatnot but it's suffice to say that it's a poor man's varnish and if you are just hosting your website you can just enable dynamic page cache internal page cache and you're good to go you don't need a dedicated reverse proxy or whatnot and you can of course use a solution like Fastly or Cloudflare, Cloudfront or even varnish but when you do that they are simply mimicking the functionality which is internal page cache providing so you have to disable that so that you can get the benefit of having the reverse proxy or CDMs so all this sums up into a lot of burden to dwellers like how and how we have to do these things and what things we have to achieve to do that and then for this I'm quickly going to discuss some of the other tools first of all while you are locally developing enable your settings.local.php so that you can see the errors properly instead of wide screen of depth set your header of debug cacheability so that you can see the cacheability data coming through in your response and you can see that whether your request has been cached or not and then there is a great module called RenderWiz it's just like a themeer if you have used themeer in D7 themeer is for the theme but RenderWiz is for the render pipeline and it gives you a full view of each every element on the page with their cacheability and cacheability metadata so that you can go and point out where the things are getting uncached or why are they being not cached properly and then here is the quick test it's just like you can normally that's what we normally do on the website we go to the site, refresh the page and we see that after the first request there is a cached response coming so in this example as you can see we go to the user home page as an editor after the login first request we assume that there will be a cache miss and once that's done we just get a cache hit and if that test is failing you can just simply add this in your CI CD pipeline and once they are there you can make sure that all the changes you are doing to your code are not rendering your page uncacheable so I'm going to quickly discuss about some of the issues which I underlined in the presentation one of them is bubbling of MaxCache right now, surprise, surprise we have this amazing API but we don't bubble anything with regards to time so here is the link to this issue if you are interested you can view that link and read the story and back history I will briefly go into the detail we have the title is bubbling of the element Maxache page header and headers and page cache it is postponed by three issues right now we cannot finish that issue and I will get into the reason why it is postponed by here is a simple statement that as a user I have a weather block on my website I want to update the weather every 15 minutes for all the user there is no way in Drupal to do that at the moment a simple solution is contrived module which was technically inherited by this patch we were working on but this patch is blocked on other stuff so this is a this is just providing you a small gap to fix it temporarily the module name is cache control override it just gives you the ability to pick up the Maxache and set up that properly if it is 15 minutes for a block and then for the whole page the Maxache will be 15 minutes but it has some limitation as well which are listed here and these are all the issues which this issue is blocked on and as you can see they are really very vast issues some of them are very specific sorry yes and then optimizing render cache like it is a very broad issue what is an optimization of render cache so the problem is the crux of the problem is Drupal is showing way too many elements on the page and we still do not have a way to identify on the page that how is this thing cacheable thing or not and if this is not cacheable just exclude that from the response and not include that which is the language switcher block which should not be cached at all because language negotiator is responsible of setting the value and you can change the value based on your choice as a user if you are dealing with site then the last issue it is another interesting one issue which is about bundle specific tags right now here is the link and this is the title of the tag introduce entity underscore type list bundle cache tags and add the single bundle listing the reason behind that is we in core we have the listing tag for all the entities like node underscore list taxonomy underscore list and all the listing pages as we know Drupal loves having listing for all the listing pages we have this one tag and if the one node on the page is getting updated it will render the response and validate the render cache and we have to recalculate for every one node so this is just making sure so this is the problem statement just explaining that like if I have a view or I have a listing page where I am showing multiple nodes it is not feasible for me to just save one node and invalidate all the listing on the site so we are just changing the scope to have a bundle specific tag so that node colon page underscore listing will be our tag once it is a page listing we make sure we are just rendering the page listing we are just invalidating the page listing not just the whole thing so we have just limited the scope we are not doing any views changes in this we are just providing the API to begin with and this and which is another listing tag on top of node list and we will add the node list if there will be multiple bundles on the page if not it is just a page listing or it is just an article listing we will just provide the node underscore list underscore page tag or node underscore list article tag and this page is in fairly good shape in core right now it is just pending on our unit test and if someone wants to help with that they can come to me and talk to me and we can finish this off during this Drupal South so I would like to thank you all but most of the Vim layers who is the mind behind all this complex API and he has written so much documentation and all this is available in very detail on Drupal.org and they are very good blog post available on Vim's website about that as well but we did a session in Drupal Dev Days like couple of years ago that helped me a lot in preparing this session how to keep the things in check so thank you to Berdier and of course previous next day sponsored my visit trip to come over here and Drupal South for giving me the opportunity to speak with you so any questions? Thanks Jibril, we get five minutes for the questions so I'm sure we get plenty I'll probably start the conversation so start something simple can you explain in a nutshell what's the difference between doing cache clear and drash cache rebuild and what's actually happening in the back end what gets cleared and not when you run those two commands so cache clear only exists in Drupal 7 and it's not in Drupal 8 cache rebuild is in Drupal 8 there is cache clear in Drupal 8 as well no if you run that it will give you error yes there is I'll show you but for cache cache clear you cannot run by itself it doesn't really matter but to answer your actual question they just renamed the command cache clear is called cache rebuild in Drupal 8 that's the only change but in the back end what we do is we just invalidate all the caches we clear all the back ends available other than your reverse proxies or CDN that you have to do the other way using other utilities in the CLI so Drupal just goes in and queries truncate blah blah blah kind of thing to the cache tables so when you clear cache it basically invalidates all the caches yes yes everything which has implemented who cache clear that was in D7 but in D8 that is not the case so yes everything is in D6 any other questions hi I just wanted to understand the difference like why would you delete a cache versus invalidating it because if it gets invalidated it gets rebuilt anyway I don't know why you would invalidate rather than deleting so invalidating there is a subtle difference delete means it's gone invalidate means that we are just leaving it there and when we need to rebuild we will but once you invalidate it it means rebuild on the next request right that means when it is required we will rebuild it but still during that process whatever response you have gotten from the cache bin you can use that as an outdated response yes so there is the subtle difference between those anyone else have you shown there is cache clear in D8 so check it out you have to target a specific thing so you have specifically theme registry router CSS.js render, plugin, bin and views you can do with cache clear but cache rebuild would rebuild the whole cache so I was just wondering is there a specific targeting because sometimes if you still use features with cache clear feature might get uninstalled I guess there is a bug in features but cache when you do cache rebuild but cache clear would get the feature back that is what I was wondering but fundamentally different between those to command in drush latest drush latest drush is drush 10 drush 9 I have been using drush 9 for a year and a half I haven't used drush 8 but I can have a look after the session okay cool thanks yeah I am interested in that what is it anyone else so in terms of reverse proxies what else do we need to do to make it work properly like aside from clearing the reverse proxy cache yeah so like there is the very thorough documentation about varnish available on Drupal.org which will work with the cache tags solution implemented in Drupal but all you have to do is you know enable your dynamic page cache disable your internal page cache make sure whenever you are doing the rendering process or creating a renderer add the proper cache dependency as a developer that is your only responsibility and rest will be handled all because they are doing the same thing reverse proxy and internal page cache because internal page cache is storing the response for the URL and it's a complete response and varnish in a nutshell is the same thing we have the URL and then the response of the URL but effectively like if we have internal page cache still turned on and we have a varnish before our web server it like the cache version will never be hit because it will be answered by varnish is that correct so Norm this is like a very so this all cache cycle of how to get the actual data and which point we are getting the actual data that's very complex to follow but in this certain example if you have varnish enable and then page cache enable and maybe dynamic page cache enable as well let's assume that and then you have entity cache and render cache as well so what would happen in this scenario if you keep this enable that varnish has a page which is the certain page and you are getting that copy as an user I have changed something in Drupal and updated my node and what it would do is it will render my entity cache invalidate that and then it will rebuild my entity cache and render cache both and then going forward it will also use the dynamic page cache because dynamic page cache might have fragments of that node and when you go to internal page cache and the varnish and the internal page cache is just Drupal cache it will invalidate that if you have a tagging solution in place for varnish it will do the invalidation there as well if not then it will keep the copy so for the end user the change will not occur but for the Drupal the change will occur and then when the max age for the varnish will lapse then it will get the fresh copy which is the internal page cache copy which is also not fresh now because we fetched that a while back so you are just adding another store behind a store when you enable that yeah just still not clear so there is a very good blog post by Wim Lear about that where he is going into detail why dynamic page cache is enabled by default in core why we have added and separated internal page cache because in D7 it was just in the small setting in the systems module you go to cache clear page and you enable 7 or 6 or something like that and now you have to enable a module to get that configuration so why there is a thinking behind that so there are a couple of good Wim Lear's blog post about that it might clear your confusion there alright let's thank Gibran thank you don't forget you can go to the website now and leave your feedback