 The awesome participation from your site, we are pleasantly surprised about this interest on the session. So, it's almost time, so let's start. I am with Kwaido Komirov, development and team lead for Drupal development in FFW software office. You can find me on this link in Drupal work, and I've been doing web development for a while now. I've been doing Drupal for 7 and more years. My main role in our office is actually performance optimizations and tuning of systems for high traffic. And, yeah, such things. And, yeah, I aim to make things faster and performant. My name is Petjo, and I'm a full stack developer. I really don't contribute very much by myself to Drupal. I create ideas, I create some solutions for it, and I leave it to my colleagues to contribute so that everyone can have those ideas. But in most cases, I am working with Drupal for more than 9 years. And, as I said already, I like to create different ideas and create solutions for them. So, I'm working currently as a dev team lead in FFW as my colleague. And today we are here to talk about something that we created when we started a project of ours. So, we have one of our projects that is on Drupal 7, and the client told us that they want to go to Drupal 8. But, they want to have the coupled solution. So, those were the requirements. So, they need Drupal 8 as a CMS and data provider and react as a frontend. The interesting part here is that they wanted to manage all of the fields into the Drupal. As they have the opportunity in Drupal 7, they wanted to keep this opportunity in Drupal 8 and into the coupled version of it. So, this was the first thing that came in our minds as a problem. And, the other interesting thing is that it should be highly dynamic page content. So, what does this mean? That means that there is no strict layout. All of the pages could have different components. They will share some of them. They can reorder them on different pages, have few of them, have more of them. But, there is no strict layout for any page. And again, path aliases redirects, they must manage into Drupal. This one should be a multi-site and actually it's with more than 100 websites. The domains are even more because this is multi-language and each language is on different domain. And, we are going to have single Drupal installation for all of them. And, we are going to have single React for all of them. And, of course, and not least is the search and optimization and the performance. So, all of those things are interesting and they are kind of a problem between the coupled and some of them. So, first of all, we are talking about the coupled template or TAM engine. So, how standard the coupled site works. So, we have our data provider, our data source. We have those data coming from there. We get into the React or doesn't matter what kind of front-end JavaScript framework we are going to use. And, we are deciding ourselves how we are going to render this one. And, we don't care what Drupal will say to us or what's going to happen if someone go and change the display view or whatever you want into Drupal. We are just receiving this data and rendering it ourselves. And, in Drupal, how it's worked is one, actually. We have, let's say that our data source we are going to separate in those three parts. So, we have the entity, which is the data. We have the view mode. What kind of fields we are going to render and the formators. How we are going to render those fields. And, we combine them and we render. And, this is our mix. So, we need to have our entity, our view mode, our formators to send them to our front-end, our React. And, combine those things in there. So, what does that mean? That means that we need to recreate the Drupal template system into the React. And, that was the interesting part. So, when we're talking about the Drupal template system, first of all, we're talking about the view modes. The view mode is, it's depending which fields we are going to render. From then, we're taking the formators and we check how we're going to render those fields. And, we actually get those things and we just describe the same thing, but in the React. And, we're not going to show everything, but we are going to show what has been done in the end. So, here's an example from Drupal, node template. So, in the node template, if we get all of our variables that we can get into the template, those are some of them. So, we have the nodes, we have the content, which is actually our fields that we are going to render. We can render only one field or we can render all of them or we can render all of them except some of them. We have our attributes, URL and label. And, this is the prep process of the node. So, if you see, there is a loop for all elements that's going to get and is going to prepare for us. This is the Drupal version. And, here is our version into React. So, again, React entity template, we have our entity that is actually a JSON API normalized from now. We have renderFields function, that's actually rendering all of our fields depending on the view mode. Or, you can, of course, skip some of the fields. We have renderField that is single field to be rendered. We have our URL, again, and we have props that is actually our alternative of attributes. And, this is some kind of our prep process into React. Actually, this one is a hook. I don't know how many of you know what is a hook into React. Okay, hook is high order components. And, you wrap your template or component, your React component, you wrap it into this different component. And, this component is just preparing for you functions, variables and different things the same way that prep process works for Drupal. And, when we have a template for the example node, we just wrap it into our entity hook component. And, we have all of those as already done for us. As a renderFields, renderFields, single properties different and so on. So, this is example from the code that is actually showing how we are preparing renderFields, single renderFields. As you see, there is a depending on the formatter. The instance actually is something additional that we need to think about it because, for example, all of the list fields, they can depend on the bundle, what kind of available options there are. So, we need to think about those. Something interesting that you can see here is actually that we are preparing template suggestions that I'm going to talk about something about this later. So, here is an example of the template of the field on the right. And, this is our on the left, our prep processor for the field. Which, if you compare to the prep processor into the Drupal, it's almost look exactly the same. So, we have a get label, we have this multiple, we have this empty, we have multiple others. We are preparing the different properties and then we render our template. And, here is the comparison between the Drupal template tweak and our JSX into the React. So, you can see that they are almost identical. A little bit changed here and there, but the idea is almost the same. So, looking like this, it's looking very easy to do those things, but we will tell you after that why this is not exactly like this. So, something else that you can see here in the end of the right file is that I'm sending to the wrapper a registry. And, this is connected to a template suggestions. In Drupal we have the template suggestions and you can have different templates for the different occasions. For example, a note can have different templates. It can be for the notes as the full plan. You can have notes templates for articles or for the different view modes. And, this is something that we need to recreate again into the React. So, we created our own template registry into React. We can register different templates and they look exactly the same as it was into Drupal. The interesting part here is that actually we create those as React components and NPM packages. That we are going to distribute later that year. To be honest, we create some of the templates ourselves to be included in those packages so that you can reuse it directly and not thinking about them. And also standards fields from Drupal as a pooling field, daytime fields, email fields and all of them have their properly formatted in them that are coming standard from Drupal. And, of course, you can override those and you can implement yourself different formatted. You can reuse this almost the same way that you do it into Drupal. The only thing that you need to do is just implement the registry, which is a class and you can just register your new template here. And now, here's the first problem. All configurations, view modes, field instances, field storages, formatters and everything need to be transferred to React and that's a problem. That's a problem. But at first, we think, okay, all configurations in Drupal start in YAML files while we just take them and transfer them and use them directly into the React. Okay, but it seems that there is kind of a problem with client sites and reading YAML files. So then we said, okay, it's easy. We're going just to transform those to JSONs and we've got that. But now, we have another problem. There's too much files, too much configurations, too much data in there and the performance was killing us. So we start optimization. And what we do is actually remove all the things that are just for the administration because if you check all the configurations for the fields, for the view modes, field storages and everything, you see that in there, there are so many things that are only for the Drupal administration. As an example, the dependencies, what labels should be there in the administration and so on. So we remove those and I think we manage to squeeze from... I don't recall the numbers. Each rank. Yeah, each rank really, really, really much. Okay, but those are the configurations for the fields and for the view modes. We have additional configurations that we need to take from Drupal that we want to be dynamic and want to have them on, let's say, almost each request. And first of all, we try with JSON API configurations. Actually, we will say after that why we are using JSON API, why we want to use JSON API and decide to use it. But when we start using JSON API to share the configurations, we saw that there is a security issue because it shared more information that we wanted and that wasn't okay. And then we decide to create a custom REST resource and plug in actually a module that can allow you to say, okay, we want to share to expose only those configurations. So that's why you can share everything that you want without any extra information. And okay, now we know how to take our configurations files, our configuration for fields, for view modes and everything, our additional configurations. And now the question is what to do with them. We don't want to have them really on each request. We need to cache somehow them. So actually, we get them when we start our React server and we're caching them, maybe in main cache, maybe already. And we use them when we need it. And into Drupal, we create a small module where you can identify your domains that are going to use this Drupal backend. And when there is some changes, the Drupal sent web hooks to say, okay, I have a new configuration you need to update. And that's how you can manage multiple different domains with one Drupal and to tell them, okay, you need to update. And the React will say, okay, I need to update. So I'm going to ask again the backend what is the new configuration in there. So this is the idea of taking the configurations. And about getting the data with the JSON API, I'm going to leave my colleague to explain more in there. Hello. So, yeah, as any of the couple of applications, we need to deliver data to it. And the first and main question is why JSON API? So there is a very nice blog post written by Dries with the help of the whole Drupal community that actually covers exactly this case. Whether you use REST, JSON API, or GraphQL, it was published sometime February this year. The problem with this one is that we had this problem about two years ago. So this information was not yet public at the time. And we actually needed to struggle with precisely the same problem. For better or worse, we managed or for the good parties that we actually decided to use the same, pretty much the same decisions and rules that the same guys decided on. But in the end, we decided that JSON API is the technology for us, mainly for two reasons. First, it was simpler compared to GraphQL in terms of how to use it and how it is implemented in Drupal. Currently at the time, it was the public version, public 1x version of JSON API module. It was not yet in Drupal 8 core. And it was faster compared to GraphQL for even simple queries. So imagine you get complicated and more complicated queries. It really depends on how it scales. And usually when there are these early technologies, they don't scale well. That was one of the things we faced with JSON API initially. Now I can say it's much better. You've seen all the keynotes. And yeah, we were part of the process to make Drupal 8 core faster. We're very happy about it. But in short, yeah, we decided to use JSON API and started to go with it. On the contributed side of the spectrum and the ecosystem there, there is the coupled router module. It is giving the possibility for us to use arbitrary URLs that are provided by the alias system. And for each of them to make a lookup and say, is that the valid URL in Drupal? And it will give us a canonical URL in JSON API that we can actually fetch by ID directly after that. And the other thing, because we are doing highly customizable pages that editors should have full control over. And that was triggering us for using paragraphs for editorial experience on the administration side. Because you are managing the whole page content through the Node Edit UI. But from that, we have many paragraph types and specifically for JSON API, we needed to have a lot of includes. And these includes needed to be, let's say, added in the URL in a specified format. Everyone that has used JSON API knows this one. You can check it in documentation, but in short, the more includes you have, the longer the URL gets. And because we're using CDN services that impose some limitations related to whether it's URL lens or whatever other reasons. We used JSON API defaults, but actually everything that could be needed by default is not specified in the URL, but actually the server adds the default, unless you override it cloud-side. So we have a lot of includes, and this was one of our future problems. We'll cover it in a bit with the guidelines. So this is why we chose JSON API, and there was already an existing ecosystem that would allow us to read any entity-related data with its relationships directly from Drupal with minimal custom coding. Because it doesn't matter how you look it, whenever you have a project with the deadline, you're aiming to reuse as much as possible. The other plus for JSON API, even though Rafael has a specification as well, is that JSON API implementation in Drupal 8 is fully going over the specification for the JSON API format, and the format is a web specification. So there is a standard how things should be formatted, how things should work, and there are valid reasons behind each decision. So we started with JSON API 1. We faced some problems with it in terms of performance problems, and we updated to version 2 that was already faster. But in some of our use cases, it was still slow. And then this was the contribution that everyone is trying to push people to. So I would repeat it again and again. It's much better to contribute. So we were having the internal discussion in the office whether we should actually fix it and move along or just hack it in some way, but in the end it was a non-maintainable solution because if you hack it or do whatever to reach your goal, in the end you're losing first your time and then all the other people using that module will lose their time solving the same problem. So when the contribution direction and it turned out very well, mainly because the whole team in JSON API were very supportive in terms of you push them an issue, you help them solve it, you help them find the root cause, write the code, actually resolve it to some extent, and they were responsive in terms of it was fast. I had cases where I submit an issue and it gets committed up to a week. It's true that it was in a let's say good time because they were actively doing it to push it in core and once it reached core it got way, way slower in terms of response times and so on. So during the country era of JSON API I think we're moving very fast. Now they're starting to ramp up again once Drupal 8 is stabilizing. So it's much better overall in terms of a module stability and everything else. In the ecosystem of JSON API there is well the extras module. It's a community one exposing the configuration UI for JSON API. It is allowing us to do configurations on a per entity type level where you can either enable, disable properties or hold entity types and then configure so-called enhancers on a per attribute level where you can tune what the Drupal endpoint will serve for this resource or expect to receive from the clients. This is a very important feature and at some point it should I think land in core but having it in country helps faster iteration times. The crucial one if you're using JSON API mainly because it is adding useful features to tune the payloads and the data you are transferring between Drupal and your clients. In short, the less data you have, the better. And I've already covered a couple of routers so it handles pretty much everything. You can ask it, give me the alias. Here is the alias. There is the actual entity behind it so it will tell you whether it's a redirect or not, whether it's a valid URL or not. It is supporting the standard HTTP status so you can handle it pretty much natively. And our general flow is related to ask whatever because client hits the browser on a URL then react is responsible to ask Drupal what page this is on and then if it finds it he is asking for the concrete data set that we need to fetch. So however you look at it you are having two URL calls. Having them... So all of these modules fully support the Drupal's caching metadata and all the caching primitives that are present in core so all these APIs are fully cacheable and are capable to be cached on CDNs or any other reverse proxy caches. And that will speed up things a lot whenever you need it, if you need it. So we've already covered this to some extent. I plan to have JSON API response but in our cases they didn't fit the slide. Nearly. They were like 50 kilobytes compressed JSON files so no way to fit here. The one big plus of JSON API module is that it has no configuration so you just enable it and it will expose all your entities including configurations. That's a, let's say, not the best case but after you enable extras expose only what you need. There is another very important bit that we used to implement a very needed feature for all the editors. So when you're having a decoupled approach you have the editorial experience very nice because you have just no dedicated forms but then you have the editorial experience a bit broken because you then don't have a preview that Drupal actually provides by default. You edit the node, you go to the node page and you see what unformatted, unstyled raw texts of some node. So because JSON API has an API to allow you to fetch a particular entity but for a particular revision of it you actually have access on a per revision level within JSON API. And because of that and some, let's say, routing of session IDs we managed to actually proxy the currently logged in user from the current Drupal installation that you're in. We have a demo on this one later to be routed through React that is doing the rendering on the current page to the JSON API. API called it is fetching the data so you actually be able to fetch a published revision and the editors can see whatever they are currently editing before it is being published. And, yeah, the ecosystem, it's already I've explained the JSON API extra so it allows customizations, allows to manage what gets exposed what attributes are exposed and how we expose each attribute. And defaults allows us to reduce or get request size in terms of URL. This is an example of the coupled router. This is subtracting the router in Drupal but it's hooking a bit earlier in the request cycle events in core. And the most important bit is that if you have redirects module enabled and configured it you will fully respect it. So it's, again, no configuration module you just enable it and it will work. It will expose an endpoint that was like to look up something, it is documented in the module and you can test whatever path you think Drupal should be available in or, yeah, should serve. And so, yeah, the most interesting section of this presentation are all the issues that we faced. I can tell you there were not that small amount. So one of the things were the preview, performance. Yeah, you can see them. So, yeah, let's start with one by one. This is a demo for the preview functionality and how it works. So you can see, you can have the page, this is an iframe that renders the public website internally in Drupal and you can change the width of the iframe from there you can, let's say, preview it in different device modes. You can switch them portrait and landscape modes and eventually if you have this functionality to have multiple domains for, to use Drupal as a content repository and serve multiple domains you can switch to domains and actually show different content depending on the domain data you are fetching. And from there you can open it directly in a new tab and see the website itself. Yeah, and now it's loopy. Okay, JSON API, these are some guidelines that we learned the hard way with trial, error, and a lot of contribution. So deep include parts. Mainly the data model that we have decided on nodes with paragraphs to model the page content because they're, let's say, hero image, that's a paragraph, there is a list of items, a paragraph. So you have a single column layout with a list of paragraphs and each of them is a section on the page and from there editors can render whatever page they like from this data model. So it's very editor friendly because all the data is in one place but it tends to create these so-called deep include parts or chains of related resources because to spare request round trips that is one of the biggest features of JSON API compared to course rest module you tend to say, please give me the node with its paragraphs, with the nodes related by the paragraphs with the media entities related by these nodes in the paragraphs and it gets very long, very, very fast and it gets slower. So the deeper it gets, the slower it is and the more includes you have, again, the slower it gets. So the guidelines are try to have as wide and as shallow as possible the resource relationship graphs or trees however you name them however you name them the idea is that they should be not more than three, four levels five as a higher level of depth and you can have them as whatever card you like one item, two items, hundred items it doesn't matter, it can handle it efficiently but if it's a deep one, not that much. The other problems we had were mainly caused by small inefficiencies that were resolved in later versions of JSON API through very greedy caches but they were related to the amount of resource types you have in the system and it was not scaling well so it was properly tested in core and in contrib with five, ten, twenty, eighty types, thirty in total but in our case we were reaching like one hundred and fifty, two hundred in total, different anti-types with bundles mainly because of the paragraphs and there were cases that it was not scaling so after the caches were added through, I haven't listed the issues here but you can easily find them on Drupal Word this was resolved but in the end if you don't need a particular resource type disabling it will reduce greatly these cache layers and they'll be fetched faster from the cache and then it will speed up Drupal overall because these caches are initialized on each request so they're in the, let's say, critical time so whatever you don't need, disable it as much as possible and that's the main rule to follow to actually have a fast JSON API service the other one, because if you start doing all these relationships and things, you tend to get bit with those objects and yeah, again, disable as much as possible you can do all of these through JSON API UI the extra UI and the last one was the caching it was not a problem by itself but initially we wanted to do it through sub-requests because any round trip to fetch data makes the time slower to fetch all the data and yeah in the end we decided to go with different requests done in parallel and leverage all the HTTP caches in reverse proxies as Drupal is very efficiently handling cache metadata it actually works much faster compared to, for example, sub-request if you want to prove all of that and send it as a huge post request another problem were image styles there is a core patch for this one or consumer image styles depending on what approach you use we went with the core issue because it works better for us anytime you do a decoupled application this is an inevitable problem that you'll face we've experienced it and that's the solution currently it's still in work so help the community reach to a design decision because that's the current blocker as far as I know I've talked today with WIMP from core API First Initiative and yeah, at the moment there is a design decision direction blocker that currently it's not designed decided how to continue and yeah, here is the example of sub-request you are sending for it a list of requests that Drupal is doing internally the problem with it is it's not handling concurrency because PHP in nature is sequential so even though you're sending them and you can handle this type of data Drupal internally is doing one bootstrap and doing all of these requests one after the other how they are defined here because of that the time to handle all requests is sequential so that's how we handled it we just said, well, we're not using sub-request and we will fully implement it on client side and this is how it was implemented and Petty will share more as you see here, actually we go with the sub-request model because this was the recommended way to go with the couple router if you go to the couple router module page you will see that there is example and this is a screenshot from there actually and as you see it's not first of all the post-request are not catchable by default and when we said, okay, why are we not doing this the same exact thing but into the React with the JavaScript and I was thinking, okay this is actually an easy one and if you know what is an Axios Axios is a library for requests from the Node.js actually it's very easy to do the same thing in there so we have Axios all and we send all of our requests and all of them are done in parallel and we are just waiting for all of them to complete and we have all the results the same exact thing as it's done in the sub-request but it's done by different requests to Drupal so Drupal is handling better this one and of course if you have different requests on each page and they can be some of them can be shared between different pages that means that you can cache them cache them separately which is actually something that it's very good for our performance system and on the left is actually our router request with the couple router it's similar to the one that actually it's not here but it's similar to the one that oh yeah it's this one actually it's similar to this one in some way and something that it's depends, it actually has in the sub-request model is dependencies of the request so here we have example with sub-request so we get in our we ask from Drupal to say okay on this URL what is it Drupal is telling us this is for example a not an article and here it is the JSON API URL from where you get your resource and we get it callback and that's it it's really really easy handling our actually a big performance issue in there so the other issue that we have was the search gene optimization and this is a standard problem for all JS frameworks for all client-side JS frameworks because no metadata so if you see a standard to react and you go check the different pages you see that actually all the pages are one and the same in the source because the JavaScript is acting after you hold the page and holding all the components rendering them and that is not a good idea so we need to go and do server-side rendering and actually we decide to use snake.js which is really really good for that one it's serving everything and the idea is like this everything is server-side render on the first request that means that you can have a really good cache layer for that you can have a codeflare you can have whatever you want in there and it's going to work really really really well but once you get the page the react is getting on top of it and start working after that as a single-page application which is very nice because you get the server-side render you can have dynamic router you have dynamic change of the data that you're going to have with the next JS and then the react takeover and do everything that you need from the react so that means that even if the first request is a little bit slower on the first time everything else after that is going to be really really fast and actually the results those are the two projects currently that we maintain with this system with this structure architecture on the left is visitdanmark which is the first project the project that you see the requirements for and actually our clients are here and it's really great as a result of what we managed to do even we have a problem recently that in some pages it's so fast working that the client doesn't know that it's something happen on the search and they said okay can you do something because the user doesn't understand what's happening there yeah so currently we're thinking how to make it slower on the right side is eggmond.com which is something interesting in there that they have some other kind of solutions built in there it's actually kind of domain access so we have one Drupal with single installation single administration that we are managing multiple domains in there from one administration and we are not using domain access we are creating something that we call currently the couple domain and we are going to contribute that when we are fully sure that it's working perfectly so as I said we have done many things some of them are already contributed as some of the patches for the JSON API for the performance some of them we are going to contribute I hope till the end of the year as the coupled preview module and the coupled domain and of course for the React we are ready to publish our NPM projects our NPM packages so you are going to have this Drupal JSON API client that is actually containing the REST client that is alternative of the sub-request you are going to have this structure of Drupal entity into React the configuration sub-system and of course the Drupal template system with the registry in it and actually we have many, many other things that we are preparing and we are trying to push into contribution but it's taking time to be sure that everything is working fine and it's not working as done for just our solution but to work in general and yeah actually we have a very small time unfortunately we cannot show everything that we have done because this is our project for maybe two years from now and yeah, there are so many things that you can share but it's not for that little time but if you have questions right now do you only show the React for content for those do you have a solution for forms? okay do we have solution for forms actually we have solution for webforms for webforms we have a solution it's working only because webforms is really a big project in Drupal so we are trying to push this one but yeah of course we are hoping that we can from the beginning of next year we can start with some administration only for content editors yeah there's how you manage the deposit so where is the React and where is the Drupal is it in the same deposit or separate depository? no it's a separate depository you can have different server for the Drupal you can have different server for the React you can have multiple reacts with different tasks, different templates if you want you said that you have a 10pm package so if there is a version upgrade every time you do some changes at the beginning ok let's say like this some of the things that we have done of our packages for npm packages we have done it maybe before let's say four months before the end of the project and we haven't updated those since then I mean we have everything that we need to work with Drupal and currently we just checking if everything works fine so that we can publish it but it seems that we have done everything that is needed to be done currently for starting so of course everything if we find the issue we need to have a new version of it and publish it with the React so do you have nested fields for example using paragraph inside the paragraph well let's go it that way our deepest relationships are 7 level deep so we have a lot of nesting and delays because of this one so yeah this is why I'm saying use less nesting mainly because it gets slower with this one I was trying to find the slide with registry template and you'll see that there is a entity reference field we're supporting this one so entity reference and percentage reference revisions do you have an edit part for when you look at the React no currently we don't have any button from the React but this is the things that we were hoping to start after from the beginning of next year to have editorial part in the React so any other questions yeah how did you solve the editor preview did you mention something about packing the sessions for them okay this is the last part that we are still struggling with the session right now so if if your backend is on same domain levels different domain levels but the same main domain you can easily change the cookie structure to be for the whole domain with all sub-domains and this is the one way to do it but if the Drupal is on different domain and reacts in different domain then that's the problem that we currently have and we are actually trying to send the cookie from your session to React and React will send it back from its parts to the Drupal okay okay thank you