 Alright. It does help. You know, if you set it to where it's audible, I think that's where they like to have it. But anyway, we'll go ahead and get started. We're at the one minute past. I know people typically trickle into these things, but thank you all for coming. This session is about the Weather.com Project Moon Racer. It's the lovingly named, does what it says on the tin. Anybody who's in the room who can figure out what it's a reference to, I'm going to give you a fist bump afterwards, so make sure to let me know. So let's go ahead and get started. This is about decoupled user interfaces. So it's not just decoupling the front end of a website, it's decoupling the back end of the website. So allow me to introduce myself. My name is Jeremy Dickens. I am the technical lead for the CMS team with the Weather Company. We're part of IBM Watson now, which is a lot of fun. I am a native Nashvillian, as you can tell by my shirt. They give native Nashvillians, when you're born, this is a true story, they will give you one of these if you're actually native. There's a coupon, though. You have to redeem it. And if you don't do that, then it doesn't work. But no, really, I'm super excited that everyone's here in Nashvill for DrupalCon. It's kind of a dream come true for me. So thank you all for being here in general, and thank you all for being at my presentation specifically. That's wonderful. I've been in the Drupal community. I looked at my Drupal.org page at one point in the last few months, it said nine years, and I felt really old. But that's probably enough about me. All right. And I am Matt Davis. I'm the director of emerging technology for media current. And I've also been around in the Drupal world for quite a while now. I installed my first Drupal site in 2003, I think. And I've also been playing with the couple Drupal for quite a long time. My first Drupal project was a Drupal 5 project that used XMLRPC to communicate with a flash application. It's crazy. So things have changed a little bit since then. But all right, let's go over a little bit about our agenda for the day and what we're going to cover. We're really going to be going over three main sections here. We're going to give a little bit of background on the history of weather.com's use of Drupal and their move towards full decoupling and then specifically how they're now been using decoupled administrative interfaces as well. So before we dig into the meat of the presentation, I just want to kind of posit a big question for the day. A decoupled editorial interface, why would anyone want to do that? If there's one thing we know Drupal is pretty good at, it's forms and fields and data models, right? So why would you decouple your editorial interface? Well I'm not going to answer that question yet. Instead we're going to go into some background on the project and we'll circle back to that. So let me give you a little bit of background on weather.com's history with Drupal. Weather.com is kind of a unique site in a lot of ways. Obviously because it's a pretty high profile site it has substantial traffic coming to it. But one of the things that makes it really unique is when a major weather event occurs the traffic spikes through the roof. And so it has very specific performance concerns that we have to accommodate and cater to. Aside from that it has very specific needs in terms of the pieces on the page have different needs in terms of cashability. So articles for example may be cashed for several hours but when you go to a weather page you expect to get the most up to date weather information and so that part of the page needs to be very dynamic. And so the way we approach this in the original Drupal build was with a project called DNA for Drupal and Angular. And this was a progressively decoupled Drupal build. That term actually didn't even exist yet when we launched this in 2014. It was coined by Dries subsequently. But the idea was to let Drupal still render the main markup on the page and the most cashable components on the page. And then those pages, those components that needed that very dynamic short lifespan we handed off to Angular 1 components at the time. And then there was also some custom magic glue in the middle using panels, plug-ins, which all of the components on the page that you see there are panels, plug-ins. And then we used some ESI, meaning edge side includes, which is the ability to have separate caching at the edge for dynamic components as well. So there are also some unique team needs in terms of the stakeholders of the site. They have an entire team that's dedicated to managing those panels' pages. And what you see here is the panels back in UI in Drupal 7. This is what the editors used for a long time with the DNA project. And I think as probably many of you in the room would agree, this is not the most elegant user interface. But we did find that with training, they really were able to hit the ground running with this. And actually they built tons of different kinds of pages configured them themselves and didn't need any developer assistance to actually launch whole new types of pages. So I'm sure most of you in the room are probably familiar with panels and how all that looks. But we'll just give you a little example of what that workflow look like to these users. We'll show you a little bit of the D7 weather.com homepage. Each of these is a angular mod. So you have this sort of decoupled, progressively decoupled thing. When we're looking at a couple of modules here, we saw a photos module. What we're going to do is we're going to go back into the panels edit page interface. You see all the variants down in the left-hand side. And then this semi-abstracted representation of what the page layout looks like with every one of those pieces of panels content is an angular mod. So what we're going to do is we're going to edit this one. And I believe this is the photo gallery. We will change the name just to make gratuitous updates, because that's what you can do if you can edit the homepage weather.com. We make that change. We just save it. And then once that completes, we're going to also say, well, the photos, we want to promote those. We want to make them more visible, bring them above the fold, maybe a little bit. So we change the position of that module. And then I think we're going to make one more change here. Yeah, we're going to go in. It's worth noting that these admin interfaces that you see are actually written in a subset of Form API. So the front-end devs would write their angular code and then basically write a Drupal form and put it in an info file. They got packaged right there along with their angular code to provide these interfaces for the users. So you hit save on that one, and then you update and save the page. Then Drupal comes back and renders you the homepage again. But this time, as we scroll down, you will see that it's our greatest photos. And it's now on top of the latest tropical updates. So that is the basic workflow that these users, I mean, they did things that were a lot more interesting, I promise. But that's for illustration purposes. But as great as that was for those users, our web team actually, they loved it at first. It really worked really well for them. And then they really needed some additional support in getting, being able to do the things they wanted to do. I love the JavaScript community. I love that it's like super dynamic and there's lots of stuff happening all the time. But what that meant for us was that, effectively, we could not support the changes that they wanted to make. So if they wanted to do something really exciting and super interesting and cool, that violated the guardrails that were put around the DNA module, or set of modules, they were constricted by that. And if they wanted to do that, and it was a business decision to do that, that meant that our team, which is quite a bit smaller than the front end JavaScript team, would have to implement something. And that was not always something that we could do promptly, to be quite honest. They wanted to move off of Angular 1. They wanted to move to Angular 1.5, Angular 2, Angular Universal, React, Vue, whatever. They couldn't do that unless we submitted quite a bit of time to get that working. There was a big drive also to be CDN agnostic. We were heavily invested in a particular ESI implementation. And what do you do if you don't want to use that vendor anymore? That was another big one. And then I think most interestingly, technically, but one that I'm going to gloss over completely, is that Drupal's really, really bad at aggregating JavaScript and shaking out JavaScript dependencies. And most people don't run into the same kind of issues that we did when you have that much JavaScript on your site. And you have infrastructure issues that can arise. So we had that as something to deal with. Yeah, we ended up commonly calling the Drupal 7 advanced aggregation module the advanced aggregation module. Yeah, they sound so similar, but it's so true. So the front-end team actually went off and they said, you know what, we're going to decouple this thing. We're going to make it a completely decoupled front-end. So Nerf was born. And Nerf is a very, very clever acronym. I've got to give it to them. NodeExpress, React and Falkor was their proof of concept technology stack. Interestingly, as it evolved, it became, you know, Coa, Inferno, Node, GraphQL, which is King, which is maybe slightly less fun. But, you know, they stuck with Nerf. It's definitely the catchier of the names. They completely eliminated any dependency on ESI. So none of that craziness. And then we were already pushing all of our content to an external content data store. So if you were in the decoupled summit yesterday, you heard a lot of talk about a content API, a separate content system. We were already doing that. So it made it very easy for them to say, hey, we'll just stand up something and read all the stuff from there instead of from Drupal. So that was good for them. But what was interesting was they inherited a bunch of stuff from DNA that I thought after they implemented it was very interesting. They kind of could have taken a cleaner break, but they had page configurations and they were calling them variants, which is very panelsy. They were talking about things like context, too, which is also panelsy. They were using it slightly differently. They had layouts, right? They had user configurable modules on the page. They were taking their module code. They kept that same paradigm where if you wanted a module that did X, you wrote a little bit of code and put it somewhere, which is a common JavaScript pattern. I mean, it totally works. But there's still modules. Interestingly, when I say user configurable, at this point, they are configurable by a developer. If the product team needs them to change, then we start to get into where this started causing consternation. NERF became, to those product managers that had had that workflow and had that capability, this became a big old black box for them to work with that no one likes a big old black box. Modules and pages. Yes, you could change them. Yes, you could put in something. The dev team could probably change them more rapidly in code than you could in a UI, but it was going to require a code release, which a lot of times in decoupled interfaces, people are like, yes, that's what you do, but we'd already let this genie out of the proverbial bottle. We have these use cases where we get into severe weather events. They want to specifically move a module up because it's important or add a new module to the page because some area of the country is in dire peril. They can't do that without a code release, which, by the way, if you're in a severe weather event at weather.com, you are under a code freeze. You cannot do a release. We had taken this away from them. That's really not all that cool. That's not a good thing to do. Effectively, we basically gave these product managers, we took away their free agency. We took away their ability to do anything in the system that was part of their job before. They could make decisions. They could say, we want page X to do Y, but they used to be able to just do that. Now, they were dependent on someone else. This is where the page management system we call MoonRacer comes in. This was our attempt to really not only give back the capabilities, but also to reimagine what these interfaces could look like. We had basic MVP requirements, first and foremost, as we mentioned, was to re-enable our product managers to be able to manipulate pages and put new modules onto pages as they were used to doing before. Also, because the new Nerf architecture was already live, we needed to be able to accommodate and be compatible with that architecture, which was actually helped in some regard by the fact that they had adopted several of the concepts from the original DNA platform, so they already had those abstracted ideas of variants and module configurations. Also, we needed to be able to support writes to an external data store. For performance reasons, the Nerf system didn't want to be querying Drupal's APIs directly. One of our overarching goals was to minimize the amount of custom code that we would have to write and to utilize open source tools wherever possible. Let's talk a little bit about how we approached this. We knew we had pretty consistent data models that we could turn page configurations into, so Drupal 8 seemed like a very good fit for this, and we were excited to get to experiment with some of the new tools that were being developed in Drupal 8, specifically around APIs. JSON API was pretty new on the scene at that point. This was coming out of DrupalCon Baltimore when we were really starting to build this, and we were really excited to play around a little bit with JSON API. We did run into some limitations with JSON API where it tends to be extremely verbose and provide all of your data in the APIs, and we actually wanted to limit that sum so that only the data that the front-end rendering system needed would actually be provided to them. We also wanted to be able to remove some of the Drupal systems that are provided in JSON APIs by default, things like field underscore or whatever that is a very Drupal specific term that we all recognize, but the two of JavaScript developers just extra noise, and there's actually a story about how this was born, how this all played out. In Baltimore, DrupalCon last year, we had Matt and I had been talking about how this was an issue and things we wanted to do and ran into Matteo on the exhibit hall floor and stopped him and introduced myself and said, hey, we're really looking at your stuff. This is really great. We've got these ideas about how we might want to modify it a little bit for our purposes and maybe provide overrides. He says, I think I've got something that does that. Give me till after DrupalCon and then lo and behold, next week, JSON API extras was released and it was a pretty awesome moment to be like, oh, good. Yeah, I think he had a branch somewhere already in a repo that already did some of this. That was pretty neat. Going a little further, we also integrated with the OpenAPI and Schimata modules for declaring out our data models as APIs themselves. And for authentication, we used SAML user auth, which was pretty straightforward to implement as well. So that's what was going on in the back end Drupal side of this new page configuration system. Honestly, not a whole lot. It's a pretty bare bones Drupal site in a lot of ways, but it still leverages that core power of Drupal, which is that data modeling capability and exposing those APIs out. Oh, and one more thing was the fly system S3 integration, that's right, which, as I mentioned before, for performance reasons, we wanted to make sure that the final read-only API was pushed out to S3. So on the other end, the actual decoupled administrative interface, which you get a little sneak peek of there, was rethought from the ground up as well. And it was built as a Node.js and Express application with React components, using Redux for state management. And this was really it gave us the ability to design completely outside of the kind of Drupal box that we're used to. And so we actually leveraged material design, which provided this sort of familiar and intuitive layout to our product managers. So we're going to take a little dive into how we broke this data up. I apologize if as we go through this, the slides, I mean, it's going to be hard to read, it's blobs of JSON, but we're going to do our best to describe what these are and what they do. And you'll see kind of how we solve this problem from a data model point of view. Infrastructure-wise, as we've said, JSON API and JSON API extras were key. Coming to agreements with the API team or with the front-end team that we were going to use that API standard was like a really big win. We normalized as much as we could. There were still some places where it was like, yeah, well, this could be this or it could be that or it could be something else. So in those instances, we actually went and leveraged JSON B fields. So we are just dealing in some cases with blobs of random JSON. Don't judge. You weren't there. So let's just get into them. The first one, this one's actually pretty simple. This is the layout document. It's really just a bunch of layout areas. You've got some objects, you got IDs, you got some names. Notice, all it does is describe what a region is. These are just buckets. We have no idea how this is going to get rendered. We don't care what a header is. We don't care what main is. You could put something in there that's left rail. In fact, that is one that I had to edit out. We have no knowledge about what the presentation layer is going to do with these. This just provides us with buckets so that when it does get rendered, it'll be rendered in the right spot. There are only three of these and they haven't really changed. The next time they do a site redesign, if we're still on this infrastructure, we'll probably add another layout. So we don't have a UI for developing this. One thing that I keep coming back to you is that, honestly, Postman is a decoupled UI guys. Think about that for a second, huh? Okay. Or maybe don't. Sorry. Module definitions is actually one of the more interesting data types that we have, I think, because it does a lot. We showed you in the D7 world that we had these modules and they could prevent or present a configuration to the user and that that was done with a subset of form API. Well, it was nice that some of the developers had learned some form API, but we really wanted to get to a standard. So what these files are is a blob of JSON schema that defines the configuration fields that this module uses. And the UI system, the React MoonRacer UI interprets this to both present a user interface to the user and to figure out what that serialized data is going to look like when it writes it back to the API. So a module developer has to say, okay, my module does x, y, and z. I need these fields. I'm going to write a JSON schema document that defines those fields. And then our UI system takes that. And not only like says, okay, we're going to make sure you've got those fields for consumption, but interprets it and presents the user interface. Talking about the actual config object that gets written, that's called a module instance. The nerve system had an idea of module reuse. So our modules are reusable. You can create them, you can name them, and you can put one module on a lot of different pages and they're very easy to use. And the cache is very smart. So it just gets reused. You can make that change in one module and it will get reflected in all of the pages where that module is used. But we'll get into some of the challenges around that maybe in a little bit. Again, the data structure is defined by that module definition document that you saw. And our UI literally writes these in. If they are creating a new one, it posts it in. And then we get into the fun stuff. So the page config is our big data model, I suppose, that really holds a lot of the meat of what's going on in a page. But first and foremost, it has just your basic metadata about a page, the title and meta tags and things like that. But then if you go a little further down, you get into this real configuration object, which is references to the layout and to the specific module instances that have been placed on that page. And you'll notice here that these references are just UUIDs, which is how JSON API entity references work by default in Drupal. This was actually kind of a problem for us because we didn't want to have to just be making a request to a page config entity object and then go and make subsequent requests to all of the referenced entities that were related to it. So we actually have one kind of additional data model that we call page composites. In the Drupal side, it's actually just still a page config. But with some handy tooling that the JSON API module provides, you can just append includes to the end of your request, and it will automatically include the Drupal component. And then it will automatically include the data for all of those referenced entities. So this composite really packages up every single piece of data that the Nerf stack needs to render a single page from configuration. And so this is the document that we actually, when we post it back to Drupal, Cron runs and it pushes it up to S3. And from S3 is where that data is actually consumed for rendering by the Nerf stack. Oh, demo time. All right, bear with us. Let's see. So this is the Nerf homepage. And we'll just show you, structurally, there's some very similar ideas here. You have groups of, you know, you have a region you can kind of make out and you see modules with, you know, probably some configuration in them under the hood. We've got lazy loading, got to wait for that. And you'll see that we've got this recommended block and we've got a photo gallery. And you'll notice that what we can't do here is find a link to go click on MoonRacer. So we have to go switch to another tab and load that. And the user is presented with a list of pages. It's pre-sizable, so we give them a search capability to find the homepage, normal site mode. And you kind of get this interface where the rubber meets the road and where pages get configured. So I think in this case we're going to, yeah, we're going to do the same sort of thing. We're going to rejigger that recommended and that photos gallery. So here on the far left you get all that page metadata. So this is the page config level information that is editable. Then you have another region in the center here that actually lists the regions that you can drop content into and you can expand these. And inside you'll see the modules that are in that region. And we see recommended and we see the photo gallery. We're going to click the photo gallery. You see this is a rather complicated form all coming from JSON schema data interpreted by our front end. We modify the name. You see, you can note it's a little hard to see on the screen but the UI calls out that there is a change by highlighting the module in yellow. We're going to drag and drop a little bit, rearrange those things. And then we're going to change the name of this one because we did that in the other demo. So let's stay relatively consistent. Because we want to recommend that content to you. We're going to hit publish. Make sure you're sure that that's what's going to happen. And then in the background it posts those module instances and page configs or patches them to the Drupal API, Cron runs and it pushes that page composite to S3 and you will see that there is a change on the decoupled page to reflect it. Those people that were in the decoupled summit, you actually got a much more real time party trick but you know, this is higher pressure. All right. So before we dive into the challenges that we encountered along the way of building this though, let's recap a little bit of what we gained from the introduction of MoonRacer. So most importantly the editors regained their ability to have that full control over what is placed on a page. And as we mentioned, this was a really mission critical need for these product managers especially during these code freeze weather events. So that was huge. Also, I think we would agree that that interface, maybe there are ways to improve it but compared to what Drupal 7 panels back end was providing us, it's a lot easier for the product managers to navigate and very intuitively understand what they're doing as well. So those were both huge wins with the system. But now let's get into some of what we learned along the way. Real talk. And the biggest thing is we were trying to in many ways effectively replace panels module with a decoupled version of panels. And you know what we found out? It's not panels. It was way out of scope to try to recreate the full functionality of panels in this project. So some of what we lost from the original DNA capabilities was the ability to control routing from within this configuration system. And what that actually ends up meaning is that while editors can modify existing pages, they had no ability to create a brand new page, give it a URL and then drop modules onto it. So that's one existing one continuing pain point I think. They're also, because this isn't panels, there's still no real configurability of selection rules for these variants. The variants exist in an abstracted state, but if you want to change the weight of how one variant gets selected over another, that's a code change. That's no longer possible through configuration alone. And then there's also this kind of magic thing that we had to create that exists outside of our data models that maps those existing variants that were already alive in the Nerf stack to the UUID of the entity in Drupal that doesn't really follow the JSON API specification. So it's kind of like this special snowflake data model that we were left with. And again, because this wasn't panels, we had no mini panel support. So one of the things that product managers like to do in the DNA system was create groupings of Angular modules that they could then reuse across pages. And then they wanted to make a change. They could change it one place, and it would be reflected everywhere. Because we don't have the notion of mini panels anymore, when that need arises, what actually has to happen is we have to create a new component in the front end that collects those subcomponents and then expose that back to the administrative UI to give them that configurability in groups. And that leads directly into this bulk editing. There's a developer step that's required if they want to bulk edit groupings of components now. All right, so those of you that have worked in the decoupled space or have been thinking or talking about it, you know, the preview is kind of, I don't think it's an elephant in the room anymore because everybody really does know it's there. And it's not that it's unsolvable, but I think everyone can agree that it's inherently difficult. If you're completely coupled, you no longer have that ability for Drupal to render your page for you and have the user go, yes, that's exactly what I want without jumping through some additional hoops. The biggest struggle with it to me is that it requires a lot of consensus building between all of your different systems and your different teams on how you want this to work. There are a lot of different ways to solve this. These are kind of ranked in our order of preference and how we went with it. So really you establish a contract between these systems and you allow the front end to be able to hand back preview to preview users without it going live, without it poisoning your cache. And we're doing that with query strings and JavaScript web token to provide that security layer. But there's some other ways that you could do this. You could, we're in React, there's enough code commonality that we could wrap some of their code into our UI. That's technically possible. But that's really kind of ugly. And then you could create a separate front end instance that all it does is create preview, but that has the burden of and maintenance costs. You've got to keep this container or whatever running to spit out these ad hoc instances. We're trying to err on the side of use what already exists, use what's there, but it's not easy. We'll just put it that way. One other thing that we saw as a challenge coming out of this project was this whole concept of layouts. So our constraint was to make our layouts be designed to be framework agnostic and kind of future proof. And what that means is that as you saw our conception of layouts is a very simple notion of just buckets of regions that you put things into. Which has its advantages, but one of the disadvantages of it I think, when you have that sort of very abstracted data structure is that it leads to an abstracted interface. So as you saw in the demo, those regions, there's no contextual way to understand what that actually represents or how it's going to look on the page until you actually generate a preview of that. You don't know, you have to make inferences based on the name alone effectively. So there's no visual cues there. And as was mentioned a little earlier as well, some modules that we create for the pages are only really applicable to specific regions. But this was not something that we accommodated in our build. So this is another step that product managers just have to understand and know which modules make sense where. And that can sometimes lead to some trial and error of them putting things in the wrong place and having to preview and not looking great when we could potentially make that something that's actually built into the system. So ironically when developer experience was one of the key drivers to letting the JavaScript team do JavaScript and the Drupal team do Drupal we found that we still have a little bit of web team chocolate in CMS team peanut butter from time to time. And we really tried. We really, really, really tried. We did our original conception was we didn't want anybody but the CMS team to have to go back into the Drupal back end. We have found that that's at least in our initial build, that's not exactly possible. And some of that you can understand, right? Like first of all, we're telling these guys that we're creating modules and module configurations strictly in code that was going to be rendered procedurally that now they have to flatten it out and write some schema document that they didn't have to post to an external system. And I think everybody would agree that's kind of like not cool to a certain level. I mean like people are like, come on man, why do I have to do this now? So we understand. But we tried to make that as painless as possible except how do you manage getting it from their IDEs and then their Git repo into our code base and our database in a way that is manageable where it's not just the Wild West where you can keep a testing process and QA in the loop. We've got an initial solution where that repository that they commit these things to is a dependency of our Drupal build and then we've got Drupal console commands that can read that into the database which works just fine and the process is actually straightforward except A they do still, they kind of require us to do something and B it doesn't account for multiple module builds where developer A has a module that needs to go to production that was committed to the repo after module B which is not ready for production yet. So sticky, not unsolvable but sticky. And then we have built some things to make things easier for us and to make things easier for us when they ask us to do things but again there's that big problem if they're still having to ask us to do things like replicating page configs. I want a duplicate of it but it needs to be for a different language. Activating pages so that mapping file that's the special snowflake is based upon whether a page is considered active or not which is a boolean on the page config. We typically do that which means that when they want to launch a new set of pages we ask them to include us in the process. Maybe not a bad thing but still I mean it's understandable it's a friction point and then we put a JSON B editor in our back end so that we could you know if we needed to go and edit some JSON from time to time and it turns out like yeah maybe you can go make that change in the dev environment and test it instead of us going through the git repository thing. So it's turning out that they are actually needing to interact with the Drupal side of things a little more than we anticipated. So one more area of challenge and I would say this is not so much specific to the decoupled editorial but I think this is really a more general decoupling challenge is accommodating the designer experience so in this new nerf front end the nerf system owns all the markup and styles and this kind of leaves designers in a delicate situation. It's a big thing in the Drupal world component-driven design and this concept of having like a living style guide that designers can have a system completely outside of front end or back end where they can see all of their components in isolation. Nothing like that exists in the system and it means that when a designer creates a new module or component they require a fair amount of developer time to integrate that into the code base. And then we have this process of moving from the JavaScript code into our flattened JSON API config and into the database you know it's its own problem and I thought it was going to be one of the huge or more difficult to surmount problems but it turns out we had a developer on staff at the time on the front end team named Joel Kanzelmeyer who just knocked it out of the park and wrote this generator that reads their data access layer, translates it all into JSON API and posts it all into our system. It was brilliant. It was fantastic. It was reproducible. We could it wasn't all or nothing. We could choose specific pages that we wanted to deal with which is how we decided to do it. We were going to phase in MoonRacer so we were going to take certain configurations and launch them earlier and then kind of go from config to config which is great. It's a great tool that it can do that but what we found is that any of those pages that we haven't done in the MoonRacer yet, there's the potential that the web teams had to make some upstream changes that are going to require either additional work in the generator to serialize them properly or even additional work in MoonRacer to accommodate for that data. I mean if you can imagine a situation where a module definition is changed or the generator decides that it interprets that there's a new field in a module but that field is not represented in that JSON schema document that means the UI doesn't know about it. That means that they go into the UI and they don't see it. So you can imagine there's a number of little pain points to not being completely in sync here but not insurmountable again but a challenge nonetheless and then the one that's maybe the more fun and more frustrating at times being in the Drupal community is we did suffer and kind of probably are still suffering from early adopter syndrome. A year ago in Baltimore we hadn't really laid down any serious code for this project outside of Matt's proof of concept and API first was getting a lot of momentum. Matteo was building a lot of momentum on JSON API. Like I said it was in Baltimore where I said hey, we need that. So he's like yes, JSON API coming soon. Which is great and it was fantastic because we were actually able to like really pitch in and help in a community effort which is something that I recommend everybody try to do. What has been the challenge though is you know literally this past week we finally bought ourselves up to date with the JSON API module and JSON API extras module. I think we were four or five versions behind it. When you have a component that you rely on that has that much velocity behind it then you've got to keep up because there's talk about moving JSON API into core and we want to be ready for that day in 886 or whenever it ends up happening which means that it's on us to keep our code base current. Really that's easy enough except for the 2A and the regression testing that you have to do anytime a major change happens. Anybody who's been working in those modules know that there's been some pretty major changes recently that you've had to work around. It's a good challenge to have. It's good to know that we're staying up to date but it was one that I will highlight. Now we've shown you all of our challenges and kind of a look under the hood. So now let's have a look forward at what's maybe coming next. First and foremost maybe we should circle back to that big question we asked at the beginning why on earth would you want to decouple your UIs? Well even with all the challenges we faced we're still really excited about the potential of decoupled editorial interfaces because first and foremost it really gives you an opportunity to completely reimagine the user experience and what's possible with forms. It allows you to make them much more intuitive and really that is so key to reducing those training cycles that you needed when you bring on new editors, new content managers. If they can intuitively be able to do exactly what they need to do with an interface it creates so much less friction with training and it also creates these better visualizations of what can be complicated data models and also importantly I think it lets developers work in the accustomed spaces so that javascript people can write javascript and that's one key that we always want to be accommodating. So you know over the last year and a half probably close to two years Matt and I have been thinking a lot about this problem space so just thinking about what you know part of that answer of why do this is think of the possibilities so just a few that I'd like to throw out there for people to think about if you could simplify your node edit screen that's fantastic. You could make it more complicated anybody who's used like tried to implement a multi step or multi stage form in Drupal knows that it's not the greatest experience either for you as a developer or the users who have to use them. So imagine being able to break yourself out of that box and build something that actually flows a user through a node entry that makes sense for that specific node and isn't just hey here's forms and fields. Another one that I think that we have a specific use case for at weather that I hope to beat my head against for a while is you could have multiple interfaces for the same data type and the example that I can present is weather.com has articles which are pretty massive node edit forms with all of these bells and whistles. Our sister site Weather Underground wanted a blog and when I looked at the content that they wanted on their blog I was like well it's an article. They said well because we don't need all that other stuff. Fair point. There are ways we could have done that in Drupal and we actually thought about doing it in Drupal and that engineering team decided to do something else but imagine being able to just have one microservice that gives editor group A an interface and another microservice that gives editor group B an interface that all populate the same data model and can search or publish or syndicate properly. I think that's pretty huge. Categorization and workflow. I know that there's a lot of current work around a workflow initiative and having publishing steps. All those publishing steps really need to be data wise. There's like a value on the field. But you could represent those visually to your editorial team in so many interesting and cool ways that it's worth thinking about like well I could have all of these different boxes and content flowing through the boxes and I could hover over it and get additional information about that content or I could assign it to this editor or I could assign it to this playlist. Really you can do that without being fully decoupled. I agree. You could make these things and make it a matter of being API first like Drupal's trying to achieve that goal of being API first to allow for those experiences. And then finally another one that I think that we've got a pretty good use case for is being able to build a form that assembles data from multiple data sources before committing it to your Drupal back end. So we had a use case where we were part of IBM. We're part of specifically the Watson group and our team implemented the first, the very first implementation of Watson at the weather company. Our team actually implemented it. And it was to read the body of an article and populate Watson natural language processing tags so content categorization, content tagging into the node. Well one of the problems becomes when do you do that? Do you do that on node safe? Well we save drafts. Do you do it before publish? Well we had this other piece that needed to read that so it kind of needed to exist before publish. And you could get into some Ajax stuff but we found that the API was not the fastest API because we were using a dev version. But anyway think about the possibility of being able to just populate data from other sources into your node edit form cleanly and easily without having to deal with Drupal at all. Just write that stuff into Drupal as necessary. I think there's a ton of possibility there. Alright so just briefly we're going to talk a little bit about what others are doing with the coupled editorial because we do see this as a space where there's a lot of excitement and energy right now actually. First and foremost the core JavaScript improvement initiative is in the process right now of building the entire administrative interface as a decoupled application. If you go to github.com.jsdrupal right now you can check out what they're building and actually install an example app. There's some question because it's still in the experimental phase around whether this will actually become an admin theme that's included with Drupal core or if it will be a composer dependency that just installs a standalone app. But there's some really exciting things happening there that I think will inform a lot of the future of decoupled editorial as we think of it. If you've been in the decoupled space for a while you've probably seen the CardStack demo that Ed Faulkner debuted a few years ago at DrupalCon New Orleans which was an inline editing experience that was actually an EmberJS app layered on top of a Drupal site. Very cool demo. CardStack has since become a blockchain project they put a blockchain on it so I don't know how much it has to do with decoupled Drupal today but interestingly in the decoupled summit yesterday we saw an awesome demo by the 1x internet team with a very similar inline editing experience that's actually using paragraphs that you can drag and drop as widgets into a react application. Super cool and they're doing a boff on Thursday at 10.45 so I'd encourage people to check that out as well. Alright so we're going to wrap up, we're going to do this pretty quickly I know we're getting short on time but just some final thoughts really if you're talking about decoupled UIs this is the key driver for what you're, the decision you're trying to make is Drupal's out of the box experience good enough for you or do you need a tailor made UI for your users. What that gives you if you make that decision for that bespoke experience is you can actually bring your editorial team or what the users of your back end to the table to discuss what it is that they actually need and you can avoid maybe that awkward situation that we've probably all had where they say we want it to work like X and you respond with well Drupal doesn't really do that or work that way. But if you do make that decision, you know one of the biggest things that made our lives easier through this is getting that API consensus, getting everybody on the same page about what your data models actually look like, how your APIs are going to work when possible please embrace shared standards. The JavaScript world knows about JSON API they know about a lot of these things so when you say well we're going to use JSON API, likelihood is your front end team is going to go hey great that's fantastic. And then finally like really watch this space and not here but globally I think my prediction for next year at DrupalCon there will be I think significantly more people who've been working in decoupled UI and will be demoing decoupled UI and be showing it off and having buffs and having sessions about it so if you're interested I don't think it's going away and I can't wait to see what you all do with it. Alright so just before we finish up here I want to give a quick shout out to this conference that we created last year. We are having another one this year decoupled Drupal days so if you're interested in this topic space we have an entire conference where all we do is talk about decoupled Drupal hack on decoupled Drupal it's a lot of fun we're doing it again in New York City August 17th through 19th 2018 so follow us on decoupled days on Twitter if you'd like to get more information about that. Thanks so much. Thanks y'all. We have five minutes thank you so much. If anybody has any questions step on up and sir may I just compliment you on your predators blank. Oh yeah they're the best second place team out there. President's trophy I don't know. Well yeah that'll be the only one this year. Oh look at that look at that. My team wins cops anyway great talk I really like that you guys focused on some of the issues in this space. I had one question and one shout out to myself so I'm doing a thing on decoupled UIs and tomorrow about a non-Drupal decoupled UI if you're interested. I did want to ask you about the language you use so like the word that you use like layout and module and did you find when working with the rest of the project team that that helped or did it make things more confusing because you would have like oh that's a module well it's actually. It's more confusing for us because again they took that ball and ran with it they were the ones who were like yeah we have layouts we have variants we have modules like oh yeah it's all this stuff that you got from D7 right so it was really. You didn't have to like make documents to keep things okay. No it is kind of funny that we're still calling the modules yeah they could have called them like sugar plums and I would have been like okay great that's like okay great but they're components we all know they're components everyone should just call them components good talk guys. Thank you. Hopefully next year Major League Soccer can contribute to this. There you go awesome. I wanted some clarification did you decouple the entire editorial dashboard? No no and I think that's a key thing in the spaces is to focus first and foremost on those mission critical interfaces where you really need to provide that extra capability to editors to have that really intuitive experience because still by default every data model you define in Drupal you're going to get a form automatically right so if that's good enough that's great you know you really should consider it on a case by case for those mission critical interfaces Did you just have like a separate URL to go to for those that bespoke experience? Yeah exactly it's a totally separate that we just link in. Cool cool alright thanks guys. Yeah thanks Thank you. Should take one more. Thanks for the work good presentation. Now my question is on the microservices you talked about did the different microservices at least did you have microservices that had say their own databases to which they pushed to and at some point you needed probably some of the data in one database synchronized with another database and how did you able to approach that if you had that? So your question's about data consistency I have nightmares about data consistency but not with this per se so that's one of the reasons why it's actually beneficial that with JSON API we can pass in those parameters that allow us to include additional data and flatten that basically it's three different entities instances of three different entities a page config and a bunch of module configs and a layout and we can assemble all those in one flat document and push it out to S3 and basically that's all the front end uses. The way that that push works is effectively anytime one of those things changes we repush all of the things that it has to deal with which in this case is not awful but there are other cases like when you start talking about content specifically where that can become a very sticky problem and yeah it's it's its own we didn't highlight it here because it's not a problem with this system per se but it is a problem it's it's or a challenge yeah a challenge. Alright one more I think we're yeah we got a couple minutes. My question is actually a little bit related to that with S3 I was wondering if you could describe what is going to S3 like a flat pre-rendered JSON response. Yep it's the page composite document with the included module definitions so it's the same thing if you hit if you hit JSON API slash page config slash UUID you get the page config then you can pass in the include and you include the layout and the module instances and it will add the relationships and give you those rendered JSON objects as part of your document and all we do is we take what that would be and write it out to S3 as a flat file. And is that a caching mechanism at that point? Yeah I mean so it was interesting the web team were the ones who said can you just make it S3 because we really like reading from S3. And that means they don't have to make an HTTP request to your server. That's right. Awesome Thank you. But I will say that is one additional thing that's sort of a challenge for me like I keep saying I want an instance of the front-end code that hits my server so that if I'd had that today I would have done that demo completely live and felt really good about it. As it is I've got service worker to think about. I've got caching in front of S3 to think about. I've got the Drupal cron run to think about. So rather than sit there and try to burn 60 to 180 seconds Right. So you have to deal with that for your testing. Exactly. And our devs complain about it too. Like why am I not seeing this? Like well it's cached. You know it's cached. Awesome. Thank you very much for explaining that. This is great. Thanks everybody. Thanks everyone. Thank you. On Thursday afternoon.