 All right, everyone Welcome So I am Michael who I am a full-cycle web developer. I come here from the delightful land of Switzerland I've been doing Drupal for 10 years now and I'm for hire. Hey Small block Hi, everyone. Good to see so many of you here today. My name is Dick Olsson I'm a longtime core and contrib developer. I love everything about Drupal. I'm born in Sweden I live currently in the UK. I've been working nine years with Drupal and I currently work at Pfizer Really excited. Cool. So what have we been doing and why have we been doing it? So a bunch of really nice people wanted me to Build an example of how a decoupled Drupal 8 site could look like We had a bunch of use cases. We wanted to try and fulfill One of the important milestones was offline capabilities Many websites can be had offline in in basic ways Where you can see is a page you already loaded in a browser But we wanted to try and push that a little bit Make it possible to use the entire site offline So you have you could actually have the whole content on your phone or wherever else And the reason for this we had a couple of scenarios for this So imagine if you're a salesperson You sell for example medicine for a living and you travel around to doctors in rural areas And you want to do a demo for them But the content you want to display them is not you can't view it because you're there's no edge or anything or you're on gprs and everything takes forever So that doesn't really make for a good demo. So it would be really awesome if you could have your whole demonstration downloaded on your iPad or whatever and similarly if you're a doctor and you Want to find information for how to use set medicine? You know dosage or whatever Information you could imagine and you're curing in a hospital and the hospital doesn't have Wi-Fi or radio devices are not allowed in the OR or Imagine such scenarios It would still be nice to be able to have at least some of some of the website information available So Yeah, that was one of our goals and Another interesting use case is to to do more than just build websites with data actually take your Drupal data and build All sorts of crazy applications with it. We didn't actually do this. We just wanted it to see if it was possible Another interesting use case is you know in these days where there's a lot of web exploits flying around and Maybe it could be nice to have a Drupal back-end that was only for your editors so you have a site deployed to the web that's Non-interactive no one can log in and mess with it. It can't be defaced So with this as we're going to explain later We have we found a way where you could put Drupal behind the firewall and have your content rendered by these decoupled apps Another angle we wanted to try out was building native mobile apps The whole thing where you embed your website in a web view on your mobile phone that works okay, but It has limitations, especially when it comes to performance and offline usage and that kind of thing So we we experimented a little bit with that as well We so we chose a bunch of technologies We will explain the more as we go along. So I think dick would like to So First technology that we obviously used I don't think it needs much of an introduction right Drupal 8 But the really core piece of the Drupal stack was the deploy module and its dependencies We call it the deploy suite This is a suite of modules that enable you to replicate content between different places These different places could be two different Drupal sites There could also be a Drupal site and another compatible application. We're gonna look into more details here before we dive into more details Content replication is really hard moving content around in a consistent way between multiple systems It's a really difficult problem to solve. There's all sorts of Latency issues. There's conflicting issues. There's All sorts of trickiness So What we've done in this stack of modules is that we've borrowed a lot of concepts from other projects other systems Because we don't want to reinvent the wheel here So the deploy suite consists of a couple of different modules I'm Not gonna dive into a lot of detail here We want to focus on sort of the decoupled piece here, but the modules are multi version. That's the base foundation It's a module that enables revisions on all content entities for your site revisions is very very important without revisions You can't really solve conflicts in an efficient way any type of replication relies on having revisions in place The replication module quite self-explanatory. It deals with the actual replication bit and piece The workspace module There's a lot to this module. We're not gonna dive into a lot detail here But it's essentially enabling containers of content. You could have multiple workspaces on your site You could have workspaces on external sites, etc We could spend a whole not a session talking about this module But not today Deploy module just adds a UI on top of this which Mikkel will demonstrate in the in the videos later and then finally and Perhaps the most important module for the audience and for the topic here today would be the relaxed module So the relaxed module. What does it do? So it extends the rest API in core With replication capabilities, it's a rest API geared towards moving content around replicating content between different applications We do this by borrowing concepts and the specification from a database called couch DB Which we're gonna talk a little bit more about later on So how does these modules fit together? I've tried to illustrate this in the best way I could So as we said starting in the middle on the left multi version is really the storage level Enabling revisions stack of revisions for all content entities being it node nodes block content menu links terms, etc All content is contained in workspaces And the replication module is responsible for sort of lifting and shifting Relaxed is the specification of the API we talk Over the wire or over HTTP and we can really talk to any compatible endpoint Again, it could be another workspace It could be a couch to be database or it could be other applications that Mikkel will touch on a little bit later All of these modules I should say not all but some of these modules Will be worked on and moved into core as part of the workflow initiative that Dries announced during Tuesday's keynote of which I'm the initiative coordinator So that's a very very exciting initiative. The focus for this initiative will be content authoring and workflow. However This initiative will enable a lot what we will be talking about here today. It will lay sort of the Underlying bits and pieces for doing replication and and for solving some of these problems So check out the link on Drupal.org if you want to read more about this initiative again, not everything That we talk about here is going to be moved into core bits and pieces will it's still very relevant So the next piece of the stack that we're going to touch on is a couch to be we talked a little bit about it So this is a database Which is very very native to HTTP every operation is done over HTTP and The API is designed for doing content replication very easily So Mikkel will demonstrate how how this How Drupal relaxed and couch to be fits together So the next piece of the stack is something called pouch to be So it's really cool that we have this database. That's online But what happens when we offline? So pouch to be is a real implementation of couch to be so PNC And it works in Java. It's a JavaScript implementation Which means that you can run it in your browser So pouch to be uses the browser's local storage as its storage and simply is able to take your entire couch to be database and Replicate it down to the browser So when as soon as the replication done, you basically don't no longer need an internet connection you have your entire Drupal sites content database available there or a More advanced use case. We did not get around to try out But it's theoretically possible at least is to have a selective sync. So to say, you know To this user whose name is Bob. I want to sync This subset of the database. So not only if Yeah, so maybe there's a lot of content that's not relevant to him And maybe there's also content that he doesn't have access to see but Surely, we're just sinking the whole lot and hoping there's nothing secret in there Well, there isn't since this is only for a demonstration And Of course, we chose to build this with the react. Yes react is Facebook's take on the next level web Way to build web applications in the browser And all the places as I'll get on to The main revolutionary thought about react is is to make things simple and Unidirectional There's a couple of other web frameworks that I've worked with personally I've worked with Angular and Emmerge as well and Both of those Tend to become very complicated applications. You have state going back and forth and two way bindings and and all sorts of creation is Causing a lot of headaches react. Yes, it's very is much simpler in the way. It's designed so you have components that can receive data and Generally components aren't allowed or Then recommended not to touch that data, but instead send messages out if they want that data changed by Those who own it and there's a couple of different methods methods where this By which sorry There's a couple of different approaches to to solving that problem One of them is Redux Redux is What I like to call state management divinity So The whole the main idea is this unidirectional flow All your data states is in the so-called store The store is just one big JavaScript objects like it could be rendered as JSON It has keys and sub keys. You can say my database connection is over here The current user is over there It's basically free for you to describe but the key is that you only have this one central data structure and It's only allowed it's not supposed to be edited outside of the reducers So the way it works is that any components? So the whole architecture around react. Yes, it's built around components So your whole website is composed of these So you have a username component Inside maybe the login component and you have all these components and they can ask to be given Either the whole state or a part of the state, but they're not allowed to modify it They're instead allowed to call actions so-called Redux actions Redux actions do something and then pass data to a so-called reducer and The reducer is the only place that the store is editable So in the reducer so you can say I logged in as user Bob and then over in the reducer you get the user and Bob as parameters and then you update the state with those Sorry with those values and Everyone who has who has access to the store will be notified. Hey the store changed so you can now see So if you have somewhere you displaying the username that component will now know that the username value changed But the whole idea is that you have these selectors on the store So if you have a component that does not display the username you can ask for that component not to get the username So each component will only get the data that's needed and that means you don't have to re-render the entire page You only have to re-render those components who asked to get the username and That gives you a lot of efficiency Because that's often the problem with AngularJS and Mages is that every time you touch something touch data somewhere the whole page gets re-rendered because no one really knows Where the data came from and and what components it might have changed So to give a short overview of our architecture So this is the simplest possible Example of of what we've built So you have a Duple 8 site with your content on it and you have the relax module installed and Then you have your web app loaded So pouch DB can talk directly to relaxed either with syncing or without so and Use that to to provide your content to the React application So the couch to be the example what which we will be demo demoing here today is a little bit more complex Here instead of relaxed in the middle we have couch DB So Drupal pushes its content to couch to be and from there it's available publicly to everyone who can talk to it and the browser part is the same and For our native app You see we have the same architecture The only thing that's changed is the bottom layer that we have now have a react native app rather than a browser based app So we have recorded a small demo Let's restart Where's the playback controls? Here we go Okay, so we have all three applications here Drupal a react native app in the middle No react web app in the middle and our react iOS app on the right So if we go and edit the title here, so let's use a more intellectually sounding word And if you are not familiar with neo-drexilin, it's part of the Star Trek series So this is not actual medicine So here we push our content to couch to be and never mind the debug info So let's see did we get through to couch to be So now we look in our relaxed database and we can see the document ID We use you IDs everywhere And if we look at the title you can see now it says received rather than gets and if we really reload the web app It will show the correct value and similarly if we reload the iOS application It will also say receive rather than gets So so all these apps are hooked up to the same data store So we learned a lot while doing this and allow me to Elaborate This combination of relaxed and pouch DB is totally awesome in my opinion Being able to have your entire websites database available in the browser Of course, there are some storage limits. So if you have a huge website So if you have like 10 years of newspaper content, this probably won't fit in the in the storage space You have a lot have available But for many smaller websites, this is perfectly feasible You can have up to 50 megabytes of storage on the on the phone on all phones and even more on some phones And of course this won't go far if you use images, but we imagine that the offline use case is You know, it's acceptable for the user to make it not necessarily have images But still have the whole website and all its content available CSS and all That's of course, you could theoretically stop images into couch DB But yeah, you might you might it might vary in any case. I think this is a huge Huge thing a Small downside or as we would say significant downside is there's there's no easy features everything you saw in the react native app or react web app Well, you didn't see that much, but It's basically been built from scratch there's no So we have all the data from Drupal, but everything all the links the menu the entire thing Has been custom built for this project This is so I don't think this is something you'd necessarily do for any website But it's something you might do for very important websites Of course when it comes to styling you want to be able to just ship something off to your front-end designer somewhere This is basically a completely separate styling process React. Yes has its own views on how you style so you still use CSS But you use it in slightly different ways and especially when it comes to react native. They have their own Variant it's kind of like CSS, but not quite and you use flexbox for layout everywhere and it's It's your I think your web developers might think it was a little bit weird another point which we Got wiser to as part of the process was there's actually very little Reusability between the web app and the react native iOS app So while the so the thing you can can easily share is logic But our application doesn't have very much logic But the whole display layer of how you lay out menus and things on the page Will be completely separate because react native uses native components and not HTML and CSS to lay out things on the page So even though the technology is very similar It's not similar enough to just be able to copy it over and and go home I would say on the upside though the two technologies react native and react What you know the DOM the web browser version are? Completely similar in the language use the structuring you use So your developers even easily matching in the same team of developers being able to do both But it is still more effort part of this process we tried to distribute as shared library Using the latest JavaScript syntax and and all that stuff and that's still an area where Where MPM and and the tools we have are still a bit primitive So I would just a word of caution It can be done and it is done and a lot of people do it But it's not super easy And I think that that goes for a lot of this stuff is this is all very beating edge technology, especially when it comes to react native There's been a lot of changes and a lot of frustrations So I don't have a lot of hair and I I wouldn't say it's necessarily only because of this project, but It certainly helped So all in all, thanks for your attention If anyone has questions, there's a microphone over there. So please step up to it I can barely hear you. Sorry So couch to be has its own replication protocol it's HTTP based as so I think they can probably elaborate more So I'm just filling the errand to get back up here So The the couch to be defines a generic protocol for for how you should move content over HTTP and There are many projects that reuse this protocol. It's very well documented and there is Very little to no restrictions on the actual Jason payload. So the Jason payload itself. You're very flexible It's very well defined on the HTTP level though how they get requests how they put requests should work and and what the various response code should be and and then there's a whole protocol of What steps you need to take when you move content from one place to another? So it's a very well documented protocol There's lots of SDKs for many languages, which is one of the big benefits To have a standardized API as opposed to Just Drupal custom because there's not really many libraries to help you interact with the API But since we're reusing a HTTP spec That makes it a lot easier Hi, I kind of have just two questions into one So one is the first about a porch DB How long does this kind of sit in the browser? Is it like infinitely and then the second question is regarding security? How secure is this data if you're having like your entire sites database being stored in the browser? So as far as data retention goes And that's very much up to the browser. So it's like anything else that is cached in your browser It could theoretically be wiped at any time And although it hopefully wouldn't won't be And the second question, sorry that was a Security. Yes, of course, there's some security concerns When you have your the entire content in the users browser It shouldn't be too dangerous, but you of course need to be Take care that you don't publish anything that you don't want to publish So I think any realistic use implementation of this would probably have a Filtering in the data being pushed out. I think did you do a proof-of-concept of that or? Not yet, but we are like 99.9 percent Sure that this is the way to go so you'd simply have a filler saying don't push Unpublished content to couch to be don't push content with the tech X Don't push all the user passwords And and this problem isn't necessarily unique to this particular setup There's a whole ecosystem of tools around pouch to be and couch to be That tries to solve this yet another reason why it's a good idea to tap into an existing ecosystem Because there's a lot of tools around security Authentication that could be reused here. So that's quite exciting Next question You mentioned Redux and so pouch DB is that Are your react components pulling data directly from pouch DB or is that all routed through Redux? It's routed through Redux. So we have Redux actions saying get me all the documents or get me a document by ID You could imagine any particular So give me a list of documents with this tag name or something like that you could define actions So that's that's how we've abstracted. So we have this meta layer between Redux and pouch to be and by the way, this this code is open source the link There is is to the git repo that houses is it's it all and of course feel free to ping me with questions If you want to try it out. It's still we're still working on Getting it nicely nicely documented and everything, but yeah Thanks. I have a question about the performance. You said on phones. You can go up to like 50 megabytes So let's say your database is like 30 or 40 megabytes. Does that mean? While that's loading on my phone. I can't use it or is there server rendering and what about using up all that bandwidth? Bandwidth is so as from From a performance standpoint. I haven't felt any effect from having all this content. It's basically it stored in so Because it's stored locally it loads much much much faster than anything, you know Just loading a single megabyte of the web versus loading 50 megabytes from local cash is much much faster with the local cash Of course the bandwidth concern is there so for the real-world use case we imagine that the The local storing would be optional So there would be a place in settings. You can go and say please get me a local copy of this website Or we might do it automatically for the most important things Or you can basically do it however you want for For convenience the current implementation just thinks everything is as soon as the app starts But it doesn't have to be that way. I've got one thing to add there and I'm I'm going a little bit outside of my experience here But the the latest version of pouch to be also works within service workers Which means that a lot of the operations that pouch to be can do Can be done very efficiently in the background as well So you don't necessarily have to block The whole rendering of a page while downloading it could be done very efficiently That's not implemented in our POC at the moment, but that's a pouch to be certainly supports that Next question Sorry Just as far as the storage goes the 50 megabyte limit is the worst case so that is Safari on iOS that limits you to 50 megabytes. For example most firefox has no limits And if you're doing react native, there's also no limit and different browsers have bigger limits Yes, but So in this example couch to be supports cross origin requests So so actually in this very demo we have a cross origin request going through Drupal has a course module which would Allow us to do the same when could talking directly to relaxed To the relaxed API rather than couch to be so I don't think this is Necessarily a problem. It's of course something that you need to configure for your sites So in this example we use HTTP standard authentication But of course the danger there is that Since this is a JavaScript application that runs in the browser any password you have in there will be accessible to clever hackers And so in this so for our use case we we're using something called cloud ends Which allows you to create read-only users on couch to be and we're using those for that So you can you can replicate the content down to your browser, but if you try to push updates you will be denied and The the API specification supports a base basic authentication a cookie authentication and then OAuth Of which we have implemented it first to OAuth is still is still on the road map Some challenges there, but there are many good libraries to use Not implemented yet, but yeah, that's that's certainly a possibility More questions Hi, sorry, I come more from backbone than react But my question was just your thoughts on a deeper integration between Drupal and some other It is something that we had as a concern with this project so Currently in our react native application. We have implemented specific templates for each node type So we don't have a generic way to render a node because we don't know what fields are on a node So purely relaxed support synchronizing all content entities in Drupal But not config entities and config entities is where the model for things are stored It's certainly something that we have considered, but it's not something we could do right now, but it's interesting So there's there's a couple of different People and initiatives working on everything decoupled Sebastian or foo be in the room here Sebastian you want to raise your hand He's working on another react implementation that does support server-side rendering Where you could take more advantage of that That that side of the coin does not Support offline storage, so there's like there's a lot different areas that touch different things and perhaps they could be efforts to sort of Magic merge of all of them. I'm not sure but there's you know, there's different initiatives going on What's what's the URL to your github is it foo be so github.com foo be slash Drupal dash decoupled Dash app That's another react implementation. Talk to Sebastian there in the blue shirt if you And of course if you have questions feel free to get in touch with the the media or dig on Twitter or Go hang out on Drupal entity on IRC And of course visit visit our this delightful marketing website to deploy the door Anything else other questions? Yes so Yeah, so so the question was if for your authenticated users, could you deploy to could you give them only the part of the website? That's relevant to the to them. And yes, that's certainly a possibility So the relaxed Module uses Drupal's own access control APIs can break me if I'm wrong So you could use Drupal's API to limit what people can see via relaxed and Via couch to be you can write scripts on your couch to be I think that's JavaScript based And those scripts can be used to determine what content gets synced to watch users. So that's definitely possible It's doesn't happen. Yes, that that certainly Would probably be possible So you would probably in that use use case you would need some comp some more complex Authentication logic for our authorization logic for the replication. So saying you're allowed to replicate comments But not new notes But yeah As long as you've write the logic of getting the updates into the pouch to be storage Pouch to be and relax will take care of the rest So there's no there's no we don't have any components or logics to build the forms and To set that all up but as long as you get it into the pouch to be storage, which it's you know You can you can do it quite trivially You could also make it very complex for yourself if you want to but Pouch to be and relax will take care of that for you. We have a full test coverage for that scenario However, the components for actually writing into The storage doesn't exist but the the other complicated bit of Transporting and syncing when you come online that's old One more question. Yeah, so the question was how does the protocol handle conflicts? It handles it very well. It's it's one of the few databases out there that actually can do By directional replication very well So without going into too much detail It tracks conflicts everywhere and It's always going to be up to the implementation to solve those conflicts because As the same would get there's only so much that you can automatically merge you are going to end up having to interfere yourself and fix Conflicts, but the API and the specification Marks and tracks conflicts everywhere and they'll let you know when you introduce a conflict as well More questions. Lovely. Is the offline storage encrypted? No There are however Additions so yet again Reusing pouch to be there's a whole ecosystem of tools here I believe there are plugins and systems for encrypting the pouch to be storage. So Things you get for free when you tap into an existing ecosystem like this Okay, please rate our session It was lovely to talk to you all and Thank you