 My name is Nick. I'm a senior developer at the Ontario Digital Service. My name is Adam. I'm also a senior developer at the Ontario Digital Service. We're here to talk about decoupling people and our experience with you at Ontario.ca. Feel free to raise your hands at any point in the talk. We'll have an answer to questions because we'll be talking about a lot of stuff and maybe glossing over things that you want to know more about. Okay, let's see. So decoupling people at Ontario.ca and losing our heads and getting me. So, I'm in a big talk here, so let's get this out of the way. There's a big, big image which has something to do with what we're talking about. There's a spider web, so you can think of spiders crawling the internet. I don't know what else, but it's there. The dew drops in this case might represent decoupling people. Also, for our Parliament Hill, there's Parliament Hill. Boom. Let's move on. So, little about our website at Ontario.ca. We're the flagship website for the Ontario government. Our goal is to make content and services easier for people to find. We do driver's license, sticker, or try to apply for OSAP. Probably just at Ontario.ca. We're currently in the process of migrating all 30 industries to Ontario.ca. Traditionally, they've built their own websites with their own web teams and we're trying to make it easier by consolidating it all to one. And currently, we've migrated 15 industries. We have six more to go on their progress. Cool. A few more statistics about Ontario.ca. We are currently serving 48,000 headless pieces of content with Angular. At the same time, we have 7,000 legacy Drupal nodes that we're making available to the public. We're about to hit our 100,000th Drupal node, so that's kind of a big step for us. It goes to show you how much content goes through Ontario.ca, how much it was curated. We go through a whole lot of content. We are serving 1 million sessions per week. And currently, we have saved the people of Ontario $4 million by partially moving through Drupal from vendor-based solutions and also via closing-out ministry websites and housingeverythingontario.ca. Also, we are growing exponentially. We create 1,000 curated nodes a month. So those are pages, books, FAQs, all sorts of different stuff that lives on Ontario.ca. We train our content like crazy. Part of it is because we are bringing over all the ministries from Ontario.ca. And part of it is because of AODA compliance. So government websites love putting up PDFs. PDFs are not very accessible. So we're turning them all into HTML. But yeah, so I just found out that stat. I found it kind of amazing. So a little bit of history on Ontario.ca. Before coming to Drupal, we were actually at an Oracle shop, in the R3. That contract came to an end. It was a very long contract. We started doing it as our vendors. And there was actually a small group of us that was interested in open source. And Drupal was the best open source on the management system. We had to be prototyped it. And at that time, Drupal 7 just came out, so we didn't Drupal 6. It was a success. So we got the green light to produce the real deal. And 2012, we launched Ontario.ca in Drupal 7. We started to run into some performance issues on the front end. And part of that was our own fault. We quoted the theme correctly, things like that. And we started looking at other options. And running headless seemed like a very promising avenue to take. So 2013, we started chopping off the front end slowly. We started serving content type by content type. Yeah, so why did we... Sorry, let's just back up and say, what is headless Drupal? I think we all know, but there's a slide here for instance. Typically, your front end is housed in Drupal. So Drupal is in charge of housing all your content and serving it to the public via the front end. Decoupled Drupal or headless Drupal looks something like this. So there's a front end that sits on top of an API that talks to Drupal. The front end is usually a JavaScript-based. In our case, we use the new JS. And there's a RESTful API. And then that talks to Drupal. There's also a long delay over there. So why did we want it to couple? Well, we wanted to improve scaling and security by separating the contribution and the user concerns. So our users visit the Angular side, the contribution team uses the Drupal side. We wanted to improve page load times. So we wanted to reduce the server load and the number of requests. We wanted to move rendering to the client side. And we wanted more efficient templating. Adam hinted at this before. We sort of jumped into Drupal at Drupal 7 pretty early. And if you're on any big projects like this, you might have developers who don't follow best practices. Big projects, that code just kind of remains there in spaghetti code. So we had pretty bad templates by some point in 2013, full of SQL queries, variables named X, weird things which basically kept our UX team from wanting to use our templates and also made our templates very inefficient. We wanted to embrace new technologies. So everyone started talking about JavaScript-based frameworks. We do want to be left behind. And again, we wanted to give the UX team better control and give you a front end that they're not afraid of. And we went with foundation actually, more on that later. So our implementation of this Drupal server was like this. We have a traditional LAN stack. I guess why just how should you use MMS? And you have a mean stack, which serves as their API. We used the last search engine, which is JavaScript-based. It works very well with the mean stack, the JavaScript. And we used foundation for the front end, and sort of layouts, mainly because it's very accessible and responsive. So a little bit of a detail, but under the hood, I'm sorry, SCA. So if we look up at the top, we have the normal sort of editor flow where editor-centric content is in the old ad form. That's sort of in the MySQL database, which is actually there. The only thing that's different is that a editor saves a node for an entity. It goes through our shadow model, which is our custom in-house model that we developed, which basically takes a Drupal entity converted to JSON and sends it to a RESTful API. That in turn, it serves through Angular. It serves connected as API. It does the client-side error, and it has been immediately jumped onto the mean stack to sort of kept both the data and both the waters as a transition. It still serves the latest Drupal entity changes for the ones we haven't migrated yet. So a little bit of our decoupled strategy. So as Adam hinted at earlier, we leveraged the mean shadow module. We customized it, and then eventually Adam ended up writing his own. The partial reason for that was basically PHP memory leaks and a few other things. It was very inefficient, especially for the amount of content that we generated. We implemented a node-based REST API. Again, using MongoDB. And we transitioned to Angular and foundation content type by content type and following our information architecture. So our tags on it. And the way we did this is we did it via Nginx and PageAlias. So with every new piece of content that we moved to Angular, we gave it a new PageAlias. And then there's an Nginx rule to direct PageAlias with this pattern to Angular and with a different pattern to Drupal. Cool. So a little bit about our successes. It works. That's the big one. PageAlias times are down to under a second. In fact, it was such a big turnaround for Drupal that we forgot to turn on caching. We didn't even notice. It was so good. Our servers and our DevOps team are much happier. They're not having to distract you to get to the end. Every day your memory leaks and you can see if you're being caged at 100%. Yep. Our UX team enjoys full control of our front-end that is modern, responsive, and accessible. Hey, how about that? And we have better security because our front-end systems are different from our back-end system and it can do things like write simple statements and themes or anything like that anyway. A nice fall out of this that we didn't really think about going this way was we have faster and easier development. We have Angular Sandboxes working. Our database at this point I think is like 10 gigabytes or something like that. So that's way too big for WAMP or MAMP or ZAMP or whatever to work efficiently. So a nice fall out of this with Angular front-end is you can run Node.js locally and you connect to our API and you can develop locally. Lessons learned. There were a lot. Angular talent is very hard to find. Yeah. Experience that. The current version of Angular has only been around for two, three years so the most experience you find out in the developer is just that. Also, sorry, because a lot of people want Angular developers. That's money. Angular is new and more of its API documentation is incomplete or incorrect compared to Drupal. The API that Drupal had already created gives you a lot of information. I've come across Angular documentation that is out of date that says something will not work and it does work or something will. The opposite around. It will work and it won't work. Which version of Angular are you on? We are currently on 1.5 but we're going to be moving to something else pretty soon. Two or something else completely? That's not officially decided yet. Or review or react. We have to make some of these decisions depending on what kind of developers we're hiring. We may have done this a bit too early. We jumped on this in 2013 so I've got climate just Angular version 1 and Drupal 7. We did this today with Drupal 8 being an API-first content management system and a newer version of Angular you're going to be completely re-ringing. Things would be a lot easier and architecture would be a lot more simplistic. An interesting follow-up that wasn't thought of when we went to this is that we lost certain features that weren't within a big like in-page editing. That edit tab that appears on the node what a lovely thing. It's gone if you build front-end architecture on top of it. You lose preview as well. We had to create more complicated contributor workflows to allow new modules to allow people to preview content and to share content. An interesting thing to follow up there. It's difficult to manage if you keep in stacks as a transition. Another interesting thing that came over this is we realized that the data layer or data model became pretty abstracted. If you're working in Drupal and you're developing in Drupal it's pretty obvious that the field might do or how a field might functionality of a field that is intended for a field or how it works in another field. Once that goes through an API and once that ends up on an Angular developer's desk he may know nothing about this. And so then he's back at square one and he's building something and you're like, why are you building it this way? And he's like, I don't know. Interesting thing that came over there. In the experience live repetition it's just like Drupal and the mean stack and sometimes I get an Angular. Another interesting thing is that we have one form on interior.ca currently it is a contact form and to get that form to work properly with our Drupal stack is a little complicated if your site has a lot of forms or if you require information to go both ways I would seriously have concerns about dropping off the front end. So, should you decouple? Yes. So, Chris posted a blog a trailer that I built you decide if you want to go ahead with this or not basically you have to ask yourself are you going to give up everything here in this big box and if you're not going to give it up are you going to prepare yourself and things are still going and perhaps doing a couple of like a better admin Yeah, so that's stuff like previewing, editing all these things. There actually was an article posted by Malabar two days ago which talks about a lot of this stuff like what you'll lose as you decouple and how are you ready to build that yourself? We have a few next steps for how to not say we want to share with you We're going to upgrade to Drupal 8 Drupal 8 puts the API first or you can't create the API first a lot easier Drupal 8 is going to be awesome We're going to upgrade to Drupal 8 for a shadow module Again, we're like with the services API you can expose endpoints and create JSON output very easily We may not actually need a shadow module that happens for a very long time to live I'm the biggest advocate of getting it We're going to upgrade the frontend to Angular for maybe view, maybe react We're going to possibly leverage the rest of our distribution I'm not sure if you guys know about this but it's a particular Git repo that has a whole bunch of modules that are meant for decoupled Drupal already bundled into it and has the theming layer chopped off so it seems like a really great place to start with headless Drupal We're going to leverage Drupal VM a whole lot more I'm not sure if I'm going to be able to play with this but it's really also, again, a great way to not have to bother with LAMP is what I'm going to bring in MAMP, all those things and microservices We're going to expand on microservices quite significantly You want to say something about this? Yeah, alright Thank you, any questions? Oh, there's one right there Have you guys handled SEO since there was a client-side renderer like how did Google index your pages? Okay, so this is really awesome, this is something that we lost again without thinking about it So we used the metatags module in Drupal So that worked to create great metatags for Drupal in Drupal We had to rewrite this We had to add this to the shadow module so all our metatags got shadowed So we basically rebuilt metatags in Angular like we had to build a whole service around it Yeah, so if you look at Toyota SAA in the store in the console you'll see these things called Angular metatags they're mapped to each other, it was a whole thing On top of that because it was a JavaScript framework we had to write special rules for the bots to be redirected to our contrib interface it's a whole thing It's an awesome idea to be thinking about it Along the same lines with the JavaScript and having to be accessible to the government Did you have to use server-side renderer for any clients, or was there open clients that you could JavaScript on? I don't have any complaints for JavaScript only The screen readers are pretty smart right now No problem whatsoever with JavaScript as long as it's not doing Yeah I didn't attend the talk earlier about JavaScript, don't blame JavaScript for your accessibility roles, but maybe they talked about this a little bit We have a pretty great UX team and so they did a really great job of making sure that we're pretty accessible From another talk today, I did run the audit on one of our pages and we had a 90% score on accessibility so I think that's pretty good So that's interesting because the good SEO sometimes is do server-side renderings that parts your page and the answer to accessibility sometimes is produced on the server-side so nobody asks you and if they'll request one you'll say my client device can't support JavaScript or I refuse to support JavaScript can you get that conversion? No, that hasn't come across actually I'm a developer, maybe someone sent it to UX team You're short on that That proves that the presentation from this morning that JavaScript is not a barrier It's not a barrier It's only an office line Yeah, I know we spend a lot of time again and this might as well be an AJAX and again if you refresh certain pages you have to put the focus on that page so there's special tags to do that it's like what's the main part of your page That's challenging sometimes but yeah, that's it Would it be a good idea to design the which pages gets to the whole directory and which pages are using Angular because maybe some pages are complicated like the forums and the others maybe you want to use Angular because there's a lot of tracking on those pages So our plan currently is just to migrate everything to Angular the reason why we have someone still in Drupal is we just haven't gotten to migrating those yet But to your point some of the more complicated pages we haven't had a chance to migrate yet for example we're having a few issues with Google Maps because it's asynchronous and sometimes it'll load faster and with content I'd say some of it's been there in 2012 when we started with Drupal 7 So the funny thing is that those contents depending on when they were created use a different version of Google Maps that require different parameters to be passed and it's pretty complicated so those actually are Google Maps a lot of our Google Maps pages are served by Drupal for that reason and the only other piece of content that we make sure is is served by Drupal's web forms again because there's the 2A binding there necessary web forms are great in Drupal you want to create a radio button you just say radio button and it draws it are you ready to redo that for every kind of web form element you have in Angular we actually had a developer get pretty far down down that rabbit hole but then he left so maybe that was the thing that the developer can't as bad What does your server stack look like is another single threaded, right? No, yeah Sorry, what is the question? Are you running multiple load instances or how are you more balancing I believe we have two EC2 instances running on it Okay They have a scale as well so these let me get a lot of traffic I was just wondering about your choice of what will be versus the Docker They're actually using both at the moment We officially use Docker as our sandbox environment so we have a Docker stack that has mean, elastic search Drupal, everything and they're all ready to go and Drupal VM is something that we're experimenting with for Yeah, we found that with the Docker stack we're relying a lot on our DevOps environment They're the ones that maintain our sandboxes and you know there's a huge database that goes with it, there's all these stacks that go with it so we wanted to give DevOps a bit of a break especially for starting up projects and Drupal VM is just a really great way like it's command line and figure just run it and you have a Drupal environment where you can hit the ground running So why did you pick Angular and not some of the other JavaScript frameworks like Ember? So at the time of its 2013, so a lot of the frameworks weren't around yet linear was Angular and React and it came down mainly due to the licensing models with the other front end frameworks and Angular had a better open source licensing model Yeah, also supported by Google Yeah, so we were just betting on Google I think a little bit But we weren't entirely sure we weren't super committed to we just thought okay this is there's a lot of JavaScript frameworks coming up and this one most likely is taking off And even now there's a lot of comparisons of the frameworks like there was one that said 90% of View users are really happy using View and 50% of Angular users are happy using Angular That has some weight to it but Angular is still I think more robust in terms of its modularity than View is but we have to evolve it all that Like this is our 2.0 user monitoring session It's going to be crazy good Any more questions? Okay, so before we leave you we just want to introduce you to the Ontario Digital Service The ODS is a new government service that will work with ministries, agencies and external partners to develop a new government action plan That's the official verbiage As part of the ODS we have a new Chief Digital Officer in Ontario Her name is Hillary Hartley You may have heard her on the radio or seen her on TV There's a big publicity campaign behind that She is going to be our ship number to help us get stuff done at Ontario and not at the bureaucracy given in the way more than a sec The ODS is going to be modeled after GOV.UK and ATF which is literally we're going to be modeled after because we are working with people from GOV.UK and ATF Hillary Hartley is from ATF If you remember the keynote this morning from the federal government they mentioned that they're desperately trying to catch up on the digital divide and they're desperate to get there We are there at this point The ODS I for example I'm actually working on an alpha project for the Ontario government Partially that's because of the people I'm working with had 2 million dollars of funding pulled away from them but I think they're also coming around for this idea of alphas and data is not delivering a huge monolithic vendor driven solution at the end Adam is working at JAL he's building microservices so he played installations little pieces coming together JAL open source Ontario government is already there and so on that note So we're looking for people If you like anything you see in this presentation either technologies or processes or hiring number of benefits So if you're interested get in touch with either me or myself also Ontario government And if you like nerf gun wars that's great, if you don't that's also great but yeah go to Ontario.ca slash difficult jobs and sign up for job alerts Current room around the office is where 30 positions short should be but that's partially because the jobs aren't being posted come join us it's really fun Thanks