 Hello everybody and welcome. Can you hear me? Okay. And can you see the screen? Beautiful. I'm seeing some acknowledgment. So yeah, okay. Good afternoon. Welcome to DrupalGov 2020. My name is Toby Bellwood. I'm the Product Lead at AMAZI.io and I'm going to talk to you very quickly. For those that know me, talking in 20 minutes is going to be a struggle. About Lagoon Insights and these are some features that we've put together as part of the Lagoon platform to help you manage large and smaller site portfolios. So for those that don't know, a quick overview of AMAZI.io. So we offer enterprise-grade, secure, scalable, open source container hosting. We work with a lot of large organizations all around the world. Obviously you've got CMS and DPC here in Australia, but a number of pharmaceutical, finance, media, communications, and financial businesses all around the world. And we provide flexible container-based hosting along with a team that's located all across the planet. So I won't say 24-7. We do provide 24-7 support, but there's always someone working somewhere. And we've got our, what underpins our platform is a tool called Lagoon. And Lagoon is a developer-friendly, cloud-native application delivery platform for Kubernetes. So we've built this platform to bring your web applications to production as easily as possible. We've really looked at it from a web developer point of view. So trying to minimize the amount of code, the amount of technology that developers need to learn in order to make their sites go live. So we'll provide a lot of the configuration, a lot of the settings, etc. So Lagoon Insights came about over the last sort of year or so. We've obviously got CMS as a large customer, hundreds of sites. We've got a few other customers who are running those sort of hundreds of sites. We've got a couple of other customers who have got sort of dozens of sites and a few that are managing sites on behalf of other people. And what we've really found with a number of the other providers is that it's quite hard to get an overview of where your whole, you know, sort of where the state is at, as such, to find out what are the individual details on sites. Certainly, when I was at GovCMS, I had a encyclopedic knowledge of the 80 sites that we managed. Now it's at 300 and something plus. I think I'd be struggling. But we're all about building tools to make that easier for people. So what I'll do today is I'll go through what Lagoon Insights are, how we implement them in Lagoon, where we can get the data for these insights, how you can utilize these insights. I've said live demo. Technically, I'm clicking through some screens, but it'll look just like a live demo. And we'll talk about how we're going to make changes and improvements to insights over the coming months. So we've got these three separate, but sort of interrelated tools. One of them is called metadata, the metadata subsystem. And at this point, the metadata subsystem was developed by the team at Salsa Digital, our partners in delivering GovCMS. And this is, it's a really, really cool way of managing metadata and annotations at the project layer. So for every project in a Lagoon instance, you can add this annotation, you can add this metadata so that you can help find, sort and the source, the individual project. So for GovCMS, they can mark sites as being SASS or PASS, Drupal 7 or Drupal 8. Some of our other customers have got different maintenance obligations. They can add those annotations to the individual projects. And then they can use external tools, AWX or Ansible. They can use to show me, okay, which sites are under proactive maintenance, which are reactive, which are SASS, which are PASS. You can add almost anything to those, to that metadata. Facts and problems. These two tools were built and developed by our partners at AMAZ Labs, sort of the other, one of the other AMAZ group organizations. And these are, they're not quite as free as the metadata subsystem. The fact subsystem is a repository of information about what's running in your particular environments. And as we go through the example, you'll see what that means. But automatically working out Drupal module versions, Drupal versions, PHP versions, a number of other things and problems is a dynamic repository of issues with those environments that can be discovered automatically. And we'll talk a little bit about where we get the sources of that problem data from. So we've built the problems as well, we've built into our notification system. So if you've got a fairly serious problem, critical problems, or you can set thresholds, you can get notifications when one of these things occurs. So how we implement it in Lagoon. Lagoon is obviously based off of GraphQL API. That's fantastic and easy for those of us to know how to use GraphQL. We've got a UI interface for facts and problems. I'll show you through that. We connect through webhooks to get the source for some of our problems. We've got a CLI tool that we can use to add and populate some of those facts and metadata. And we've not done anything in the problem space yet, but the CLI will come along. So that's sort of how you access those individual tools. The problem system has got two real sources of data at the moment. The first one is from Harbour. Harbour is a Docker registry. Docker registries obviously have hit the news in the last couple of weeks because of their rate limiting decisions. Each Lagoon instance is capable of running an inbuilt register for all those Docker images. So every time you build your own site, the site will push up the image to a Docker registry. In our case, it's Harbour. And our Harbour is installed along with a scanner called Trivi. Both Harbour and Trivi are open source. And Trivi will perform a full vulnerability scan on that image. And by a full scan, I mean it's scanning the operating system layer. So the actual Linux installed in the container, it'll scan PHP, it'll scan Drupal. And if you've got themes, it'll scan the node packages inside your theme. And it'll collect all of this data, it'll run it against all of their databases. And if there's issues with it, if there's CVEs or problems, it will pass those back to Lagoon. Lagoon will add those to the problems database. And then it'll add those to the individual projects. And depending how you've got it set up, you'll be able to view each project's own vulnerabilities from Harbour. One of the other integrations is a tool called Drutini. Drutini is a remote scanning tool for Drupal sites. It can access the Drupal site and it can run through a series of policies and profiles to check the audit, check a site against a predefined set of audit conditions. So GovCMS uses this, a couple of other customers use it, and they say what modules should be enabled in Prod, what shouldn't be enabled in Prod, what settings should be set, what shouldn't be set. Drutini will run that check or that policy or that profile against an individual site. And then Drutini is being configured to send that payload, again through to Lagoon. Lagoon will surface that through the problems database. And if there's a severity threshold met, consent and notification to a user. So if someone has enabled the stats module or the DB log module in Prod, the person who's responsible for that site can find out straight away and so can go and action a kind of remediation for that. So obviously talked about problems, talked about facts. We use that trivy scan via the webhook. Drutini comes via a webhook. Facts, we've got two ways of populating those facts. In these facts, it's important that these facts broadly should be computed. So we should be running a tool that can calculate that fact. So the fact isn't as manipulatable as the metadata. We've got AWX and Ansible tasks currently for calculating those facts. But one of the things we're working on is a post-rollout step. So every time you deploy your site, your site goes through and does this quick check and says, what version drew, pulled am I running? What version PHP? Is there anything else I need to calculate and need to put back to the project? And the metadata is added either via GraphQL or CLI. So because the metadata is more flexible, it doesn't have the same kind of calculable source that the facts do. So I'll run through my screen cap so quickly it looks like I'm doing a live demo. So for those that use Lagoon currently, this is the UI. This is the main screen. This is what you'll see day to day. So what I'll go through is I'll go through a couple of branches. Our main site, if I click here, it looks like I've gone to the main page. Our main site was deployed on the 13th of October. We can see it's fairly standard, it's set up. And I can go through and I can have a look at these problems and these facts for this site. So under my problems, I see I've got this list of problems with my site. I've got 42 problems. I think I joked on Twitter that I don't have 99 problems, but it was really tempting to try and load up my site with lots of bad modules and bad decisions to get myself to 99 problems just so I could crack a funny joke on Twitter, but that seemed like a lot of effort. So we've got these problems. We've got these no criticals, thankfully, but we've got 17 high, 7 mediums and 3 lows. So if we have a bit of a look at this list, we can sort of say, well, one of these problems here, we've got this CVE 2020, 1366. And that comes from Drupal Core. So Drupal Core has a security advisory on this CVE. So at this point, Harba's gone in, passed the image to Trivi, Trivi scanned it and said, hey, you're running Drupal 901. Did you know that there's a security advisory for Core out for that one? And you really should be upgrading to version 906 or above. So that's an actionable piece of insight we can get out of this list. So we can see straight away 901, the severity mapping between Trivi, the data source for Trivi, Trivi and Lagoon doesn't come through quite as well for Drupal as we'd like it to. And that's why 3 plus 7 plus 17 is 27, 42 is 15. So there's 15 problems in here that have an unknown severity. Now, we know that this one is moderately critical because we've seen the Drupal security advisory. A couple of other things that this scan has also picked up. If I go back a screen, I can see it's picked up lots of things to do with Drupal. It's picked up some other things. So if I do a quick search, I can see that there's a curl of vulnerability here. So there's a password prepended problem that's actually got a proper severity because it links to an external CVE. So I can see there's a problem with that curl image. So I'm going to have a look and see what I can do to fix this Drupal and this curl issue. I can have a quick look and see my facts, see what my last computed state of my site was. So I can see, indeed, I've got a whole sway of the Drupal modules here that are running at 901. Now, these are all the built-in core modules. If I do a quick search on version, I can see that I've also got a check there that will do a Drush status to tell me what version of Drupal I'm running and, yes, 901. I'm running 1030 of Drush. And I'm running 7410 of PHP. So I know there's a new PHP version out as well. The old version wasn't insecure. So the problem that didn't tell me that I had an insecure version, but that's something that I know. Ideally, a future way would be to link that fact to a problem to say, hey, there's a PHP update available. You might want to think about that. I can also see non-core modules. Now, this is a really basic site. There's only one contrib module in there, and that's our Lagoon logs module. But again, I can see that that's running at 8x1.1. So I can collect these facts for all my environments and bring them all into one place to manage centrally. So I've got this piece of information. I've been told, Drupal example 2, it's not good enough. It's got an old version of Drupal. There's vulnerabilities. You need to go and fix this. So I'll make sure it all works. So I'll go to my test environment first. So I've made a heap of changes in my test branch. I've pushed my test branch. You can see it was deployed on the 4th of November. So what I can do now is go through my test branch and see, well, first of all, did that upgrade work. So we can see in the facts that my Drupal modules are now at 9.07. We can see that my Drupal version 9.07. We can see my PHP version is now 7.4.11. So upgrading my site has also pulled in the latest version of PHP from our upstream images. So we'll get the benefits of having a later version of PHP. And there may be some other benefits to pulling in a fresher version of those images. Okay. So I'm comfortable that I've got that new version of Drupal. My updates worked. The deployments worked. Let's check those problems. And hopefully, we'll see no Drupal problems in there. We can certainly see that the number of problems has dropped from 42 down to 16. So not only have we addressed all those Drupal problems, we've picked up a few other ones and solved a few other ones as well. So we can see the ones that left Freetype, Audegaruma, URL, Regex. So these are all things that have come from the base Linux. So these modules have either been installed as part of Alpine or they've been installed as part of the modules that we install for to run PHP in our individual base images. I remember I had a curl issue. So if I go back to my curl issue, I could see if that... Yep. Indeed, the latest version of that image has now fixed my curl issue. So no longer do I have that curl vulnerability to worry about. So my image update has really pushed through a whole heap of fixes. If I know exactly what I'm trying to target and I've got control over my images, I can also address those individual vulnerabilities. So if we'd installed a particular extension or a module ourselves, we could install a specific version just to mix that individual critical issue. So looking at these insights, obviously the problems, we've got this notification, the UI. Me as a non-technical user, as a project manager, or a junior developer, or someone that just likes poking around in UIs, can go through and see exactly what the site looks like. I don't need to SSH and I don't need to log into a site. I don't need someone to generate a report for me. Those of us that are developers are going to find this horrific that someone can actually see that much of your site information exposed there, but trust me, it's a good thing. We're working on an aggregated view. So if you've got access to 20 sites, being able to view an aggregated stream of those problems is going to be really handy. You can see where your Drupal vulnerability is, which of your sites have that curl problem, which of them could potentially be fixed just by redeploying and pulling in a fresh image. Facts give us this mechanism for querying the state. So again, we can look at those facts. We can look and see who's running Drupal 906, who's running version 1.2 of the C-Tools module, who's running Drash8 still. This can change deployment by deployment. And generally, the teams that are looking after the sites aren't necessarily the teams that are deploying the sites. So individual development teams will be continuously deploying. So the team responsible for site reliability engineering may not have day-to-day view of what goes on in the individual sites, but this gives you a way of being able to see across the whole breadth of your holding exactly where everything is at. And the metadata. The metadata is really handy for being able to drive some of that reporting, building those inventories and doing that kind of audit across multiple sites. So I've given a bit of an oversight into the vulnerability and the audit side of things. We're really keen to start working on sort of profiling and analytics as a source of problems and facts, being able to identify slow loading sites, being able to identify sites that are attracting additional traffic. And also from an observability point of view, being able to put some metrics behind site performance. So if we've got a site that's got good uptime, that's got good deployments, it's generally healthy. Being able to put those kind of measures against projects. So at a glance, you can see what are your healthy sites, what are your unhealthy sites. As a team managing 100 properties, you've got to work out where to target your energy. We're going to work a little bit on permission levels, how to manage those problems from a top layer. From our point of view as AMAZIO, we know that if there's a problem in one of our base images, we want to be able to rectify it and we want to be able to let everyone else know that we're in the process of rectifying that. So we can acknowledge some things at the platform level and say, the next release will fix this up for you. So that was a really quick overview. I think I've done pretty well to get there in 20 minutes. If you want some more information, come visit our booth. Most of us in the AMAZIO booth all day, more than happy to do demos. Find us on social, we're in all the usual places, send us an email, hello at AMAZIO. But also check out our blog, we've got a blog on our website and we're trying to really post more information that gives more insight into these things for all of you. So I think my clock says five seconds, so that must be time for me to wrap up. Thanks all for coming. I've just been sent a message saying I can keep going over time, the session won't end.