 All right, is that working? Yes, but it's nice. Okay, hello everyone. Welcome to my talk. Today we'll be covering Drupal Defense in Depth, which will be a security framework for hosting Drupal at scale. And this is something we've been practicing internally at Salsa on our own hosting products, Salsa hosting. I just wanted to share our approach with you all. So on the agenda for today, we'll have some introductions. We'll go through the five key phases outlined in the NIST cybersecurity framework and just wrap everything up. So as for introductions, who's Ming? Who am I? I'm a DevOps engineer with Salsa Digital. I've been working with government slash enterprise Drupal deployments since about 2020, mostly with Lagoon. I've worked in projects such as the Victorian government's single digital presence. I've been doing some GovCMS work here and there. And I also work on Salsa Drupal hosting product, Salsa hosting. Oh, something in chat. That's, Lee has hosted a Drupal link. So I'll go ahead. So I realize I forgot to advance the slides. So before we dive into it, we need to cover a couple of concepts. So our first concept is something called defense and depth. So what's defense and depth? It's a layered approach to security and it's traditionally composed of physical, technical, administrative controls. However, with the move to cloud computing, cloud computing platforms, the physical security layers often taken care of by the cloud provider. And it's not really something that us and DevOps have to be overly concerned about when managing our Drupal sites. So we'll really just be covering the technical administrative controls. Salsa's defense and depth strategy consists of seven layers. So the first layer is infrastructure. You know, the infrastructure, compute infrastructure forms the foundation of the entire platform, needs to remain robust. I'll contain a hosting platform such as Kubernetes on EKS or AKS or GKE. Ensuring the security of that origin host prevents unauthorized access to any of our host applications. Of course, the application itself, Drupal in this case, needs to be secure up to date to prevent the exploitation of vulnerabilities and so on, like code execution, memory corruption, edge protection. We want to have edge protection as well via a web products, such as QuantoWeft, which will help detect and block potential attacks before they can even reach application. Content delivery, something like a CDN, like Quant CDN, Cloud France, number of CDN products offers DOS protection and traffic filtering. And these two are strictly, they're more of the administrative side of things. So people, we need to have well-trained and security-aware personnel that will help reduce the risk of incidents on your Drupal site. Someone's falling for a phishing link and in a process as well, in a well-defined security process in a well-defined disaster recovery plan, help you identify and manage risks associated with Drupal hosting. The next concept we need to cover is the NIST cybersecurity framework. So it's highly respected and a highly adopted cybersecurity framework in the US and contains five key phases. So let's identify, protect, detect, respond and recover. The NIST CSF isn't a certifiable technical standard, but it's highly flexible and it can be tailored to, you know, specific tech specs and unique threat landscapes. So what's our strategy here for these two things? We'll be aligning a Drupal defense in depth strategy with the NIST cybersecurity framework. Why are we doing this? So for one, defense in depth is an effective strategy to contain many attacks. If attack is supposed to compromise a single defense, the effective blast radius of the attack will be limited by the latest protection. The secondly is the NIST cybersecurity framework acts as a stepping stone to begin the journey to more stringent certifications. Many of the practices and recommendations from the NIST framework are actually echoed in stricter standards. So aligning a defense in depth strategy with the NIST cybersecurity framework, what applies to layers or your defense strategy while helping to streamline the path to compliance for more advanced security standards like ISO or PCI DSS. So the first phase of the NIST framework, sorry, I should also clarify, so for the purposes of the presentation, we're just focusing on these five layers, infrastructure, application, container hosting, people and process because these are the things that most organizations usually have direct control over. So infrastructure, we need to identify assets in our control that might be at risk. So some infrastructure components that might be targeted include computing infrastructure like web servers and worker nodes in addition to networking components like network load balances. Application, the vast majority of vulnerabilities in Drupal sites come from contrib modules and themes and libraries. It's important to keep track of those that in use and monitor the Drupal security advisories for any vulnerabilities that are disclosed. And we also need to be aware of what Drupal sites we have, like how many times have you received a support ticker, some sort of obstacle escalation for Drupal deployment. And it's like, well, I never knew we had that. It hasn't been updated in like five years. Our container hosting as well, parts of our container hosting or orchestration infrastructure that might be particularly exposed include the Kubernetes API server as well as components of any observability stack that's in use like Grafana or OpenSearch. Other cluster management APIs in use should also be considered, for example, if you're using Amazey's Lagoon.sh, the Lagoon API, for example, is exposed publicly to the internet. Your people are an asset that might be at risk as well with the level of sophistication of phishing and spear phishing these dates. Personnel are often the weakest link in the security strategy and your process. Although it's not something an attacker can attack directly, outdated processes are a risk that could negatively impact your implementation of later stages of this framework. So moving on to the protect phase. So we need to protect our five layers. So starting off about infrastructure, we want to try and leverage cloud provider concepts when we can. Make effective use of things like network security, policy and security groups. Try and rely on things like cloud provider managed operating system images because these are often configured with best practice and are kept up to date. We can also do things like regular rotation of our working nodes to ensure that the OS is always up to date. This can happen manually by a maintenance window or it can happen automatically using tools like carpenter where you can set an expiration policy. So every seven days, for example, we'll find a node that's hit a certain age, like 14 days old, drain it, kill it, bring in a new one. So it's got an updated operating system. Our application, so for our application, we need to have things like configuration management and auditing, ensuring that the configuration of our Drupal sites in line with the security policy. Celsa actually has a tool for that called ShipShape that can monitor any running or any Drupal code base to ensure that all the exported config or the config in the database is actually in line with what the Obscene or security team expects. Regular and automatic patch management for core and contrived. We want to do these as often as possible. We can make use of security modules like password policy, username and enumeration prevention, login security, the TFA module. These provide quite a few extra security features for Drupal. Like Drupal Out of the Box is quite secure but these modules help build on that good foundation. And finally, we also want to use static content if at all possible using tools such as Tom Aucoin CDN which will generate a static representation of your Drupal site. For our container hosting, we want to put sensitive endpoints like our API server on a private network accessible only by a company VPN. We can do the same with Grafana and any logging dashboards. We can put it behind a VPN and further protect them with things like OLO. We can use an IPS to detect connection to certain blocked addresses. For example, someone breaches your PHP container and deploys a crypto miner and it tries to phone home using a known crypto protocol. Your IPS can detect things like that, block them and send an alert to your upstream. And in a container hosting scenario, you usually have an ingress controller. You can implement things like mod security which is an open source web directly on an ingress controller. So that no matter how traffic gets to your cluster if it bypasses the CDN or if there's host header spoofing that gets through all traffic is checked by mod security before it reaches origin or at least you have some baseline level of protection. For your people you want to have things like proactive security training, role-based access control, Drupal's access control system is very robust with roles and very granular permissions. Managing the use of things like password managers and to factor authentication. And your process as well, you should instate a security policy that enforces the standards listed above from things like your application configuration to network policies to user access which will drive all these other layers. Our next phase is detect. So on our five layers, how can we detect a potential breach or an exploitation? So for our infrastructure, a good way to actually detect anomalous behaviors, your cloud cost and usage alert. So if your cloud uses fairly stable, a cost and usage alert is a fairly useful identifier to determine if that's a breach, especially if a tech is still a compute by spinning up things like crypto lines. Amazon also has tools like GuardDuty which will actually monitor your instances that actually scans your volumes and monitors outgoing traffic to identify known malicious signatures. At our application level as well, the login security module for Drupal is very robust. It can do things like send out automatic alerts when it detects that multiple accounts on your sites are having the passwords brute forced. Also things like the security kit module can be used to configure a content security policy in a reporting endpoint. So if someone does manage to compromise an editor account on your site and they embed a third party script or an iframe, they'll be blocked by the security policy if the user browser will actually report the violation to your endpoints so you know that, oh, there's some content on my site that violates our security policy and you can take action on that. At the container hosting level, you can also ensure that logs from Drupal and PHP are aggregated and stored in a single place. So Drupal can forward logs to a wide variety of endpoints through the use of community modules. So the benefit of having centralized blogging collected from all your Drupal applications is that your options support team can proactively create alerts and monitors for events that might indicate potential issues, breaches, you know, just general instability. You know, people like content often should be trained to spot unusual or suspicious contents and be encouraged to report it. Like, you know, no shaming here, like that was actually legitimate, but that's fine. But yeah, your staff should be encouraged to report things they find suspicious. Same with emails if they receive a phishing threat, they should be encouraged to report that. And that's what process you wanna have as much automatic alerting and proactive detection as possible by IPS, things like Prometheus, Elasticsearch. OpenSearch can monitor metrics and you can create lots of thresholds to monitor for specific events and actually send proactive alerts to your ops team. The next phase we're going to cover is, you know, responding. So there's a brief famous quote that I like. It's not, if you get hacked, it's when. So we'll need to respond in the event of a security breach. And at your infrastructure level, since we're in the cloud, we can, you know, leverage lots of cloud infrastructure concepts to respond to the attack. So for things like a DDoS protection, we can, instead of, you know, adding a web server rule and blocking the attack of the Ingress Control or Nginx, which will place a lot of loads on your origin infrastructure, you could use things like a security group on the load balancer itself to drop the incoming attack traffic. So the advantage of this is that this is, this filtering is done at the cloud provider level on their hardware, so it's not putting strain on your web service and with alleviate a lot of pressure. For your application, most of the time, if your Drupal site is, you know, pretty badly compromised, the only immediate response that can be taken is to just take it offline. But if you've actually taken a static snapshot of your site beforehand, using the tools that you described earlier, it can actually serve as a pretty good fallback. Although, like, functionality for some very dynamic applications might be reduced, the site would continue to be accessible in some capacity as opposed to completely offline. At this point, you'd also want to look at initiating restoration of backups while the static snapshot does its job of keeping your presence sort of there. At a container hosting level, it's easy to ensure that your site can be restored to a known good state using things like container images which are immutable. You can't change a container image unless you compromise the image registry itself. Also things like version control help with that. So we can very easily redeploy, restart your deployment and instantly your site's code is replaced with a known good copy that, you know, that you're sure your devs have authorized and worked on. That being said, your site's still vulnerable at the stage to the exploit that the attacker use again exists in the first place, most likely. We can make use of our centrally collected logs and metrics to perform a root cause analysis. All staff should have clearly defined roles and responsibilities during an instant response. Like, last thing you want is if you've replaced your site with a static copy and you're trying to respond to an attack and the content team is confused and they're trying to log in to try and fix things as well and they send in a support ticket and then ops is both trying to respond to the attack and also trying to respond to your support requests coming from in-house. And at this point all documented and hopefully practiced disaster recovery plans should be executed. And the final phase is recovery. So as part of the recovery process at your infrastructure level, you probably want to recycle all your work notes and ensure they're updated to the latest machine image. Although containerization should have protected the host note, you know, better be safe than sorry, there's still a risk of container breakout. Recycling all the notes will return them to the original condition using the cloud provider's operating system image. And of course before we can put our application back online, we'll need to make sure it's patched and updated so it doesn't get breached in the same way again. We can use information gathered from our root cause analysis earlier based on your collected logs to actually figure out how they manage to gain a football in the first place. Once that's done, we can bring the application back online or cut back from our static version. At a container hosting level, we can retrieve the backups I described earlier using a centralized backup solution, such as K it up. So the advantage of this is that the backups are completely to couple from the application I start off site. So even if they are executing code on a Drupal site and they have access to the entire file system, they can't affect your backups because the backups are never on the file system in the first place, they're shipped off to S3 or some other blob storage and encrypted so they can't get to them. And you can know that those copies are completely safe and tampered with. In terms of people, you should ensure that your staff are clearly informed about the cause of the breach. There's no point hiding, it's happened. And this will just help people know what to look out for in the future. And of course, if you've identified any weaknesses during this whole process, should make sure your plans and processes are updated to address any of these. So in summary, the NIST cybersecurity framework is a great security tool. We've applied the five steps in the framework across these seven security layers or domains. And it's time for you to start defending your Drupal site. Hopefully this gave you something to think about, the way we've broken down security domains and the five phases of how you can, the lifecycle of each of these layers of your hosting infrastructure and how you can proactively apply the security to them to ensure that your platforms as robust as possible. Thanks. Any questions? Dick-Mian has a question. Hi, I was just wondering on, I think it was the first slide, you had about nine points. I was just wondering if there was a significance to the order? I don't know, so it was the fifth slide. The fifth slide? I was just wondering what the order is. Yeah. Yes, it's numbered here. It goes from, how do I explain that? It goes from like foundation and we slowly build up from that. So we've got our compute infrastructure and we've got our container hosting platform like Kubernetes use. We've got our Drupal application and on top of that we have our itch protection on the web. And then on top of that, we have our content delivery system. Does that answer your question? Yeah, if that makes sense now. So I couldn't say that before. Oh, okay. Sorry, I might need to make it a bit bigger. But yeah, that number's on the side here. Right. Don't think there are any other questions? Yes, I mean, I'm hoping to feedback. What did everyone think? Any general or more specific one? Actually, I had a quick question. I probably did notice at one point that spearfishing attack obviously on a fishing boat. Can you do the spearfishing attack? So spearfishing attack is, it is conceptually similar to a fishing attack like they're trying to trick you into something where spearfishing attack is very targeted specifically to your circumstances or your company. So for example, it might be, instead of Nigerian Prince trying to steal your money, it might be someone that's impersonating your CEO and knows that your CEO is currently on a business trip to Japan. And they might write a message saying, hey, I'm on a trip to Japan. I need $5,000 very quickly. I'll pay you back. Something like that. Let's spearfishing. Actually. Any more questions? Thanks a lot for the presentation, Ming.