 All right, let's get started. It's just more appropriate that straight after the Drupal Security panel, we talk about more in-depth security and having to look at securing your Drupal ecosystem with essential aid scanning. OK, so this is more because it's in the development stream. I took more like a hands-on development approach and yeah, just a little bit intro about myself. I've traveled 6,000 k's to get here from Perth, but I'm actually in Melbourne currently working as an enterprise architect for an IoT multi-cloud company and a tech lead at Hide and Seek Digital, been working with Drupal for about 13 years or so. And yeah, that's me. So a couple of topics or discussions. So I will start with a horror story of Log4J 2021 just to get us started. And I'm going to kind of explain how Log4J has shaken the government and has then gotten us to a point where everything started to require to be audited. All of a sudden things and frameworks that were in place that were silent or ignored or not looked at started to be scrutinized and we started getting huge emails, 40-page papers to explain are we compliant, are we not compliant? And especially with federal governments, the state government became very difficult. Even though Log4J is not actually a proper thing, every ecosystem became very scrutinized. So we'll talk about what's the worst that can happen if you don't do security. We'll look at what essential aid framework is, which is, yep, and then we'll also talk about different strategies and approaches to become compliant with Australian cybersecurity essential aid framework. We'll talk about lots of different types of mistakes that I've done over my career and I've seen other people do. And yeah, we'll discuss the caveats of those mistakes and we'll look at different ways that we can also protect Drupal websites from attacks. And yep, we'll also looking at different approaches and solutions to scanning dependencies and scanning the ecosystem as a whole to make sure that we're compliant. From my point of view, I feel that as these essential aid requirements started to be enforced, we found out really quickly that some of the tools are not in the market. There's actually no tools, no solutions that exist currently that provide end-to-end compliance for Drupal. We'll look at composite dependency scanning, reporting. We're looking at updating modules on time, how to do that. And we'll also look at different weaknesses of platforms, looking at self-hosted commercial enterprise types of hosting and how you can shoot yourself in the foot by picking the wrong host. We'll summary and Q and A. The topic is huge. The topic is vast. I was just saying it's like a degree in itself, cybersecurity, so yeah, I won't have all the answers for you, but at least hopefully we'll get into the right direction. So I will start with a horror story. At the time, December 2021, I was working for a federal government client in Australia if they were commissioned and we've delivered a large website of 30 progressively decoupled applications inside it and everything was going really well until the government got hit really, really badly with Log4j. What happened is, yeah, so over 80% of the ecosystem has been affected. And as you can see here, a very famous meme. You have a modern and fancy ecosystem sitting there and the Log4j is just a tiny little dependency down there at the bottom of which allowed basically full admin access to all the servers, allowed you to download the database, allowed you to drop the database, allowed you to access basically environment variables just by running a couple of commands in the terminal. I've joined some meetings, a lot, a lot of meetings, and some of the interesting comments from those meetings were, we've got a phone call from Canberra, a really, really high place and we're screwed. We need to shut everything down, right? Do we even have Log4j? What is Log4j? If we have Log4j, what version Log4j? What version is vulnerable? If we update it, is it gonna crash? Do we have a dev environment where we can test it or we don't? I had some of the, this service been running fine for 11 years, I don't know what will happen if I'm gonna update it. And this is, we're talking federal government, right? Yeah, let's see. Some of the IT technicians were going, we just update Windows by clicking the update button going to sleep and the next day everything's updated. So we don't know whether we've solved the problem or not, whether this particular module has not been updated. Yeah, so some of these questions in our meetings we found out that the government was really not ready or at least a lot of the clients that we were servicing were completely not ready for this type of scrutiny. So even though Essential 8 and so many other frameworks that we just discussed in the earlier talk were available, they were not used, they were not implemented, they were ignored. So I'm gonna go into the red screen of death now. Anyone seen this before? Has anyone have to deal with this on their own project or their own site? You can raise your hand. Yeah, there's a few hands. Yeah, so this is basically the worst thing that can happen to your site. By the time you get this red screen of death sponsored by Google Chrome, your website has been delisted from Google. It's off Facebook. It's the Google ad campaign has been stopped. All of your paid traffic is terminated and your reputation on Google will never be the same. Also everyone is going to get this red screen of death in their browser. Yeah, so it indicates that your website already has malware, it's infecting clients. The website might have scripts that will be stealing your data. Yeah, you will have complete loss of reputation, significant drop in Google rankings and you will need to clean your website up. Seen this several times and when I went in, I saw some of this magic in the PHP code. This is actually a file from WordPress, but anyone can see any issue with this PHP file here? Yeah, thank you, that's right. Sorry, it is a WordPress, but the concept is the same. As you can see, it has all the correct information plus a little bit extra lines of code. Now this is a base 64 or whatever the method there is to encode the injection or virus. Essentially, a very common pattern is the website gets injected. It then gets all the PHP files receive a nice little injection like that. It's really difficult to decode it. You don't know what it does, but essentially it will then result in the red screen of death. Manual cleanup is really difficult and costly. Now it is not so important with the enterprise cloud hosting platforms existing that don't allow to write a file system, but this is very much still the case for self-hosted. I can do it myself or low kind of GoDaddy, registry those really cheap blue host servers. Okay, so when the log4j happened, I was asked, is today's build secure? Are today's dependencies secure? When was the last time we ran the build and have all the dependencies have been compiled as part of the composer? Are they all secure? Is there a log4j? If yes, what version? We need to know. We need to know every day as part of the compliance. But we could not answer those questions. And then after digging, there wasn't a module or anything that actually existed to be able to do that level of reporting which was required for the essential aid compliance. So yeah, exactly. Are you 100% certain all the dependencies are secure? Can there be a report? There was no report, we had to build that solution. So going to go through what actually is essential aid, I did not know until 2021, but I found out very quickly being a delivery lead, a technical lead, Drupal lead on this project. Australian Essential Aid is a framework developed by Australian Cybersecurity Center, essentially help organization assess and improve the cybersecurity practices. There are eight strategies, but it has not been tailored for Linux or web applications. Essentially, it's very Windows based. So some of those, I believe two out of the eight are really not applicable because they're talking about Microsoft macros, it's not a Drupal thing. So yeah, so I didn't know that Australian Government Signals Directorate exists. I didn't know that Australian Cybersecurity Center exists, but I found out fairly quickly once I had to meet the compliance. So yeah, I'm going to go through individual each individual strategy, but of course the topic is extremely extensive, like Michael Richardson mentioned in an earlier talk about the Drupal Security Panel. It's very comprehensive. There are documents that are 400 pages in length. There are multiple frameworks. So this is just to cover the surface and to get an understanding of what compliance looks like. Essentially, I needed to complete a 40 page document for a federal government to tell them that we are compliant. This is how we handle in compliance. And yeah, just to mention as well, some of the things are very straightforward. Like, can we do multi-factor authentication? Yes, we can. Can we do a daily report and viral antivirus scan? Maybe not. How do you do that on an enterprise platform, enterprise cloud platform? If you're self-hosted, do you have enough virtual machine CPU to be able to execute those scanners? Unknown, unclear. And at the end, some of the really obscure ones that I'm going to get into and really ridiculous ones. Like, can we have MFA on a SSH connections? CICD needs to use two-factor authentication and things like that. So anyway, we'll start with the application wide listing. So strategy involves creating a list of approved software application, allowing those on a list to run. So wide listing is the first one. Patching is the second one. We've got configure Microsoft Office Macro settings, which is the third one, totally not applicable. User application hardening. And yes, so applicable or not. We've got restricting administration, administrative privileges, patching. Everyone knows about that. And finally, the multi-factor authentication backups. Yeah, so essentially it's a model. It's a framework. It's a guideline that as a developer you can get and then you need to answer the questions in order to kind of understand whether or not you are compliant and not you might, you need to take steps towards compliance more and more and more. Yeah, so, you know, Drupal can be exploited as, you know, if anyone still remembers the Drupalgedan story, zero day vulnerability. For me, I was working in a company that happened at 7 p.m. and I think I was awake for 20 hours straight from 7 p.m. manually cleaning up the site. And I'll go into that topic in a second. We're gonna talk about the bad architecture and how bad architecture can really shoot you in the foot. Even you're, you know, compliant at the server level or ecosystem level if the application or application architecture is not there, you will, you know, not be, yeah, you can still have really bad after effects of attacks. So, yeah, I mean, we can exploit Drupal through cross-site scripting. SQL injections are very common when you just put some SQL code into, you know, contact form, denial of service, distributed denial of service and file inclusion vulnerabilities too. This is the approach to protecting, right? So, we've got the white listing, the patching, restricting admin privileges, multi-factor backups and network segmentation as recommended by the central aid. If you translate this into the Drupal, the Drupal needs to be kept up to date. You need to secure your database, probably not have the database open to the internet and only open to the instance that actually has the Drupal on it. Strong passwords, two-factor auth, using security models and scanning, make sure you restrict access, check your logs regularly, implement backup and recovery plan and stay informed of security vulnerabilities. But it's not gonna help you if you have bad architecture. So I'm gonna tell you a story about I was working a full-time job about seven years ago and when I got to my company, I SSH'd into the server the moment I got my key and I saw this on the left here. I saw dev, broad and stage on one virtual machine. And I asked the technical leader question. I said, um, why? And he said, cost, it's cheaper. We don't have to run three servers. And then he said, it's really good because if you run a shell script, you can copy the database from one instance to the other really quickly. So easy, right? No problem. And then there was an HD access file here on the right. You can see, depending on which URL this hit, it would redirect the traffic to that particular folder inside a C-Panel server, which was self-hosted via. Sounds like a perfect architecture. So what happened is there was also a backups folder. I forgot to show that and all the backups were also put in this folder. After three months of working at that job, I tried to change it. I said, no, don't touch it. It's okay. So when the Drupal get-and-happened, the whole backup was deleted. Ev, and when I did a scan, we had 500 files infected out of 4,000 in both staging dev and production, no backups. So I had to do manual cleanup. So I'll get into manual cleanup in a minute. Now you'd think that now with the more modern enterprise grade platforms like Acquia, Ironstar, Platformer Sage, Pantheon, it's no longer the same. So you probably no need to worry that much about this. But still, if you're self-hosting, if you are using C-Panel, if you're using Plask, if you are running a smaller site, and this is very normal. So, yeah, no matter how you are, if you're architect badly, it's not gonna help you. There's also another level of vulnerabilities that's really difficult to pick up. Like when we're talking about the hosting, yeah, your hosting can be secure. You can have all sorts of security monitoring in place. But if you write bad code, like for example here, then you won't see any issue with any lines here. It's very optimized, you know? That's it. So, see, you got something like that. Even if you have the best enterprise grade platform, load balanced, and multiple regions, you're screwed. So here, of course, you need to use proper Drupal Database API, and you just sanitize the input before you are pushing this to prod. But I've seen this before in production, and yeah, so that's another thing, right? So as we talk a bit more about the remediation strategies, remediation approaches, we need to search for this and report on that daily. We need to also report and search on, you know, look up whether all the composite dependencies and dependencies of the dependencies are all up to date and also not vulnerable, don't contain zero-day vulnerabilities. Yeah, that's explanation of what we should be doing and what you could expect in terms of, you know, exploit in the second, in the second black box there, Drupalgetin 2. Yeah, so Drupalgetin 2, really kind of, I was showing previously an exploit. I don't have time to show an exploit here, but it just to show you how easy it is when their vulnerability happens, an exploit is published normally, and then you can reverse engineer it or just, you know, use that to literally gain production access to any website. And this is kind of an example of what happened with Log4j as well. There was a zero-day vulnerability, there was a big disclosure in a patch, people reversed engineer the patch, and in the next two, three days, there were exploits available on the internet to be downloaded and just run it on your system to gain production access to any site. But then because of lack of scanning, because of lack of awareness, they weren't patched, which resulted in major, major breaches across different systems. See, yeah, so this was a Python script I was writing to showcase the exploit. Yeah, scanning and reporting is really important. So as part of the Essential 8, you need to scan and give a report daily. And this is the part that I found was really difficult because it just did not exist in the market, or maybe you could run a compo, you can run a composer thing to do the scan, to do the reporting. However, it was not automated, it did not result in the correct report. So I made a module, which I've open sourced, and just an example of why this composer dependency scanning is required. So here's, we've got a composer file here, right? And it looks okay, but if you're running a build every time you do a production deployment, there is a little problem here, right? Anyone can see how this could be vulnerable? Yeah, that's it. There is a, we've locked one of them. I can't remember which one, but because we've locked it, that means that every time you're gonna build it, it's gonna stay at the same version. So even if it's vulnerable, you're gonna compile that. You're gonna composer install that. And then that could be used to gain product access and drop your database, so easy, so simple. And it's just two signs here. So something like that, we need to report on that. If this is done, we need to know, but there's no tools that currently exist to do that. Sometimes GitHub would tell you, but providing you using GitHub, what if you're using something else, right? So I made a little module here, which you can check out if you like, essentially what it does. So what my strategy was to really quickly solve this scanning for composite dependencies, was to create a module which would expose securely, compose a log file, updates, the required modules that have expired and modules that have vulnerabilities from the Drupal, and then use a CI CD pipeline to consume that, which will allow them to run a whole bunch of scans. And you can use publicly available tools in the CI CD space, which we used Azure DevOps for that. This Azure DevOps would consume those files, it will consume the output of the module and run the scans and then output. Output the report, you can then schedule the pipeline to run daily or even twice a day, once an hour, if you like, and you're gonna get a report once an hour. I have not seen this solution anywhere else, so I ended up building it. Yeah, just a couple of tools, because I think we're running out of time. I've found that once you've been attacked, this is a really good antivirus. It's a lamp stack antivirus that works on any PHP application but also works on WordPress and Drupal. You can install it. It's gonna do a scan. It's gonna give you a report. It costs about $6 a month. It also does something called the Horistic Logic Scanning, which means that if it looks like a virus, for example, it's got a very long base 64 string, it might not exactly match any virus signature that exists at the moment in SVEs, but if it matches it, it's gonna give it to you in the report. It does file change monitoring and it does that reporting daily. So very useful, very cheap. And yeah, we wanted to talk a little bit about the Cloudflare that we've mentioned in the Drupal Security Panel as well. I believe that Aqua is using Cloudflare quite heavily. Cloudflare is amazing. It has a AI-based web application firewall that literally looks at deep packets and look at using AI, analyzes the traffic. If it matches a pattern of a vulnerability, it's going to block that request and send it to a big black hole without impacting your Drupal instance completely. So it does that at the edge level. And the cost of this enterprise plan is expensive, but if you wanted to protect your own site or a small project, it's about 20, 25 US dollars a month. And that literally will catch any packets that match bad signature and it is gonna stop it. Very, very good and saves a lot of time. Kind of like an instant, instant fix really. You can see here at the dashboard, you can see unique visitors and you can see all the traffic that it saved, but it stops DDoS attacks and also has that AI-based attack detection, if you like. Yeah, so I've talked about the heuristic logic, talked about the antivirus solutions. The reporting and scanning, I figured it should be done using a pipeline, not to load the Drupal instance itself. It also might be that you might need to change the pipeline. So sorry, you might need to change the code of the scanner. So I find that as running as part of the CI CD, whether it's CircleCI or Azure DevOps is really nice. The module that I've built exposes securely the log file and the JSON file and then gives you like a token that you need to use to make an API call to that module and that you can W get that from the pipeline, you'll get the composer file, you can run the scanner on it and then get your results. This is one of the example, you can run the security checker like this and it's going to give you your report and then you can just send an email or put it into JIRA or whatever you like to do. Got very little time left, so I'm just gonna talk about hosting. Self-hosting is the most dangerous one of all. People are smiling is true because you need to maintain so many levels of dependencies and you're not getting that protection that was talked about in the previous talk here. Tenable is really great for Linux package scanning. It's an agent that self-hosted essentially meets the government requirements because it's not sending data somewhere else to for analysis instead it's installed on the agent. It runs on your local VM and gives you the report securely. However, it will be tricky to install it on because you need root access to Linux. It's quite tricky to install it on a managed services. So that's a caveat there. Yep, so I think talked about most things. Huge topic, needs to be scanned, needs to be mitigated and needs to be discussed further. Essentially, it's just one of the frameworks that's multiple frameworks available. Yep, so CDNs are great. Handling vulnerabilities at the age are really good. You need to do antivirus scanning and you need to do reporting. Yep, so thank you very much. I hope you found it useful. Howdy, thanks for the talk. So with the real-time scanning that you need to do like for the static code analysis or how do you do that like in a containerized solution? Is there, does it apply or what do you do there? Yeah, so real-time, it doesn't have to be real-time but I guess there's a few things, right? It could be a module that needs to be updated. It could be a custom module that has vulnerable code. It could be something weird that you're pushing upstream just now, quick fix and that actually might make your website vulnerable. Okay, so let's say if you are pushing it to Git you can run a CI-CD pipeline on pull request that is going to execute the security checker and then gonna send you the report. Looks good or hey, this looks doesn't look good. So that's normally the approach. I mean, of course you can run it locally as well to get your report to see if you wrote something strange or if that's gonna pass. When you, before you deploy to production CI-CD pipeline could run and also give you the report then you can scan production every day which is the government requirement. They wanted a daily scan, they wanted a deep scan and get you the reports. So just in case for example, you push something today and tomorrow there's a zero day vulnerability that gets discovered and you've targeted that without knowing. So maybe it passed yesterday but today it doesn't pass, right? So then you want to know about it. So that's why we should do the production scans daily as well. Oh yeah, thank you very much. And yeah, you can catch me around if you wanted to ask any more questions. Thank you.