 So this isn't finished, I was expecting to have a bit more time to get the polished product done, so there's going to be a number of points in here that are like leading questions for you to go out and do your own research if there are things that sound appealing to you. It assumes a base level of infrastructure knowledge and Kubernetes concepts. The target audience for this was people running Kubernetes in production. And it is not a comprehensive guide to security on Kubernetes. It's just, I guess, a collection of ideas and things that people can do to improve their security posture and, you know, fix up any gaps that they might have in their own stacks. So start off with a picture of potato done. So it's tip one is going to stop the bots and this is primarily about using a web application firewall to kind of stop malicious traffic at the edge of your stack. So, you know, there are a ton of bots out there just running scripts and trying to hit known vulnerabilities for every framework and application out there. So these types of things can just denial of service your site when you hit by one of these bots just because it's hitting pages that are not cached and, you know, it's like computationally quite expensive. So the idea with a web application firewall is you can configure rules and when the traffic comes in the firewall checks or analyzes the request matches it against the rules that you have configured. And if it hits then it blocks the request before ever even comes to dribble. So that can save you a lot of traffic and a lot of compute. So one thing that I think, you know, most people should do is set up your waft to block common paths for other frameworks. So block your WP admin dot PHP, block your CGI paths, block your auto discover stuff for those Microsoft products. So those are all like, you know, you know, minimum level of configuration required and you'll get, I think quite a bit of value out of that. A lot of these waft products also have managed rules. So, you know, that is like the OWASP top 10 vulnerabilities. So, you know, there are most of them have some feature that you can turn on. So that will, you know, monitor for like a known set of exploits, you know, CSRF, XSS and all those types of things. There's also not 100% sure on what the status of this is. So try to do some research. The blogs are quite old, but Drupal steward. So this came out after the remote code execution exploit last year, maybe the year before. The idea was that it's a commercially supported waft rule that, you know, when we have a big, you know, remote code exploit or something coming up in Drupal. Behind the scenes, these waft rules can be developed and automatically protect all the Drupal sites that have Drupal steward enabled. So that's something that is super exciting concept. I'd love to know where it's at. If anyone knows, please feel free to give them. Let me know at the end. These are just, there are a lot of waft products out there. They're usually the one that you're going to choose is tied to the content delivery network that you already use. So, previous next in Skipper, we're using AWS waft product. I have heard very good things about cloud flares waft product. There's a link there. I'll put these slides up later, but there's a link there to an excellent talk by Sean Hamlin at Drupal South camera. And he goes into some serious depth about very cool features and things that that product provides. And if you're on GCP, as a lot of Kubernetes users are that Google has its own waft product too. But I have not used that. It's meant to be text here. It's meant to say hydrobytes, hydro, no, hydro bits, hydrobytes. Okay. It's an old name server that checks out. So talking about all things encryption here. First off, the rank is encryption at rest. So this is talking about stuff that is stored on disk. And, you know, when a machine is imaged or, you know, someone gets access to the file system. You want to protect your data and make sure that like they can't just run off with your, you know, database volume or stuff that's in your private file system or your backups. So there's a, if you're using S3 in any capacity, we use S3 for our backup storage. There's a few options. Service site encryption would be my recommendation. So this is just an option that you can enable on a bucket or like on a backup utility to send a specific header with the S3 API requests. And then just make sure that the stuff on disk at Amazon is encrypted with a KMS key. So the advantages of this is it's super easy to use. It's basically just a checkbox. You don't need to manage keys yourself. And it does give you, like from a SecOps perspective, like a pretty good level of protection. The cons are that it might not satisfy the security requirements of your organization. So say you're, you know, you maybe your organization doesn't like the risk of Amazon itself having the ability to decrypt the backups. Then you might have to go to the other option, which is client site encryption. So this is where your application is the one that's actually doing the encryption and then pushing the encrypted blog. To your S3 bucket. Yeah, so the pros of that it's flexible. It's portable. It doesn't have to just be S3. You can push it to Google or Azure's products. But the cons are that the key management is all up to you. And please don't roll your own crypto. That's never a good idea. Just a few links there to ideas you can have if you want to roll with any of that. At previous next and skipper, we're using elastic file system for our private and public storage. And that has, again, a checkbox that you can tick to have the volumes on, you know, the underlying instances encrypted. That's a good idea to check. And RDS has the same option to, again, checkbox and you're good to go. Okay, so now getting on to encryption in transit. So this is talking about TLS connections to protect data while it's going over the wire, unencrypted traffic. Someone can sit on the network and just monitor the traffic and, you know, if there's any sensitive data in there, take it and use it for nefarious purposes. There was a, many years ago now is a great example of some Firefox plugin, I think called fire sheet. And you could go to like a, an internet cafe or like a, you know, McDonald's and sit on their open Wi-Fi and just see all like these people logged into Facebook or Twitter and things like that. And it'll give you an option to just say, oh, hey, Nick Sanat is logged in on Twitter. Do you want to log in as him? And it would steal their session and you could post things as them. It was amazing. But anyway, if you've got encryption in transit, those types of attacks are mitigated. In Drupal land, we've got connections coming into the app from the outside. So those are your browser saying, hey, I want to request index.php. And the other type are Drupal making connections out. So in this context, we'll just cover ancillary services that are inside your cluster. So that might be Redis, Solar, Clamav, things like that. So we're running on Amazon RDS for our databases and this is a really neat little couple of lines and settings.php that will encrypt your database connection. So Drupal talking to your database, fully TLS. There's a blog post there where I go into more detail about how to set it up, how to add the keys to your container images or however else you run it. So that's a really low hanging piece of fruit that gives you just one extra pop that's encrypted. So, yeah, talking about the like incoming HTTP requests. So in a typical Kubernetes setup, this is kind of the bits that are encrypted or indicated with the ticks and the bits that are not encrypted or indicated with the cross. So this is not too bad. Like at the point at which your traffic hits traffic, it's inside the cluster. So at that point, you're only really worried about bad actors inside already the network perimeter. But, you know, if you're running untrusted code or, you know, there are people who I've lost audio. Yeah, similar to me. Where did he go? So yeah, so at the point at which the traffic hits traffic, it's already inside your cluster. But there are some instances where you might not fully trust everything in your cluster. I mean, you probably shouldn't have a fully trust anything inside your cluster because there might be bad actors who have managed to penetrate your network. So completing that last little hop there is important for protecting your customer's data when they're submitting personal information or credit card details, things like that. It gets even worse when you look at when Drupal is then connecting out to other services for like your case or search or AV scanning. Typically none of those are encrypted, but they would potentially still be storing data that is sensitive and could be stolen or yet used otherwise by an attacker. So there's a few options here. I guess I'll go with the one on the right first. So do it yourself. TLS for that first for this like hop here where you're coming into Drupal. So you can set up an SSL certificate in front of Drupal here in that Drupal pod and have traffic connect over HTTPS. Now that's a quite a simple thing to do. It's essentially the same thing as setting up SSL on a static virtual private server like we used to do back in the day. But it only encrypts that one hop that traffic coming into Drupal. It doesn't manage or doesn't look after any of the other traffic then going out into your other services. The kind of the more holistic and robust solution would be a service mesh. So a service mesh is a description of a tool. There's a lot of products that do it, but essentially it that you can configure network policies and and have certificates deployed to containers that get attached to every single pod inside of your cluster. Then when Drupal is talking to Redis or to solar, it's not talking directly to the Redis pod. What it's doing is it's talking to its own little container that attached to itself. As a proxy and that proxy is talking to a proxy that's attached to the Redis pod. And so those two proxies talk over TLS so that over the wire connection is encrypted and then Redis and Drupal kind of don't even know that that's happening. They they're just fully blissfully ignorant to the fact that they're like way more secure now. So the cons are so the pros are that that manages everything, you know, you can configure any network connection inside your cluster to use a service mesh. The cons are that there's, you know, obviously it's a quite a complicated system to set up and maintain. You do have additional latency added to your to your hot path, you know, you've got extra TCP hops, you've got encryption of overhead. And there is like a non negligible amount of additional compute resources that are running like all those proxies need CPU and need memory to be able to run. So that does kind of balloon out how many how many nodes you're going to need in your cluster to run. So these are some options for service matches out there. So Linkerd and Istio are probably the two most popular ones. Istio is a is deployed by default on GKE and Google and it is it's a behemoth. Linkerd is trying to go with the more like turnkey type solution. So that's definitely one that like I personally would look at over Istio just so I don't have to understand a huge project to make it work. These are the ones here like console and Kuma and Misha kind of newer on the scene and probably wouldn't be deploying them into your production cluster just because things moving pretty fast and, you know, yeah, they're basically still for all beta software. This is a excellent talk from the Kubernetes forum in Sydney from last year. And this guy is absolute gun and he goes into extreme depth and detail and all of those options and basically gives a breakdown of his recommendations for particular use cases in which service mesh to choose. All right, so on to the next topic. So this is a tool that's relatively new called open policy agent. So opa is like a policy engine that allows you to set up rules. And when a new object is created in the Kubernetes API, it validates those objects to make sure that they conform with your rule set. So cool things that you can do with this. And then you could ensure that images that your pods, you know, have defined in their spec only come from a specific registry. Like you can, you know, use rejects rules and things to say can only come from, you know, skipper.io or something like that. You can ensure images are configured to run as non root users. You can ensure that that host names, like if you've got a team and they've got like a specific sub domain inside your corporate domain that they're allowed to run wild with. You can make sure that anytime they create an ingress that it is inside that domain space. Otherwise it could have people, you know, trying to create things where they shouldn't like, you know, maybe they can accidentally deploy www and sit on that with a dev site, something like that. And it is like a basically it's own true and complete programming system for these these policies. So it's just options are limitless in the types of rules that you can enforce. Now this type of thing is in my opinion is very important if your team is deploying with tool account, you know, where they can be quite free to basically run images from Docker hub and things like that. There is a there is a risk in in running images like that. So yeah, definitely want to be having some guard rails around what they can do. This is an example of a policy. So this is what I mentioned first. It's basically checking that the name of the image starts with holy.com as in trying to make sure that images are only like holy.com slash Drupal rather than Docker.io slash Drupal. Okay, the next one is runtime monitoring and anomaly detection. So this is a tool that monitors and what's called the like sys calls on Linux. So, you know, all the really low level stuff that the kernels doing is a tool called BPF which can check what's going on. So this is kind of again another policy engine where you can set up alerts for specific things that look kind of wrong. So, you know, examples are, you know, unexpected processes, you know, if someone, you know, managed to manages to escape Drupal PHP into a shell, you probably want to know about that. If someone manages to put a crypto miner on disk and run it, you probably want to know. If someone manages to become root, probably want to know. Those are, you know, all examples of things that that that runtime monitoring can do and and there's stacks of other stuff. I'm not a real expert in this area, but yeah, definitely worth looking into. Skipper we're running Falco. So again, like all those things that I've mentioned are things that we're monitoring for. And I believe that aqua has a similar tool, there may be others as well, but again, I'm not a real expert in this area. Okay, getting on to some simple ones. Read only file systems. So this is essentially stopping contained like files in the container from being modified. So the benefits of this you can stop developers from, you know, jumping onto production and changing a file inside the container to fix the problem. When if they do that they'll get a message saying permission to mark tonight read only container. And this, if you're running read only containers, it really does defend against like many classes of attacks where an attacker can always trying to write a file to disk or change index dot PHP to then run arbitrary code. So they get it. If you got a read only file system, they can't do that and it's, it's just like, you know, again, not saying you shouldn't patch your Drupal site but if it gives you that a little bit extra time. And to, yeah, not have crypto miners been deployed on your infrastructure. This is an example of how you would define that in in your pod spec so you can do this. You know, in your deployments or however else you're deploying your code. Okay, a couple of things around images. So image vulnerability scanning is like a process that will scan your image files and identify like operating system dependencies that are vulnerable or your project dependencies that are vulnerable. And select these tools can be deployed in different ways. You can tell them to just scan everything inside your registry and identify things that are broken around a day. And you can also put them into your CI CD pipeline. So, you know, an interesting idea might be to have this at the end of your build process and if there are any vulnerabilities. Just to avoid deploying any vulnerable code. Two options there to do this. I believe there are others. Yeah, definitely worth looking into. Okay, and the next one is image signing and notaries. So this is an approach to defend against an attacker. Somehow managers to get access to your registry like they still credentials or something like that. And they push an image with some kind of exploit or, you know, some sort of arbitrary codes they want to run, and they push it into a registry that you kind of just implicitly trust, you know, so you're wherever your Drupal code your Drupal images are going. They just push a crypto miner to that. And, you know, when your Kubernetes cluster pulls that image, it doesn't know that it's meant to be Drupal it just goes images that cool runs and the crypto miners running instead of Drupal. So what this approach says is that when you build your image, you cryptographically sign it, and you send that signature to what's called a notary. And the notary remembers that and associates that with a specific image version. You then have Portieres running on your Kubernetes cluster and when Kubernetes pulls the image, it checksums the image tells the notary, hey, I've got this checksum doesn't match what you have. If it does, good. It runs it no problem. If it doesn't, it won't run the image. So that's just a, you know, defending against that particular factor. Cool. Now is it. Thank you. Any questions? Comments. Thanks, Nick. That was great. Yeah, I have a question. Well, I guess a comment when Drupal Stewart and I guess the there it is a bit contentious if like these large hosting companies are getting like firewall rules before.