 Hi, everyone, and welcome along to this talk. Thanks very much for coming out this afternoon to hear about security standards. As Ian said, this is our talk. It's too much to choose, making sense of a smorgasbord of security standards. Both of us picked the title before we realized that we have problems saying that word. So we will try not to say smorgasbord too many times. But we're OK so far. So who are we? I am Rory McEwn. I am a cloud native security advocate at Aqua. And I try to help out in various places in the community. One of the reasons I'm doing this talk, or I was very excited to do this talk, is one of the things I do is help maintain the CIS benchmarks for Docker and Kubernetes, which is something I've been doing for about five years now. And that's kind of where I got a lot of ideas for this talk. Hi, everybody. My name is Anais Orles. I'm the open source developer advocate at Aqua Security. Before that, I was working as side reliability engineer. I'm also a science ambassador. And I run a YouTube channel where I explore different technologies. Now, I got started in the security space when I was joining Aqua. So all of this is still fairly new. And that's also why I got really excited about giving this talk because it gave me the opportunity to dive into security standards. So I hope to take you along and then you find our talk interesting. Cool. So where do we start? I actually want to take one step back and say, why do we start? Why do we actually care about security standards? For me, this comes from, well, I'm guessing that a lot of people here, maybe even everyone here, is running their workloads and their applications in Kubernetes clusters and their running production environments. And once you've been running a production environment with Kubernetes security, with Kubernetes for a while, one of the questions someone is going to ask you is they're going to say, is this secure? You know, you're running all your workloads if what you've got is secure. And they might say, well, you're going to tell me it's secure, but how am I going to prove it's secure? You can you prove that this is secure? And that's where security standards come in, right? Because you can say, well, it's not just my opinion on what security things have to be set here. This is an authority who has said that this is the secure way to run this platform. So that's, I think, why we're interested and why we want to think about what standards are, how they work, and which ones we might want to use. So we're going to take this through three steps today. We're going to talk about making sense of security standards, a little bit about what they are, how they work. Picking the right one. So we said there's a smorgasbord. Let's see if we can pick out which one makes most sense for you to use. And to do that, we're going to talk about how they work and why your Maya may or may not want to use them. And then, and Ace is going to really help and take us into the practical side of this about securing your cluster. You've picked your standard. Let's talk about what tools and techniques you can use to actually apply that. So what is a security standard? This is, there's lots of different definitions. This one will work for this talk. Guidance for vendor or third party on secure configuration and hardening of a product or service. So this is going to be some authority. Maybe it's the vendor or the product. Maybe it is an independent third party who are going to say, this is how you secure this. This is how you make this a secure because basically almost no software products come out of the box super hardened down. They tend to come out of the box in a fairly easy to use model and Kubernetes is no different, right? The developers and the Kubernetes project obviously want you to have a good first experience when you run up your cluster and you don't have to spend like two hours taking off all the hardening just to get something working. So typically you're going to need to apply more security in order to lock stuff down. So we're going to get guidance from third parties on how to do that. What type of standards are there? Because when I was looking at this and what I've been thinking about this for a while they're essentially in my mind two different types of security standard. The first ones we got are checklists, right? And you can spot a checklist because it will typically have specific guidance in numbered sections with specific actionable settings like you will set this setting to this value. It will have a fail and it will have a pass. So it'll have audit guidance and remediation guidance. So that's a compliance checklist. And those are really targeted at people being able to say is this product configured in a way that complies with this document? And then we get hardening guides and a hardening guide is a little bit different. So a hardening guide will cover the same kind of area as a compliance checklist but it probably isn't as prescriptive. It might mention key security standards, you know, flags or settings but it won't mention every single one. You know, it might just talk at a more high level. You should look at this area. You should consider this area. So one of the problems in assessing against a hardening guide is hardening guides are typically not written in that way. They're not designed to be pass fail. They're more designed for you to read, think about and apply. So those, I think, most of the ones we're going to talk about today fall into those two categories. So what do we have for Kubernetes? There's actually a variety of options which is our smorgasbord. We have the NSA guide. I'm sure people will probably have seen this, it got quite a lot of press when it came out. The NSA have produced a security hardening guide for Kubernetes. There's two versions of this 1.0 and 1.1 so they're keeping this up to date. That's a hardening guide. We then have, as represented by the vendor logos, vendor hardening guides. Typically, if you're using a major distribution of Kubernetes, you will find that the vendor will produce some kind of security hardening document for you. So AKS, EKS, GKE, OpenShift, all of these, if you go to their website, you'll find hardening guidance. We then have a couple of checklists. The logo of CIS isn't brilliantly visible. That's CIS. So this is the one I've been helping maintain for a long time now. This is a compliance checklist. This is a CIS benchmark. The CIS, if you haven't come across them, is a center for internet security. They have a wide variety of these benchmarks. They're vendor-neutral and you can get them for pretty much every product, but a lot of different products. In the computing IT ecosystem, we'll have a CIS benchmark. And then we also have the DISA Stig. So the DISA have come up with a essentially compliance checklist, which is really targeted at their users. But it's similar to the CIS benchmark. Down the bottom there, I've got the logo for the Kubernetes project. Now at the moment, the Kubernetes project itself doesn't have what I would call a full hardening guide. We've got lots of great documentation about security and there's stuff you can go and read there, but it's not really structured in the way of a hardening guide. With that said, I'm gonna do my punt for SIG Security Docs right now, which is we do have an open issue to be starting a hardening guide. So if anyone comes away from this talk, motivated in thinking, you know what, I'd really like to see one of these from the Kubernetes project. We'd love to meet you. We have SIG Security Docs meetings regularly, totally come along and we'll happy to help. I think it needs to like broken down and split up. I think we can totally do it and we've already got some good starting points. One option you won't see here is you won't see logos like PCI up there. So a lot of companies, I'm sure if you've been out in the floor, they'll talk about doing PCI compliant Kubernetes if you're in payments industry. PCI doesn't currently have any Kubernetes specific guidelines. People can take the general PCI and they can apply it, but that's their opinion of what Kubernetes compliance looks like in PCI. There is a PCI draft document, which has been in draft for a quite a long time now, which might come out hopefully this year, which will cover that gap. So there's no actual PCI guidance for this at the moment, but people will still talk about that. So that's kind of like our available options, what we've got to work with. So having said that, which of these should you choose? Which if any of these actually makes sense for you to apply to your environment? Now, obviously if you're regulated, that question might be answered for you by your regulator. They might say we expect you to apply X or Y, but not everyone works in a regulated industry, not everyone has that. So there's a couple of questions that I think are worth talking about, and there's also a couple of things that are definitely worth knowing about the standards process. First one, which distributions of Kubernetes are covered by the standard? There are 132, I want to say, the last time I looked at the spreadsheet that gets maintained, different Kubernetes distributions, product services, which give you Kubernetes. Not all of them have got a standard. CIS benchmark covers QBDM, AKS, EKS, GKE, OpenShift, OKE, which is Oracle, and ACK, which is Alibaba. So if you want to comply with CIS benchmark, and you've got one of those, you can find the document which actually applies to your distribution. If you're using any other type of Kubernetes, you can still try and use the CIS benchmark, but just be aware that some of the findings might be false positives, and they might be false negatives, because it is not designed for those. So things like file paths, you know, we will say, for example, admin.com should have permissions of 600, and we expect to find that in ETC Kubernetes. Well, if you're not using a distribution that puts it there, that setting and check doesn't make any sense at all. So first point to be aware of is CIS benchmark careful what distribution you're using. DeciStig does not specify its distribution, but when I read it, it's QBADM. It's fairly obvious that most people tend to use QBADM as a nice vanilla Kubernetes distribution. It's about as close as you can get to vanilla Kubernetes. Again, if your distribution puts things in different places from QBADM, be aware that your tools might not work properly, and your checks might not work properly. NSA is generic. It's a hardening guide. It's not a compliance checklist. They do have some specific file paths, but typically they stay away from that, which is great, because it means you don't have to worry about this too much. And then, of course, we have our vendors, and vendors generally is only their distribution. So that's one challenge. If you want to standardize on using a vendor's hardening guide, is the hardening guide only generally applies to their distribution, right? Microsoft are not going to write 130 different CIS style checks for different distributions. They're going to focus on AKS completely understandably. That's their product. So that's the first question. Is your distribution covered? Second question, which versions of Kubernetes are covered? This one is important. The Kubernetes project adds new features and removes features all the time, right? If we think about some of the things that used to exist in Kubernetes and don't anymore, Insecure API port. From 123, I want to say, definitely in 124, the Insecure API port settings on the API server do nothing. They don't work. But if your CIS benchmark, or you use an old version of a security standard, and it says you need to have the Insecure API flag set to nothing to avoid it being set, well, you might have a false positive. Your person runs a check. They say, hey, you've got a finding for this and you don't have a finding for this because that setting doesn't exist in Kubernetes, that version. Pod security policies. If your standard talks about pod security policies, it doesn't apply anymore if you're using later versions. PSP is deprecated, forever doesn't know, and it should be removed in 1.25. So it's important to think about what versions of Kubernetes are supported by the standard you use. CIS benchmarks. We have benchmarks for 123 and 120. Typically, we do it every three versions, which is a resource issue. If someone's thinking, I want to see a new one for every version, come and help. CIS is an open community. We are more than happy for someone to come along and say, I want to do a new CIS benchmark for every Kubernetes version. DeciStig. Undefined includes options not present in recent releases. So I would recommend being very careful if you do have to use the DeciStig. I found at least one check that references Kubernetes 1.12. So challenges, they're pretty out of date. This is a big challenge in standards world. In standards world, do you have the resources to go back every four months and re-update the standard? Do you have even to do it every year? And sometimes there's a lot of bureaucracy in changing standards. So be very aware if you're going to use that one, you might get some checks that make no sense whatsoever in modern Kubernetes. NSA Harding Guide. Again, this is undefined, but generally up to date. I would make the point of make sure you use version 1.1 of the NSA guide because they fixed quite a few out of date issues. They got a lot of good feedback when the first one came out. Things like they were using the old insecure versions of the scheduler and controller manager ports, which made no sense in modern Kubernetes again, but they fixed all that in 1.1. So if you're doing an NSA thing, make sure you use 1.1. You should be generally fine. Don't know how up to date they're going to keep that. I hope, obviously, that they won't maintain that, but I've got any insights into that project. So, the last thing is what areas do they cover? The first question after yourself is what is actually covered by the standard I'm using. Hardening guides typically can cover wider areas. So Hardening Guide, for example, the NSA guide will talk about security incident and event monitoring. It will talk about threat detection. Things which you can't really make prescriptive guidance on. I can't tell every company in the world this is what you should do for a seam for Kubernetes because that obviously doesn't make any sense. How could I do that? It's specific to your environment. So Hardening Guides are good because they can go slightly wider, but at a slightly higher level. But the downside is they won't tell you exactly what to do. They'll just give you pointers in the right direction. Configuration benchmarks, however, typically look at just the product. So the CIS benchmark looks at just the product. The Docker benchmark looks at just the product and they stand alone. So the Docker benchmark assumes you're running Docker standalone. It doesn't assume you're running Docker as a CRI in Kubernetes. And so some of the settings, again, don't make any sense if you do that. There is a gap currently in Kubernetes, which is that there is no CIS benchmark for container to your cryo. So this is a gap. I personally would like to see it filled by some kind of CRI guidance in the Kubernetes CIS benchmark, but that has not yet been accepted as a concept. So again, be aware of what these things won't cover. CIS benchmarks are quite limited. Things you can set in Kubernetes, not all the stuff you need around to make a complete environment based on Kubernetes. So you might end up needing to mix and match. But that's kind of like two, there's kind of three questions I would be asking myself when I'm thinking, which of these things, which of these makes sense for me? But let's talk about the details now. So how do we actually go about securing our cluster and choosing a tool that can help us to see whether we are compliant with one of those standards? Well, the standards itself, they don't help us to secure our environments. It should start far earlier when we are in our development process that we integrate with different tools to see whether or not we have certain vulnerabilities, misconfigurations present in our environment. So we can, for example, integrate different tools into our processes. Another way to go about it is to adapt our current processes or workflows to become more security focused as well as assigning ownership within our team, whether those are specific security teams or we have somebody within each engineering team focused on security, but ultimately the process of securing our environments starts way earlier before we think about security standards. So who's responsible for actually implementing those tools? Well, like most things, it depends on our team, on who's responsible for it, who has to know how, who's available, as well as our objectives. One thing we all struggle with is finding somebody who has to necessarily know how to actually implement those tools, understand them and then implement them and build upon it with our environment. And other obstacles to implement security standards, to follow security standards are competing priorities between different teams. So one team might want to check whether or not we are compliant and other team might just want to go about, let's release the next feature. Another thing is budgeting, whether you're using vendor solutions, such as Aqua Security, Sneak, or you're building an in-house team, you will have to allocate budget and other resources to implementing security solutions and checking for security standards. The other thing is vague and generalized guidelines. So while CS NSA provides us with a checklist, kind of steps to follow, it's not clear, it's not to the point of like what do you actually have to do. And then the last thing is difficult to use tools and platforms. We don't just want to add new tools to our existing stack that we then have to maintain, that we have to understand, that we have to maybe use even separately from our existing tooling. So those are the issues. And then, which tools are actually out there? Which tools can you use to check for security standards for your compliance? Well, they're different tools and they differ in the following aspects. So the first one is installation. How do you install the tool inside of your cluster? How do you operate the tool? They mainly differ between CLI tools and those are different to your usual CLI tools. CLI tools that check your cluster for compliance, they will install Kubernetes resources usually inside of your cluster. And that's across different tools, the case. Now you can also install Kubernetes operators. You can install Kubernetes manifest to have those resources living within your cluster. Then the other thing is the way that those Kubernetes resources operate within your cluster. What types of resources are there? Are those jobs running inside of your cluster? Is it an operator? Then the scan coverage, depending on which tool you use, you will get a different outcome ultimately from the tool, from those scans. And then do they integrate in your existing environment? What tools are you already using and do the new tools integrate with them? And then the focus, are those tools focused on engineers, on cluster admins, or on security professionals? Now let's look at some of the CLI tools that are available to generate CIS reports. The first one is Coupage, which is an ACO security open source tool that is used across different vendors that are also built upon Coupage. Now here you see the Kubernetes resource that basically has run the CIS report scans. So with Coupage, a CIS report is gonna be generated for each node. If you have a three node cluster, you will have three reports. If you have a one node cluster, you will have one report. And then you can describe ultimately each report and you will have a Kubernetes resource. So Coupage really generates Kubernetes resources inside of your cluster. If you run it as a Helm chart, as for example, part of the starboard operator, then you would have a COD cluster resource that you can then use. If you can also run it instead for the CLI, but in this case, we're using the Helm chart. Now, who here knows what an operator is? Should I clarify it real quickly? Okay, most hands go up, just clarify it really quickly. It's basically an operator automates human processes within your Kubernetes cluster. So this is what the starboard operator will ultimately do. It will continuously scan every three hours or so your cluster to generate those reports. Now, some of the highlights of Coupage, it works as a standalone tool or you can use as part of the operator. There are multiple different installation options. You can also run it as a Kubernetes chart just by itself if you don't want to use the entire operator and it's used across companies. Now, the next tool is CoupBacon. It's a project by a coworker of mine that I came across who's here also in the audience. So if he has later any questions, you can also approach him, I guess. So he created this site project. Now, it's a fairly new tool from what I could see when I was doing my research, which means that you can also probably contribute to it, help shape it. Now, it creates or it has specific CAS scans for different cloud providers. So from what I could see, the goal is to add more of a time and it runs either as CLI or as a job inside of your cluster. Now, moving on to NSA reports. Starboard also has now an NSA add-on which basically creates cluster compliance reports inside of your cluster. So before the starboard operator, you could also generate NSA reports and it's the same thing to the CIS reports. It's ultimately a custom resource definition that's living inside of your Kubernetes cluster once you generate it and you can query it like other Kubernetes resources. Now, with the NSA reports, they are part of your other cluster audit reports with starboard, like I mentioned. And you can ultimately, because it's a Kubernetes research, it allows you to integrate the reports with your existing tools. Like other Kubernetes resources, it's not a separate tool that you have to use and maintain. Then it also creates cluster compliance and cluster compliance detail reports. So there's a more detailed report than this one that will be generated. Then the next tool is Cubescape by AMO. Now, you can install as a CLI tool or also as a home chart. From what I could see, if you use the home chart, you will have to sign up to their hosted UI to get the token and then install the home chart. But you can also use it to the CLI and this is the full output that you will receive but you could also get a really detailed output if you add additional flags. And with the submit flag, you can push the reports also from your CLI to their dashboard. Now, this is ultimately the resource that will be created in your cluster. There are several jobs that will run every once in a while to perform the scans. So CLI tool, they are integrations for the integrations with the UI possible. They are plans from Cubescape to open source the UI but I'm not sure if that will be also a hosted version or if you can also self-host it then in that case. Then another thing is that Cubescape is producing human readable output. So it's far easier to see what's going on versus checking out the custom resource definitions from within the cluster. So the target audience is a little bit different. It's more focused on engineers and cluster admins versus security professionals who would want to build upon the scans and the tool itself. So I think personally that's suitable for multicluster environments if that's what you want kind of out of the box open source because you can ultimately install every cluster and then aggregate it within the UI. Now, other decision factors that you might want to take into account like mentioned is how much does this tool integrate with your existing environment? Will you have to maintain the tool itself? Or can you actually build upon it and integrate in your existing for example observability stack? So this is a screenshot from another demo I did where I basically visualized the vulnerability scans from starboard inside of a Grafana dashboard. So I basically used starboard exporter to get the metrics out of the cluster and inside the dashboard. And similar you can do with other scans from starboard. Cool, awesome. So that was essentially kind of the, there's a couple of different takeaways I think hopefully come across from what we've been talking about today. Security standards are useful, right? They're useful because you might well be asked for this external validation. Whilst you can have, I always say that there's a difference between security and compliance, right? I can be secure without being compliant and I can be compliant without being secure. So for example, I could be secure because Ian here could be running my cluster and it's going to be secure. But if Ian doesn't write everything down, it might not be compliant because someone can't walk in and go, I've checked all the things that've been done. I can also be compliant because I follow a checklist but if that checklist isn't appropriate, it doesn't cover everything, I'm not secure. So security standards are helpful but there should be taken as something you use as a starting point. They'll give you advice about where you can go, the sorts of things you should consider. I always recommend against where possible, taking those being the only thing you do. Don't just say, I've done the CIS benchmark, I've been through it, I checked or crossed everything, good, my security is done because it's not. With that said, I also recognise that you work in people working in compliant environments or regulated environments and you're going to get asked to prove it and having something like CIS benchmark where you have gone through and said, which of these apply to me? I'll do them, which of them don't apply to me? I won't. And as long as you can explain that to an external security auditor, hopefully they'll take that as being a reasonable indication that you've done the work necessary to secure your environment. It's important as well to understand with the tools what they do and don't do for you. What Anais showed was these tools are very useful and they'll deploy in your cluster and they make some checks, but they're limited in what they can do depending on how you deploy them. So for example, if a tool only checks the Kubernetes API, it can't do checks that are on the local host. It can't say what file permission you've got because the API doesn't expose that information. So again, taking a bit of time to try and understand what these tools are gonna do for you but also what they won't do for you will make the whole thing more useful and will give you an idea of what you know, what you shouldn't, shouldn't be doing. So yeah, I think tools can definitely be helpful in that regard. So just two links, we've got them up there. Slides will be up on the site, they kind of weren't because we were finishing them yesterday but the slides will be up on the website on Shed. So that's there as well. And I think that's awesome. Do we have any questions? Or are we all good? Is it Friday afternoon and everyone's thinking about their flights home? We do have time for a question or two. Anybody? But there. Yeah, so what the question is, what about the workloads running in the cluster? In addition to the cluster control plane itself and the cluster nodes, you need to do security checks quite rightly for the workloads. The answer is yes, some of the scanners will do that. Off the top of my head, I wanna say that I know that Trivie, we didn't talk too much about Trivie but Trivie actually has just added support for doing exactly that. And I'm fairly sure Cubescape does as well. What you can get a scanner to do to check the security context settings of all your workloads and to see which of them aren't optimal. Obviously here, one of the things I'd recommend very much looking into if you haven't already is pod security standards, which is a generic guidance around here, all the settings and they've got some ideas about levels. But yeah, you're right. I would say things like Trivie do do that now. So they've added that. So essentially, if you think about like what's actually needing to be done here, from a technical standpoint, this isn't super difficult. Pool YAML, check YAML for setting, return, good, bad. But you know, having that automation means you don't have to do it yourself. So yeah, definitely two, I'm pretty sure Cubescape does. So yeah, they do cover workloads as well from that perspective. Whether the standards do mention it as well. Thanks. Yeah, no, no, several questions. A good question. What other gaps do you see in the market that you'd like to have a tool for or built so that other people could benefit from it? Are there other things beyond the things you showed So the question is what gaps do I think we are or do we think there are in the market of things you won't want to fit a tool in? I would say for me, I suppose in open source, it's a question of whether we can cover more of the distributions. I think there's a slight concern to my mind now that, you know, I can blithely say it covers QBA and a couple of others, but there's people running lots and lots of clusters that aren't covered. The challenge is, you know, we need someone to actually go through and do the rule writing. I think there's some engines out there, but rule writing I've all seen is like maybe not something people are as fond of because it does get, I've done it myself a bit, it does get a bit boring when you're writing rules for an entire product. So yeah, I think coverage there, and I think maybe as well some slight cross cluster coverage. I don't, the open source tools at the moment don't as much do like comparison, you know, this cluster is better than that cluster or this cluster over time. That tends to be more where the commercial software I think plays at the moment in that kind of like long term, you know, my cluster is doing like this and then it went like this and you know, so. You mentioned earlier correctly that the different versions of Kubernetes have different security relevant default configs that the benchmarks need to keep up with or maybe behind and you need to make sure you run the right versions. Is there anything on the Kubernetes side to make it easy to know what security defaults change between releases and those security relevant settings so that if you are running ahead of the benchmarks that are provided, as you just mentioned it. It takes a while to get those rule sets updated so that if you're trying to stay like current with latest case releases, you could update your compliance benchmarks to align. So the question. Without having to dig into the code and like. Yeah, so the question is around, I hope I've got this right, is as new Kubernetes transactions come out, is there any way of keeping up to date with the new defaults and chain settings that have changed? I'm going to say that I haven't come across one. I hate to say I read release notes. I honestly reading the release notes for each version because typically security settings will be mentioned in there. I know that's not like a nice answer. But that I personally, what I tend to do is read release notes and kept. So I'll read the kept coming to each version and say which new things are going to land in this version and then do they come out? Are they going to make a change? You're right though it would be nice if that was perhaps like presented because I think the information is definitely there for sure. I just don't think that it's necessarily like summarized in a nice way. So yeah. I'm passing the mic to the next question asker, but hi, I co-chair Six Security and we do actually in the most recent couple of releases anyway, have as part of the release blog a summarization of security changes. And so I think that's the thing we're going to continue to do. Next question person, sorry about that. Thank you very much. A follow up question to the first one regarding workloads. You said the tools, for example, Starboard scan the cluster API or like the API at given times, I think it was three hours or something. What happens with workloads that ran in between two scans, for example, like one of jobs that could have malicious payload? Are they picked up upon? Are they considered or are the tools just looking at workloads currently running? Do you know how this is handled? What I would say is that, man, I need to correct me if I get this wrong because I'm just thinking about how I know how it works. The answer is they wouldn't pick it up if the workload wasn't running at the time the tool ran. What I would say for that is typically the control I'll be looking at there is admission control. So typically, hopefully, so in my world of what does the cure Kubernetes look like? You are running the same checks in IAC before thing goes live. You're running the same things in CICD to make sure bad things aren't happening. The same thing in admission control and then the same thing as your scanner. So hopefully, you know, there's other stages that you're gonna catch that at because you're right. Otherwise what you would need to do is actually have something which reflexively ran. And in terms of the open source tools, back to the question about gaps in open source tools, I don't think there's one that will reflexively scan. But I would hope that everyone, hopefully everyone who's running in production is running admission control. If you're not, I thoroughly recommend that that should be the thing you should go and look at doing immediately after this conference because that's how you would stop things like that happening. We have one more minute and at least three more questions. So I'm going to arbitrarily pick one of you and encourage all of you to talk to them after. Both of us are on Twitter, by the way, for follow ups, totally happy to try and help. And arbitrarily picking the excited person in the front. And Nate, from your research on DX of security tooling, what do you think the trend is towards the change in developer experience with these new security tools and this new generation of tooling? I don't understand any of the questions from IPM, I'm so sorry. Can you repeat the question? My question was just, what do you see as the trend in terms of the developer experience with security tools? Are they getting easier to use? Do you think that's part intentional or is that just an outcome of the rest of things getting more mature? So the difference of the developer experience between the different tools. Okay, so how did it differ? So I mean, the main thing is really like, how do you want to operate a tool, right? You want to run it more automated and you just want to have kind of the aggregated source and you don't want to check the cluster itself or the CLI, right? Like I, for example, prefer not to have the output in my CLI and I would have to, I don't know, take it somewhere else from there which is very complicated and annoying. So what I would do is I would integrate into an existing dashboard or into my observability tools. That would be my ideal case. Now, from the user experience, a lot of these tools are not documented in the best way and that's also something we're working more towards the open source projects at Accra because a lot of them will mention like, here's how you run NSA reports but a lot of people who get started with those tools and that's, for example, mentioned in the case of Cubescape, they have a lot of young new users and I would imagine that they would struggle having just kind of the output of like, here's how you run the command and that's it and no further explanation of what you do with the output then. So answer the question. Okay, thank you. We are at time. I would like to encourage everybody to give our amazing presenters a round of applause for this amazing talk.