 Welcome, everyone. I'm Surya Sitaraman. I'm an engineer working on the OpenShift networking team at Red Hat. I'm also a contributor to the SIG Network Policy API working group. Welcome to our talk on the Admin Network Policy API. It's a new Kubernetes native API that is designed for comprehensive, cluster-wide network security. Hello, everyone. My name is Nadia. I'm a senior software engineer at Red Hat OpenShift networking team. Hi. My name is Yang, and I'm a senior software engineer at VMware working on container networking and security. And my name is Andrew Stoikis. I'm a senior software engineer at Red Hat in the office of the CTO. And we are all part of the Network Policy API subgroup to SIG Network. That subgroup is fully devoted to the future of upstream policy-based APIs. Today, there's a lot of downstream variants of policy-based APIs. We're trying to consolidate everyone to increase portability and help the whole ecosystem. We additionally have a maintainer-track talk on that subgroup tomorrow afternoon, so please come check us out. OK, so hopefully everyone heard the theme music as you were walking in. We're really excited today to go into a little Hogwarts story. Who here knows what Harry Potter is? Hope to see a lot of hands. Who here would classify themselves as like a Harry Potter fan? Some good hands, some good hands. OK, awesome. I'm a huge Harry Potter fan, so I forced everyone here to kind of go through the Harry Potter story, so bear with us today. For today, we're going to have a Hogwarts Kubernetes cluster. At the head of that Kubernetes cluster is going to be Albus Dumbledore, of course. So he's going to be our cluster admin. We're going to be telling this story from the bottom up from the perspective of the cluster admin. So just try to keep that in mind. It's going to be really important to remember throughout the entire story. We also have some really important namespaces, right? So we have the Forbidden Forest namespace. Ooh, scary, right? We don't like it, just remember that. The Professor's namespace, obviously, that includes Professor McGonagall, who is an important developer. Now, they're a namespace with some developers, but those developers have slightly more rights than the rest of the developers, right? They're professors, so they're kind of important. And of course, we have Gryffindor and Slytherin with our favorites, Harry and Draco. This is what makes up the Hogwarts cluster, and it's going to be integral to the rest of our story. So let's hop into kind of ground zero. Today in Kubernetes, the basic networking model is every pod can talk to every pod, right? And I think most of us are pretty familiar with this, but we wanted to start from ground zero. So here you can see Gryffindor namespace pods can always talk to Slytherin namespace pods. Everything is kosher. By default, everything's allowed. So let's look at a little bit more complex case. Not that much more complex, right? Harry doesn't love Draco. Harry's like, ah, Draco's a little evil. So we're going to block traffic from the Slytherin namespace. What do they use to do that? Today, they really only have one tool that's part of the core Kubernetes API, and that is network policy, right? Most of us here know what network policy is. It's been around for a long time. It's a stable API. It was originally made explicitly and only for the application developer. And I want everyone here to remember that. And it was also explicitly made to be namespaced. It was not cluster-scoped, and it was only for the application developer. So one other funny thing I want you to note here is that when Harry creates a network policy in the Gryffindor namespace, it implicitly isolates the Gryffindor namespace from everything. On this slide, we're just seeing from Slytherin, but on the next slide, we'll also see, oops, kind of accidentally, kind of not accidentally. We also block traffic from professors. But wait, shouldn't professors be allowed to talk to every student namespace? This is kind of a problem. So let's look at that a little bit more. So now, what do professors do? Even if they really want to talk to student namespaces, they might try to use those existing tools that we have, right, network policy. But that doesn't work very well. As you can see, even if professors try to write and allow in their namespace, that arrow is still red. We have some major problems. So this is where kind of the work we've been doing comes in. We're moving away from the application developer role and looking at the cluster administrator role. And Nadia's gonna kind of take us down that journey of some of the new stuff we've been working on. Thank you. So we've seen how professors try to solve their problems themselves. That didn't quite work. So now they go to cluster admin Dumbledore and ask for help. And actually, Dumbledore is able to help thanks to the new API called admin network policy, which is created specifically for cluster administrators. And it lets you enforce security policies that can apply to multiple namespaces and that cannot be overridden by any other personas. So Dumbledore actually agrees that professors should be able to talk to all houses and he creates a cluster-scoped object admin network policy that explicitly allows that. And you can see the arrow is now turned green. But let's take a deeper look into how it works. So we have introduced three layers of precedence. They are evaluated top to bottom. And you can see that the first one is admin network policy. It is managed by cluster admins. And as soon as admin network policy has a decision on a connection, it will be applied and nothing else can change it. But if admin network policy didn't match your connection then the network policy is the next layer that will be applied. It is managed by application developers and that will take effect. And you can see there is one more layer that is called baseline admin network policy. It is also managed by cluster admins and it is evaluated last and we'll talk about that in more detail in just about a couple slides. Okay, so let's go back to our professors and see how it actually works. So you can see that admin network policy has the highest precedence so it will be evaluated first. That means that when professors are trying to connect to the Grifendor namespace, this connection will be matched by an admin network policy and the connection will be allowed. But for Slytherin namespace it is a bit different because their connection to Grifendor namespace is not matched by an admin network policy. That is why the Grifendor namespace network policy actually applies and that's why this connection is still denied. You can also see one more detail. It's a little field called priority and admin network policy object and all it says is that different admin network policies are prioritized and this field basically defines in which other admin network policies will be applied. So let's keep this picture in mind as Yang will show us a little demo on that. Thanks Nadia. So as you look at the admin network policy in CYAML form, you can see that it is really kind of like similar to the current Kubernetes network policies but there are some key differences. We wanted the API to be explicit first of all and second of all, we're seeing some fields that are new to the admin network policy which we highlighted here. First is a priority one. The priority field actually signals the relative precedence of this admin network policy compared to the other ones but admin network policy objects in the cluster will always be evaluated before the namespace Kubernetes network policies. So that's why we can achieve something like strong allow that we're demonstrating here. And the second one is for the rules which is in the ingress, the ingress section, there is a explicit action field that says I want this to be allowed or denied and it actually comes really handy with the priority model so that you can do a denied rule underneath and then some allow rules on top of that which we'll show later. But for the purpose of this demo first, this is a strong allow story which we just showcased here in this diagram and we want to dive in to a pre-recorded live demo. Let's see if it works. So we're showing this on a cluster that's running and share as a CNI. And share is open source CNI based on OVS and along with other Kubernetes are two first implementers of this admin network policy API. So you can see that in the cluster we have entry running as a CNI and we are actually also showing all the namespaces that are in the cluster which includes the Forbidden Forest, the Gryffindor and Sres ring houses that we just seen and the Professor Namespace. We wanted to make the demo a little bit easier to read. We wanted to, sorry. We are actually looking at the Gryffindor namespace and there is actually the network policy that we just talked about which says I don't want to allow any connection coming in to me. And this network policy actually does that. As you can see it allows nothing as an ingress traffic and it's applying the Gryffindor namespace. So for the purpose of demo, we're first of all exporting the IP of the Gryffindor a Harry Potter pod in the Gryffindor namespace which serves as our Harry Potter IP. And then to demonstrate the network policy effect we can see that the Dracoal pod in the Sres ring namespace cannot actually connect to the Harry Potter IP because of the network policy that are in the Gryffindor namespace. Similarly, if we want to initiate a connection from the Professor Namespace to the Harry Potter IP it will got denied for the same reason because of the same network policy that's in place. Now let's look at the admin network policy that I just showed you guys which is we have the Poverty 11 admin network policy which is applied to the two houses and it has a strong allow rule that says the professor should always be able to talk to the houses. Now if we do the good old Kupkado apply of this policy and then we try the connections again now you will notice that the professors are now able to talk to Gryffindor because the admin network policy has a higher precedence and overwrite the network policy implicit isolation behavior. On the other hand, Dracoal will still not be able to connect to Harry Potter because of the network policy that Harry Potter has placed. So this is the first part of our demo and in addition to something like strong allow, oops, we also wanted to show another use case which is strong deny. This is a totally different storyline because we're now looking at the forbidden forest. As Andrew just mentioned Dumbledore comes in and say forbidden forest is really dangerous. Like nobody should just be able to talk to them forest but people like Dracoal and the professors they are actually having their own allow policies they are trying to talk to the forbidden forest. So as an admin how can I lock this down completely? Well the answer is a strong deny admin network policy. So right here we have a admin network policy that's really like the one we just showed before with a slightly higher priority but instead of allow rule we have a deny rule for the egress. It's selecting every namespace in the cluster and basically says that nobody in the cluster should be able to talk to the forbidden forest. And we also has that as a demo so you guys can better understand how this works. We have the, oh, wait a second, is it? Can you guys see that all right? I think the resolution is a little bit low for that one. Okay, cool. All right. So yeah, we have the same cluster as before. We have the four namespaces. This time we're exporting actually the forbidden forest pod IP so that we can refer to it later easily. Then we're actually looking at the admin now policy that I just showed which is a strong deny one. And as I said it applies to all the namespaces, has a higher priority which is trying to deny people to talk to the forbidden forest. And as we showed before, we wanted to do a Kukalo apply for the policy so that we created a cluster level. Okay, now it's created. What it does now is that if you try to make a connection from Draco in a certain namespace to the forbidden forest IP, it will get denied. It because the admin now policy is taken into effect. Similarly also like for the professor namespace, if you initiate a connection to the forbidden forest it will also get denied because of the same admin now policy that's applied there. The important part though is that we are actually also gonna try to put a now a policy in the professor namespace which basically says that as a professor I wanted or as an application developer I want to put a now a policy that actually allows myself to egress to the forbidden forest namespace. Well, this now a policy has been already created in the professor namespace and that's allowing egress traffic to the forbidden forest. But does it work? Let's find out. So we're having a connection from the professor namespace to the forbidden forest and it's timing now because the admin now a policy strong deny is actually taking effect before the now a policy evaluation. So that's how as a cluster admin or demo door you can ensure that if you have a strong deny rule it's always gonna hit before any developer tries to do anything different. So that concludes my part of the demo. And I'll hand over to Nadia. Thank you, Yang. That was a nice demo. Let's get back for a second from Yamal's two pictures. That is what we've just seen on this demo. So we have a cluster why deny everyone to the forbidden forest policy. No one can talk to the forbidden forest even though they have their own allow network policies in the namespaces. But at some point professors decide that they actually do want to go to the forbidden forest. Maybe they need to feed the giant spider or they have some other business there who knows but they go to Dumbledore again and say that yeah, we want to decide by ourselves and actually Dumbledore thinks yeah, professors should be an exception to these strong deny rule. I want to delegate this decision to them if they want to allow themselves to go to the forbidden forest or not. And that is now possible with a new pass action. What it does is it basically skips all the following admin network policy for the matched connection and jumps straight to the namespace network policy layer. So let's try to see what happens here. You can see that pass action admin network policy has higher priority than the deny one. So it will be applied first. And then when professors are trying to connect to the forbidden forest, pass action network policy will be applied. And the network policy in the professor's namespace will be actually the one that decides if the connection is allowed or not. So now you can see that professors can connect to the forbidden forest namespace because they have this allow network policy. It is again different for the slithering namespace because their connection to the forbidden forest is not matched by the pass admin network policy but it is still matched by the deny one. So this connection will be denied without even looking into what slithering namespace has in it. Okay, now keep that picture in mind again as we go to Surya's demo. Thank you, Nudu. So like Nadia mentioned, we have a pass action and a normal traditional firewall usually allows you to do deny or allow. So the pass is our shiny new feature that is a part of this API. And this is that feature that lets admin network policy interact with the network policy. So this is exactly how you delegate to the network policies in your namespace. So this is how the sample demo looks like of kind admin network policy. And here we are trying to pass all the egress traffic that's coming from our subject, the professors and going towards the forbidden forest. You can see that it's created at priority three, which means it's higher compared to the strong allow and deny admin network policies that Yang created on his cluster. And we can also see the explicit action pass, which is different from what you've seen before. So I know I'm the last to present, but I wanna make sure everybody is with me here. So how many of you use network policies? Can we have? Cool, that's a lot of you, I'm happy to see that. So this is the one thing that lets your admin network policies interact with the network policies, right? So literally if you figure out that your network policies are not working as expected, you might wanna open a ticket to your cluster admin or I'll be stumbled on, right? Trying to figure out what's wrong. So let's look at the pass action demo that we have prerecorded here. And like Yang mentioned, I'm gonna show the demo on the OVN Kubernetes CNR cluster. So here we have a kind cluster with three nodes, one control plane and two worker nodes. We also have our OVN cube agents that are running on each node in our cluster. And then we have our demo pods, right? So these are the four namespaces that we've been coming back to again and again. So we have the forbidden forest namespace, which has the forbidden zero pod. We have the Gryffindor namespace with the Harry Potter pods. We have the professor's namespace with the Milvano-Gonagal pod. And then we have the Slytherin namespace with the Draco-Malfi pods. The ones that we care about in this story line in the demo are the professor's namespace, so this one right here. And the forbidden forest namespace right here. We also have a network policy that is defined in the professor's namespace like we saw earlier, so this is namespace scoped. And this one is allowing all traffic to go to the forbidden forest. And this is, I'm just taking it from the same spot that Yang left in his demo. We also have the two admin network policies in our cluster at priority 11 and five. So this is the strong allowing the strong denying cases. And I think you've already seen the strong deny Yang that looks exactly like that. So we have basically denied everybody in our cluster from being able to talk to the forbidden forest. So all the egress traffic is denied. So clearly when Milvano-Gonagal tries to talk to the forbidden forest, she gets a tie knot. So this is where our Albus Dumbledore comes in and he says, okay, I wanna delegate, right? So I wanna create a pass action admin network policy at a higher priority number three, which allows Milvano-Gonagal to be able to talk to, well, not really talk to, but to allow Milvano-Gonagal's network policy to take effect. So the pass A and P that you see here is exactly the same thing that I showed on our slide deck. So let's go ahead and do a clip cut to create of our pass action, sorry, pass admin network policy. Let's also do get admin network policy command to make sure it's created and that we have all our three A and P's in our cluster right here. And now when Milvano-Gonagal tries to connect to the forbidden forest, you can clearly see that it passed, right? So it actually worked as in it was successful. The reason is because the network policy in the professor's name space is what took effect here, not the strong deny, not the, it was actually the pass admin network policy which first took effect and then it actually delegated it to the network policy in the professor's name space and that took effect, right? So that's two matches that happened there. So going back to our picture, so this is the same thing that Nadia explained and this is exactly what you just saw in our demo right here. So we have the professors who tried to talk to the forbidden forest name space. It worked seamlessly because we have a priority three pass action admin network policy that took higher priority compared to the deny admin network policy in our cluster, which was a priority five. So we keep coming back to this diagram because it is kind of important and this is how the interaction works. So from the admin network policy layer, so let's say you have an admin network policy where you passed the traffic which is what we saw in this demo, you fall down to the network policy layer, right? But let's say you have no admin network policies on your cluster, in which case, there's nothing matching that layer, right? So then you evaluate the network policies anyway. That's the two conditions for you to fall back to the next layer. And then if there are no network policies in your cluster, then you fall down to the third layer which is the baseline admin network policy. So this is the second resource that we ship as part of the admin network policy API. It's also a new one and one of the use cases and one of the main reasons why we shipped this API is for you as cluster administrators or for Albus Dumbledore in this case to be able to define default guardrails in your cluster. So maybe there are no admin network policies and network policies in your cluster and maybe when you delegate stuff, it doesn't really hit any network policies. Even in that case, you wanna have a zero trust policy sometimes, right? So the baseline admin network policy which is a singleton resource in your cluster allows you to do that. So let's also come back to our storyline here, right? So we had a pass action admin network policy at priority threes, that's where we left off. Now we are creating another resource which is the baseline admin network policy and this is saying I wanna deny everybody from talking to the forbidden forest. So this is the default guardrail that Albus Dumbledore is laying down. In case professors don't have any network policies, you don't know, right? They could just delete it for all you know. So you wanna have the default guardrail layer on your cluster and that is what this shows but still we still have our admin network policy and the network policies in our cluster at this stage. So right now the pass action admin network policy is what is taking effect right here in this picture which is why you still see the arrow being green which means it's passing it to the allow to forbidden forest network policy in the professor's name space. So that's what's taking effect right now in this instance. Like I mentioned, what if you delete the network policy, right? What if the professor's name space has no network policies present? In that case, that's when the baseline admin network policy fondly known as BANP kicks in. So now you're gonna deny all the egress traffic from everyone to the forbidden forest based on the BANP. So it's the point to note however is the admin network policy at priority three is still being hit and then you are passing it. It's just that there's nothing on the network policy layer right now. So you're falling down to the baseline admin network policy and this is how a sample YAML for a BANP looks like where you have, like I mentioned it's a singleton resource so you can have only one BANP in your cluster. So BY is to define it, right? In our subjects part of the API we have selected all namespaces in the cluster and we are saying we're gonna deny egress traffic from everyone to the forbidden forest. Let's also look at our part two demo for the BANP on a OVN Kubernetes cluster. So just recapping the whole thing we have three admin network policies on our cluster right now. We have a network policy in our professor's namespace and this is how our baseline admin network policy looks like which I showed on our slides here. And so let's go ahead and create our BANP in the cluster right now. Let's also try to get the BANP to make sure that it got created correctly. So we have our default BANP right there and now when Minawama Garnagal tries to connect to the forbidden forest she's gonna get connected because we have not really, because we have not really had the BANP take effect, right? There's still the network policy in the professor's namespace that's kicking in. So we have created the baseline admin network policy but the reason why the connection still works is because you have a network policy in the professor's namespace. However, if you delete that network policy, any pass action that's happening from your admin network policy layer is gonna fall down to your third layer BANP and that's why the connection doesn't work anymore. So the key takeaways, right? So let me try to recap it. I'm sure that was a lot of information in the short time that we had. If you are a cluster admin persona, please check out our admin network policy and baseline admin network policy resources and you can clearly see that if you define A and P's, you can either use the pass to interact with your network policies or with your baseline admin network policies if there is no matching network policies, right? And then if you have no admin network policies and no network policies, you can still fall down to your baseline admin network policies. So these are the three layers to keep in mind and the three takeaways of our presentation. Over to Andrew who will talk about how to get more involved in our project. Yeah, so thanks, Surya. I think this was our first introduction to admin network policy at our main CUBE contract. Our overarching goal is to consolidate on network policy APIs. We know in the ecosystem today, many CNIs implement their own sort of global network policy or cluster scope network policy. We wanna bring it back to the upstream and we wanna make it portable for everyone. So how to get involved? I kind of split it up into a couple different levels here because we want everyone. Like we truly need a lot of help. We've had some exciting new contributors recently and they're able to kind of hop in and take some big roles. But even before getting involved, if any of you maintain a CNI or know people who maintain a CNI, we wanna get the message out there. We wanna work together. We have some really exciting new features that are coming in with the admin network policy API that we didn't talk about today. That'll be for another talk that kind of make it even more useful. But for getting involved, I've laid out a couple different layers here. You can kind of be completely new. Start at some of the generic Kubernetes documentation. You also can be a little bit more advanced with that. Hop right into our documentation and even further if you're sitting here and saying, oh wow, I'm a CNI who has a cluster scope object and we are implementing a feature that we haven't talked about here, we wanna hear that. Like we want everyone to get involved. We wanna create new NPEPs which is kind of our version of a KEP. We've designed it after the gateway API's GEP and it is way less painful than a KEP. So it should be a lot more enjoyable process to get done in my opinion. So yeah, I think that's all we have today. Thanks so much for coming and please leave us a review if you'd like to. Take some questions now. Shem, but I did just wanna let you guys know how much I'm looking forward to trying this out. It's gonna make my life so much easier and I just wanna tell you guys so much I appreciate your work. Thank, we appreciate you. That's exactly what we wanna hear. Thank you. For the baseline, you have a rule in there that says you can deny or allow or whatever. What's the default from that? So if you say you're forbidden from talking to the forbidden forest, is everything else then allowed or does it, how does that work? If it doesn't hit a rule at baseline, it falls down to the default stance of the Kubernetes cluster which is always allow today. But what we kind of foresaw baseline is being as a way to kind of flip that, right? So network policy, it's implicit. Whenever you make one, it blocks that pod off in the namespace. We wanted to make everything in this API explicit. So if you wanna do that at a cluster scope, you can basically create a baseline admin network policy that says don't allow anything to anything and then developers and admins need to work together to whitelist or allow list the things that should be allowed for the cluster to function. Right, so I think the major thing is usually the baseline policy is supposed to be used for denied rules. Though the reason why allow action is in there is because we have power these, right? So it's really hard sometimes to make a whole lot of a bunch of denied rules where you can now with the baseline admin now with policy, you can actually stack an allow rule which is really narrow and then we have a universal deny everything rule below and then it would just work as expected but it's just a way for you to more easily apply those kind of policies. Yeah, but you can only have one BANP so you also wanna be a bit careful with how that's defined. Sometimes you can shoot yourself in the foot and deny everything to everything. So yeah, that's also why we don't have priorities on that API, yeah. Or if you have a very nice use case while you may need more than one BANP reach out to us, we're happy to discuss. Okay, it's not a question, thanks for the sessions. So I'm part of the maintenance Andrea projects that you saw here. You're all invited for the dinner tomorrow, food and sweet dinner on here, free dinner, free drink, two, two, one, six, sour, Michigan, jet and cherry. That's awesome. Two, two, one, six, sour Michigan, four PM tomorrow. You're all here, invited for dinner and drinks. Offered by VMware, Binance and Google. They're three talks. We have a speaker here. He's one of the speaker. Go deep and do this. Thank you. We'll talk about admin or posse, yeah. Yeah, so thank you. This is actually very good. I think it's very needed. So I had a question about your earlier comment about user versus admin. So how often do you actually see actual users, quote, users? What scenarios do you see users writing to the Kubernetes API? Because something I've seen in the industry, and I just like to see your input on this, is that there are people who talk to the Kubernetes API and then there are the containers running in there. And a lot of people like to force users to be just in the containers and not know anything about Kubernetes. But clearly this is, you have to know quite a bit about Kubernetes to do it. So I just, as far as that line, I don't know if you have any inputs or thoughts on it. I mean, basically a person making a home package. You might consider that a user, right? Because that's what that is really applying to. But the question is how often, like is there a particular workflows that you've seen where they're like more other than that? I mean, I guess because like a lot of the GitLab stuff and the bunch of stuff I see out there, it's a clear wall between Kubernetes API and the actual application running in the cluster. I think like that's kind of an interesting case for us as core developers versus what's going on in industry on the ground, right? And to be honest with you, like we're thinking of it from when we say a developer, it's sometimes lumped in there synonymously with something that you might wanna call a namespace administrator, right? So someone who's in charge of a namespace and who gives pods or resources to someone who can't talk to the Kubernetes API, right? Does that make, does that clear it up a little bit? Yeah, that's to me, I guess that's kind of like there's one level of cluster admin and then there's maybe some sort of another level of admin. Right, there's like, and the personas are, they do are a bit flexible. And like if we need to be more clear on that, please, please come talk to us. And that's, I mean, it's a great question, really good question. I think a lot of times we have the cluster admins define our back roles for the namespace resource in general, and then each namespace owners can do their own thing is what we actually intended, yeah. Okay, so this stuff is really cool. My question revolves around that new priority that seems like adding a lot of complexity. First question is, do you foresee that adding quite a bit to what CNI providers have to implement? Do you see issues with having to implement something like that where before it was, hey, I defined one set of white listed rules in order to evaluate a policy and now I have a chain to filter. So I think today, different CNIs already have their APIs proprietary and one of the goals that we have is to be able to commonize all of that and be using the Kubernetes native API. So for example, Calico has their own global network policy, CLAIM has their own, so today implementers already have an extra layer, right? And we are actually trying to make it simpler by having just that one API that they have to implement. So almost all CNIs today, like most common ones have it. As it pertains to the data plane, it depends, right? CLAIM already has priorities. That's one concrete example, whether it's easier for, or sorry, I misspoke, Calico already has explicit priorities in their global network policy. Is it hard for CLAIM to do it? Maybe harder than Calico, but it is on a basis level. I think the other part of that question is complexity and use, right? It's gonna be tricky, right? We're gonna have more priorities, more objects with different priorities. You, as an administrator, or even as just a viewer, observability viewer, how do you understand the relationship between those objects? We actually have kind of a commitment to create some really nice developer tooling along with this API. And we have a cool tool called Cyclonus that we've kind of been building off of in the upstream that gives you a really nice table output view of the relationship and what's going on with your cluster. That's a place we need help. It's gonna be really exciting. I think it's gonna be cool for the whole ecosystem, so. And I think if the concern is on the poverty, the scale of poverty, know that right now in the API, it's not unbounded. I think it's like a hundred, zero to a hundred something. Like we can't think of a use case where people will create 90 different priorities in the cluster already, not to mention like say 10,000. So there's not going to be too much of a data-playing trend, in my opinion, for most of the CNA implementers. We just want to control it so that people write policy that are actually making sense, right? They're not tries to put every rule in every namespace as a different priority. That just doesn't make any sense, so. And I think it's kind of like, what I've seen in production today is the proliferation of thousands of network policies. Like customers are rolling out entire systems just to manage their network policies. And we want to cut that down. One more detail to add to that is that it's a bit of moving the complexity to the implementation from the customers because it seems like it is actually easier for customers to use priorities. They are very used to it. They know how to write their policies in these terms, right? So that's another reason to do that. My second question regarding priority is priority unique. And if it's not unique, how do you handle two policies that might conflict or do you have to leave space in order to and plan out eventually what could be, if there's changes and you reorder, you might have to reorder all of your priority numbers? The API today itself is actually, if you define two admin network policies at the same priority, it's undeterministic. The definition, so it's a great question. However, implementations sometimes take over and they do the implementation specific stuff. So the way we did it in the OVN Cube CNI is we say that you cannot create two admin network policies at the same priority. We do not support it. If you do it, it's at your own risk. So we call that out. I wanted to add on that. It's only undeterministic if you sort of like step on own toes because there are use cases where you have to admin now policy, create the same power, but they're regulating completely different traffic. And that's fine because you will never have a conflicting rule with each other. But let's say, you know, cause a pod can have 10 labels, right? So if you are writing different labeled rules, which are essentially selecting the same traffic, but one is two at the same priority, one is allow and one is deny. Well, you kind of like step on own toes. And that will be undeterministic. It really depends on how the implementations choose to write data pass rules for it. But in general, we just advise against doing that. And it's important to remember that rules are also ordered within a single admin or policy. You can write multiple rules at one priority level and they're explicitly prioritized in that single rule. So we hope to avoid that as much as possible. And we hope to write really good tooling, really good user documentation to avoid it. Thank you. Great questions, by the way. Thanks everyone, that was awesome. We have the SIG Network Policy API intro and update stock at 3.15 central time tomorrow. So please join us if you wanna come and know more about that. Yes, we want contributors and help and implementations. A lot of cool proposals that we are trying to add to the admin novel policy. So thanks everyone. Thanks. Thank you.