 Okay. Thanks everybody for joining today. This is the CNF working group weekly meeting. Thanks for joining. Things that will be going over today is the process and the structure for ideas to become adopted best practices and kind of jump into some of the issues that we have. And so the read, I guess before we jump in is there anything anyone else wants to add to the agenda. Okay. This is sorry this is Ruben speaking. So quick question. So the goal is really to discuss how to turn lots of the discussions we had into something more actionable right. Yeah, that's correct. And then that fits. Yeah, so I guess Ruben to jump right off of your comment is, I think what a lot of people have been seeing is that there's been a lot of conversations flying around both in this meeting on the slack channel and various different ways but it's been a little bit hard to make them actionable and in terms of how we want to move forwards with them into kind of like proposals for best practices. So we do have the, like, the best practices like, like, or like proposal template, but as so far, no one has really like open that open that up and so what we're thinking of the actual process is kind of like providing like a smoother onboarding and we're just kicking off discussions here in the CNF working group discussions board, and we can actually see a couple different, like topics on here. And so best practice like CNF should not rely on local persistent storage for data retention, best practice keep application configuration outside of the image. There's probably a couple different discussions around like defining the telecom actors. And so the idea here would be that first to start the discussion in a more informal way around the best practice is we would start start a discussion thread. And so what this would allow us to do is to basically have a written way of to look at the different best practices that we have rather than someone to have having a complete idea formed. And what this allows us to do is to kind of iterate on the idea before we need to come up with a more formal proposal. And so, in this way, all the ideas around certain best practice can be kind of contained in one place. The group thinks that there's enough information discussion around a certain best practice, then a more formal proposal can be made. So if you think about it, kind of like the, the proposed process right now would be first, like either would be starting with something on this discussion board, creating discussion discussing it in this meeting, generating enough ideas and interest, and then the next step would be actually defining a actual proposal, and in this way, instead of having a whole bunch of conversations in Slack that you'd have to search through each best practice kind of has something behind it. And then obviously the final part would be reviewing and adopting the best practices. So, kind of with that in mind, what do people think of this newer kind of like less formal process. Oh, the, just to add a little bit to it. So the, I don't know that every, every topic in here is going to become best practice, especially ones that are meant to be like defining the talk one actor so any, anything that we're discussing is going back and forth and we keep asking, let's try to get it in so that everyone is getting those seeing the conversations, whether those are on the meetings or in Slack, it's fine to keep having them. Of course, any, any area that try to capture something so that we can have that shared understanding. That one of the main reasons to get it in here besides making sure that everyone can see it is to make the hurdle for, I guess, sharing the ideas broader is the formal adding a proposal may feel like you don't want to do it yet because you don't have all the information so to lower that hurdle for that. And of course, if anyone see something that they feel like there's enough content. I think that we can put something forward we have all the references we have the use cases we have all the stuff that we feel is necessary to communicate that this is a best practice that we want to put forward, then open a PR. Do a PR an issue, put it forward. We'll still have discussions just because it comes out of here doesn't mean we'll stop. It's just lowering the bar so that we have a shared area. Hey Taylor bill. Hey, so one thing I would like and it's going to be contentious and rough. But I think it's necessary before we start defining best practices is we put in our charter that one of the things that this group would do would put forth our definition of what a CNF is. I think that's really important. I know I've talked to you a little bit about this Taylor but I'd like to, you know, put it in front of the group is if we don't have a definition for what we're proposing best practices for. There's a lot of ambiguity that can get us into trouble. For instance, like, you know, there's best practices around the amount of privilege that a container gets, and we could get into specific, you know, nuance discussions around least privilege and the container having exactly how much privilege it needs. But we talked about like these kind of like, you know, least privilege, you know, blanket statements. If we don't know like whether or not the orchestration for say the infrastructure comes along with the CNF itself if we don't say that you know those are two separate things. And some CNF developers put their orchestration pieces in with the CNF among it all together some don't begin this weird thing where we say it's a best practice to not have, you know, privileged containers. And all of a sudden we're in a scenario where we have a container that's you know an orchestration platform that's been quote unquote labeled a CNF, because it was you know just bundled that way. And it's now you know breaking a best practice because it's provisioning block storage for me or something right like, so I think we need to make sure that we a define, you know, what we're saying this is a best practice for you know is it a single container, is it a collection of services like everybody has a different opinion, but it would be nice if the CNF CNCF gave us like a, this is what we think it is. And then from there, we can provide context in these best practices to the type of workload that the best practice applies to is. I mean obviously there's going to be times where you want a container to do certain things. So we can just make it, you know black and white by saying this is for the CNC. This is for a CNF, and we've excluded these types of containers from a CNF, and then therefore you're not breaking something with the best practice like, you know we're calling this an orchestration container or something. That sounds good. Jeff, I think that there was something that you talked about last week. In chat about exceptions to so that context so this would be tying in with what context do we need to know about a best practice. So that you know how it applies. And it seems like you're related. So, so yeah, context, how it applies and then I mentioned exception and then I'll let Ian speak to it because Ian was actually talking about like a formal process he's used in the past but the big thing is right is we don't want something to instantly be disqualified if it's a really good idea because it breaks a current best practice and maybe that best practice wasn't exactly supposed to fit to what that specific container was trying to do so. Yeah, exceptions in context I think are super important, because if not we end up with these really broad brushstrokes that either limit what we can do, or at least things so open to interpretation that you know we could both be quote unquote following a best practice and have completely, you know, polar opposite results. I mean, when I did this in. Well, it wasn't say it was safety mitigating which is a little bit of a get out clause for the companies I were working for but safety mitigating software then what you would tend to do is you would have. If I put it in the words we're using here you'd have a set of best practices and then you'd have a baseline of these are the best practices I'm applying in this particular circumstance, for whatever reason, not necessarily all of them. And it meant also that you could potentially have conflicting best practices something was a best practice in the circumstance but not generally, which meant that you you comply to a baseline which would be some subset of your best practices or maybe a bit of context about how they were applied, which meant that you didn't have to get it right. Well, among other things for us, it would mean we could evolve over the course of time the best practice list would not be a list that we couldn't change because it was already in use we could come up with a V2 or a context specific list for certain types of application, or certain circumstances. And I'm thinking about this. Because, you know, it's nice to think here we're in a relatively academic environment where we're talking about stuff that doesn't change the world. Or alternatively the world will just naturally comply to it whichever way you want to look at this. But we know that what's going to happen here is that whatever we write down ultimately somebody's going to throw at me or another vendor as an RFP. So we have to ensure that effectively it doesn't stop us doing the best job we can do because there's an awkward bit in it that just turns out to have been not a good choice in in practice. This seems like the way to make sure we could potentially evolve so that the bits that become, you know, turn out to be practically difficult, or not as valuable as they looked like we can throw away over the course of time. So the idea that we have best practices. And also we make a baseline that says these are the best practices that get you to, you know, a 2020 gold standard CNF. If we keep those two things independent of each other we'll get I think a little bit further, which can also be read as saying, you know, if you've got a best practice that you think is, you know, the best we can do today, as opposed to the best we might ever do, or alternatively a best practices that doesn't apply in every circumstance all the time. That's still not an excuse not to write it down. You should write it down put it in, and we will find the moment to apply it and if it turns out it gets outdated, we will have a means of saying actually it turns out we won't be doing that long term after all. We can get more ideas in there and we can choose our baseline it's not we haven't written something in stone it's such that it can never, it can never be altered. I'm just trying to capture what you said either that's a horrified silence for everybody else or you will think I'm absolutely genius and you're speechless I don't know which of the two anyone care to come in. I was wondering whether you could recommend because I think we just have to start and do it. So my question will be more whether it is whether there's any any recommendation to start codifying this this basically coming on the one hand from the definition of a CNF. And on the other hand from codifying how to come with best practice and probably will end up meeting in the middle without chopping off our respective ads. Yeah, I think. I mean, Taylor put something up which I did like the definition of audiences which is context. And we definitely want to get that context written down, because best practices relate to context you can't have a best practice if you don't know what it's a best practice for. So we have to approach this from both directions at the same time and then that baseline will be saying, you know for our understanding of context for this list of best practices we recommend these ones right now. So effectively, three tracks. One is the context behind the CNF how it's going to be used who's using it what matters to them, and what audiences we have. One is a set of best practices that are probably more related to the software engineering practices of working with cloud native applications and using containers to implement them. And then the baseline that we'll eventually put together which is going to be the standard we release and tout to everybody and say you should be compliant to this. And then there is where to start or where to focus. And my view is wherever you have, wherever you're reading something and it seems to affect you as probably, or whatever's interesting. So we don't have to stop, start with any specific ones. I, some of these like this actors is going to affect everybody but so please at least review it. What a CNF is but maybe we should start with what a CNF does before we talk about how it looks like how it's supposed to behave what are the best practices around it. What does a CNF do what qualifies as a CNF Indian without putting in a definition for for the world but what qualifies as a CNF in our view. And maybe we can, we can start from that because once you know what it's supposed to do you can start working towards how I think that was Christian bill and I agree. Like, I think that ties into the, the requirements thing right like, what do we need us like, not just what does a CNF do but what do we want it to do right like, why am I sticking this in my network. Because what I don't want to hear is that a CNF is a VNF that follows cloud native principles. Like, and I've heard that, you know, spoken as can and more times than I can count but like, you know, Christian hit the nail on the head, like, you know we keep coming like why are we doing this and I mean we all know what we have to make this cloud native transformation so like, we get into the whole motivation discussions I mean it's common vendors are making it this way service providers are deploying it this way like it's just whether it's a chicken or an egg scenario it's happening so like, we need to start identifying like our requirements on like how and why we're doing stuff right like, what is this CNF supposed to provide us, you know, from a developer standpoint from an operator standpoint, all those people that are in the actors page like, we all collectively are saying this is the path we're doing. And, like he said, what does the CNF do. And then from there, we can start to describing, you know, what it is. Since we're talking about network here, can we summarize it as CNF moves packets around from a to be with that qualifies a very, I don't know 10,000 foot view of CNF. And the data plane though what if it's like a control plane or a management plane portion of it, it doesn't move packets. How do we modify that. I don't know, but I'm just pose playing devil's advocate. Well, that's why I asked the question and I was just reading through the monster discussion that button chain we're having on the subject of, you know, the fantasy that I think we all subscribe to we all wish we were there is. Obviously enough from whoever supplying it with it's a vendor or it's another team in my organization, it really doesn't matter the point is it's an independent unit that's not me. And I want to get it on my network in 24 hours. How am I going to do that. The practical problem that you've got there is that you're integrating it in a wider system and you would like to make sure effectively all of its interfaces work the way they're supposed to do all of those touch points. And then you actually meet with the expectations of the thing that's consuming them. And then you'd like to put it through its paces as much as you possibly can so it's got to effectively be easy to test, which means it's got to be in turn easy to deploy easy to upgrade this sort of thing. And for me that the way to determine what requirements exist is to actually put this in the context of problems like that this is the thing that I would very much like to do. And I'm assuming that even if I don't get to, you know, a 24 hour deployment cycle or a two minute deployment cycle, whatever it happens to be, then at least traveling that journey will give me something that continually gets better. And what is going to help me on that journey. And then you get to a definition of what I'm looking for from a CNF and then you get to best practices in the CNF. But doesn't that differ from one type of application to another. I mean, in the end, why would you want to put a minor change in your network or a new application in 24 hours. Yeah, fine. But 24 hours one second the point is fast right. I have a critical security update I want it on my network, and I don't want my network to be less stable as a result. That is effectively your aim. Isn't it. I mean, is there any case where that's not true. Well, you do that in basically when you say 24 hours for an application that I want to patch. That's lifetime. If I want a new application that I have no idea what it is and I want to put it in my network in 24 hours, then I do have a problem. But if I want to patch something, there's two different workflows here. One is patching so rolling out an upgrade in 24 hours, which is fine. I think we're beyond that and we should target for a lot less but fine with it. Don't get tied up with the number. The number was just the important thing about the number is it's less than three months, which is not an unusual length of time for a physical device. So I'm just picking a number out here. That's all. Of course, the point I'm making is there's two different workflows. One is patching, which is driven by some considerations like security like compliance like bug fixes and the other is new deployments. So when you do have a patch, you're kind of assuming that you have that application in production already and you're figuring out the qualification mechanisms and upgrade mechanisms that should pretty much be allowing you to do this in line in place without any sort of service interruption or service interruptions and the questions are around how do I qualify this against my previous baseline? What do I need to test? And what ensures that I can move forward in the steps? And what are the checks between various canaries that I'm going to do until I reach a full out rollout? And then if for some reason that fails somewhere in the intermediate steps, how do I roll back? And how do I ensure that everything rolls back cleanly? Now, when you're talking about a new application, that's a completely different workflow because you have to understand how that application integrates into the wider ecosystem. And that study and the definition of those, I was just looking over one of the discussions and one of those is around dependencies. You don't have to worry about them or you have to worry about them in a different way when you handle an upgrade, whereas you have to worry about them in a completely different way when you have to instantiate something new. In one place, you want to ensure that you act on the components in accordance with that dependency so that you don't break the dependency chain and you end up at some point causing a service outage. You want to identify what are the dependencies, who's going to, what am I going to consume with this CNF and what am I going to provide the CNF. So the two workflows are in my mind at least are radically different. I don't know whether they're radically different because you've got to be at the end of the first with the workflow you're talking about when you've eventually worked out how to deploy it for the first time. You've got to be certain that you're ready for that second workflow that the short duration upgrade. Of course, you've got to, you know, it's not just about getting the application running in in your network. It's about understanding that there's going to be another copy of this application turns up, you know, next Wednesday, and I'm going to want that in my network as well. So at the end of the first process, you're going to have to be prepared to start from the second. So it's not, perhaps we're not comparing light with light. They will absolutely be different, but the first one has to end at a point where you're prepared to start the second one at any moment. So, okay, we want to tackle both at this stage or should we pick one like let's pick the incident one let's pick the one that we start with. So if we can model that and understand it and then we move on to the second one which obviously has slightly different behaviors and maybe we'll understand something else after we model the first one. That was the original. Yeah, no I see what flow or should we look at the instantiation workflow first. What I've been saying here is a specific workflow and you must follow it. I think what you're talking about the needs behind those workflows is more important because we're not saying we're going to copy over what's there and add to it. What we're looking at is, if you have something like we need to get a security patch out. So on the life cycle management or you already have something out a application running, you need to get a security patch out. So what are best practices that help with that that one goal so that's one thing that you want. You have something else where you say a misbehaving app is running and we don't know what the problem is so what are best practices that help you observe and get information about that application best practices on Kubernetes. So those are the underlying concerns so whatever level you're at a new deployment, a running deployment, rollback all of these things, what are those underlying needs. And now we can start looking at what are best practices that already exist in Kubernetes that are applicable. These are all ones that we should invent because we need them but all I would say is there isn't necessarily anything stopping us from writing both of those together, but what you're saying is that what I write in one is necessarily going to have some effect on the other. I don't think that stops us making the attempts simultaneously we just have to understand that the first try at this, you know that there's going to be a revision process it's going to take some time to get to a place that we like. I would write both of those and it was what I was saying earlier this is these things individually these are not best practice these are context effectively these are the things we would like to do this is the context that will determine what best practices matter to us. So it looks like we seem to be going all over. I thought that Jeff started the discussion with the CNF what is the CNF. And now we are we are going into all kind of workflows and the best practices for for workflows and whatnot. And we completely drifted away from what I see enough is and and the, the point which I wanted to make for a while ago, as soon as Jeff started a discussion that was Taylor, during the time. Didn't you define or did didn't we define those CNF where we call them bronze silver and gold and whatnot. And there was an attempt to make that we're what what these CNF are what do they look like what do they have at least, at least an attempt was made there. So where we're not picking up from there why we're trying to reinvent all over again. There was I mean, so people still will I mean I said into a lot of meetings, and I hear people argue about it all the times that guys VNFs are not going away. They are here they're going to stay here they're going to stay here for a very long time, and, and vendors need to see an incentive to throw away their VNF implementations and start getting into CNF implementation with not even knowing what a CNF implementation looks like. If I, if I get into room I bet you right now we have a 40 people or 30 people whoever we have on the call. If I asked, you know, that what is a cloud native, and we're not going to get one answer we're going to get at least 20 answers. So, and I do that all the time. So why are we, why are we not starting with there and I thought tug was doing that. Tugs, tugs started to do that, especially when they got into gold CNF. No, so they were bronze they were silvers, but the gold CNF gold CNF is pretty darn close to what we are trying to look here for what a CNF should be. So, so why do, why don't we start with that. And then, and then at the top of the hour when Jeff started he started with, with CNF, and now I think we've gone to security patch installing and we've gone into upgrades of workflow. So what, what is the goal of this group. What, what are we what breakfast practices are we at the end of the day, we can meet every week and we can talk all kind of different things and we can even do whatever we want to talk. And by the end of the day we have to set up something out of a goal that what what we want to achieve, so that we can set up some time frame, like by by q one end of q one, we want to be able to achieve this, and by end of q two, like Robbie is on this call, who was leading CNPT. And the only reason we made tremendous progress there was that he will step up and he will start setting up the goals. You will say, Okay, guys, let's define this release. Let's define this content. Let's go with this and work with that. So, so here my suggestion is, we should take some kind of similar approach. Right. And let's start writing it down. Let's start defining it. But then, then we take it as as it comes rather than because everybody seems to be wishy-washy everybody seems to be going all over depending upon what our personal preferences are what our backgrounds are where we come from. Right. So, so there's a lot of bias based upon what what we have been exposed to what we've been using to, but but there are certain things which are in front of us. So whether we pick what is available and then start to work from there. And my suggestion is that there are write ups on on what what gold CNF are. Let's take a look at it. Let's review it. Do we agree? Do we not agree? You know, if we don't agree, why we don't agree, let's fix it. Let's call it as a CNF. And then, then we've communicated on the goals just to I agree on the tag side. We've done a lot of discussions about all of this. There's a lot of information. Geoffrey's posted something there. That's now that's what's used in the text. So that's kind of the starting point for all of this. The principles to go with what you were saying as before. Yes, there's a lot of people that will say a lot of different things. But if you go and look at the I'll say the experts in the field. Whether you're saying specifically cloud native, whether you're going into Kubernetes usage, whether you're going to stuff that builds into cloud native dev ops and wherever you want to look. There's a lot of content that go right into what the principles are about. It happens that CNCF doesn't have those expanded out directly in their definition, but it doesn't mean that the experts that put forward these haven't said a lot. So this is a good place to start if we want to look at some of these things, but I don't think Geoffrey was specifically looking to say, do we have any idea it was more of are the best practice that we're putting forward going to be applicable or are we going to cause problems. So it's a more specific thing. And then as far as the goal of the group, we've kind of communicated this again and again it's in the scope for the charter. We're looking to determine what are these best practices that are going to allow the telco applications to most efficiently use Kubernetes. And they have, why are we doing this? We want to lower OPEX primarily, I would say, but it probably affects CAPEX. We were talking about life cycle management. So how are we going to lower the OPEX cost, whether that's reducing risk, whether that's making it easy to manage changes on the network because you push more stuff out to the infrastructure that already is handling some of those things. So if you're building an application that you're selling to a SP, you know that you're focused on the part that matters the most that you have the expertise on and you're not having to do additional work outside. So it's all around those same things as far as the reasons, but the goal, that shared goal of how to best utilize Kubernetes, the framework, the community. So outside of just our group, how do we bring more and more in that already have the knowledge on running these applications. So in effect, the goal is identify best practices. You just summed it up. It's not defining what the CNF is, but we kind of settled the CNF is what was defined in the talk and the goal of this group is just to identify best practices. And this is something that we kind of went all around the place before. But that's fine. If that is the goal, then yeah, we can focus on that. Where is very clear stated that is where is the tugs definition of what a CNF is and how extensive is it. I think it's right there in what Jeff put. And so, I think to both Christian and and the what the reason why we're bringing up CNFs a what is a CNF or what are the goals of CNF or all of these things are because we need context the same same as everything else we're talking about during this call for why a best practice would be put forward it at all. How does this affect me, is it going to cost me problems, is it going to benefit me. So it's all context so we do need the definition, and probably it's pointing at it so in, I can't right now drop a link and that says here is the type definition which is a problem. We should make sure that that's very easy to find on the tag, and any type of context and content around it, we have all of that so that'll be something that I'd like to put on at least my plate and if anyone wants help to look at. And then we can do the same thing from the CNF working group where we go. Here is what we're referencing. Along with what Jeff is saying, but what we probably don't have anywhere else is some of the context that Jeff was worried about where we put a best practice forward. And it doesn't apply to a CNF. That's very common so we have something out there, we put the best practice, it only causes problems if you use that best practice on the CNF. So how do we communicate that that it's not either not applicable to this one you shouldn't use it, wherever, however we deal with that. I believe Jeff the main thing that you're wanting to get forward on this is, it's not applicable. You got it spot on. So I just confirmed my own fears. I did not speak with enough specificity with my initial discussion thing and so then lots of room for interpretation which mock things up. And I think what Jeff said on Taylor's like, would a CNF that's specifically targeting layer two, and potentially, you know, interacting at like the Nick layer going to have the same best practices is maybe one operating at layer three, and is you know looking at rap tables, etc. I don't really interested that right now just hypothetical right silly. It's exactly what you said is, you know, do we expand upon it, and ensure that we, we just give like the proper context for when we say best practice of like, this type of workload because it's going to give you the optimal results based on what this specific, you know, container function, whatever it is trying to accomplish. And then, if we need to expand definition, we can, you know, if we need to contract it whatever but like the big thing is if I just say no, no root, you know, privilege in any containers. Well, is that always the best practice. It might be that the cases it is it might be the case that it isn't and I just want to make sure that I'm hoping that this is a little bit more lower layer than like the more abstract stuff where it's like here's a reference architecture. You know, here's a white paper like my hope is is, when I come from this group I have things in my toolkit that I can actually go and you know pun fully intended put into practice. Versus you know just abstract concepts that I'm like okay you know these are great guidelines that I'm going to follow it like follow that like a broad level actionable versus just thought something to think about. Do you have something concrete or is it abstract. Yeah, exactly. So for looking at an example. Okay, specifically is this is this like the type of content that you're looking for like CNF container should run on privilege. Yeah, absolutely right and once again though it has to have the context because Ian's concerns are valid. I'm going to put this in an RFP, you know, are these containers unprivileged or are they going to try to mess with my, my infrastructure. Are they going to try to talk to the kernel and do things that I might not want to do. If we have the context and I would try to tailor that RFP to make sure that once again we're calling a least privilege model versus a no privilege model. But you know this is something I can test to like I can go in, and then I can try to like go into one of these containers and try to execute system commands that I shouldn't be able to do I can test this. And then I come away with, you know, a slightly more warm and fuzzy that if I deploy this container, I don't have to worry about it going in and doing things that maybe I don't want it doing in my infrastructure so I do think that you know discussion number 25 here is a great example of the type of thing. I personally and maybe not everybody feels this way is trying to get out of this endeavor. Another thing, again that you can, we may be able to steal from my past life is when we were doing compliance in those days, then there were a set of rules, largely around right software that, you know, doesn't break in horrendous ways and kill people. And there were a set of exceptions you could, because no one's assuming that a rule applies blanket across the board perfectly, you could absolutely write a set of exceptions but there was a formal process for writing the exception down, explaining the exception and supplying it along with the application you were shipping. And, yeah, it's a get out clause. It basically says I can be compliant without being compliant, but on the other hand, if we had a best practice along the lines of this is how you document why you're not following a rule in certain circumstances, how to identify that how to demonstrate its and so on and so forth. And that also might work for us. Because Christian right there on the screen and put in an example of a perfect idea where like, if I'm, you know, and this gets into the whole other best practices on the dev side might come out of like do I compartmentalize. So there's not one giant homunculus container that has all these things because it's doing like layer one and two stuff it's doing layer three stuff but you know he puts like thing like if we decide we're going to go down an SRROV path, I spin up a new container. It's going to have to, you know, be able to interface with the system and pull that the FID into it etc right so that's Christian put like a great example of the types of stuff I'm just curious and like 99% of the time, it's going to be applicable but here's an example of where it's not so we document it as an exception and we provide the context of what it is a best practice. One other thing that for me personally is a pain point is the dependencies. I think Jeff started as discussion or it was started from a thread and slack, but one of the biggest issues I have when looking at a CNF or a VNF is what are his dependencies who does who needs to talk to this CNF who needs to consume it. What happens if it doesn't. How do I ensure that its dependencies are before I started. Today we use a very wacky work around using staconities from OpenStack. And that kind of, I mean, most of the time that works, but when we're starting to look at layer 2 VNFs that falls apart because it's not able to probe layer 2. So can we think of some sort of a generic interface is anyone else facing this dependency problem. And is it a real problem or it's just something that I have. Can I pull up out that word dependency for a second. Are we talking about system interfaces that all CNFs must consume as in that there's a common dependency. Are we just talking about a relationship between two CNFs or something like that. My idea here is more around relationships not interfaces to the system but when you have 235 CNFs on the same Kubernetes cluster, how do they know about each other. How can we do maintenance on one while not impacting you ever how can we suspend ones up operations or how can we let's take an example from the real life world. If my upstream link is down I want to ensure that I put the ports on the switch down so that I immediately take the entire data path offline so that my applications can fail over to the active one. Can we do something similar for CNFs. Can we inform one that its dependency went away. So it shuts down and commit shifts over to its standby. Christian, are you talking about a, I'm going to call it a product of vendor sends a specific type of application to handle things it may have many different components and parts. Do they all need to work together. It's more about multi vendor let's say I have one component from one vendor another component from another vendor and the upstream component needs to inform the downstream one. All right, that in a generic way, so that I want to swap vendors from both upstream and downstream and not have to worry about the fact that I'm going to need this interdependency to work and those vendors need to be certified. I want to, can I do my qualification 24 hours or not. If vendors respect that interface, then it's easier. So if you, yeah, and you've hit the nail on the head here it's not about the two pieces of code. It's about do they respect the interface. So you would, you know, I'll use BGP is a fairly traditional example here in theory, if not in practice, every BGP speaker you can pull off the shelf will talk to any other BGP speaker you can pull off the shelf. And you're already 95% confident that's true because they both talk BGP and you can check that they're compliant with the standard before you even get to talk to each other. And what you're talking about here is kind of the same thing. You want certain link up down behavior from a CNF, and it doesn't matter what's on the other side of that link, whether it's another CNF or your switch or anything else. The behavior is something you can document it's the way it must respect that interface. So something a bit more generic so not let's forget the network world but say my control plane goes down and I can no longer manage my data plane. I want the whole thing to go down so that if the control plane has a BGP session with an upstream data center gateway it can fail over somewhere else. Basically, let's think about this in a more abstract way than just BGP. For sure BGP can solve this at 95%. I was using it as an example. I wasn't sure. I mean I know what you're thinking that you could play with roots that's not my point that's not the point I was making I was saying that BGP is a defined protocol so rather than defining anything else what I'm actually defining is the place where these two potentially huge pieces of code touch. And because they touch in a very small area, you know the protocol that is BGP, then you can document the protocol and you can be almost certain. Before you've ever brought them together on the same system that they're going to work together because they both respect BGP and you can demonstrate that. I can do this with API is the same way. Theoretically, I'm struggling to think of a fine example of an API but the idea would be there is an API definition for a. All right, let me take a weird example but it will do for now right. There are many implementations of cron out there, but I don't have to know the details of each implementation because ultimately, they respect the same CLI, I use them in exactly the same way. So regardless of which quantum cron implementation I happen to be running on my box, it will do the same job if I tell it to do the same thing so all I have to document is how I use it know not what it's doing behind the scenes. And so theoretically if I write strict the programs cron and I and I install whichever version of cron I personally favor on my box. I know these things of 95% likely to work together because the API definition the interface is what I've written down. And so in your case you're saying are there some generic interfaces. I know interfaces are loaded term in networking but you know what I'm saying here. Are there some generic interfaces I could write down where specific kinds of touch point could be documented. I had in mind something a bit more it side. This is from Juju from Canonical, which has providers and the consumers kind of the interface. So it's a declarative way of saying my VNF provides load balancing services. And my web server is a consumer of load balancing services if I tied those two together I know that the load balancer will load balance traffic, coming into my web server. The perfect example of this of the proper implementation is the ingress controller. I know it's crappy everyone hates it and I'm streaming and wants to rewrite the ingress interface but it actually does what what it's supposed to it says the ingress controller provides this kind of service and I'm the consumer of the ingress. For sure one of the issues of the ingress controller today is I cannot tell from the ingress controller to the back end that the ingress controller is down. The back end has no way of knowing it's no longer receiving traffic except for observing the fact that it's no longer receiving traffic. But it's a it's going along those lines of declarative dependencies where I have one component offering a service and another component saying I'm going to consume that service. Yeah, it's a contract between the two exactly. That's where I was trying to get. We need to define this guy or at least in my mind I think we need those kind of contracts and as a community we need to agree on those kind of contracts so they can be honored by different vendors. Yeah, I think there are a few other really boring contracts that we need to remember as we're doing this the one that keeps coming up with me with my grot lot is logging right I know I absolutely am 100% certain that whatever application I write with it's a CNF or anything else it's going to spew logs at something and something's going to want to pick them up there needs to be a contract between the two so that way I send logs and the way it receives logs we both agree on how that's going to work but it's still a contract. Yeah, that's, that's a perfect example of logging is really painful today we in my case I have two types of logs. Basically, one structured logs and Jason structured logs and figuring out which goes where and telling log collector, this is Jason structure this is not easily painful. Today my log collector really needs to parse land by line figure out if it's proper Jason tried to interpret the Jason otherwise it falls back to just sending the message on parse. And this takes enormous CPU cycles when you're dealing with text. Just a question, would it make sense to start thinking about the table of content that need to be created for the artifacts regenerating I mean this discussion are great. I think, if we try to convert them into pro request instead of just discussions that might be better focusing on the attention effort into making something more tangible. So is there any table of content being created to define how the structure of the release will look like. So, the thing that we have right now is these proposal like definitions. And so this is similar to like a Kubernetes cup, where like each one would define like a best practice. And so we're just thinking of seriously numbering them but if you think there's like groupings or something else that we could have. Feel free to open up or press Robbie was there a specific idea that you had. I'm thinking, like to work backward from the delivery that would like to make now if we say, we're going to release something as part of the outcome of this working group what would that look like is that document with a certain structure, different sections. And if we say okay, if this is the overall definition of what we're going to deliver. I think going back to the discussion we had in the slack channel. In the last time call, we decided to look at the guidelines and best practices for the CNF developer himself. I think if we just focus on that and think about what would the content be look like and start with some real content in. We will be able to identify the relationship with other parts of the platform for time to think about what other areas we need to work on next. I think that is this discussions are great, but I want to be very clear and blunt here. I mean those discussions does not really lead into tangible outcome of something that's you can actually refer to and point that. I think it all starts in my mind if we create that structure on and document structure with the different chapter sections wherever it is. We can work in particular section of particular area, even if they can have an off all or something that for those want to see to work more into the technicality of that part to be more like tangible outcome that we can lead to. Probably that's great. We didn't go into all of that earlier I guess when we're talking about structure what's next beyond the process to get to a list of best practices. What you're saying is is needed so if we determine a set of best practices that are helping to most efficiently use Kubernetes. What is that mean so one part is what Bill is saying is similar to caps. So this enhancement proposal, but the other part may be more similar to the Kubernetes conformance program. So if you go and look at any new release where they say here is the version of Kubernetes and what tests are required for conformance. There are only tests, but you actually have the test and why they were there so why is that feature part of Kubernetes. That's not immediately apparent if you just look at the test you'll have to dig in, but that's a similar thing here. So if we end up with a set of best practices that you can go and look at. And in this folder will probably have something similar to what the Kubernetes conformance which is a large document which list all of the test that are required and then you can click through and go look at what those features are for us it's probably going to be. Here's a list of best practices in a document and some maybe a subset of information so you can quickly see here are the ones that we're recommending that people use and the audience and and just a small blurb about why it's good. Click here to go read in details about this best practice that would be similar in my mind also to 12 factor apps. So now we're just helping people do better. But beyond that how are we trying to be on the document like that which I do think is good 12 factor, if you go look at whether you're looking at 12 factor, you have a list you can go dive in deeper, and so on. Beyond that is the test suite. So the test suite is currently developing and working on based on feedback it's already getting so talking to vendors talking to Kubernetes folks looking at stuff that seems like good things to test, but one of the things it'll do similar to Kubernetes so Kubernetes is writing tests all the time. And then eventually they come out with the best practice or not a best practice they'll say this is a conformance so this is a required feature this is something we really think everybody should have for interoperability. So if we come up with the best practice, it may be a one to one to a test, or it could be something like the idea of least privileges for CNS, and then a subset where we go here is something to do that. So now we can say what are some things that we can actually test. So the test suite is primarily to allow CNF developers or we'd say vendors to go and run the test suite and see how well they're doing on these best practices that they think are also good hopefully so that they can improve. Maybe it's about storage data or how you store data and cloud native way, but those will be working hand in hand as a result of the best practices. So Robbie are you interested in helping created table of contents. Yeah, I'm happy to have a look at that yeah. Okay. That'd be great. And I'd love to have that discussion when you're ready to have it. I think other people would, I think welcome the structure in it too. In the meantime it'd be great to move forwards our discussions in the discussion board around the different best practices that have been pro so far on the definition of a CNC of a CNF. I know we're at time. You know, we are having one meeting and then we're skipping the next two weeks and starting again. Not on the fourth because I'm assuming everybody's just going to be back into the office, but our next meeting after that will be on the 11th. So, thanks everyone for your time today I really appreciate it. So there's a meeting next week and then meetings for two weeks. Okay. Yeah, exactly. So, thanks. Cheers. Thank you.