 Okay, let's go ahead and get started. So item one on the agenda as always is agenda bashing. So if anyone has anything to add to the agenda, now's the best time to do so. Hello, speaking here. So I'm new on the group and thank you guys to inviting me on short notice here and I'm happy to join. And it was an idea to have five to 10 minutes about a use case we are doing with a more left-homed or on service mesh, or on a group of service mesh, if anyone interested. So we can probably add a spot on the use case mapping below for you to talk about. I think that was already added, right? Cause I think I saw someone put that on there. Exactly, it was a quick hack just four minutes ago before the call to add some use case drawing to the use case map. It's not final, obviously, it's just sketched in there. But if we want, we can share use case even as a call here and talk about what we're thinking or what we're doing for this stuff. Yeah, yeah. I think that would be good. I think that would be good. Yeah. Yeah, and of course, last week we thought probably we'll have a detailed discussion on the use case document. And I think this comes handy for you to present too. That'll be great, yep. Or talk about it too. And Prem, I agree, I was hoping that this, we might spend the majority of this meeting on the use case stuff. Right, right. Yeah, I personally think that right now the most important thing that we can focus on in network service mesh is working out what those use cases are because that really will drive the development. So I'm happy to devote the majority of time to use case and use case discussion. Okay, so is there anything else that needs to be on the agenda or should we continue on? I'm good, if everyone else is good. Nothing for me. I think silence is good to move on. Okay, so the next thing is there is a, for those who are attending the open source summit in the end of August in Vancouver, there is a cloud native network function seminar that is gonna be held the day before the summit starts. So the summit itself is on Wednesday through Friday on August 29th through 31st. They hold a couple many sessions, I guess you could call them, or workshops on Monday and Tuesday before. And so the Tuesday before is the seminar. So feel free to join in. I believe you have to register for it when you register for your open source summit bus. And as a, and one of the topics is going to be about the network service match, or at least there'll be some discussion on it. Okay, so I'm trying to understand what this is on the agenda, AIs or Al's review, one to one or the other. So, Oh yeah, AIs. Yeah, so for those first two, I actually opened up issues. I looked this morning and it wasn't clear who was gonna do that, so I just did it. So I put links to them there too. I do apologize, I have not gotten to the AIs assigned to me from last week about getting the namespace and fry inside the pod. I will go ahead and get that up there on the wiki. I just haven't had a chance to do it this week. Okay, so much better with it than saying action items. Even though we do have an Al on the call, I do feel the same. So I apologize for that. So anyways, so the action items, so at least as far as I know, nobody's gotten to verifying the ability to change the CNI driver. Certainly there's been no changes on it in the past two hours if the issue was created. But I'm pretty sure someone will get to that. If anyone wants to take ownership by all means, do so. Let's see. I should probably be a good idea. If anyone wants to take ownership, add yourself on the agenda and we'll assign it to you on GitHub as well. And to be clear, this is an open source community. We all get that all the actual items you take are inspirations. And so don't feel like you're signing in blood when you sign up for a thing. Just do it if you can and if you can't let folks know it's up for grabs again. Yeah, I'm perfectly fine with that. And in fact, they'll be surprised if you take an action item and then someone else completes it on your behalf. Yeah, that's also totally cool. So, probably won't happen for the more complex ones, but for the simple ones, certainly can happen. Okay, so let's see, open a GitHub issue to verify container runtimes and so the issue was created. As add to documents in the wiki, the kind of getting the namespace from inside the pod, I don't recall seeing anything on the wiki yesterday, so I know who added it between last night and now. That's what I said just a few minutes ago. I have not gotten it done yet and I apologize. I will attempt to get to that. I've got a bunch of things that have been backing up for this afternoon that people have been asking for around this stuff. So yeah, definitely. Okay, so we'll circle around with that later on. No problem. Prem sent the poll off to the mailing list. Yeah, so I can probably quickly give an update on the current status test for the benefit of all. So until now we have 17 responses and then out of which we have close to seven responses that said no. The remaining 10 people have said yes. So, which means from a majority perspective, it is still leaning towards the current time slot. So I'll probably send out an email to announce. We should probably talk about that a little bit more because I think the whole point of sort of getting the yes and no was not to sort of vote on the time slot per se, but do we actually have a problem with this? And I would tend to feel that seven out of 17 responses saying there is a problem, probably means there's a problem. Does that match how other people are reading the situation? Yeah, I would say seven out of 17 is a big enough fraction to be concerned about. Yeah. It's to 40% yep. It's quite concrete. So, and I know you'd made a comment, Frederick, about people not getting to weigh in on times if they said they were okay with the time slot. Do we want to try and just run a quick doodle poll? For a new time slot. And we could include the current time slot in that doodle poll so that we can get a sense of where everything stands. Yeah, that's what I thought we were gonna do. That would make most sense. Yeah, because I feel like we're losing information on that. I'm personally, I'm open for this time, but I'm open for other times as well and we lose apps. Right, so I think some kind of weighted voting would be best, like give everyone three votes or something. I mean, that's the way they usually solve this kind of problem. Well, I don't know if Doodle will do that. Doodle does that. Doodle definitely does that for us. Oh, cool. Effectively it lets everybody see what they can make and then we figure out what's the best solution for the group. I guess I've been weighted. So for some people, you know, sometimes they're doable, but more or less painful. Do we have a tool that we could use for that? So far the numbers are small enough, just manual analysis of results would be doable. So one other thing is we also had another Adelaide Doodle poll. Probably I can do a quick correlation on what were the times there and then try to see if we can. I think the earlier Doodle poll though was a little more restricted in time because my big recollection was it was restricted enough in time that literally none of this lot's proposed in the last Doodle poll were times that I can make. But I think the set of times that you were suggesting more recently on the poll, if you actually did click no and go through were a little bit broader in terms of the possible times. Okay. I see what you're saying, Ed. So one thing what I can do is I can probably remove the restriction irrespective of the rest of the room. People can go ahead and then do it, but that by the way, everyone expresses their time. Yeah, we just got a comment as well from the chat from a part of I get the name wrong from Lucina suggesting Google Forms saying the Vulk Co-op had used it for weighted voting. So that might be another solution to look into. Sure, we can look at that, yep. Real quick question here. Prem, you've been extremely generous with your time in trying to coordinate the meeting. Right. But if I were in your position, I would kind of feel like I keep getting pushed back on it. Would you be okay if we could find another volunteer to pick this up? Sure, I'll be happy. Yep. Is there someone else in the call who's highly organized? I am not. Who might be interested in picking this up? Well, okay, I've made enough noise about it. I probably should pitch you and do some work here. Awesome. Yay. Thanks, Mike. And Lucina have volunteered as well. So I suggest that both of you could. Well, we don't need to. We don't need to. If Lucina wants to do it, that's fine with me. Thank you. So saying, yeah. Yeah, I'm fine with either one of you. So. I appreciate both of your willingness to help. Are you okay picking it up Lucina? Yes. Excellent. Thank you so much. Yeah. And just to, just to clarify, like we're very happy with the work that you, if you did Prem, so please, please don't go bad. Absolutely. But you do, you're, you're doing so much for the team at this point, but it doesn't, right? It's on to your, on to your head. So. So, no, it's just, yep. Sounds good. So. Okay. So. Final accident. John to crisply express the invisible network and via prompts. To, to ML and or next week meetings and see network service mesh document. So. Do you want to share the agenda as you're walking through it? Oh, sure. I can share it on. Actually. I'm actually talking on the phone. So if someone can share the agenda on chat on my behalf, that would be very useful. Somebody willing to volunteer. So Frederick, the agenda for today. Right. So you want to. Okay. Nice to chat. Cause it's, I'll do that. Yeah. Doing it on the phone. Yeah. Sure. Sorry guys. I think the formatting is going for a toss. So I hope it's okay. Right. So for the, for the invisible network. And via problems. Is that something we want to talk now? Or do we want to, do we want to add it to. To. The use case discussion or like, well, what do you, what do you want to do with that, John? I had some comments. Online and document. And probably I think that's fine just now. To have people look at it and make comments. I think. I even from Intel me some really comments and try to, we tried to sort of narrow it down. I mean, the problem is in the data plane. What, what do we want to expose the data plane. To STL from NSM. I don't have a solution, but perhaps as we do the use cases. Something may. Jump out at us. Any other thoughts? Okay. So. My recommendation is. People are interested in this. To read the document and comment on it. And what we can do is we can schedule some time in the next. Or in one of the following weeks, once, once you're ready to discuss this, if you feel that the topic is something we should discuss in the meeting will, will that work? Okay. So moving on. So we've had another week of active. Development. And as you can see from. Some of the. GitHub issues that we've, that we've ran through. The. The discussions that we've been. Did we lose Frederick? Oh, you're on mute Frederick. That's what happened. Please. And so that bug was, was squashed. There's also work that was done to refactor the. Some of the code and get it so that it's. Reduce some of the code, some of the code duplication that we had and just make it a little bit more, more robust. And there was some more work that was done around handling. Handling errors. So. So a lot, so a lot of. As you would say. Ensuring that we don't accumulate too much. And there was a lot of technical technical debt in the process. And things that were uncovered through. Through some additional testing. And. For the pole, for a pull request that were, that were merged and as well. See, it was. of things. The only other thing was we've also we also updated the Kubernetes dependent and version and the client dependencies to use the semantic versioning. So Kubernetes for some reason releases multiple versions and some are semantic versions and some of them are not and depending which ones you pull in you unfortunately depth defaults to the non-semantic version. So just so just as a heads up if you want to know how to how to do semantic versioning Kubernetes properly go look at our dependency files and in this particular repo so we've worked through those issues. I have a sorry I have a question that regards to the dependency with the 111 there is a there was a change in the client go and basically with the 111 the latest like a beta 2 client version 7 doesn't work so you really have to do like a release use branch release 8.0.0 and I was kind of curious what's the plan right now you're using the Kubernetes 110 so you guys planning to move to 111 at one point or what's the plan? So that's that's good question at this point my my recommendation is until 111 is released that we not that we that we because the semantic versioning specifically I believe that they cut a release of client go after 111 would would be released and so it really it really depends on what the state of the system looks like at that particular point you know I ideally do you know if the changes like if there's any intention with within the client go in order to make it backwards compatible with 110 and 119 and so on I don't think so I mean at least based on on the change between the release 7 and release 8 I mean it seems that it's a kind of a breaking change one of the reason why I looked into the client 8 is because with the 111 they introduce a new dynamic client like when you create the kubernetes kubernetes client you know to talk to the api server you return kubernetes interface and then arrest interface and then like basically three types of interfaces and the version 8 there is a dynamic client but that can be used to access pretty much any type of object so I mean that's what that's what I think it could simplify a little bit the code you know when you deal with the client so that's okay that that makes sense so yeah like right now they're documentation and the documentation may be wrong and their compatibility matrix claims that they will support kubernetes one ninth and 11 in their head version and but yeah I think we have to wait until and until they do their release and and see if there's any to see if there's any issues and I my gut feeling on this at this particular point today it would probably be best if there's if it's incompatible it would probably be best to just jump over the the jump over that hoop now since we don't have any production clients at this point as far as I as far as I know if anyone is using production please pick up but I'll go so far as to say is if you're using this in production you're a braver man than I right now but yeah so if so my my suggestion would probably be to like let's let's move now and not and not wait so that way that we're we're working on what will what our key customers will more likely run on and that likely will not be 110 by the time that we cut a first release so just thank you for for bringing this up and we'll we'll spend extra time to focus on this on this issue to try to make a good technical decision when more information is available great thanks so I I think we do we I think it's not like we should take an AI to open an issue for this to make sure we track it and we can use that to discuss it right that's a fantastic idea so cool I mean go you just put a generic one down there and one of us will get to it so that's added something on that we'll get to it love yeah thank you for freaking out I that was bothering me as I typed it so since we're done on on that the last one was never service mesh on GKE so if anyone wants to speak up on that I think that was John I believe right sorry I was I was in mute so yeah I did I just from the help from Kyle and Frederick I just copied down steps and kind of documented it I wasn't quite quite sure where to put it so if anybody else is running in GKE a couple of gotchas you too this is so John this is this is really cool would you mind basically creating a markdown file with this and we can check this into the docs repository this will be super useful cool exactly what I was thinking just give me an AI it's just not a meeting until John has taken an AI so let's move on to use case mapping so I will pass it on to uh which one of you three wants to take it hey uh Frederick uh just one quick thing before we move to that sure I just wanted to bring up that uh I would love to get some reviews on pr91 because that actually there's currently some of the stuff is broken due to that so getting that one merged would be would be great if anyone has a chance to take a look at that sure I will pop on and take a take a look at that after the after the meeting cool thank you okay so so use case mapping I will pass it on to to Premphabian and John Mcdowell so use case document is on the the link is on the agenda so we also chat on the chat and and just a heads up use case document is also available on the link is also available on the uh get on the github repo so if so if you can't see the chat or or so on feel free to go to the github repo I don't think it'll so whoever had the problem with accessing google docs I don't think that it will help for you to take to go through that path because that's also a google doc so sorry about that anyways it's headed to you Premphabian sure um so quick update on the use case document there are a bunch of comments and then I've incorporated the comments for the use cases related to that of the cloud networking I have also added the distributed mesh or incorporated that on just to reflect on how the use case looks with respect to the distributed bridge model I just want to briefly cover about that and then probably pass it on to John to share his updates so this is in continuation to the presentation that it touched upon last time so the idea is basically use the route rules are you sharing on the zoom what you're talking to oh sorry I thought uh could one of you share because my uh connection is a bit flaky I can absolutely share it just thanks a lot I do I find it easier to follow these things with the pictures in front of me so give me one I'll get it up there yeah so and then you'll have to make it stay on the right part of the document sure I'll mention that and uh cool so uh is this let me give you the page number so that uh page number is good okay page nine page nine well there are no page numbers on the document so I'm not sure um so uh further down further down okay uh oh sorry I think you just passed yeah here here here the same okay good okay um so um the use case uh basically talks about the how do we build a distribute uh or using distributed bridge how do you build the same use case uh here uh that can be two types of um two types of meshes one is the persistent full mesh what is meant by persistent as um as a as a prerequisite of the of this particular use case you do need uh vx lands uh between all the compute nodes um so the only downside to this particular approach is that when the the number of compute nodes increases uh it's going to increase the um number of mesh between this because of the numbers involved um so the idea that is basically mentioned here is that how about an on-demand full mesh the idea of on-demand full mesh is that uh let's assume that we'll play with the use case uh let's assume that uh the one of the uh application exposes a l2 channel and the others would essentially want to connect with that uh so in this case what happens is the information is available in the service discovery and any other client uh or the part that wants to have this request when they request the vx lands are uh connected on-demand um and uh this is going to happen for the first request uh for such any uh such uh channel and then uh it'll continue on um uh for uh the different for the different tier down policy by this what happens is you avoid the uh full mesh uh problem that you have in case of the persistent full mesh um so these two use cases would probably fit in well in case of our network service mesh and the recommendation would be based on how you want to have your mesh created um so that's what I've captured here and then the diagram again explains the uh interaction between the uh various components here you have the bridge part which essentially exposes channel and then a part a subsequent part would uh that wants to have a connection would essentially request for the connection and then it uh continues on in the life cycle so this is the update I have added in addition to that of the uh our conventional model I've also incorporated the other comments from people have uh given until now thanks for the comments um also I think I just touched upon the cap-wap use case I just saw the cap-wap use case I think this seems to be similar to that of the bgp-vp and use case probably when we touch when we discuss the cap-wap use case we can probably further discuss on whether we want to have some collaboration between both the use cases um that's all I had from the update point of view probably I'll pass it on to John if he has any updates John yeah I don't have any updates the only thing I was doodling with this week is trying to tease out some design patterns from the all the various use cases to try and categorize them I'm not sure if this is really useful but it's just just my way my mind works I try and look for I'll speak up with somebody whose mind works all that way um I would find it super useful because I tend to pin it very rapidly up and down the abstraction uh tree and so you know I do tend to sort of look at a lot of concrete things and try and squeeze out what the common patterns are so it would be really super helpful for me I don't know how everybody else's brain works plus one to that I will also be interested John okay Frederick give me another action item we do appreciate all that you do John very much yes yeah and we love working with you John cool so one other uh just to add to that John uh I've always been doing a lot of work around microservice patterns aligning so that is also one of the favorite areas so yeah so let me take my doodles and put them in a document and then people can comment on them I mean what I have so far it doesn't feel quite right but I think maybe having other people make suggestions might will probably be the better way to get to get it done so okay different viewpoints excellent always good so Prave back to you uh yes so I think um I'm sorry missed the person who had added the cap up use case probably was me basically okay hey nice meeting you so do you want to present the cap up use case do you want to share um I'm more than happy to have you share so you can drive if that's totally fine uh it would be great to to have a chat so we need to change I need to change that yeah basically uh it was just a quick hack to bring the information in here let me let me go ahead and let you share so you can drive that ends up working better when it works okay okay I need a name for that action item that we just discussed the patterns was this the one for john for john yeah for john okay all the actual items are belonging to john I'm happy with that I have to have an event in the alias so I can join these meetings okay sorry john I'm actually I'm out of uh I'm traveling so I'll probably help you join you uh soon and then work with you on the other as thanks bro all right I got a new github script to reassign everything to john so let's continue on okay and don was that so we can quickly talk about use case uh we quickly added before the call um yeah basically uh as already mentioned by prem it's uh mostly a bridge or a layer 2 use case uh what we have here which is um at the moment in time absolutely not related to uh natural service mesh from implementation perspective but from the use case perspective the thing what we need to do here is obviously to transport um uh cup up encapsulated frames coming from the fields basically outside the Kubernetes cluster to a cup up controller um cup up is basically a UDP encapsulated protocol for the control plane a user plane I think the guys in the court probably know or need we do we need to go into it a bit I'm not used to it but yeah it would be good if you could probably say okay so we make a quick what it is basically it's a wi-fi uh control protocol for wireless access points which is uh um standardized and an idf and basically as as anywhere you have a cup up control protocol which tells the yellow box which is a wtp a wireless termination point to bring up bring up a wi-fi network so that the user equipment the UA can can attach to and all the control home elements for authentication authorization and channel management etc are going to the cup up c channel it's a classic control plane and the user plane it's encapsulated and cup up u which is uh a quite uh usual uh tunnel protocol which has a tunnel id uh which relates to the control plane etc so and and both uh both protocols needs to be terminated in a Kubernetes cluster which already creates some usual problem because it's UDP and you need I think we just lost your audio well I'm mute I hear you now you were saying they create some problems because it's UDP and then you broke up at least for me ah okay uh bringing UDP uh traffic into the apparently the world does not want you to express what it is about UDP because you said bringing the UDP traffic into the and then but I lost you is it is anybody else losing his audio or is it just me I also lost I lost okay sorry about this I'm well I tell you what we I think we go away with the camera the perks somehow it helps so can you hear me now here you are now so basically the problem is uh quite often in the cloud to bring UDP traffic into a cluster to a port uh um because of uh layer 4 load balancers etc sometimes time timeout issues on UDP but besides the fact this can be managed uh uh uh we have done it in a bare metal deployment with some special local answer configuration at one day the UDP traffic reached the the cut up uh port basically where the control plane and data plane is running as a container and uh just represented by the cp and dp box here and this cp and dp talks to each other over normal kube network over c and i to to chair informations is basically an internal internal method protocol so now we're referring to who's a layer 2 because now the the the the data path decapsulates uh the catwalk u traffic and after decapsulation you have basically the traffic or the layer 2 frames of the device which are from the mega addressing perspective foreign to the community's network so basically uh it wouldn't be forwarded at all so that means if we want to give this traffic to another network function in this case is a cgw ip second g re that's called uh in our terms connect pp gateway this is a really good use case because you've got a classic illustration there of what we mean when we say that your payload is l2 yeah exactly right so it's a beautiful use case yeah that's a problem yeah your payload is l2 you put the l2 payload into the cloud and you're dying because all security settings will not allow to have remote payload from foreign mechanisms any number of crazy things could go wrong absolutely for nearly everything from mac learning tables it's it's it's it's okay almost everything goes right i'll buy that i'll buy that i think i think you lost a vacuum again yeah we lost your audio again which is really sad because i'm excited about this use case and you heard me now it's better i hear you now okay cool so sorry about that next time we make it better anyway what we have done then i said okay us we have dynamic ports there are no stable endpoints etc so we we have started to label to label the ports with a simple label that call it vxlan through and there's a another controller watching for the labels and have some annotations and then the controller access into the pot which is a little bit hacky and spans a dynamic vxlan between two ports that's basically represented by the vxlan to dp vxlan do tp which is a which is a port or a sidecar in the container which creates a vxlan link and push in the interfaces into the port so from use case perspective different naming but exactly what you're proposing was was netmatch but done differently so from there we the the layout to traffic or payload traverses where the vxlan to the next port was then creates an ipsec gre tunnel which is terminated externally in the service controller box and this way we're leaving the cluster basically because we need to hand over to a foreign system here so basically what we have in the use case is in our Kubernetes cluster l2 payload which needs to be distributed or leak across ports and then we need to leave the cluster and forward the traffic through a gre tunnel to a remote system and to achieve that that basically in production at the moment for a couple of months or even a year now but has no mention to go in this in this way in this implementation upstream it's just to demonstrate the use case um 10 minutes before the call we uh as a head of anton he also he also joins a call here um we push the vxlan controller uh public to the to the open cnf group which we are have created on github just to look at this uh how it be done um yeah basically that's the use case and uh if there are any questions or about the implementation i think anton you are here you are happy to to answer this as well and it's about the use case i can also answer i have a quick question if you don't mind uh so you're saying that your controller builds the vxlan tunnel between the ports so basically uh they're leveraging the same the interfaces which are provided by the cni right you just built on top like a the tcp tunnel between the two ports right exactly it's nothing to do with cni cni is not uh involved here at all and uh basically the controller access in and make some commands to create an interface with a given name which comes from the manifest uh uh in the port oh so your port then has multiple interfaces like one the regular cni and then another the new one vxlan exactly if you interest if you look on the edp interface which uh along distribution protocol what we use internally this goes via the cni interface but the payload it's totally separated over this vxlan interface so maybe if we want we can go into the manifest here so it looked like well this is actually very good and you're right this is very much up the alley of the kinds of things thoughts we're thinking here in network service mesh um because effectively what you've done is you you've come up with a way to sort of hack standing up what we call a connection for l2 payloads between your your capwap ts and your cgw ipsec gre and then likewise to stand up a connection you know of type ipsec to your external service controller if i'm understanding correctly exactly we have two different implementation of a link let's say one goes external one is internal one is vxlan based other ones ipsec gre based so that's i presume the vxlan is carrying an l2 payload and the ipsec link is carrying an l3 payload now in this case as ipsec is carrying the gre payload which then carrying uh still the l2 payload because l2 has to go to the controller control let's hear it but i i'm amused to note that i will add that to my my very long list of ways that people carry l2 payloads because gary was not on the list yet but it should be yeah it's still the case so especially in this use case yeah so um so other use cases we have in mind we have a quite similar one which is carrying in gtp so generic handling protocol payloads and and uh bringing up ports implementing um uh in gg sn and mobile core and if it's failed comes up on a different port and gets a vlex lend and then controller to to uh come up with the same endpoint ip address and and stuff like this but this are different use cases all right if we try to put in the l2 use case but because as you say everything will break this l2 payload in a in a cluster so it's uh this is this is the everything it's even worse than the everything breaks is an l2 payload in the cluster in that and i've occasionally had this conversation with people uh kubernetes actually makes no has no concept of an l2 segment so if you try and stick a mac you know an l2 frame out there god only knows what would happen right even if nothing broke it certainly has no guarantee of getting where it's supposed to go exactly i've seen a nearly all virtual elevation environment uh there's nothing kubernetes related uh we have deployed the same thing in open stack uh on a not in containers not in ports etc so uh and even in vm where if you are not careful and you put layout to payloads in there it's either doesn't work break things or you need to have special security settings and it's all all in mess that's certainly fair now this is very cool i appreciate you're bringing this to us um so our what are your forward going interest series it sounds like you're interested in sort of um you know looking at making sure number one that we can meet your use case with network service mesh um overall are you interested in the medium to long term with in terms of using network service mesh yeah it basically exactly is that because network service mesh from this perspective is not not not really our use case so it's all but we want to leverage uh the cloud native environment communities other as it is uh we have a strong opinion that's network service mesh you call it you guys call it has exactly similar functionalities from a pattern perspective and the classic service mesh let's say the the tcp and tcp ones but uh uh but for l2 and l3 payloads so the medium term goals would be to to join to join network service mesh here and one day face out the vlex land control as a homegrown stuff to to go there we are also in a project currently which is a running which is a plug fest about this stuff where we are on a research project so we could people like to promote to start to bring up this lab environment with service mesh because numbers of l2 a3p payload uh cnfs will come up in this lab and that's uh why we are it's hugely exciting frankly on all fronts um we're delighted to have you guys involved um you know and and and just don't be at all shy about asking anything uh that might help you move that forward we would love to see you guys move us to the plug fest space there yeah that's whatever you really don't try to be shy it's just about timing sometimes and uh if you're getting out of the console but i have the the the interesting part for us would be to learn how your activity your working group uh is received in the signal networking as a something which is besides beside of that or have as a chance to get let's call it upstream or get uh get forward here or where we where you stand at the moment uh with was never true i i i could tell you my viewpoint and i would actually encourage others on the call to express theirs we did present i think two weeks ago roughly to the to signal working um and i think we got overall a pretty positive reception there um from folks the the one of the benefits of the network service measure approach is that we don't actually need any changes in kubernetes proper um which is really helpful to us because we don't have to go try and convince like three four five different groups in kubernetes to change something for us but it's also seen as a good thing by uh signal working i know tim made a comment that he really liked that about this particular approach um you know since then we we've actually been starting up conversations and trying to have a conversation with the signal working group about whether it makes more sense for network service mesh to be some project of signal working or a kubernetes working group or sort of what's the right formal structure there um we were going to have that conversation yesterday at the signal working meeting but the turnout at signal working was very very low this this past meeting um and so we we didn't get to have quite that conversation but we're we're actively working to line up with signal working and so far they seem pretty warm to us does does anyone know what i would add i can give you my experience for that uh uh uh there was if you go back to mating list and signal working there was three attempts and even in the channel on slack to say hey sfc service function training and service measures basically the same yeah why not uh thinking in this direction and um but this as well mostly up in the bones and and and no way was following this due to the htp htp folks usually are there and uh so networking is not highly represented in in this group in my experience well that's that's fine i mean the thing is the signal working guys have solved a very important class of problem really well um and just like the esteo and the envoy guys have solved a very important class of problem very well and and even though i'm hugely in favor of sort of borrowing by analogy the things they've done i don't think trying to ram l2 and l3 payloads into they're already functioning really well for them system is like a happy experience for anyone yeah to to further that uh the kubernetes use cases are primarily around enterprise which primarily calls for a very specific l3 l4 pattern and so like i know it sounds a little bit negative but it's actually the was was a right approach for kubernetes in order to keep it simple in the grow uh you know it just it affects us by saying not having anything like l2 or so on so like this is an attempt to lift those but lift those use cases uh but yeah from a kubernetes perspective like if you if you go up to them and you ask them for a feature uh if that feature has wide enterprise uh uh use cases then there's a good chance it'll get in but if it's if it doesn't or it complicates those use cases then the chances of getting it in i mean it's still not impossible but it's it'll be significantly more more difficult because like telco and so on is not is not the main the main use case but uh i think that we'll gain as we gain more traction we have the capability to to add into some of that some of that influence around areas where kubernetes may not be as strong so i would second everything you might buy perception and matches yours redrick with one exception which doesn't think it's the perception of broad enterprise use cases i think at various points there are things enterprises will discover they need that may not be perceived as being a needy eye and i hope we can help with some of those oh absolutely you know the container startup is an excellent is an excellent example so our pod startup i should i should say rather than container startup um for like how long it takes but um yeah so sorry go on no sorry go ahead and i'll just add it up to you yeah um yeah i think like from my from my view as well like this particular approach like we we went with the term service mesh because it makes it easy for people to latch on to but the goal was not to stop to stop there but it was something that was a lot more flexible so use cases like yours can can be built so this is like totally in scope so don't feel like you're diverting us or anything like that by bringing up these type of use cases like this is actually a really really great example of something that i believe that you'd like to support okay which leads to the next question i said we starting next week was to bring up a plug a plug fest environment here which has run for about two years and adding more and more features and in the future given this use case we we we shown here how ready for prime this nsm at the moment when you think we should start to make the hands dirty on what you expect from a from a steep or flat learning curve here for people already have done it doesn't make sense to mess around with that already from a implementation perspective to work with that or should we wait a little bit more which just goes basically back to anton on the call because he has written the vxl controller for from our side this is already there versus an experiment or what's the status of nsm well we're still very we're still very new like in fact just from a timeline perspective um uh the first conversation that ed and i had about this was in uh mid to late march and so all the work that you've seen from then to now has literally been between the past literally the past 70 days and so from a from the implementation side uh we're you know we we already have we're we're building up the primitives at this particular point in order to in order to describe this so like we've added in kubernetes crd's and we're adding we've built up protocol buffer apis which those will actually be really good for you to review as well just so you can get a sense as to like what's like what the core functionality is that we're that we're working with um and uh so but yeah but we there's still there's still some more work that needs to be done before i would be comfortable saying that yeah this is ready ready for prime time yeah so two things kind of the mind for me like number one is you know obviously you guys need ip sec over gerry and people with real concrete needs at hand who want to try things tend to bump up the priority of things as i mentioned ip sec over gerry was not on my list before it is definitely now um the other one that i actually wanted to throw out there is we would really welcome your participation in the development community you sort of have an opportunity here to to shape making sure that um we meet the kinds of needs that you see and that you see from other folks um by participation in the community and i can tell you having often arrived at communities after the the um after things have hardened after your stuff is already in deployment it's really nice to have that opportunity for early influence and so we'd welcome your participation um and then also you guys look like the essentially prime beta customers for us um in terms of you got a use case um you know when nsm does actually get to a point you can try it out you guys kicking the tires would be extremely valuable to us so but before we continue on um let me just wrap up the meeting and we can continue the discussions afterwards if you're both uh if you're both interested so first thank you everyone for for attending is is there any last minute stuff that we that we didn't get to that we should add to the to the agenda for uh for next week i think meeting time planning was the it was really the only one okay so anyways i'm i'm gonna stick around for this conversation so if i don't know if it has time and historically you have to drop off uh i i actually do have time on this occasion and i'm delighted that i do so so i'm i'm willing to stick around for more conversation as well so let's uh let's say this concludes the uh the meeting and then we'll we'll continue this particular conversation uh i will take this opportunity to remind folks that the meeting is recorded in its entirety from the very beginning and that'll include this after meeting conversation i don't think that's a problem i think in many ways it's a good thing but it's always nice to make sure people are aware that's right um shout out to the preamble of i'll i'll create a preamble at the start of every meeting and that and it'll include that we the people of the the cloud native networking world in order to form a more perfect mash um hey they actually um in when when you talk in uh ham radio and that's uh many of them actually have set preambles that they say at the beginning of every of every start uh first asking if there's any emergencies and then two describing what what is it that they're there for they say it every time cool so anyways uh diverging so okay yeah so i so i was just saying before like i think these type of use cases you know we like we we talk about network service mesh and we've given some some examples but you know like the examples that we've given are like by no means like saying this is in concrete like the rails that were that we're setting so we we want to make something that is ideally my my viewpoint on it is that we want to make something that is uh it's flexible to handle these type of use cases and if you can if you can think of it and aligns with uh the patterns that of the parameters that we've provided then there's no reason why you shouldn't be able to to do it barring up barring some technical technical reason like if you say we want to send l2 over some environment that doesn't support them of course not you know but ideally you know we want to have um we want to have the sdn and the services and clients all work out and essentially negotiate uh the transports so that so that you can build whatever it is you want to you want to build in this particular like like this particular use case and uh and get things working so so we definitely definitely appreciate this particular this particular use case yeah sure that's that's that's understood and thanks for well receiving that for for us that's basically the question when and let's say how to join uh the the activities here so therefore the the answers that uh Tim and the guys have says well received so which was more or less uh I think easy because it doesn't affect uh the the cni's underlying background etc so uh and I say it we we are starting not in production and we are not intended to say hey please make uh nsm stable next week because we need to migrate the production servers uh over next week that's not the point here but I say next week we starting with uh with a group of developers even students and junior developers which can dive in whatever we decide and if uh nsm uh a good fit then they can dive in there and using this uh framework this technology to solve the use cases here that's where the the question was coming from where rare and when we wish well if we would be in a situation where a bring up of a system every second breaks and it's very very early etc etc and only to understand from one lead developers etc then maybe it would be the not the right moment to do to go in here but if if the environment is already in a shape where I say okay we can start with that and adapting uh and helping for the use cases and bring back uh uh issues problems etc and ideas we see uh then I think we we are happy to join rather sooner than later maybe you should know we all are part of the vpp community so we we know the underlying data parts elements quite well working with them already for the bnfs itself and therefore having a network service mesh which is based on the same uh same uh data would be the premium as well because otherwise you you end up in too many tools so quite honestly so nsm itself is agnostic as to the data plane that you choose so they're there as to you choose to use there's sort of the there's the data plane inside your your cnf your cloud data network function and obviously we're agnostic as to that that's whatever you're going to do um nsm is also agnostic as to what you might call sort of the underlay data plane in other words the thing that is connecting the connections that said um as you might imagine there are quite a few people in the nsm community who care a lot about vpp and so I expect that to be one of the early data planes supported um so you're you're going to get basically what you're looking for um I have a question just out of my own curiosity um and this is because often so you're dealing with wireless traffic right now um and one of the interesting things that we've gotten from some other folks is what I would call exotic alt tubes so for example if you exotic l2 protocols so for example um in talking to the cable guys um they have use cases where they would like to be able to pass doxus frames as the l2 payload okay do you have exotic l2s like that in the wireless space that it might be interesting to pass um over an l2 you know over a connection and then we're service mesh I think from a from an outer frame perspective it starts with classic isonet l2 when the traffic arrives us so because doxus goes beyond that um maybe on wireless framing as well if you look at 8 to 11 frames yeah it's of course a little bit more but from what we need to carry I don't see it for the current use cases we we have in production so that's that's completely tough tuck in the back of your mind that one of the design intense here is to be able to support exotic l2 l2 and l3 protocols if need be because there are a bunch of them around and if you just leave the architectural white space for them supporting them is super easy um and and if you don't then you make it very hard for people like I've got similar things talking to fiber channel guys where you know they've got their own l2 and l3 and and if your attitude is there are kinds of l2 and l3 payloads then it's a very easy game to play yeah so I said what do you mean with exotic or you mean from from a framing perspective going beyond so is on a two frame off from the payload itself or from your gun now you're on mute oh sorry when I say exotic I mean non-ethernet because there's a very large percentage of the world that thinks there exists one and only one l2 protocol and that protocol is ethernet um all right are here as we go into into the mobile core network elements there is upcoming uh non-ip transport for for vendor data in the narrowband iot which is 128 byte center frame which is encapsulated somehow and then will arrive somehow and you need to forward it somehow that's exactly exactly you you see precisely my point and that's that's entirely why we're trying to keep it because it's cheap to keep it generic if you think to but a whole lot of people think either that it's the only l2 there is yeah it is so yeah that's uh at least from the iot perspective yes we are all in this in this area and as you say it maybe doxxus is coming around as no current customer need at the moment but our customer base which our carriers are usually although in this in this area but not direct at the moment cool what i also think just this mapping out ideas here what we have discussed and in our offices uh uh that with network service mesh or with the principles of network service mesh which is basically encapsulating any kind of traffic in in any encapsulation that call vxlan and bring it to the next part could be also a transport primitive for classic service mesh because usually what you do with the classic service mesh is very expensive here what you do is that you need to pass protocols and you need to put in hcdp accelerers and then you create a new packet and then you send it forward yeah why not doing it's the other way around encapsulate encapsulate the traffic put an hh around that yeah and no need for for even uh uh decoding the packet and encoding the packet again because you have this uh you have this trace id's or whatever id's on the outer frame just where you even could transport tls frames or whatever you you you want you you should absolutely talk to john mcdowell he's actively trying to write down things in that direction i think he was the guy who got all the action items on the call um so yes action item so add another action item to him i don't even leave the call without getting action item but exactly that's that other ideas how to how to make this happen or even put trace id's for open tracing and they're somehow arguable in in user plan if it goes to high speed however if not going to high speed for whatever tracing scenario you you can you can do it this way actually very good things for that this this this might be an interesting use case as well where perhaps if you have something that doesn't support those tracings but you want to to add them in and that's all that you're going to look for except when maybe you get a new a new flow you have to initiate a new a new id but there might be an interesting use case to to show as a uh to to add in a example where perhaps you encapsulate and encapsulation and add in these particular headers and then transfer as you and make your decisions as you would and then de-encapsulate on the on the other end and so you know i it's it should be you know and it should be trivial to do this in uh in our in the architecture that we're that we're proposing but at the same time uh be able to you know and you know basically i think it'd be a really great way because to to also demonstrate some of the flexibility because we're showing here something that that it's an l2 frame that no one else in the world has ever seen but here it is handling it without any without any issues yeah does that make sense yeah but that totally makes sense and and the thing is we we've got some really fascinating tools for that as well um because not only can we do sort of a thing that essentially encapsulates you in a way that you get crazy but we've already got built into things like vpp stuff like the i o a m uh protocol from the itf where we can not only trace what's happening sort of above the the the tunnel we can actually trace where the tunnel is going because you know to the degree that you have i o a m support which is starting to come online in your networking devices you will get per hop information including latency information for where the tunnel translated to so the amount of tracing you could inject in network service mesh becomes frankly insanely huge it's really cool next afternoon then you put it on the normal old tracing api and put it away but but but you have at least the meta data in your hands but based on the i o a m stuff yeah so um absolutely yeah just just remember when we do that we have to define the apis in a way that other stn's can eventually implement such functionality oh no absolutely i i that i totally get in you know for example if you were going to do tracing you would probably want to negotiate tracing the same way that you're doing tunneling so that you're basically you're actually doing tracing in a way that both sides can deal with but this actually brings up a new matter which is we've talked about the negotiation of tunneling and it's all fine and handy to wage your hands at being able to do something similar for tracing but as we're building out the the the what we see between nsm's we probably do want to be able to express a preference on tracing and i think tracing is a little orthogonal to underlying in cap um yeah but i actually think i've at least uh distributed or correlated trace id and that's that's what you need yeah so at least you need to bring both things need to support the both things need to support the tracing um what the tracing mechanism is and then there is an exchange of of the correlators and right now the way the negotiation between two nsm's is mostly shaking out is the requesting nsm says i can do this the you know the nsm on the far side basically comes back and says okay well of the things you've suggested to me in preference order this is the one i picked you know because of preferences and here are the parameters related to it and for the tracing it would be your your trace IDs right um and so effectively the the requestee is in charge of the parameters um in these situations because it's the one who ultimately has to make the decision okay well yeah that would be to keep to one round trip yeah one of the other uh use use cases that we're going to have to think a little bit about as well is um it's i can see potential use cases where you might have one nsm that's managed it's being that's managing let's say vpp that's you know everyone manages odl and you want to do tracing across the both of them as an example and so i what would what that use case like is that even something we want to do in the first place yeah i would say that that's something that looks like that is very likely um you know because you know in the again in that scenario whatever the nsm that's that's controlling or talking to odl it would it basically have to um have some set of things it's capable of and you know so if the nsm on the pod it's using dvp it can do i oam and it comes across and it says okay i'd like to trace with i oam as part of this connection and the um the far end comes back and says well that's nice i've never heard of this i oam thing that needs to fail gracefully and my suspicion is failing gracefully means you just don't get tracing but you do get in cap yeah and i'm and i i think something that we may end up being able to do is um if we if someone were to use um i'll use a higher a higher level example like suppose that we were to define a protocol buffer that describes tracing that could be encapsulated into a into a network service mesh connection that then handles the uh that they can then traverse across all all nsm's and so that so there may be interesting ways that we can utilize our own architecture to smooth over the differences between different different sdns and still get the same uh the same functionality and so uh yeah we we have options you know yep this is this is very exciting may come up we have some time because leaving a tracing area a little bit we have another use case which is a little bit pressing uh i i like to address or like to discuss basically which is which is about well let's call it dependency across the pass yeah uh let's say you have microservices which do different things the right hand side is establishing an ipsec or whatever routing pass the connection to a remote end and on the left hand side you have your cup up termination or gdp termination so basically if the right hand side is loses the connection then the left hand side is not allowed to to uh receive traffic anymore because otherwise you're stuck yeah because this is a more or less disconnected microservice-based environment yeah you need somehow tell the left hand the left hand uh we and f that they should shut down or not accept any traffic anymore because the next one the pass is broken somehow yeah so uh which means i think frederick has thought a little bit about this already i might suggest you would be that you inform the left hand the cnf that you can note that you can no longer service this connection in some manner maybe it's a disconnected the connection but or whatever but you can't service it anymore and then it's up to the left hand cnf to decide what the right response to that is right in fact we can we can borrow a lot from uh from enterprise use cases in this scenario because they have that that same path and many in many scenarios so i'll give an example from from netflix someone watches wants to watch a video and suppose that they have a failure that prevents them from being able to service customers in a given region or or maybe watch a specific set of videos rather than allow the customer to continue on and press play and then buffer and wait for it to never arrive and upset the customer they wanted to return an error fast and so there's a number of techniques uh the one that comes to mind that would probably be most helpful is probably what they call circuit breaking uh but there's there's numerous techniques that we that we can borrow for them it's always just a matter of picking the ones that uh that we think would best suit uh and then we can see if modifying for this use case for use cases and pick and see if there's a good way we can pick them in easily uh that being said i i get the feeling as well that we have to be a bit careful with some protocols because some protocols may be sensitive to uh to to this thing as well so like there may be a specific way that a certain protocol may deal with an error and pass that up and if we could preempt that you know that that would uh not preempt but if we can if we can follow those channels out that would probably be better in some scenarios so but basically i'm pretty aware about circuit breaking et cetera et cetera so we we developing other applications which has authors super wiser trees et cetera et cetera but it's always inside an application so so from the principles are quite clear you need to signal something uh right hand fails you should signal that this has failed and so the question here if you're leaving your application environment which you usually use it might might support that and say hey we have independent parts which are created in a in a different language in a different environment uh let's say you open up an ipsec on the right hand side using line of kernel and some shared scripts just to give an example and turn the left hand side you have the running port implemented in c or vpp doing another thing so and and no but this parts have a pass and have a connection maybe maybe they have readiness probe tells probes lifeness probes so you need to coordinate or somehow or across language boundaries in in the orchestration environment which tells us let me ask you a question because that you're you're a good person to ask this question because you have a real problem right so one of the things that i've occasionally used about um is a possibility like the following which is um for situations in which the thing you're doing is effectively stateless right i'm going to say some things that may be unsure about your use case just for illustration so correct me at the end um so say for example that your your left hand capwap uh c and f you know it's getting a bunch of you know data that's coming in as l2 frames that was to pass on to your gateway um now you may have five replicas of that gateway um and we happen to have routed your connection to one of them um and since it's just a vx lane connection my presumption is that you don't have any magic state you're just shoving frames at somebody who will then be able to shove them into an ipsec gary tunnel to where they need to go if you happen to connect we connect you at first to replica number one and replica number one dies we discover via lightning this probe that it's gone maybe the node caught on fire for god's sake right um you know it strikes me that there are some scenarios in which just seamlessly and quietly connecting you to replica number two is probably doing you a favor um and and it seems we should be able to do that in nsm um quietly and seamlessly reconnect a stateless connection to um another replica that provides the service you're looking for um yeah is that that would be useful to you yeah that's exactly the case and i think usually in mobile call protocols and even in the cop up it's much simpler because the client makes a retry so the request coming over anyway uh keeping the state and putting it to another stateless element would be even better but that's more or less easy because a failing port knows he has failed and and so the problem is sometimes sometimes he knows he's failed he's knows he's failed for certain kinds of failures right so if you're an example of i can no longer my ipsec tunnel is down on the gateway there you know you fail but if if you know something went wonky on the physical server where your note is um and so the node went down ungracefully and so the pod just disappeared um then it strikes me that you know it would be a favor to you if you're truly stateless to when your capwap sends another you know basically sends another um uh frame we just you know having realized that that one is gone we put you on to a new vx lamp tunnel that takes you to another replica uh and you continue to get service and so instead of having to think inside the capwap server and do a lot of logic um essentially you get a very brief period where shit's just not working followed by shit working again and it just works right and so you get out like a packet or whatever it is it looks like a packet drop or yeah it just looks like a a blip of packet drop um followed by packet drop it's no longer dropping um thank you yeah and kubernetes does provide some of the primitives that we can that we can rely on so for example if it's a pod if it's a pod we can use uh readiness and liveness probes to to control my problem is basically i have a liveness readiness or whatever other state from the application on one pod on the right hand side and if this liveness or this let's look on the picture we we we we we we shared um again sure sure do you want me to share sure it again or do you want to share it again uh i'm hang on a sec yep i shared again yeah oh sorry go on if this pod here or is this connection let's say is failing maybe not even because because the pod itself fails but so the outer service other servers say let's say this connection is not there anymore this should delete you i don't accept more frames on the left hand side because otherwise they will send traffic and traffic and traffic and have the impression everything is right but the state of this connection it's it's it's gone it's it's actually a practical use case even even with with the stock equipment at the moment yeah that this will still try to send traffic even you have a redundant second copy in another data center etc etc but you think because you don't know that this connection on the right hand side has been failed they still put traffic in here so basically effectively what you're saying what you're saying is if so think about this from the point of view of the capwap pod you see enough i find it very useful to think about local points of view from the point of view of the the capwap pod it does not matter why the connection it has to a gateway service is not working like why it's not working is not its problem if it is no longer a good connection it needs to know that right because it may have steps that it needs to take in order to behave properly and in your case you're saying this step is to refuse to accept more incoming wireless range of spine but but i'm a firm believer in localizing intelligence wherever possible and limiting the need for global vision wherever possible and so i think all the capwap pod needs to know is i no longer have a connection to a gateway right yeah that's what you can do if you say i'm not ready anymore because whatever the the the reason for that let's say cgw die as well then i don't have a connection anymore than i can't send traffic so if the gateway for whatever reason is no longer showing as lively which can include in declaring itself not lively because it's lost its outgoing connection or could include you know somebody took a sledgehammer to the note it was on and that pod just isn't there anymore right it doesn't matter which one the nsm essentially notes that it has failed its liveliness check and having failed its liveliness check that means that we need to look at the connections it has and either um notify the pod that they are gone or reconnect those connections to someone who is passing their liveliness check who provides the same network service that gateway was providing yeah yeah like yeah like the way that i'm looking at this like there's there's multiple areas and this this would be uh something that whoever is designing this particular path would have to decide on so like the ipsec connection goes down you know is is it possible for it to resolve the connection issues itself open up a new ipsec channel and then silently deal with it you know and that's that's one option another option would be to signal up its upstream connection saying i no longer have connectivity uh i'm going to go away now and you make a new request for a new can making the new request for a new connection or passing the error upstream would be the decision of the of the next hop up so like in in essence that like the the ipsec like imagine that the ipsec path itself that you had there was like another nsm like that that connection that context of that and that connection it doesn't know anything about about what's above it other than the limited amount of state that's and metadata that's been passed to it and so it doesn't actually have the context to deal with it but the next hop up definitely or may have that context or the next one up from there so it's like you know you have to pass that information up until you get to a point where where so where something can make the decision like i want to try to reconnect or we should fail up the entire uh we should fail up we should fail up the chain and eventually you you hit the uh the customer where you might fail the connection the worst-case scenario but yeah yeah so i think like we still want to capture statistics on on all this stuff like if we see the ipsec tunnels dying all the time you know we definitely want to know this and so the tracing is still it's still important but like you said orthogonal but the actual decision itself uh to me it it sounds like like we we want to we we want to hand the decision to whichever service has the best context to deal with that and the i think the most effective way is to just keep handing it up until such a service is found and can make that decision yeah it's almost analogous to how you throw exceptions up the line until you meet the person who actually knows what the fuck they're going yeah i know i thought this was just an open discussion about how to do that there's a poor man's approach would be the ipsec plot is not ready anymore it loses the readiness if the ipsec tunnel goes down it's still live but not already and then if you can propagate i'm not ready or the readiness is it's gone then yeah this should be probably just so just so you know good the the management channel that we're using at this particular point uh we've built up through protocol buffers the uh the management path like how do you make a new connection or so on so one of the things that we need to build out as well is uh is passing some of this state information back up about about a connection so we haven't added any primitives to that just just yet but uh like i think this is exposing a hole in that in that area where like it's not it's not that we didn't think about it it's that we haven't in our development cycle it's it's on the it's on the agenda and i'm part of so what we're using is we're using g rpc which has a dual uh it's a bi-directional uh an rpc mechanism and so in this particular scenario it sounds like what we need to do is ensure that the that there's a some way that we can communicate this information to and from each uh each uh service pod i guess we'll call them in order to make sure that this that this information can get propagated up and that it's understood what those error messages uh uh are and what they mean and perhaps perhaps we can we can we may even be able to add a little bit of additional functionality as well like if the if the client says i want to connect to an ipsec network uh and that's handled by another network service mesh and dies uh i mean we could we could potentially even add some functionality functionality to say hey if this thing dies don't even bother returning uh returning back just try to connect to another one and i'll need return to me if that fails and that yeah i mean that that's kind of my the point i was suggesting about the for stateless connections because for a stateless connection it's entirely possible the network service mesh itself can handle the error without disturbing the pod on the the pod that has its connection to the network service that went away yeah but ultimately the the one that decision needs to be made by that by the client requesting the service and the client should specify that as i want one of my failure strategies to be to retry new connection on failure no that's absolutely true right i mean because the the the client is the one who knows whether a reconnect on failure is going to be a problem or not right um you know again it's back to the the local knowledge um you know capwap understands whether this is okay or not and so it should indicate if that kind of thing is okay when it requests the connection right and because it's on a per connection basis as well that means that uh you can have multiple connections from a from a single client so that means that you could set up based on your sla requirements or so on exactly what you need and even if it's the same data path perhaps one customer you have a different recovery strategy because of contractual obligations this and that could be that could be added in to your to your intent so you can try to service it as best as possible exactly that that's that's what we what we promote anyway for other services as well so in the intent you already know this as a la et cetera et cetera and hard which drives some behavior so see that and what i also like on that is this is not on the power pot based from a from a site comp perspective doing that or it's regular then we have it on a connection based yeah so it's like okay this connection fails from nsm and has this information on this level and not on let's say a vpp plot has failed or something as a global say connection absolutely the other thing that i would suggest you it sounds like you're already starting to think this way is one of the things that i found really profound here is the very dynamic nature of these l2 and l3 connections in network service mesh opens a whole world of possibility that we just haven't thought of before because the world was too static right so what is it useful for example to have a connection per client right coming out of your capwap mm-hmm you know i don't know why not you know it's unthinkable in the way we used to do things like you literally couldn't have imagined it that's exactly what's happening with soft green soft gary you may say a pair of client you make a make a soft gary connection which is not preconfigured you're just bringing it up because this 12 client needs to go to another hop as other ones and as you can establish them dynamically that's entirely possible and it wasn't before that was over cni only possible at bring up time and not not on at runtime anymore so absolutely absolutely cool so we've run fabulously over but i think very productively so we look forward to having you guys get more involved in the community going forward and thank you so much for bringing this use case particularly on four minutes notice because i think it's been a very productive one okay yeah and and just to circle back to your first question is doing in terms of involvement like i think it may be a little bit too early for junior developers who want to implement a service to jump in at this particular time i mean if they want to help with building out the network service mesh you know we have plenty of tasks and and part of what i want to do is is help people you know even if they're very junior to to learn how to contribute and be effective contributors uh if you have the bandwidth to help build this out fantastic if you don't have the bandwidth to to help build it out you know the use cases alone are are invaluable so yeah so don't don't feel bad if you don't you don't have that at this particular time timelines i don't have a good answer that to to the timeline at this particular at this particular moment other than you know this is a to me this is a high priority i want to get this up and running as fast as possible but i want to temper that with making sure that we get it right so that's that's why i can't really commit to say hey you know this should be usable by october or november or like a sort of specific date so um so so i apologize about not having something concrete on on the outside but it means from a from a time being from a current state the environment is not even usable it's just about defining the apis at the moment or this is already using a very limited scope and can you can you push a packet from a to b array that's a question as you imagine in the long run yeah so we haven't built out the uh so we're targeting uh vpp is one of the as the first uh as the first sdm at this point uh we haven't built the back end for that just yet so there's no there's no packet so that's so that's that's where we're at so from a production perspective um i mean that's your willing to build out such a such a component it's not really it's it's not really ready yet right now we we do have the capability to push uh information into uh network service mesh in terms of some of the intent some of the state and we're doing that through what's called kubernetes crds so crd is basically an extension to the kubernetes api okay and so so we're going through the crd path in that scenario but um we also have some protocol buffers that have been spec that have been added out we have co-generated uh where we have a service that's that's up and running uh and is injecting things in two queues but there's nothing on the other side of those queues yet so we're so we're so we're still in the process of actively billing out the core functionality so so i mean the the the nothing i would say is that there there's some placeholders right now for the apis don't take them too seriously we just needed something as a placeholder we're building up the information to be able to handle those apis um and and i my my suspicion is that once that infrastructure is in place there will be some rapid iteration giving very serious about those apis that will occur um you know because they're mm-hmm and and what and part of what it's what's going to happen as well is that once once we're happy with the data injection of where we're just about done with with that particular path uh in terms of the crds uh being written so the next uh the next step immediately after that is let's start billing out say bpp provider and demonstrating demonstrating it and so uh so we should have the ability to push packets very soon especially with the expertise that we have and the and the recent and the access to resources that we have uh in the team uh including uh including ed with that um and so so well we'll have something we should have something relatively quick barring any major stoppers that that we find but yeah it's not it's not quite it's not not ready to be to be demoed actively at this point what one other thing i will mention to you which is the way nsm looks at the actual cnfs they you can sort of divide them into two classes immediately there are what you might call from nsm's point of view the smart cnfs these are the cnfs that are intelligent enough to participate in the conversation with the nsm um and then there are what you might call the dumb cnfs these are the cnfs that are not smart enough to participate in the conversation with the nsm they expect a static set of interfaces presented to them um and for those we intend to write an init container that would take a config map that would set up whatever the dumb cnf needs it sounds like you guys are wanting to be smart cnfs i think we definitely at the end up there it's basically like building smart cloud native applications or doing a parser and start up something i totally get it and i'm happy you guys want to be smart cnfs the reason we have to make sure we support both is we get a lot of feedback from some of the larger operators basically saying but our vendors cnfs they aren't even dumb yet um you know that's like it's like a sfc proxy in a service function chaining saying yeah you either have an nsh aware vnf or you need to have a sort of sfc proxy and to that so i think both is there but as you see with open vnf group already or organization we are about the vnf cells so basically we see us as a vnf or cnf provider so we can make them as smart as we want there is no there is no vendor vnf there is no there shouldn't be a vendor cnf there so because we are about to boost the vnf obviously if something comes in which needs in a system integration perspective if you say hey but we have this vendor cnf or vnf still here then from system integration perspective okay this can be represented by an interface but if you talk about our own cnf and vnf cell would always be smart and they are already as they're consuming directly community resources they are pushing matrix orders from from each of its format so they do all the cloud native things already and if it possible they do nsm of course as well so otherwise this doesn't make sense to have a cnf at all it's a vnf then absolutely really quick when you're talking about a plugfest was that the opnfv plugfest or a different plugfest no it's nothing it's a it's a local it's a local uh research project which is uh state funded here and it's about bringing up uh and it's about the partners creating creating a plugfest uh for themselves our fixed partners universities and other renderers etc is there is a very local okay is there something i could google to find out more about it yeah i can can it's similar yeah so and at open nfv we we have not much contact we we have contacted dan kohn about one and a half year ago uh and do a lot of circumstances we lost a little bit of traction here because of real projects to deliver but uh we like to influence our work with open nfv as well but we don't like to orchestrate open stacker so it should be it should be i feel you i feel you all right cool i do have to run it has been such a pleasure gentlemen okay so i have to have a run as well so thank you guys i think it was pleasure to to to meet you and we we until we get forward from here and discuss what that would mean for us and thanks a lot for all the information yeah thanks for stopping by on such a short notice and just so you know it i had a conversation with him about 10 hours ago for the first time and i saw i saw um that's why that's why i poked him um you know because i saw your conversation when you were you poked me so i do appreciate it okay yeah so we definitely appreciate you know short notice and the amount that you've done up to this point you know it's uh strongly appreciated and with that uh thanks thank you both and uh you both have a good uh a good day or in your case a good afternoon or night yeah we'll begin so yeah okay and see you on i'll see you then guys definitely take care take it