 Let's go ahead and get started. I had two main questions I wanted to ask. Unfortunately, we didn't get the notes included from our... Oh, no, we did. Yeah, from the call last time, which is discussion of Heat and Tosca regarding cloud native network functions. But before we dive in, can I just check here? So, Leia, I appreciate you joining from Huawei, Sweden. Interesting that we're not getting anyone from China this time. And Leif, is this the first call you participated in? Yes, that's correct. No, that's all. Okay. Welcome. Can you tell me about cinch.com? Yeah, so we provide messaging solutions. We have like a two-sided business model. We are like one of the largest segulators, but we also provide like infrastructure towards operator primarily done in the messaging within SMS, MMS or CS, but software operators. Great. And so I guess I would... But we have a few more folks on now. Could I just ask everyone to please enter your info in this Google doc to sign in for the call? So I would love to just have a short discussion on the concept of conformance for CNFs. So I've been beginning the conversations with LF Networking and CNTT about their plans and with VNFs, they are essentially demonstrating software or conformance against either heat or Tosca templates. But my impression is that the heat part definitely doesn't make sense for CNFs. And even Tosca is a little unclear whether there's a fit there or not. And so I guess I would love to hear from anybody on the call and maybe Robbie would be willing to speak up first on the question of your thoughts on CNFs with Vodafone and how you're looking at going about conformance, demonstrating that they work correctly and then also how they will interwork with other parts of your network. Thank you, Don. Yes, so CNF is conformance specifically. I think Tosca has been chosen in Vodafone as the language for service description. And this is why in LFN, if you look at the OPP program... I'm sorry, Robbie, you said Tosca has? Yes, Tosca. Okay. Yeah, so if we're trying to apply this to the CNFs, it might not make sense. It might make sense. It depends how really this is being defined. So Tosca as a language, it could sit on top of other description language like Helmstratt, for example. It does not have to be pure Tosca to be able to define a service that is composed from CNFs as only Tosca language. So it could be any language underneath, but Tosca will be the umbrella language that will define the overall service end-to-end. And the individual components, it could be other languages. This is one way of doing it. The other way is go natively Tosca. So everything is just defined as Tosca from the GitGo. And that will make it simpler really to perform compliance testing and that will make it more compatible with the way we're working OPP. Now as Vodafone, I don't think we have a strong opinion for now how CNFs should be presented since this is still, we're trying to figure out really the right way to do that. But I think within this forum, it would be good to understand what would be the best architectural decision to take in terms of supporting a conformance testing and which way to go. Okay, great. I appreciate that feedback. Would anybody else be willing to share an opinion? Raymond, I don't know that you've joined a call before. I'd love to have you introduce yourself as well. Hi, yeah, sorry. Sorry, yeah, I haven't joined a call before. So I'm from Expo. We do test and measurement software and hardware for telcos. We actually found out about your telecommunication group at the recent KubeCon. So myself and my chief architect, Alex Devagori, have been trying to get a little bit more active with the Tug. So we're just at the moment in very much observer mode just trying to figure out exactly what's going on and what's happening. But yeah, we're trying to just see how we can fit in and how we can hopefully contribute to building a better community. Great, we're glad to have you. I guess I would ask, have you were at KubeCon in San Diego? Have you looked at the process of containerizing some of your current network functions? Yeah, so I think we're less in terms of actually building network function stuff. I think we're building less network functions as sort of in a post-network function service assurance space. But yes, there is a lot of work going internally to containerize a lot of our stuff. I keep telling the NFV guys that yes, we need to be more container happy but we do have at least one group that is working on containerization of a lot of stuff. So yeah, there is definitely some work going internally as well as the fact that we have fairly recently sort of tried to shift our architectural focus on heading into a cloud native world. Great, yeah, I'm glad to hear that and that is sort of the point of this group. So let me open it up for anyone else then I would love a question. Have any of the rest of you worked on TOSCA in terms of using that with a Helm chart? I personally have not used that but we're looking at reading some papers and articles. It seemed to me very powerful way of doing it because you could mix not just CNFs, you could mix other technologies like PNFs for example within the service definition or even BNFs. That makes it more flexible in that way. Personally, I have not used it directly with CNFs, we use it with BNF and Vodafone but all the evidence shows that it provides some more flexibility in definition of the network. Are you going to be at the CNTT meeting in January in Prague? Yes, I will be there. Can I ask you if anybody else in the call is planning on going? Yeah, I will probably also join in coming from CINCH. Great, I know that they're planning to begin to come up with a plan for CNF conformance but you don't have any more insight into what the plan is that that, what's likely to occur at that meeting, do you? Well, I do actually because myself and Mark prize from Maxis, we're organizing the meeting. So the plan is to, yeah, the plan is in Prague to get together, Jim Baker is sitting up a session that we can sit together and do some brainstorming. We would like to have as many talk people as possible contributing into a discussion. We know we have a lot of the RA2 folks from CNTT will be actively part of that discussion. But yeah, this is the plan, kind of trying to think about it in there and see what would be the general direction of the community feel like. And this is something we could report back to the talk here after the discussion that happens in Prague. Yeah, do you know Manoj Nair from Netcracker? No, I don't. He has a document. I'm not going to share it now but I have encouraged him to post it into the Tug Slack channel talking about current approaches for a CNF modeling. And he's looked at a variety of different scenarios of how you can use Tosca types or a Tosca Kubernetes profile. But I'm just a little unclear whether any of those have a kind of traction right now or are going to be the preferred way forward. Yeah, I have not seen the documentation unfortunately. I will look for it. Yeah, so hopefully that'll occur in this Slack channel in the next couple of days if we could have a set of conversations there around it. He had just given me an advanced copy but I don't want to paste it in without his okay. So I guess I would just check again with this group. Is there anyone else who would be interested in sharing an opinion on that question on CNF modeling and conformance and any alternative opinions? Yeah, I wonder if this group has just want to pay more attention with the entity or because I think we have more discussions than before on the entities progress now, right? Lacey, a little bit more please. Yeah, I mean for this talk meeting I think we are currently talking about more and more stuff around the same DD as work. Like the CNF modeling, maybe the same testing also. Sorry, maybe the deflation of the templates. Something like that. So it's like we're going to more cooperate with those entity activities. Yeah, it's a little bit up in the air to be honest. I mean, we're definitely interested in collaborating with CNTT, I think the key question is that CNTT has been very VNF focused today. And so it's just not clear yet for a model on how that would work with CNFs. So just a comment on actually in CNTT we do have a whole reference architecture focusing into a cloud native. So we do have two tracks, one of them is completely to your point VNFs and the other is a CNF focused. So the only thing we talk about in there is CNF and cloud native. Well, so Robbie, I mean, I might have missed it but when I last looked at the RA2 documents about that second track, there wasn't really any information there yet. Yeah, it started after Antwerp. So now we do have at least the first three chapters drafted and I talk about the model and approach taken to use Kubernetes to run cloud network functions. So I'm not sure what was the last time you looked at it but there's been a lot of work happening. I'm just pasting a link in to, and I think it was Antwerp was the last place that I looked. So I guess, which of the chapters here would you say most of the work has taken place in? So it would be like chapter three, the high level architecture? Yes, so chapter two had just finished the requirements and chapter three and four, where the focus currently is on. Just looking at it from high level and then going into the component level specifics for the search and architecture. If you look at chapter three, there is more information than chapter four at the moment because of nature of it. Chapter one discusses the approach that will be taken to do that and will be useful really to look at what would be the approach in terms of multi-clusters? How do we deal with the lifecycle management of a cluster itself and what the pressure model will look like? Right, so I definitely see more material here. Right now, it looks more descriptive of how the Kubernetes manages environments and such. I'm not seeing yet a discussion, for example, around Helm. Yeah, so for like 4.8, you say the reference architecture must support the usage of Kubernetes application package manager using the Kubernetes APIs, like Helm v3. So I'm glad to see the reference to Helm v3, but I don't quite see the details here about how you... So I'm particularly focused on this conformance question, which I guess does seem like one of the most important outputs of the RA2 work. Absolutely, so if you look at the way CNTT really position itself is looking at the infrastructure layer more than maybe the orchestration layer. Now, having said that, one of the things that will keep coming up is how do we help within CNTT specifying requirements to assist programs like OVP phase 2 or OVP phase 1 to do the compliance certification program. Now, although we in CNTT, we said this is not something we're going to directly cover because it's our view of this orchestration kind of level, but as we start looking at OVP phase 2 with the CNTT tag group, the hope is really to collaboratively come up with a way to move forward. So I'm not expecting CNTT to specify in details what would be the right mode of... Is it Doska or is it Helm or is it a combination of both? We're hoping that collaboratively we can figure something together and that will be driven by the waiver verification program at the Tug and with LFN will come up with it. Okay. I mean, I guess I'd love to get your current thinking on it. Let me just go ahead and share a little bit of my thinking on the subject. So there's a project that CNTF has been investing in called API Snoop. And just to be clear, I'm going to give you a quick overview of this, but I have slides that I'm working on to walk through this process in detail and provide a lot more context for it. But what it does is use the audit logs in Kubernetes to evaluate the process. And look at every Kubernetes API call that it's making. And then it's able to classify all of those and say, okay, there's these stable APIs and these beta APIs and these alpha APIs. And so the user interface I've pasted it into Zoom is admittedly a little confusing for this use case because it's a little confusing for this use case because it's currently being used more for a different use case which is trying to look at the other side of certification, the platform side, and looking at which of the APIs are not being covered by the current tests. But the same software infrastructure could also essentially be used to look at a CNF and say, okay, does the CNF only use publicly accessible APIs and what version of Kubernetes would be needed in order to support the necessary APIs at the stable where the beta level. Yes, actually, I like that. I just looked at the whole page. I think this is certainly one of the things that can be leveraged a lot in the conformance application program. I do feel there is different level of verification needed. One of them is the conformance, compliance to just Tosca or HIT or whatever that format we decide to test against. And that would be just compliance. That means like offline testing. And then there will be the validation testing which is the second category in my view which is looking at the APIs. And one of the things that are important for CNTT is to make sure that all CNFs using the same APIs. So this kind of tooling will help us a lot to really identifying if a given CNF is really using the APIs that we standardize as CNTT moving forward. And the third level would be the performance. And I do imagine things like Testbed project, for example, which is mainly focusing on the performance can also fit into that verification program to deliver that performance metric and measurements that needed to start also defining how the characterization of the CNF will be within the accepted range of CNTT. But yeah, absolutely, yeah. Great. And I would just mention, Robbie, that the API Snoop project is entirely open source. And so as I said, it's not quite set up for what's needed yet, but it's something that I think could be pretty easily adapted to it. It also kind of follows the model of the current Kubernetes certification where, which is a self-certification that any organization can run, but then it can be validated by a third parties to ensure that Kubernetes platforms are actually conformed the way they're supposed to be. And the other side of that is, so we haven't actually done that for the application side. But there's certainly a scenario where you could use API Snoop to do so. Yep. I might call on Watson for a minute. Watson, I appreciate you being willing to wake up so early to do this call. Is there anything that you might add so far into the conversation about API Snoop and Toskey and such? Yeah, as far as API Snoop, I think I would recommend select what you said as far as trying to maybe validate the use of APIs as Kubernetes and it's interesting when we're talking about the different, let's say, I don't know, problems between maybe the vendors and service providers and the whole idea of should be for Kubernetes and all those issues that could come up from that. Trying to make it to where we're all having some kind of like a fair place to where vendors can make and collaborate their CNS and try to avoid some of the issues there. API Snoop seems like it could help to contribute there. And things like that. There's so many problems or ways to describe how the different vendors collaborate. Like if you were to take, say, the Etsy standards and all the different standards that we try to make to ensure that everyone plays nice with each other even though they might have closed source offerings and things like that, it seems it comes back to making sure that APIs are strong and validating each other. So API Snoop, I think, would help with that with Kubernetes. Yeah, I'll just make the point that API Snoop itself is open source, but it definitely works completely against a closed source CNF. There's no need, so it's a runtime application, API checker, so there's no need for the source code, the CNF, to be open source. So I guess the question I might ask for Robby and also Lea and anyone else on the call who has an opinion is how you do see, let's say that we have a CNF and it's installed via a home v3 chart and it's doing something like the broadband network gateway or some kind of routing of traffic or firewall or something like that. I'm curious your view on how that is interacting with the existing MANO platforms that operators are using today. I guess Robby, as you've looked at this work, do you see it occurring as orchestrated by ONAP or what are you seeing as kind of the, I think it would be southbound connections, but I'm not sure that's the right metaphor. Yes, to be fair, I think ONAP is an orchestration platform. If you look at Kubernetes, it has some orchestration capabilities, but I don't think the capabilities that are covered by Kubernetes at the moment is equivalent to the flexibility we have within ONAP. Just to interrupt for a second, I find that the fact that ONAP and Kubernetes both describe themselves as orchestrators, very confusing. Yeah, it is. And this is what part of the confusion of the answer I'm going to give you in a minute, which is it's not clear yet exactly what role Kubernetes will play in the existence of ONAP. And we do feel ONAP is still important in the CNF world and having that way to have a southbound to your point from ONAP to a Kubernetes platform to orchestrate CNFs is really important. Now, how they will end up really implemented in real life, this is still to be seen and explored. I don't think we do have a clear answer, even within CNTD itself to what's the recommended way of doing that. This is something we would like to also highlight and we did highlight as a gap in the industry, trying to figure out really what would be the relationship between Kubernetes and ONAP moving forward. Now, if we look at it from different approach, if we look at Kubernetes as standalone platform to starting with and then think about how do we hook it up into ONAP, what benefit do we get by using hook it up to ONAP. I think this is the way we start to look at it. If we just forget about ONAP for a second and then look at it from pure cloud native concept and then when we start to think about services, I'm sure ONAP will have more important role to play there. It's not clear to me at the moment how would that be felt. Okay, I mean, I was hoping you would say yes, it is clear to me and here's how it's going to work. Yeah, I hope that too. Le, would you like to speak up on that question of ONAP and CNF connections? Actually, I think we have been working on the ONAP for this CNF for a while. Actually, we recommend the way that they are currently moving forward to the containerized working nodes. They try to using this, how to say the CNF, to treat it as the artifact inside the whole existing electrical management so that they can directly adopt the current credit way or how to say the helmet way to provision those CNFs. So I think that's what they are currently doing and actually in our internal implementation, we are actually using not only the helmet but also the TOSCA, the heat, but even actually we even developed our own version of the TOSCA to help our product because we still think they are not good enough to feed for our special requirements. So I think the helmet is good enough for maybe some general purpose application but for the telcon applications are still some requirements there cannot be met. So we still need to investigate on the helmet to see if everything can be done via the helmet. I guess our policy is just become more and more open so we are trying to transit to using the helmet but still I guess need some time to make it more how to say to meet every requirement from the product. That's not an easy thing. So it's still ongoing progress. Yeah. Could I appreciate that feedback? Could I call on Michael Peterson for a second? Michael, would you be willing to tell us what we should be doing here? Yeah, I think it's a tough discussion and decision to make so I'm honestly not really sure at this point what would be the right approach forward. Yeah, that's a recurring theme here on the call today. Yeah, but any views on, I mean, have you worked with Tosca much in the past? No, I briefly looked into it. I think it was a few years ago. Otherwise, I haven't really used it much to be honest. And the work you've been doing in the CNF test bed, you haven't yet been trying to package up in Helm charts, but in fact, you've been more focused on getting the actual installations correct, just doing it manually. We've actually created quite a few Helm charts at this point to try and try and shift away from running everything using a mixture of Ansible and just shell scripts. So a lot of the examples we have now actually based on Helm charts. Great. Okay. But whether it's all working on Helm or not, the goal is to get there. Yeah, there's still a bunch of out of band things that you're having to configure ahead of time. Of course, of course. Although I guess even there your aim is to have some of that work containerized and running that it would run on the node to set things up first, correct? I think that would definitely be the end goal. Again, having everything in like a modular fashion that also makes it a lot easier to do custom deployments or do modified deployments without having to redo everything. So again, having building blocks to set up what you're planning to use should definitely be one of our end goals. But at this point, we're not really there. We did some updates to I guess just the Kubernetes deployment recently where we've tried splitting that up into hardware provisioning and then the software stack of Kubernetes provisioning on top of that one. But I guess that's it's under to do for 2020. That's for sure. Okay. Hey, Robby, could I ask you one more question before you have to depart? Sure. Yeah. It was the CNF definition question. Where can you point me in where in the CNTT work? I presume you're referencing the Etsy specs in order to define a BNF. But I'm curious if you have worked on a definition of a CNF. Actually, within the CNTT, what we did, we referred to the definition that been defined by the tag group. So that's very nice of you, but I don't know that we have the best definition yet in the in the white paper. Yeah. So I've been in a few calls within the tag group to define that. I don't think we still got to a conclusion. But within CNTT, we thought this is where the right way of defining it, not within CNTT itself within the tag group. So yeah, we did not really define it in CNTT. We were relying on others to define it for us. Okay. Because I think we were still having a debate. And I know we were still having a debate within the tag group on, I think they were at version nine or something on the white paper in definitions. I just posted the link. Actually, if you look at the section on the document. Yes. Well, no, that's the, that's the cloud native definition, which is great. And that one, I think there is uniformity on and people find that useful. But it's specifically for the definition of a CNF. I think was a little less. Yeah, if you scroll down a little bit, I agree. If you scroll down a little bit, it said the CNF is also working on a set of cloud native principle and CNF definition. So this is where we really added the link to there and hopefully once that finalized, we can just copy and paste it here. Yes. Okay. So I do appreciate the site by reference and now, but of course it means that the tug group actually doesn't need to, to come to consensus on that definition. Those were most of the areas that I was looking to cover today. Would, is there anybody else on the call who would like to, to voice and Robbie, I understand you have to go now, but thank you for, for joining the conversation. I would like to share an opinion on any of these issues. I still just wonder why would, why we wanted to understand the, the usage of this helm or the Tosca or something like that. The purpose of this discussion. Well, the, the, what was driving it is that CNTT is going to be moving forward very likely with the OVP, the verification and compliance program. And so I'm trying to understand how that program would work about how you would be able to demonstrate CNF's as being conformant. But I definitely don't think we have a consensus there. As I said, I am going to put together. Some details about API snoop and talking about how that could be used to show the conformance of specific helm charts, but that doesn't speak about how to show interoperability with the man of systems. Right. So actually it's kind of like we are going to use the OVP to to certificate CNF or what? I guess they haven't figured it out, but I think the natural thing that they're looking at is what is the smallest changes they could make to the, the VNF tools that would allow them to certify CNF's as well. Then what about the CNF testbed? So what's the relationship between the same type that were in the OVP? Yeah, so today there's none. I mean, we've certainly talked about using the testbed to run CNF's and demonstrate that they do work. The question is just that it's not set up for that today in any sort of automated way. And so, I mean, one of the advantages of API snoop is that it doesn't need the same level of infrastructure and set up to spin things up and demonstrate it. I guess Michael or Watson, would you, either of you want to make a comment about the CNF testbed and using that for certification? Actually, I guess if we are going to do something like that, we'd better just set up, you know, like some relationship or mappings too, that they can describe those relationships because it's a little bit confusing, you know, there's so many programs there. Oh, I totally agree. And I mean, I think it's, it's problematic. So we definitely need to have clarity here for vendors, particularly for vendors who are just getting started in the space and trying to come up to speed, that we would like to have a simple story for them about what the process is to get certified. And as far as the CNF testbed, yes, it's a bit too set up still for the vendors and service providers. Also, another thing that we're working on is how the output is viewed. So something a bit nicer for people to consume on what's valid and what has been validated or what conforms and what doesn't. So, yeah, I mean, let me just make a quick statement on that, which is that one of the huge strengths of the current Kubernetes conformance program is that it is a self testing framework. It doesn't require very many resources to run. So essentially any Kubernetes vendor out there who's created their own implementation can just run Sonabui or there's other ways, other test harnesses, but a relatively simple container and run through this whole test and within half an hour or so get a detailed log showing where they pass or fail on everything. And where the testbed, Michael Watson, how many servers is it right now? It's like four. And it takes quite a while to spin up and have everything get configured just on the Kubernetes side. I think the Kubernetes side right now we use two servers, one for running as a master, a small server for the master and then we have a bigger server running as the worker node. Okay, but my only point is they're quite substantial. It can be. It can be. I think at this point it's manageable, but there are definitely plans to scale it up and have some bigger clusters and some cross server communication going on in the cluster as well. Okay. So I guess I would be curious on your thoughts then on a conformance program. I mean, it seems like there's at least three things that we're discussing here where something like a Tosca template could be essentially entirely a static analysis ahead of time to just look at that template is conformant and parses correctly. And then you have the runtime analysis that you could do with API snoop of looking at what API calls are actually being made. And then you do have the test bed that you could look at some performance analysis and look at the ability to actually move meaningful amounts of traffic. Yeah, and I think I think all three approaches should be considered and it's something that I think we should aim for. At this point we have some performance testing in place and that's still being worked on. And I think we're not really at the point yet where it makes much sense to try and take in the conformance testing on both the templates and the runtime, but it's something that should be considered when we get to the point where we start having developers actually deploying and testing on the CNF test bed. And then we should definitely figure something at least a bare minimum in place to begin with. And then of course it's something that can be scaled up afterwards. Yeah. So, Watson, could I give you the last word here? Anything that you would add? Yeah, again, as far as the test bed, I think it's important for obviously for us to get it working easier for people to install. One of the things obviously is it's a little bit different than maybe the other conformance tests because you need smart mix. There's hardware involved, that kind of thing, and maybe in the future even A6 and stuff like that. So that's where it kind of comes a little bit more difficult. And again, we need to make it a bit easier to consume the results that people are a bit confused about that part. So. Yeah, I definitely agree. Okay, would anyone else like to comment or any other feedback on the meeting? I definitely am the one taking an action here on trying to document a few of these questions in more detail. Okay, last chance, and then I'm going to end the call. Really do appreciate everybody joining today at this awkward time, particularly you Watson, I think it's 545 a.m. where you are. Yeah. So we will look at whether we want to move this an hour back for January. But appreciate the call and happy holidays. Thank you very much. Thank you. Thank you. Thank you.