 Hi, I get a drink. Yeah, you're good. Hello, folks. No. Hey, good day. Um, has anyone got started today? Sorry for getting late here myself. Um, has anyone got the meeting started? No, I do not think so. All right. I think the leader wasn't available today. So, um, this call is recorded right now. It's currently uploaded to the CNF working group playlists that's hosted by CNCF. We'll have to figure out what's going to happen with this call as we move over fully into Elephant. There should be, if you have access to the mailing, um, the mailing list or the calendar, you should have access meeting notes. You can drop your name and any agenda items. But I think today a couple of things probably want to continue with the, the topics we had in the cube con with the community meeting and we did that a little bit before the U.S. Holiday last week. Yeah, actually, I'm interested too. I've been joining this group multiple times. I got to learn a lot about what this group does. Can maybe discuss how, how this group is going to play in the overall initiative going forward and new initiatives and the overall, uh, like CNCF networking at our edge. What's the, what does this group, this project plays in there in that picture? Well, I think that's, that's what we try to figure out what this group fits to the LFL landscape. So maybe the minus standing of what this, the work being done in this group is more of a, because I see a lot of a kind of telecom standard and testing. Basically, this is to test the, how CNCF, I mean, not how Kubernetes and related technology is used in the, in the telecom world, right? That's minus standing of what this group does. Is that right? Anything specific? Yeah. So the high level, which is ties in with the Gurge is we're trying to transition and merge things from CNCF with the LF, LFN to have one area for the telecom specific things. We know that we're going to have a test catalog continue based on the work from the CNF test suite and pulling in at least part, if not most of the etiquette RC2. So that'll be a test catalog that'll continue. So any testing that can help with validation and verification of functional and non-functional tests for telecom. And there is going to be some type of certification. And that'll start with continuing and supporting the existing certified products that are in from the CNCF CNF certification. Those are going to be moved over and we're going to, you know, launch something from there. Beyond that, that's what we're trying to figure out. What do we want? What's going to be valuable to either continue or is there anything missing? We've talked about supporting maybe other projects like NFIO in the future. But I think we right now are more trying to build a list to find out what do people want so that we can see beyond that basis of that initial set of tests and the certification that we have now, what do we want from here? And then as far as like what does the program look like or whatever, we are saying it'll be something like the whole thing is a cloud native telecom program and LFN. But how that's organized I think would be part of this as well. We don't want to wait too long to say we have to have it all figured out before we continue. And that's why we're trying to build a list and saying some of it we're going to keep as is or I won't say keep as is. We'll continue it until we're ready, just like this call. We don't want to just say we're not going to do anything until we know what's there. Does that help at all, Victor? Yeah, I'm just learning, but one thing I find is just observation because I've been trying to listen to all this community doing. The biggest either problem or kind of need is the integration of the solution that's already established. So because the telecom before containerization, virtualization has been used mainly in the telecom for open source community then containers. So the problem I can see is all those development is done without the standardization of being kind of test done by this group. So if how does this project is going to help the make the integration of existing stack like a cradle, you know, all those own app, all those different solutions. I know those may not be the one F. No, I'm sorry. Just the focus of this group, but how to make all those stacks which involve not only container but also virtual machine, how to make those work together. Is that leading the scope of this group? I think we can probably after we have a list of the different projects and tools and then maybe you're saying also the challenges, what do we want or what would we like if we look at everything just add this large list then we would be saying what is in scope. So I don't think we're deciding that today, Victor, that something is completely in scope or out of scope. The one thing that has been said several times before the community meeting that was maybe the larger one, I was hearing some of the stuff before it led up to Chicago, KubeCon and since then is a reducing the scope or limiting the scope was desired. So one thing that we've heard repeated at this point would be not having, not trying to take over all networking, we are not takeover, not try to say that we can handle and everything within networking in this group that we are going to be able to cover it all and take it all in right now. We may expand. So we've said we do want to go from the central core all the way out to the edge. So that means we will have overlap with something like Acrena, but you're mentioning VMs and other things. I don't, we have said this is going to be cloud native focus. So if there's something out there that's being used now actively, there's development or whatever that's not in the cloud native space. If someone said it's not really, this isn't really for cloud native, but it is useful. We're not saying stop doing that. I would think that would be out of scope though. So we're not trying to do all telecom applications. There can be best practices for running in a environment. And I don't really want to even say VMs. I think you can use Kubernetes. You can use software that people may think of as cloud native normally. And I don't think you can use that soft just because you're on Kubernetes doesn't mean you're using it in a native fashion. But it may be the best way to use it for your particular use case and you don't want to follow cloud native best practices. Well, then what we're doing here is probably not for you and we're not trying to encompass everything. So there is some type of scope. So I don't want to say that we're not going to have a scope, but the only thing I can think that it's been agreed on is we want to narrow it down to cloud native best practices in telecom. I believe the own app. I'm not very completely up to date on it. And so if someone else here can speak to it, but I believe the own app has been moving towards trying to work closer in Kubernetes environments to be closer to being Kubernetes native cloud native. But you know, I can't say that it would definitely be part of this or not. I think it deals more like with the network management layer of things. And I think this group so far was dealing with whatever, like how cloud native, a CNF is, and that is only partly touching this network management domain. So according to at least my, the picture in my head is not part of this. It's another layer on top of this. And I think that's one thing that we need to discuss in these discussions what kind of problem do we try to solve with this initiative in LFM because I think the targets are a bit different from what we tried to do so far in a look at where we were focusing on the whatever like runtime compatibility of the platform and the CNFs while the working group is focusing more like best practices and design patterns of CNF. So these are little bit different things. However, there is a small overlap, but I think we should one thing what we need to figure out is what problem we try to solve with these initiatives in LFM. And I think from there we will then see what is the scope and what is the target. Well, maybe I can provide my point of view in this particular case. The way that I see these things is I mean in the past, for example, especially in Anuket what we have seen is we prepare the infrastructure to try to cover all the possible scenarios. Try to cover multiple nicks, like SRIOB, all the possible workload scenarios which was a little bit hard for try to cover everything at all. So eventually at some point we need like we have three examples like how the CNFs are required or on needs. So on the other hand the same thing like for the workloads we need to understand like what is what is the infrastructure capable for like maybe we have some limitations or we don't have any particular limitations. So that's usually the main dilemma here. So both parties require like some knowledge of each other. But I guess this particular initiative is trying to clarify that correlation. It's not trying to provide like a implementation per se. I don't see like we're trying to define like what is the right way to do or right knowledge to use. It's mostly to define like okay you follow these practices in infrastructure or maybe CNF in the workloads you're going to suffer less. Yeah, that's my point of view. Maybe I'm wrong about that. Yeah, I think the main thing is whether the implementation, the reference implementation in Aniket is probably the biggest difference from our current focus. But the scope of where best practices will probably go into a lot of what Aniket cared about. So if you do care about those things I think that'll still be within scope. I think Rani went over this during the KubeCon presentation. But we're looking at this more as a full stack where best practices. If when you're looking at onboarding a CNF then you're thinking about how does it interact with does the application have specific requirements with the environment it's going in? Well then you have to think about those how the environment set up challenges that CSPs have are not just an application running in a workload and all environments are exactly the same. So the environment set up I think will be part of it by set up I mean what are the different components but that's more of the capabilities of the environment versus the implementation. There's some gray area there that might seem closer to what the Aniket was doing towards the implementation but more of it's on the I think if we think of the testing and stuff are you capable of testing what's going into this environment? So some of this could get into like Nephio best practices. So what do we mean there that could be stuff like the GitOps patterns or KPTs all about declarative configurations. So is your application set up for being deployable from Git and what is that? So you need to be able to communicate from the start that your application has all of its needs communicated up front so whenever it comes in or there's a change to it then this can be picked up in an automated fashion. So are you ready for your application to go right into the CICD pipeline? This is more thinking about production versus what is the group going to do? We're not saying we're going to test all of these things but we want to help guide people to that. I think from vendor creating and developing to consumers that are taking these there's been a desire to provide more automation and stuff across the board. So I think anything that's helping to test and promote best practices that allow automation is going to be at least topics of discussion in the group. Yeah I think at least in my again my mental model the reference implementation is only a like a Lochmus test if the requirements defined in array 2 are implementable. I don't see the reference implementation anything what would be used for actually running CNFs and I really like the idea from the reference implementation to project contributors that they would like to make the silver stack the reference implementation so we would not maintain our own stack in Anuket but silver would be the reference implementation for array 2. I think for me the main difference between what has been done and what has been done in Anuket is that Anuket states requirements for the environment also and it also tests these requirements of the of the environment and that's because if you would like to have compliance we need to be compliant against the way how we are sure that the expectations of the CNF are matching to what the environment provides while in the working group it is more like compliance against a set of design patterns and practices therefore there is no testing of the other end of the interface that I would like to say. That's what we have seen. The other thing that I was thinking is like at least it's something that I have seen this particular trend where now the CNF implementations are trying to cover the infrastructure portion like in FIO they are trying to not only define what the workload is going to do in terms like the container definition the source limits and all these things they are trying to consider like what are the infrastructure requirements so if you remember in FIO there are two different swim lines and one is from the infrastructure and the second one is from the workload and the other one is for the configuration of the workload so eventually I guess I don't know if it's going to be an industry trend but I think it's more like application centric so I think the infrastructure is going to be applied by the application per se and not vice versa. Yes, I see also this trend in FIO that they are aiming to set the requirements for the platform in the CRDs and automate the life sacrament of the platform itself. At least for now I think this new program isn't going to decide on what the requirements are for a platform so that Silva is doing that and FIO is going to have requirements when we look at compliance with your Sanger Gay it's not in scope as of today I don't know if it's ever going to be in scope but the group is running a direct compliance program and claiming this is the one platform at the moment it's providing upstream tools and tests so I don't want to just say best practice it is going to have tests so a test catalog is a goal and some of the tests and the test catalog could be used by an FIO it could be used by Silva or anyone else that does want to provide a compliance program so Silva has already communicated that they want to have a compliance and verification program for the Silva platform and they are already using tests from Anikadar C2 and from the CNF Test Suite so that's fine right now the goal is to increase the number of tests in the test catalog and best practices so that downstream programs like that can use them so when we say certification as far as within this program we've tried to move it away from the meaning of compliance you're certified that you're following you can pass these tests or pass following best practices doesn't mean you're compliant to one platform ideally right now it's going to mean that you're going to work on many platforms that all incorporate those best practices and the tests are checking those practices some of those are not just non-functional best practices so we could have non-functional and if you're looking at something like the CNI spec which is covered already in the CNF Test Suite and it's a requirement in the Anikadar test matrix which I think it may be one of the missing test requirements but that would be a more of a functional so that and that's closer to like APIs a lot of what's in the Anikadar test matrix I think 99% of the testing are all external pulling in stuff like Kubernetes ED test beyond just the Kubernetes conformance to test specific API coverage and that is something that could be included so these would be everybody upstream so if you're running a Kubernetes based environment and you're wanting to follow you wanting to utilize the open standards the community wants well that means you're probably going to start with the base Kubernetes APIs and the extensions from beyond there and I think that's all valid stuff that's already in Anikadar C2 and supported by the current CNF test suite that we can use some of the ED tests are already there they're just not part of the CNF certification but when we're looking at test catalog so think test catalog and not just aren't the test catalog to be anything in Telcom that's going to help you to be more native in your environment that's going to allow interoperability with anything that's Kubernetes native and cloud native and its utilization is I think in scope so that's why I mentioned FIO earlier there's going to be a lot of upstream tests that can be part of the test catalog that would be useful for something like there's also projects like Flux workflow I'm sorry Argo workflow Argo CD so there's overlap in those projects with what FIO is doing and CSPs are already in production so I think any type of testing that we can do that'll be upstream of those projects so then we say here's your application or your environment and it wants to be able to do automation that's useful for FIO or Flux or Argo if we have tests that can validate those things as well as document them in best practices it's going to help anybody whether you're developing applications or you're consuming them and you want to use those type of projects does that differentiate a little bit between say silver which is focused on a specific implementation and requirements and then us being an upstream group that can be utilized by silver or an FIO or even specific cloud providers this could be maybe Google or Microsoft Azure for operators they've actually talked with the CNF working group in the past I don't know Gurga if they've talked with the Anakit team but they've talked about using the testing and best practices for their environment and when people are onboarding so they could actually run and say if you pass these tests then you're likely to work it'll give you some assurance that you work in an environment because it's not specific to them it's more general purpose Red Hat, OpenShift, any of those would be downstream of what this project is yeah we were working together with the group from Azure who was moved there from AT&T like Pankaj for example I saw that he is also active in the working group group repo but they are not using according to my knowledge they are not using the RC2 as it is but they are using RA2 the documentation as the kind of like whatever like design document for their cloud infrastructure they are not following it like requirement by requirement but use it for inspirations and I think one application I think having conformance program for the platform doesn't mean that we would like to glorify one platform it's the exact opposite we would like to have several platforms which are compliant and that will guarantee that a look at compliant application can start without any problems there basically does the whole dream state of the look at specifications project yeah that makes sense and I think I can't see that Silva is trying to do something different maybe expanding on that same idea to add more requirements for what the platform would look like at least previously it's been based on building from etiquette at least in part taking pieces of it and then adding more requirements so it's the same sort of idea and what they are doing so they do not have a platform compliance program but they have an actual implementation of a platform so Silva provides you a Silva platform and therefore they do not need to have a compliance program for the platform they have the compliance program for the verification as they call it and they are ready to compliance but they do not take pieces from RITOO so they build their own implementation which is mostly based on RQA and they are using FUNCTEST for the verification center at least that's the current state and there is now a set of test cases contributed by PETAR which are testing the RITOO requirements to the platform and these contributions are in a bit of a limbo state because it was contributed to an internet and now I promise that they will be integrated to FUNCTEST but I think that's the logical next step that the platform compliance test is testing more than just the API compliance and it tests the specific features which are required by RITOO and the test frameworks developed by this group is there any other test procedure mechanism in the ARF networking and ARF Edge community? For what purposes? I mean this group focuses on testing and best practices related to cloud native network functions yes similar but maybe for that project not like standard best practice but kind of a testing for that particular project I think historically we had a few maybe testing frameworks or initiatives under DLFN but right now it's expected that the relevant parts of them will become part of this program yeah that's actually where I think my limited knowledge about the ecosystem that seems to be a really good use of this group's work is to merge those tests framework into this one and then because the actual best practice is probably hard to enforce right away because there's so many opinionated implementations involving virtual machine as well my understanding the biggest hurdle to for adoption of open source software overall is because of there's a lot of already ready to go ready to run open source projects but for them to be adopted by the telcos and any private cloud providers integration is probably the key so this probably is the work by this group is from the foundation for the test framework but the actual best practice might need to evolve yeah so there's already work on going to include projects like nephio from within DLFN and silver that was mentioned from DLF and not try to create some parallel testing framework but work closely and provide the testing frameworks and tools that they require I think integration is one of the biggest hurdles as far as adoption Victor you mentioned integration people are using open source software and Kubernetes and lots of different the technology is being used all over but having it I won't even say widely used I think it's already being widely used in some fashion and open source is embedded in any product that you get out there it's some type of influence but challenge wise as far as a challenge that is shared universally the integration of the different software and that also includes ongoing operations and maintenance so whether it's directly about the software that you're utilizing or ensuring that your environment continues to function and you continue to have services for customers well that the integrations is there so the interoperability testing I think is one of the key things that we want to do with the group and we want it to be vendor agnostic we may want to ensure that we're encompassing different vendor specific things that we see out there but that should be done in a vendor agnostic way so when we look at stuff like APIs and things that Gurgay was mentioning that have been covered in the Anakit test matrix and requirements it already did pretty well the Anakit side in trying to look at what are the different APIs that are being used by vendors today and what are different APIs in the Kubernetes platform and I'm going to just say the environment, the Kubernetes platform because you can look at extensions and components that go beyond the core Kubernetes that are critical to run a live production environment well you have APIs that have the attributes that need to be tested so I think all of that's going to be in scope and a lot of it goes right back to what you were saying Victor on integration integration is a challenge and interoperability testing would be part of that go ahead Victor because I think ten years ago Telcos was really spending a lot of money on virtualization and then comes container now there's a lot of solution if this group can make that easier for Telcos to adopt a more standard way this is the right way to do it making cloud native Telco infrastructure I think it will be really useful for both existing Telco and new providers because otherwise all the money already spent a lot of money has been spent there's no standard and there's still no good use for this existing software so I think this group could play a key role in the integration standardization part I think one area where the CNF working group has been successful has been following the model of other groups within I think Kubernetes and CNCF is one but this is just an open source model when you're talking about standardizations you can build from a top down approach where you have committees and groups that are deciding on what's needed there and then build out the standard and then everyone needs to implement to that standard and then you can do it from a bottom up where you have a lot of implementations that are already happening you have APIs that exist and they're being utilized in production like you've talked about Victor as far as things are out there already you may have new development as well that's fine but people are trying to work together and then you start having APIs that are communicated here's what we have what's been working for us so there's a lot of communication there and you end up with a standard that's built based on open sharing of the information that everybody can implement these similar APIs or integrate between open APIs so a bottom up open standards are created and you see this with a lot of what's on the internet today as far as protocols and stuff that everybody utilizes HTTP would just be one of the largest biggest ones where anyone can implement and if you look at the very early days you did have implementations that were not all compatible but it kept moving forward until now any of the browsers are going to have some implementation there and now we've had stuff like and it's of course old at this point today but REST and other compatible protocols for HP that can do RPC calls for applications so function calls over that and we have the same sort of idea from things like GRPC that are expanding that and I think the success that we've had so far in the CNF working group with products that are certified and people adopting the practices and so forth and using the test catalog base has been because of building up for that interoperability and integration so if we can have those integration challenges Victor if we can expose them and make them like clear here is where we're having a problem with integration then we're more likely to come up with a solution for the interoperability between those rather than coming from the top down and saying hey all of y'all that are having problems re-implement yourself to a new standard bottom up open transparent community implementation standards and do it based on testing you can start testing and this would work with what we're seeing in Aniket already when you're looking at the test matrix and RC2 you can go through the whole set of requirements that are there they're using E to E test from the different special groups within Kubernetes C and CF communities those are all open APIs and testing against running tests that you can go run against your own platforms they're there I agree I think this should be a good place to start so on top of the hour I think two of the things that I think keep coming up is building out a list of what are tools and resources that we currently have so you know one of them funk test would be one of the tools test frameworks we have sets of tests already listed in the Aniket RC2 test matrix we have the test catalog in the C&F test suite which is beyond the certification so listing what are the resources and what do we think would be useful is one thing if people have those where they think here's something that might be good to keep that in mind if you have a better idea of where we could put them if there's a wiki page I may have set up a page let me find that you could drop that in so we want to build a list of what are the resources that we have now and you could even say here's a resource that we had but maybe it's something that's not being maintained and so we probably don't want to use it that's fine we're really building a list not deciding on this is definitely used we need to have that list to make it decisions and then the other part of the working group notes as well the other thing would be those challenges or what type of problems would we hope to solve we may decide that it's not in scope but I think we need to build a list up first before we make a decision on for sure on what's in scope or not in scope so Victor when you're talking about integration or any other type of problem existing applications or systems that are running whatever it may be those we need to start listing those and then get them defined enough that we have examples and we can get down to a point where we can say this is something that we want to tackle now later, never, whatever it may be so those are the two lists so if you have specific integration issues that you're running into or know about that would be something that we'd want to put and Ranny probably we could have the same thing as an asset list as far as like challenges challenges and problems that we would like for this group to help solve or provide tools for solving I don't want to claim that we're going to solve everyone's challenges but we may be building tests or best practices that someone can use to help solve those problems whether that's on that same wiki page or another wiki page we should have a place to start putting that together yeah let me create something real quick okay just create a page and paste it in the chat thanks Ranny alright so if folks can be thinking about whether it's resources that we can use or you'd like to use so there may be something that we're not using within Aniket or the CNF Working Group now but we know it's available that may be just be an open source project NFIO has been using Free 5GC for something that could be tested with not everybody must use it but it allows them to test CNF Working Group started doing that so this is more proof of concepts there's potential lab areas other things please add whatever your you know if you think this could be useful for us then add it to that asset list document and if you have challenges problems whatever you may be seeing please add that to the challenges wiki page and then I think we could next time we come together maybe go through the list of the asset list and challenges see what's there maybe take some time to add and then we'll have when we get started and then maybe discuss some of those to move it forward does that sound good as a way to move what we're doing forward Gurgay Rani Victor? Okay for me all right at some point of time we have to somehow channel all of these activities to the talk level so if you decided to create new projects in an FN or stuff like that we have to use the processes what we have. Yeah so as far as organizing it or creating a group or moving it into a group leave that up to you know Rani you may need to go communicate higher up and figure out what we can do to get some of that started the one thing that I heard repeated multiple times it seemed I won't say consensus but it seemed like a majority were wanting this was some type of new group a new program rather than trying to move into an existing but something specific for this cloud native telecom focus and then taking the pieces that we think are relevant for that to be within there and then you know we could continue helping existing groups for instance is going to continue as a specific project so but this new thing would probably be a new group and then the only other thing that I was hearing was there was thoughts that it may be it may not look exactly like anything that's currently existed with an elephant so it may need that may meet me like a higher up finger gay where it doesn't exactly follow anything that's been there before so that may also need to have people at the top of elephant to decide how is that going to happen definitely because for that they would need to change the charter of elephant because currently elephant has projects projects and resources and those sort of things so how does this fit within how they're currently named and that sort of thing alright that may be something to discuss as far as the topic what would we like to see versus what's possible an elephant today so you know these assets and challenges find these are what we want to solve would we like to see this group continue what would we like it to see and if it can fit within something that's currently exists gay great someone says it seems to fit there if it doesn't fit then we just need to write out what we'd like and then that's pushed up to the top of elephant so that they can say okay we need to update things to support this type of entity alright we're well past the hour let's stop here and then we can pick up with assets challenges and then what we'd like the group to look like on the next meeting and we do want to continue with building out the test catalog building out the application practices we already said at the start we want to continue with where we are but if folks have thoughts for expanding on what we currently have then please bring that forward I think that could be something to start talking about I know that there's three or four tests that are missing from etiquette RC to matrix but we could look at those if you have familiarity with silver whatever let's bring those forward so that we can at least continue in the test catalog and certification help that we can provide to everyone else thanks everyone we'll see you next time see you