 Hey, good morning everyone. This is Uma from MyData. Good morning. This is Taylor from Vote Co-op. We're going to give it a few more minutes and then we can get started. Sure. I did it. I'm trying to merge the agenda. It looks like they wind up with two areas. So I'm going to copy some of this down. Taylor, I added, this is Ed. I added a slide to the deck for a short agenda item. Awesome. Thanks, Ed. Nice to talk to you, especially. Hoping that you're doing well over there. As well as could be expected under the circumstances. Yeah. Well, hearing voices is a good indication at least. If you haven't done so already, please add yourself to the meeting notes in the attendees section. And if you have a topic you'd like to discuss, you can add that into the agenda section. Posted the notes and zoom chat. And I believe that they're posted on Slack already. Yes, they are. And if you have any slide content, feel free to add those into the shared slide deck, which is listed in the meeting notes. We can get started. One moment. I'll. Share my screen here. Okay. So this meeting is being recorded and it'll be posted to the CNCF YouTube. Organization and the. CNCF CI. A list playlist. If you'd like to share some items, just add them to the agenda and call them out. If you're having problems accessing the doc and you'd like to add something, then I can add it. Or someone else can. Seeing a chat message. That's just mine. Okay. And I think that covers it. So this is something to bring up for folks who haven't been attending. I'm seeing some names that I haven't seen on here. This group. Is for discussing any CI topics specifically related to CNCF projects. But this could be projects that are. Focused on CI. Or tooling. Or talking about. Maybe problems that you've solved or you have questions. That you're trying to resolve issues that you think would be. Useful for the whole community. And. I think that about that about covers it. There's service providers. Like packet. And they do a lot of work with CNCF. So. There's a lot of items where they. Focus on trying to help different, all the different projects. So I think with the community that we have, we can cover everything from. Development and tooling all the way through. Running staff on different. Platforms, including arm and X86. So with that. Seeing the first agenda item is. On the litmus chaos. Do y'all have a. Slides other than it looks like you're dropping them in. It was. Yeah. I'm not going to take a lot of time. This meeting, it was more of a question than. You know, a big topic. We presented about. Litmus chaos in this. Our group meeting. Some time ago. And we also went ahead and raised. Our PR. To add chaos stage to one of the projects. Thank you. For giving us comments on. Arm support as well. So I think. We have added that. The litmus team. My question really. Here is. You know, some kind of a coaching that we're looking for on. How do we actually work with. Other project teams. For example, code DNS. We do not really know. The deep testing required. Inside code DNS. We are chaos. Infrastructure providers is a project. As you might have observed. We are now trying to donate. This project into. C and CF as a sandbox project. Last week we had a meeting. To propose it. Hopefully that will get through. The whole idea here is. We want. We think litmus will be useful. As a CI project to introduce chaos for a lot. Many other. Projects, but we don't know where to get started with them. We don't have resources to go and write chaos for every project. That's part of. Since here. So is this the right working group? Or how do you actually guys. Work with this project teams. Can you plug us there? Those are really our questions. Taylor. Thank you. For that. And thank you for the adding the PR. And I. I saw that you've made the update. So we're asking about for arms. So thanks for that. And I think as far as the PR goes. We're. Probably at a point where we can review that and merge that. So maybe the other part is the, the focus then. So engaging other CNCF projects. Right now, I wouldn't say that. This. This group and meeting. Is covering that. Where. You're going to be able to present here. And it's going to ensure that it's getting it to say core DNS or. Any other project that you're wanting to. Try to help add any chaos testing. That said, I think. There's probably room for that. Where. Maybe this group could move towards something where. If there's general CI. That could help all of the CNCF projects. And how do we disseminate. That information. And I think this is probably. More than what we could handle in this call, but it's something that we've been talking about how to, where do we want to go with this group? And some of that would have to do with, is it. Is a working group the best option? Or should we look at switching this to be a. A user group. Or something else. So within the CNCF, there's different type of communities and they serve different purposes. And potentially what you're talking about may fit better for a user group. And I think we need to explore that. But I'm happy to add. Add chaos. A testing is one of the items that would kind of drive that conversation for what the CI working group would do. And then in the meantime, I can say. Going, there's a, there's a few places. Number one, of course, you can go. To all the pages or the websites for those projects. So core DNS has a contributing guide. And all of the projects have their own contributing. So that would be one place that you can go. The core DNS I know is active on Slack. So there's the CNCF and Kubernetes. They have a lot of the CNCF projects are going to be active in those. So if you want to have some type of live conversation to. That would be one of the areas. And then probably the user mailing list would be one of the first places to start. A conversation where it may be kind of a longer async. Type of conversation. If you're wanting to raise that there. So the GitHub, which allows you do like issues and at whatever they're contributing guides are. Slack for live conversations and then mailing lists. And right now it would be targeting each one of those. There's, there isn't. Anything at the moment that's going to come in and immediately get applied. To all of the, all the projects. So I would say. My opinion on this. Would be if you went to the CNCF. Main CNCF page and you started the, the projects. I would focus on. The graduated projects and then move forward to the incubating so you could just go down the list. So core DNS is one. And reach out to these individuals. And you have all the links and stuff here. And I'd probably get some type of pitch on engagement. That's general, but then look at each one of these. So you can see the contributing guide for core DNS and those. That's where I'd start for right now. And in the meantime, or I should say at the same time. I'm interested in looking at what would the ACI common group for CNCF be able to do for stuff like this. Okay. Great. That'll be helpful. I tell her, I will reach out. The common entries of all these projects. And then if you could make the chaos testing is a common. A topic that might drive more community engagement as well. But our goal probably is, you know, we've been using litmus quite effectively and to bring out the bugs or restrictions in a couple of projects, namely opening BS. So we think at least for Kubernetes main project itself, this could bring out a lot of value. So if I go to CNCF.ci, we see the pipelines, right? Or those pipelines, the tests for that, whatever have been written in the CI pipelines where they contributed by the projects or you are mostly they've been written by your team. Mostly I was interested in how well the project teams are interacting with you. So I could be a platform for us to at least advocate chaos testing. Okay. So I'm going to step back here. I can go back to CNCF.ci and talk about that as related. One of the things that to keep in mind is all of the projects are running independently. So they all have their own contributing guides and they all have their own process. There has been some interest in what are common services that could be provided. There's no structure right now. Primarily in my mind that's because the projects are left to run independently for most of what they need. And some of the needs are quite different. Prometheus, for instance, does some pretty large scale performance and scalability testing. Core DNS doesn't do the same type of testing for say their integration and E2E tests. They do a different type. There is I think some possibilities to do something like a service where you could say all of them maybe could get chaos. But that sort of thing I think would be much further out. So right now I would think if you want to get chaos testing used by more projects you're going to have to go after them directly. And then specifically with CNCF.ci, the first thing to state is it's right now in a maintenance mode. There's focus has been moved to other CNCF initiatives right now as primary. It's in a maintenance phase. We do have, there was some planning on the roadmap in the next phase is beyond this, which would allow for and prior talks we actually showed some mocks and stuff what we were looking at. But would allow for adding more tests that could be run. So it would be a perfect way of saying we want everything to do chaos testing. We also want them to do whatever other testing is desired and plug those in and have those options available. That's future. So right now what you're saying is you're going to have builds, compiles, builds, unit tests, which are either going to be running within the system, which is based on GitLab, or it's going to be all on the remote system. So Core DNS has their own CI system. And if they have everything we need, including published status and artifacts, then we integrate with those and pull them. So we're not rerunning them. Then we pulled artifacts and do deploys. And then there's some type of smoke tests. So very, very minimal tests to ensure that it's up. You can see that we have some failures right now, some stuff going on, at least on this set, whatever it is. And, and then the final what we have here, which is you can see is all NA. There was a point in the past, it's been disabled for a long while, but there was a point in the past where we were running E2E tests. That would be, I think, what you're asking about with regard to what tests do we run? Those tests, where possible, what we wanted was to run upstream tests. And some of the projects had E2E tests that we could run, but it was very limited on which projects. So those were disabled. The long-term goal was to work with the projects so that we could have the people that actually know the details of the, how the application should work to have the E2E tests that would run. And then we would source those and be running them across the matrix. So that, that's another future item. If, if and when we move out of this maintenance mode and start working on the other, but right now those are disabled. So some projects do have E2E tests and if you're interested in contributing like other types of tests, then I, I definitely think some of them like Cordeon S, Hermitius, or two of them that I could think of immediately. And that, I don't know, Denver, Watson, or if any of y'all remember any other projects that were, had E2E tests. Feel free to speak up. Taylor, I have one question. So we are adding the PR for the test stage. So will it trigger in the dashboard? In the CNCF CI dashboard, there's one stage called test. Sure. So the PR that you're adding for this chaos stage in the pipeline, that would end, that will end up affecting the test. So as, as far as the pipeline goes, you would end up, if, if there's a failure, let's say that this is specific to Cordeon S where right now we don't, you're not doing chaos for all of them. But if, if the, this stage fails, then we would show a red failure up here. And then whenever, let's say, I'm going to click on this. So this goes to the GitLab. We would end up with the console log on the job and the, the stage for the chaos test failure. So it would end up affecting the dashboard. Is that what you're asking? Actually, this disabled test stage. So I'm asking for this. Actually, I'm adding that chaos for this stage, test stage. I may be misunderstanding. Are you asking? In line number four. Yeah. Oh, to run it. Yes. Yes. It's part of the test stage. Yes. Yeah. Denver. Do you have any comments on that? I think the question is, will this affect the Cordeon S? So maybe let me step back. So the dashboard is a custom view. Of status information from multiple places. It's not just GitLab, but we source the data from multiple places. The upstream. We have our own. Status repository that has an API that we, we combine all the information and it's retrieved from there. So this is not a specific view of the stages. It's, you do see it's similar bill deploy test. It's not exactly. They're not exact names. We could name these anything. So it's not exactly specific to GitLab. But I think you're asking. If, if the PR is merged, will the test column for Cordeon S then show. These badges. Okay. Yes. So then Denver, is that something you can respond to? It won't. Yes. Yeah. Yeah. Not yet. We haven't enabled the test column yet. So. Yeah. I think there may be. That's what I was thinking Denver. So it seems like there's probably another step before. So this is enabling it within the GitLab pipeline. But. Because of the disabling it in the past, there's some other stuff that we'll need to do to enable this on the dashboard side. Okay. Yeah. Fine. But it will run on the GitLab, right? Yes. It'll be on the GitLab. And once that's passing, then we can focus on enabling it. Okay. I understand. Yeah. Thanks. And we can follow up with you on that offline because there's probably some items that we want to talk about where it's going to go rather than going into it's basically not going to show up on production from the start. We have multiple environments, but we can follow up with you on that and as we're testing. Yeah. Sure. Thanks. Yep. Thanks. This is very helpful. I'll take your advice and reach out to the individual projects and we'll keep lurking in here as you come out of the maintenance phase and we look for the contributing chaos as a general stage for whatever we do here. Thank you. Sounds great. Thank you. All right. So next item. Overview of the integration testing. For the CNF conformance test suite. And that's Watson and myself. What we don't have in here Watson is a quick intro to the CNF conformance suite. Do you want to give it or do you want me to give it? You can go ahead. I've been talking for a while. So, yeah, the CNF conformance suite is a bunch of software that is made to see if a CNF, which is a cloud native network function is compliant. Now there's multiple aspects to that. But basically, it's trying to see if some networking functionality or some networking software is cloud native. Now there's a bunch of principles that go along that and Taylor's displaying some of those there. Compatibility, statelessness, security, scalability, configuration life cycle, observability, installability and hardware resource and scheduling. Historically, some of those things within the network world were incompatible with cloud native. A lot of that is not automated, not elastic, and not a bunch of these other properties. So the conformance suite is meant to help users within this space that would be the service providers like telcos, like a Verizon or AT&T or something like that to be able to see if a network function that they purchased or if it's open source that they downloaded and installed is compliant and cloud native, let's say. So along the same lines as maybe the sonnabooly and the conformance suite for Kubernetes itself and seeing if a flavor of Kubernetes is conformant to what Kubernetes was meant to be. That's it. Does anyone have any questions for that? Okay, we can go to the next slide. So the conformance test suite itself needs to be tested. And another thing I forgot to mention is the CNS conformance test suite is a CNCS initiative. It's not a project. The CNCS initiative is still open source and everything like that. But as I was saying, it's a test suite and it's kind of weird because to think about it, the test suite itself needs to be tested. So supplying, say, sample CNS and one of the CNS we use happens to be coordinates. It's the first one that we're using as a test as a layer seven application layer network function. The conformance test suite is a CLI right now. So everything's at the command line. And so our test suite has to test the CLI and has to install CNS and has to test the CLI and we're calling that the integration testing. We're not doing much unit testing right now within the suite. Next slide. So for the conformance suite, right now we're using Travis and everything that we're doing can be seen inside the Travis YAML. Let's see what else. Within the Travis YAML, Travis installs crystal, the language that the CNS conformance suite is written in is crystal. I'm going to talk about that a little bit later. And then we have to manually install going. We have to install going in order to install kind and use that to of course install the network function to create the Kubernetes cluster to install the network functioning. And then we run the integration test, which is just the command crystal spec. And then I guess we can go to the next slide. So crystal is, it's similar to Ruby. So it's made for humans. It's easy to read the communities that are out there that are familiar with maybe dynamic languages like Python or Ruby. And within the CI CD community would be more familiar with it and may find it easier to use. It's statically typed and compiled. So in that sense, it's similar to a goal line. And Travis, we were very familiar with it just setting up all the complementary projects to the CNCS CI. So we just reached for Travis. We're probably, we're not tied to Travis. We're probably going to change the GitHub actions pretty soon. And yeah, that's pretty much it. Any questions? Is there a, this is Ed from packet. Is there a corresponding set of interoperability tests that's also going on perhaps in a different project? So you say interoperability. You maybe you're saying maybe platform type tests or what do you mean by that? I'm thinking more of the, in addition to testing conformance with the specification, you demonstrate that. To conforming applications can actually talk to one another. Right. So we were calling, so within the network function world, we were calling that a use case. So if we were to say, take core DNS and then say, maybe try to use it with a, some other layer to CNF that uses VPP for instance. And we would take both of those and put them together. The CNF test bed has a bunch of use cases and we wanted to be able to maybe grab some of those in the future. There's lots of use cases out there and we want to have a strong narrative to be able to test multiple. What we call CNF that together, which would make a use case. So yeah, the answer is, yeah, it's in the future that, that we'll be looking at that though. The CNF test bed is, is where we would test use cases right now though. Okay. Thanks very much. That answers the question. Okay. Anyone else. Okay. I think that's it then. Okay. I'm happy to start up here. I'm at Ville Medi from packet packet is as of. About a month ago and equinex company. We just got acquired. So a few changes coming my way, but nothing specific of. Nothing specific to note. So the project that I've been doing for the past couple of years is called works on arm. It's focused on arm compatibility for various projects. Quite notably for CNCF project. So next slide gives a quick overview of things. So I've been following the CNCF arm dashboard and trying to figure out where to put energy in to. Improve the state of things I spent about an hour before this college is walking through each of these projects. And this is a working document so far. The good news is that core DNS, which was mentioned earlier, is an exemplary project with as far as I can tell from. Everything I can see full support in their current release for arm systems. And therefore. CI testing on top of that should have a very good chance of success. For many of the others. And there's details all through here. There are one or another issues involved. Typically in the build process. So in the. In the CI part of CI CD. Where the infrastructure that's available to. The individual projects does not easily lend itself to building binaries for multiple architectures. It's not completely true everywhere. There's at least one place where I saw project distributing. Arm and S390 binaries. So they had some multi architecture capability. But typically in my experience with other project development, there's a somewhat sometimes messy build process. That spits out a binary for X86 and that. To re architect it for spitting out binaries for multiple architectures takes a bit of work. So there's a bit of work out of us. The Prometheus is another example. So. The CI process produces a. A binary or it can be coerced to produce a binary. But one of the downstream consumers of those binaries is. Registries. And up until relatively recently. Not every registry and I'm looking at. Quay right now. Had full multi architecture manifest support. And so there's been some delays in getting full architectural. Diversity. As a result of that. I don't want to go into lots of these in lots of detail. And I don't have all the details yet. But what I do know is that. I'm going to recommend to the works on arm project folks at arm. That we go back to each one of these core projects. And understand a little bit better. What their limits are and how they could be improved and. See what we can do now that we have a pretty good baseline from the CI process. Just to see what we can do to. Urge some of these things forward either with. Providing build resources providing. Manpower for. Projects or what have you. To bring each of them and all of them up to as good a state as core DNS seems to be in right now. And with that, I will take any questions. I'll just comment that this is similar in some ways to the chaos question. Of how do you reach all the projects. And my experience today has been you reach all the projects by reaching out to each of the projects. And there's no. Necessarily easy way to reach them all at once. But it's not. There's not really a single centralized. Nob that you can twist to make something new. Thanks. Yeah, that really helps. And also the new CNCF services desk. And that's something that we can all. Someone can approach maybe. It's a common way, but. That may be futuristic. Right. So. It really helps in this comment. Yeah. Yeah, it's the experience I've had has been in. You find the issues for each of the projects. You read some of them to see what they're actually doing and see how, you know, receptive they are and see where they stand in terms of project maturity. And then you start to chip away sort of one at a time. To figure out, you know, which of these. Potentially. 10 projects. Although really own app doesn't fit exactly here. Say which of the nine projects has the most likelihood of. Being in the spot where you could describe a. A port or a test. That would make a difference to their, to everyone's lives. That helps. Yeah. I'll reach out and. Let's see how it goes. Yeah, please do. I should be easy to find. But if you. It's just edit packet. Yeah. All right. Okay, good. Okay. We can advance the slide. I'm done here. Oh, Ed, I was asking. And I was apparently muted. Are the no binaries built? Are you referring to artifacts during the CI pipelines? I'm referring to artifacts from the original distribution. So in cases, for instance, envoy Jaeger linker D. It may be the case. That the code compiles for 64 bit arm. But they, the project itself. Does not do that compilation and does not release those artifacts. As part of their official build process. Okay. So that's, that seems to be in sync with what we were saying as well. We did find some. Areas where things were built, but maybe not released and other, other items on, if you go and look at their systems versus. What's published. But the, the goal that. You're talking about was something that we were wanting on the same CFCI. So that we could do integrations. Directly for each of the platforms. And found the same problems. So I think getting those projects. To. Have. Artifacts published. And the status for. All the platforms. Publicly available. Is a good goal. That would be helpful for many projects. And that would be helpful for many projects. And I think one of the things that we were looking at. Beyond that was how do you get them to. Publish and have. This would kind of be like that maybe onboarding for chaos or arm or anything. Making it easy to publish because you maybe follow us. That says we're going to always say the architecture. We're going to. Give artifact URLs and other information. There's no real standard around that. But I think that there's some value. And maybe pursuing that. I agree. There's certainly support. That needs to happen in both directions. It really helps for projects to see. You know, if you have customers or partners or other folks pulling and asking for things. And as well, you have to do a certain amount of pushing and. Providing requests for. Looking at code and. And other sorts of artifacts, you know, artifacts that can help things out. Generally speaking, projects that are written and go. As long as they have at least some conception that they're building for. More than one. But often the default assumption of people in the world is that there's one computer. Out there and it's a virtual machine and it's a Intel virtual machine. Which is different from, you know, the actual world that we actually live in. I'll follow up with you. I'd like to talk more about maybe some. Ways to move forward on. And getting more projects to build and publish. I'd be happy for that. And. I could say we've made a whole bunch of progress, but there's more progress to be made. I'd look forward to. I can share this worksheet. It's a working document. So. If that's a helpful start or just a helpful. Spike in the ground as a way of having something to talk about where we stand and what the status is and. What the next step would be. I think that might be a good, good way to go. Yeah, sounds great. And, and we have some. Some data too that we can share based on our research. All right. I think that's it. If there's no other. Questions or. Comments. Topics. Thanks everyone. Thanks to. You're welcome. The next meeting is scheduled for Tuesday. April 28th. Same time 12th. I would add only one. I would add only one thing before everyone goes. Before that meeting on April 1st, there's an online conference called virtual rejects. This is a public event. It's called virtual rejects. Was to have been held at the same time as. Q con. Or actually the weekend before for everyone's paper, who was rejected from Q con. This is a fringe festival style event. I don't know entirely all of the details of it. But it's some number of days away and should be of interest. Thanks for that. Thanks, everyone. Thank you. Goodbye.