 Good morning. It's Lucina Stricco. We'll give folks just two more minutes if We expect anyone else to join the CNCF CI working group status update call today Good morning. I'm gonna try to share my screen here. Everyone see that Yeah, looks good. Lucina. Can you post the? Notes for the CI the link to the CI working group agenda notes. Yes, thanks. That Today I'm trying to bring that up. The only thing I've seen was update on the cross club CI so we can give that We may have some other items topics For next meeting if you have some ideas for what you want to talk about you can post in the that public CNCF CI working group agenda notes Just add yourself there for discussion cool so No, and here they are. So there's the this is the zoom meeting. Hopefully it's gonna It's changed a few times the actual ID. So I would make sure and use if you're Going to URL use the short URL they have and that'll get you into the right place This also lists on the public calendar Second port to say so from the cross club CI project We want to go over the current own app integration sort we're working with the own app project, which the Linux Foundation project to integrate with their CI systems and Do some testing of their service orchestrator component eventually adding more So The current goals for the initial integration is testing One component and whatever the minimal requirements are for that We'll be trying to use the upstream Artifacts that they've provided from their their build system and Their registry and Then try to deploy those on top of kubernetes and run e2e test All is on the call. I'll do a quick overview of the cross club CI Unless is there anyone on the call who hasn't is not familiar with the cross club CI project Okay, I'll do a quick go ahead. Yes My name is Francois to I think we met already during the cubcom at Austin I attend today and maybe I will attend next a meeting But my focus is typically on core DNS and I was expecting to have To have a better or better or something we can extend the integration test of core DNS So that's why I attend today. In fact, I don't want to interrupt your Usual agenda. I just want to have someone to talk to on how we can go ahead with core DNS Absolutely. Yeah, I recall our conversations good to hear from you and I think the Some of the integration that we're doing with own app will relate to what we want to do On the with core DNS for what we had talked about previously on the e2e test I know there are several ways of testing including as a plug-in for Kubernetes running standalone and we had talked about several ways that we could test and We'll definitely follow up after this Let's go over some of the own apps stuff And then maybe if we have some time in this call We could talk a little bit about the current status of 80 tests and core DNS. That's on good Okay, thank you very much. Yes Okay, great. So I'm cross-cloud system the cross-cloud CI system it's Split into three main parts build builds cloud provisioning and then app deployment which it does the e2e testing and All of these parts can be used independently. So For systems that projects that don't have their own build system We can I'm going to kind of skip down in here. These slides will be available for a further overview Let's see I think I went too far so On the build system itself. I Can bring this up same CF CI You'll add that same CF CI the build system we can mirror and the projects Source code and do builds and compiles. So for say core DNS, we're just running the upstream build Scripts make or whatever it may be depending on it and then creating those artifacts and pushing them up and then once we have those will do We'll be using our own Provisioning for Kubernetes and we provision the different clouds and then once we have those we'll take the artifacts from the builds on each project and deploy those and deploy Potentially an e2e test container or whatever we need to do to run the e2e test so for own app itself, let me go to a different environment here What we're doing is We are Using and we have some builds running right now because this is in progress For the build status what we're doing is we are pulling in the information from the own app as CI system, so they use a combination of Jenkins and Custom Chef Deploy scripts They have a nexus for their registry container registry and some other items and For the stable release what we have here v11, which is what they call Amsterdam we can their releases are not too frequent so we actually can Specify those like we do for all the other ones and say there's a new release. It's going to go and Check that that's available in the registry make sure it's downloadable to the system Externally and does some other validation so that we can use that artifact in later stages for the No, we've had a problem here. Maybe check that out. So for the let's see it looks like there was a when it went out for the The nightly release is what they have so they have nightly builds of their master and What we're doing here is we're actually going and looking at those nightly builds on Jenkins and we check the status We actually had a failure. It's showing that there was a failure on their last nightly. So this is whatever they ran yesterday and We can see we tied that into their commits for their the service orchestrator project here, which is a mirror of Garrett, which is what they use internally and They had some errors here and that's what caused the build So we're not running the build But what we're trying to do is integrate with whatever they're doing so that we can update here if it does pass we're going to set all those six success and then we continue on and Validate the container that we figure out dynamically what container did they create based on this and then they registered that into their or they uploaded that to the registry and We'll make sure that's available and then we create all of our pinning configuration for both of the releases that we can then use for app deployments and run those Deploys of the containers Once that is complete then we're ready to do The e2e test and that's part of the app deployment phase. So if we In these pipelines here Well, actually I clicked on the wrong one go back. So if I look at say linker day, there's an app deploy and That didn't work with tap in here And we're let me know if I'm going to the wrong area looking at some live stuff here Okay, so the app deploy phase in this stage We actually run not we run the actual deployment using helm to put the application on kubernetes and then by default what we're running is We'll deploy a e2e container and then run those Tests that are within the the e2e container for own app they actually have and This isn't something I can show you because we're in the midst of doing this but they have their own test framework and they use something called robot and Have a large set of test robots a piece of software that allows you to define test and you can split those out into different files if you're familiar to cucumber and Our spec it's it's similar to other things except for a kind of a higher higher level defining the test and then you go in and Implement those so we're going to be use utilizing those for our test and We're doing a subset of those tests that are just specific to own app so once we get done with the app deploy for the different releases will be focused on the the actual e2e test and running those containers and Hopefully not having to do much modification. It looks like they go through and Do most of the health checks and and validation all the way down to the database for the service administrator and if possible, we'll be able to use those as is and that would allow us to Let own app run the the actual The test themselves and I think that's the main thing so let's see we're having Bring this up. Here's the Main ticket here. We've kind of split these out but if anyone's interested to come and look as the progress goes so from a core DNS perspective this project the own app integration that we're doing is Quite a bit more complex. I think than what we were looking at for core DNS and I think if we Look at what we're doing For own app is kind of a reference. We could probably go with something even simpler If if we can define it sounds like we have the three different areas that we could test so I'd be happy to switch to that if there's no more questions about own app the own app integration itself Does anyone have any comments or questions on what we're doing here? Let me I guess let me finish this out and then maybe we can focus on do a switch to core DNS There'd be another topic. So We're gonna after finishing or I should say besides the own app Integration that we're working on and currently that deployment We will be enabling the IBM cloud and Soon and then open stack is We're hoping to have that up soon after we're collaborating with One of the open-sack folks to get that integration Fluent D is in testing so that should go active pretty soon. We have a face-to-face and CI CD workshop some of y'all may be on that That's in LA just before on us and then we're gonna we're helping at the CNCF booth will be demoing the cross-codd CI project and some other items during on us We're planning on attending cube con in May That's in Copenhagen any questions before we switch over and maybe talk a little bit about talk a little bit about Core DNS and ad test. Do you want to give us status update on where? For things are with ad test on core DNS themselves Yes, I can so sure core DNS We in fact we have two kind of tests plus the Kubernetes end-to-end test so If you go yes in core DNS, so what is sham we make a commit There is a CI that start and that run kind of tests that are defined in this repository But we have also if you click on core DNS, we have a core DNS CI repository That run a bunch of tests. We did not want it to put inside directly inside our code If you go to no, sorry to get up core DNS CI Here you are in core DNS slash core DNS. So yes, let's see. Oh Yes, there we go this one. Yes So these are a bunch of tests not that we are running that we we have a specific environment packet IO to run this integration test that are mostly not not only but mostly testing The interface between core DNS and Kubernetes but it is something that is a side of the regular Travis CI if you want and We are maintaining this part. It's only on one platform packet and Then you have end-to-end test, but these one are the one of Kubernetes. So maybe we don't need to To take them as part of core DNS what happened is core DNS is going as the default DNS service of Kubernetes Cordiness itself is is a DNS service that is not linked at the beginning to Kubernetes But it has a kind of plug-in for Kubernetes and we are very I Just say that very very attentive to to to make it work with Kubernetes So that's why we have deployed a specific CI and then we will We want to extend this CI to have some long run tests some performance test So I tried to know to not overlap with your work or to consolidate with your work So because you are deploying in several In in you are already deploying on several Type of container or several platform. I would like to take advantage of this deployment to make run our CI into your environment or your deployed already deployed environment So my question is how do we tie it together? Is that clear enough? Yeah, thanks for the overview and I think What we'll need to do is I guess work with you to Try out this CI and fully understand it and posting a ticket That we previously had and we wanted to follow up on So yes, there was a ticket and I had I had a Comment maybe in January after our meeting on on QCON. Yeah, but looks like it was not on the wrong. It was on the wrong Location and and someone just move to the problem Yeah, there was some reorganization as we're releasing some of the other components So I think that was probably still a good start to pick back up with what you had put there and then maybe reviewing this ticket and then saying I Don't know if if all parts of the CI testing that y'all are doing are Applicable it'd be good though to see what's reusable Yes, I was reading it in another window here, but I think we could schedule maybe a call after we've reviewed and It looks is this something that we should be able to run directly. Is there anything that we need to know No, yes that right now that is we have a I understand is in fact I am making the link be here It's not to me that that did that that work, but that is a kind of Permanent system that is just waiting who Kevin to run to take a new and you build and run in Existing environment. So it's not exactly how you are doing. You are doing a deployment of Kubernetes I understand or deployment of something and then run the test, but I Don't know we are open to find what what what is the best to to to keep on your Organization but what I want to know is how where do I write some tests to make them run in your environment? for example at first and then can we share this one if if there is a way that these tests are are Running in your environment, then we will avoid to maintain the this environment for our test if they already run on some other environment So that is I think the tying part can be talk what we can do is okay read this document read the the my ticket and then I will you can either add to to you to the When you're ready just say it on the issue on Sorry, the cost cloud issue CI issue 23 and I will we will schedule a meeting with also Chris However, this guy here that did this for this work He's working with me. Okay Sounds good. So in general how the tests work or will truck We will figure out what we need to run So if if you had some things as simple as make test If you had a task runner of some type then we would add that to the Deployment so during the app deploy we would have a job That starts and it's the only thing it would run would be say make test or make e2e test Or if there's a script that we run, that's fine Essentially, we need to know how do you run your test and then we add those tasks into That that phase so I was I was showing earlier the the pipelines here, so Right here so when we get to After this app deploy and we're going to run this e2e test job We will define that job based on whatever information that you have so if that meant I'll go back here, so if What what I would anticipate is whatever runs whenever you add this flash Integration in the comment for the PR that's triggering some set of commands It seems like that might be a good place to start Whatever command those triggers to run so that's probably in here somewhere Let's test deployment test kubernetes, so there's probably yeah, let's say like this test kubernetes So we probably don't need this fetch coordinates So there's some other things build Docker that we're handling so it's probably this test kubernetes Job, which looks like right here this line 75 might be it so Potentially, this is it and all we need and then if that's the case then what we can do is take whatever the needed parts are and this this looks like the code is all here and so it's it looks like we'd package up this test and Then we would want our Job to then run the make test or make test k8's and That should be a good starting place. That's kind of what I'm thinking, but that's Generally what we'll do we'll figure out what is the project currently do for 80 test if anything and then try to run those as Is so if that works, that's fine and The thing concerns that I'd have or what are the requirements for this to run does it expect for the For the core DNS to be set up in a certain way I see some stuff on the this is the repo here, but if we looked at okay the build So is there anything specific for core DNS that we need to know when we're deploying core DNS Should anything be started up? But I think this is a good start. We can take a look at running go test from this repo on Our kubernetes to play with core DNS. Okay, so but in your if you go back to your So your workflow for the test is there is something to deploy Before it is something that happened during the end-to-end task. No the container you deploy the compile I'm deployed Yeah, so So we're gonna deploy both the container the a container that has core DNS and whatever containers are needed for to run the application and then we've Normally been deploying a container which has the ed test and then we run those tests and which is often Though the start command for when the container Starts up. That's usually what just immediately starts if for a Project that doesn't need a separate container or maybe it it should run the ed tests that are in the actual application container We could do that as well. So I'm not sure exactly what y'all have it may be that rather than Deploying an ed container separately and then that starts up and runs it may be that during the ed job we actually go out to the core DNS container and Run something that was already built and inserted into the app app container Okay, but when you say Kubernetes when you test core DNS you always Launch a Kubernetes container also Because Cody always run So are you referring to what do we do to test Kubernetes? No, no, no in the case of Cody and as you say oh we run so We we use the Kubernetes container, but that means in in case of Cody and as Testing you you deploy your Kubernetes container or you deploy if we need it so After deploying after we have Kubernetes up on all the clouds on their provisions Then the next step is to deploy a container with core DNS and That happens right here. So this is in the cross project. So and then that gives us like right here. We have a deployment to AWS for and We deploy the actual a container that has core DNS and that's then running on top of Kubernetes, so I realize that core DNS could be a plug-in for Kubernetes for its DNS It can also be run independently just as a DNS server. Yes We're not testing all of those in one go right now That's probably something that we need to look at like all the different I guess the different types of tests would require different deployments So if we're going to test core DNS as a plug-in for Kubernetes rather than using another DNS server for Kubernetes then We would need to do another set of tests and probably another deployment Right now we're talking about core DNS Running as a just an application independent Yeah, I've Kubernetes Okay, but anyway on on Kubernetes side, so let's that is core DNS itself as as a DNS service running in in a pod is what you mean But on Kubernetes side, you are running the end-to-end test of Kubernetes, I guess If we are looking only the Kubernetes tests you are doing You're running the Well, so what we had been running and so this is what we need to talk to you about what would be the different types of tests that we look at what we were running was DNS tests your general DNS tests and not Kubernetes specific Yeah, we weren't running tests that are validating and that Kubernetes can talk to for DNS Okay, what I wanted to maybe something that we need to do a follow-up call to dig in a little bit more, but yes go ahead No, what I wanted to underline is Core DNS as the DNS service of Kubernetes is already tested how in Kubernetes part of the Kubernetes test So part of the core DNS test then we need to test core DNS as a server, but not really as the DNS service of Kubernetes I Well, I would like to make a scheme up for next time too to not confuse you Yeah, absolutely. So when we're testing Kubernetes so from the Kubernetes side here When we're testing doing conformance tests of Kubernetes We're not currently plugging in all DNS Servers that could be used for Kubernetes and testing all of those So we're right now. We're only selecting one that might be something that we're going to add in the future So we go we've tested DNS With this DNS server we're going to test with core DNS. We're going to test those right now. We're selecting one And yes, then saying that that's working, but yeah, and then I mean that makes sense right now the default DNS service of Kubernetes is named QB DNS and I guess it is right So soon for next version of Kubernetes the default one will be called DNS at that time You just change this QB DNS by core DNS and you run the same test. I guess Absolutely, so from the standpoint of of the cross-cloud CI testing like what can we do and what have we done We test we've tested core DNS as a plug-in. We've used we've gone back and forth Since we don't have no nothing currently displayed from a dashboard side So when we're looking at cncfci here, we're saying what is tested here We don't have a way currently to display Which DNS server is being plugged in From the back end though. Yes, we can we can select what's there So build process and everything else we could use either one So we're good to go when it goes to fault the core DNS And the conformance test as you're saying part of what they do is actually validate that DNS works So when that goes into place it will be tested Yes, okay. Yeah, that's good Okay, let's let's focus on core DNS test and the way to my my my main focus right now is when I want to add Test I would like to consolidate to is what you are doing. So we do not make two branch and two two kind of Two kind of ci test So that's okay. Let's let's um look on your side. I will comment I will look with Chris however, so we can explain you what we are doing right now And I make a plan so we can Have all our all our tests or the one that makes sense run inside this dashboard or dashboard. Yeah, so I think The next step would be what the what what happens what runs whenever you Get this pr that has the slash integration What are the actual commands that are run? And that's I think what we would want to plug in here for our ad test And try to see if we can use those upstream The way that you're running them now And see what may need to be adjusted if we can and if that makes sense Um, and like you said, ideally we can use these rather than having two sets of tests Yeah However, you run your dashboard is running on after commits. It's not on pr Oh, I'm wrong That's right. It's after the it's after the commit So whenever by the time that you get to this stage, we've already done a build the builds have passed And we're now doing the integration test For the passing builds the commit Okay So during the build phase is when So during this phase whenever we do compiles and everything is where You would say unit tests would be um run And if everything passed here, then we're going to make those the compiled artifacts available And then we go through this stage and we would run those integration tests Okay All right, let let me uh Look with on my side with Chris however, so we make a small plan and we I will welcome back to This week, I guess I will send you an email or add to that if I add to that issue. That's okay Yeah, absolutely. I Think that's in chat that was on the cross cloud ci. That's issue 23 That's posted in the zoom Zoom chat Yes, 23. Yes, called cross cloud ci 23. Yes Right. Okay. Thank you very much. I'm happy that we can go ahead on that Great excited to see it Appreciate your patience I'm glad that we're going to move forward again And anyone have any other questions or comments? I think otherwise we're Good to go on this Meeting. Okay. Lucina. Was there anything else that you wanted me to go over? Uh before we end this call I may have missed a slide here That's the one. Yes in progress. We are preparing the next release for the ci status dashboard v 102 and That should include adding ibm to the cloud providers adding fluency to the projects And updating the version numbers for the cncf projects So we'll post an announcement to the mailing list when that release is Up and running Okay, here's france. What's this? Thank you very much. But I have a last question. I see on the dashboard cncf.ci, for example, core dns is already right now It's me to get in I guess to to see what's going on The error seems to be related to Something called a caddy instance And I did see that there was a bug fix A couple days ago My thought is that it could be fixed when we update the version 106. Yeah. All right. Yes, it's fixed Yes, we added the vanderization press make file adapted. So yes, it should be in the one there were six Thank you so much. We will Add the 106 version and if we're still seeing any errors, um, perhaps we'll ping you in the ticket and see if there's something else That has changed Okay One thing that is strange though, but I can see because You have the 106 but anyway, you have the ed Um So the bottom line should work now They have the same issue on the bottom line I'm sorry on which line Um, you have two lines one is a stable one. Oh, you updated 106 already and the second one as As the head. So this one is not working neither status Oh, there is another issue. You have another issue. We'll scan it fine. It's another one Um, okay, I will I will have a look at that because Um, we don't see it. But if you see it, that means we still have a tuning to have on make file The make you are running is a make off core dns Yes, I see that Yeah, we think that maybe something that y'all are running. Uh, we had we're not Haven't incorporated potentially it. Um Into our build process This is what we had been using and it seemed to be good Oh, no Oh, I don't think it is a make file.trillies It's make file um, okay, let me Verify that with a guy here. I always call the ns and I let you know Because my idea is no it's just make file Make my gym Okay Are we looking to so the very interesting? No, I was not looking at that but I will now have a look more often on this test Thank you Here's the make file release that we were using Yes, but it is a make file for releasing core dns So I think you should I think but I will ask the guy you should use the make file itself. No, yes make file Under license you have a make file note. Um, we were using the make file release To create a release that then we Added added to the container that we deployed Well, maybe This is the right thing to do but in that case it is our release that has not been updated. I know we updated the make file very Maybe last day or two days ago. I don't know But maybe yeah, let me You're right four days ago for the make file and then the release was a month ago. So maybe they're not They are not synchronized You're not synchronized. Yeah Are we look anyway, maybe you have the release just those commands And merge to make file so that they're They can call the make file um task to do the builds and then create a release Okay All right, and then the only other is 106. What was that? I mean, I will follow up with for this make file and and see looks like we have not updated correctly all the make files Okay, and you're the same issue Okay Well, good to see Good to focus on that Sounds good Appreciate joining. Um, and if you have Available time for the next Meeting it's March 13th Um, besides um, reaching out to us, of course, we'll Be collaborating just on the 80 test Thanks everyone for joining Goodbye. Thank you Have a good one Thank you so much you too. Bye. Bye