 All right, hello, everybody. Thanks for having us. I'm Mary Parsons. I'm basically the head of DevX at Dish Wireless. And I have my partner here, Watson, presentation partner. And we're going to talk to you about crawling the CNFs. So who are we? I'm going to take a first couple of minutes to tell you about Dish. So I think most people who know Dish know of us as a satellite TV provider, right? So developing CSP technologies in the US. What a lot of people don't know and are just now starting to learn is that we've spent the last decade building a portfolio of spectrum holdings in support of building out a wireless network. So for the last two to three years, our deployment teams have been architecting, building and deploying sites across the United States. Thousands and thousands of them, which you can see on this map, which basically are all edge nodes of a network, right? And this is our 5G cloud native O-RAN network. It's not a telco. It's not a cloud. This is a fully virtualized cloud native ecosystem that was architected by developers for developers. So this is a game changer, right? The idea is that any developer in telco space or otherwise can configure and program the network to achieve their requirements in their space. With the intentions of, we've heard this word a lot today, speed, fast, improve the velocity of development, and then also open. We fully support O-RAN. We're very committed to O-RAN. We're very committed to an open environment and supporting multiple vendors. So I mentioned by developers for developers. This is one of our internal development teams. Just FYI, I am their talking head today. This is their work. I didn't take part in any of this. So if any of your technical questions aren't answered today, come talk to me afterwards, and I'm happy to connect you with these brilliant developers so you can get your questions answered or you can have more information. So this is what we're calling a net blocks project, short for network blocks. And they took on the task of coming up with architecturally unique testing environments that can support multiple applications and test multiple applications at once. That's Dash. Just a show of hands as everyone here, who all here has used the CNF test suite or is familiar with the CNF certification? So a few of you. Okay. So the CNCF CNF test suite is a community is amalgamation of service providers vendor partners and CNCF hosted project contributors who are interested in focusing on the properties of high quality CNFs. And so why are we using the CNCF test suite? So as I mentioned, our network of networks as we're calling it, our developer ecosystem is inherently cloud native. So I won't go through all of these with you. You can read more about our 14 principles on our website, but number 12 here is what I'll draw your attention to. So each CNF should be accompanied by test cases to validate the functionality and load test it using automated tools. We want better CNFs. We want the velocity to be able to modify CNFs. We want more CNFs. And so that, you know, at its baseline requires repeatable testing with reliable outputs that developers can compare against. Okay. So a little bit about the origins of the CNF test suite. Several years ago, the service providers and vendor partners got together and said, hey, we want to have our DNFs work in Kubernetes at right about that same time, the CNCF had a very successful test suite for certifying Kubernetes and the CNCF decided to create a test suite for enabling or verifying vendors interoperability for CNFs. So a little bit about. So if you notice this visual on the left, if you know what that is, I'm sorry. This is the old way. We're moving to this brilliant new way, right? So summary of what Nutblocks did. They went from slow to fast. It used to take months to spin up these testing environments that you can test your one application in. Nutblocks is doing it in 35 minutes. They're spinning up a testing environment in 35 minutes that can test more than one CNF at a time. And then when the test is completed, it's destroyed. So slow to fast. Keep hearing about speed of innovation, right? Here you go. Manual to automated. It would take multiple teams, multiple resources, hands-on processes, handoffs, dependencies. We're using CICD automation. It's repeatable. It's scalable. It's flexible depending upon the CNF being tested. Restricted to open. So again, you were held hostage by what your vendor was providing you. Now we can have multiple vendors in one space. They were going from running approximately 10,000 tests today to the world's oyster. It's unlimited. You can have as many environments as you want that are scaling to your CNF and you can test as many applications in one environment as you would like. How many tests can you do a day? You tell me, right? Let's see what that limit is for us. So common themes, right? Slow to fast. We're increasing, completely increasing the velocity at which developers can do their tests, get their results, and then therefore do their development work and we're automating everything in this open environment. Okay, so as far as our process or the dish process and really any onboarding process of new CNFs for a service provider or anyone that's consuming CNFs, you're first going to have to go ahead and read the documentation obviously and locate how to install those CNFs. So that's hopefully within helm charts or some in-band type installation process. You're also going to have to locate how are the install parameters in order to figure out how to completely install everything. So for dish, their process now includes Lambda Functions. So they're going to take from the initial onboarding process, they're going to take the information they got about the CNFs and inject that into the Lambda Function. What we went ahead and did was create an example CNF that we're going to talk a little bit about more later. We used Open 5GS and the UE Ransom in order to validate the dish environment. So they also use EC2 and EKS for their Kubernetes. So after they go through and do their configuration of the Lambda Functions, for this process we'll talk a little bit more about later but then they go ahead and review the output. So why even use Lambda Functions? Lambda Functions are pretty easy to set up and tear down and you don't have any lingering resources hanging around just to show a hands, who all here has used Lambda Functions before? So it's you. Okay, cool. So really the Lambda Functions in this use case, they really just replaced the jump box for calling the CNF test suite. So one of the most often repeated questions is what's tested within the CNF test suite? So here are the categories. We have configuration that's declarative configuration. So making sure that the CNF, the CNF's configuration is all declarative. So using config maps, operators and declarative interfaces, so on and so forth. All of these categories are pointed towards upstream projects within the CNCF. So as far as compatibility, we make sure that the CNF works with certified Kubernetes. So it's just broadly compatible and or interoperable and then also uses what we call in-band deployment tools instead of like bash scripts. So using something like Helm or CAPT or something like that. Then as far as microservices, we essentially just make sure that the CNF is not a tightly coupled monolithic application. So we have a bunch of tests for doing that. As far as state, we really try to shun CNFs that use local storage. So we promote using or managing state with custom resource definitions and also just separate databases. And then you have reliability. We heard the lightning talk earlier from the litmus chaos people. We essentially use them for testing, doing chaos testing for resilience. Then as far as observability, we try to make sure that the CNF externalizes its internal state so that it can be captured with metrics logging and tracing. Hopefully in a way that's compatible with open telemetry standards. And then we have security. So just like an example of security would be making sure that the CNFs are isolated from one another in the host. Some of the projects that we use that are upstream would be a curverno, cubes escape and maybe Falco. So that's really that there. As far as the example CNF that we used, it's open 5G S as a 5G core. And then the UA ran sim to send traffic to the 5G core. We went ahead and open sourced the configuration for the helm charts. And we also created a GitHub action script. So if any of you want to see how to test the 5G core simple one like this within GitHub actions, spinning up a kind cluster and everything there, it's all in one place. Back to the link at the bottom. And just your usual subjects here as far as our suspects here, as far as containers and pods, all of these are inside of the Kubernetes cluster. If you wanna spin something up and take a look, it's out there. And then now for the demo, essentially as we, let's see. As I'm trying to get this going. So essentially before I run this, there's really three components that are used within the dish process. It's really an early process. We have the event bridge, the Lambda screen and the system manager screen. And this is gonna be a really rough demo created by harsh. Let's see if we can get this going. So on the event bridge, there's a rule that at this point in time just essentially schedules the Lambda function. That's the CNF trigger there. And that Lambda function, there's a CI CD pipeline that makes sure that the EC2 and EKS resources are already started. That's all supposed to be composable. Here we have the Lambda function. They have to excuse the blurring. There were some email addresses and stuff on there, but essentially just the screen, it's just the Lambda function. It's just calling directly the really three commands for calling the test suite. You go ahead and start it by clicking the test there and then you'll be running the test suite. This would be scheduled. So within the test suite, this is the system manager screen. Essentially, this is where you would see the output. One thing to note about the system manager screen, there's a 48K limit with that 48K limit for the CNF test suite. That's a week ago a little bit further than that, but you can see the output here. This would be something that you can, if you're using AWS pipelines, you can go ahead and compose this with a bunch of different pipelines. Here's the output. And then you can also download the output so you can get the full output. And I'll just show that here in a second. This is essentially, would just look like something like this. If you're familiar with the test suite, essentially tells you all of the tests and what pass would fail and then you get a summary at the end. And I'm just gonna add in that essentially, the CNF test suite, as far as the reason why you would use it is it's really intertwined, intermingled, with the upstream projects within the CNCF. So, what we do at KubeCon is go out into the vendor room, go out to all the CNCF projects and try to get, and we're also attending the working groups and try to get the best practices from the CNCF project. So from the stateful working group, to the CNI working group, to the all these different upstream project security and so on and so forth. And we try to integrate those directly into the CNF test suite. We'll be back up the presentation as well. Yeah. I just want to point out, our way to spiralize our developer teams do have a get-hug up also for all of our ongoing cloud native work. So I have that link on the last page, as well as some other links to dish wireless resources. For those of you who aren't familiar with the work that we're doing on our network of networks on this developer space, please visit those to learn more. That's all we got. Okay. Thanks. I got news for the audience. We got the audio setup for a Q&A. Now, I'll ask my final question and then we'll improvise a bit more. We got time for that. I'm interested in the net blocks. I couldn't quite figure out what that exactly is. Is it the framework? Is it the product? I tried to Google it briefly, but no trace of it. Sorry about that. So it's essentially the development of those testing environments. So that is the net blocks project. Are those testing environments that can be spun up and down on demand in an eight hour lifespan, basically. And so the team did a lot of work into not just how to develop those environments, but also the automation involved and then ensuring that the CNF test suite was being used in a reproducible standard way. So the net blocks term, the project in and of itself is the development of those testing environments. Thank you very much. So please still stay there. As I said, I'm going to try to improvise. We got some time and we got the mics and the speakers. So we could organize taking the questions from the audience, but since we missed it for the first three presentations, I would like to invite whoever of the presenters from the previous presentations is in there in the room to come on stage and we'll try to do a Q&A round. So if Tom, you are there and then Josh, I don't know if he's around. Come over and Roman Dorian. So we have now opportunity to ask anybody of the initial, or from the presenters in the presentation block, the questions, hopefully you saw the presentations and you have some questions. So I can hand out the mic if you're interested in anything. My name is Evan Anderson. I had a question about this most recent dish testing stuff. I noticed that you're using AWS Lambda and EventBridge and so forth to orchestrate all of the testing. Is that configuration open source and available? Or is that kind of dish network secret sauce? I don't think it is on the GitHub open source yet. I would say stay tuned on that one. We are very committed to not having a whole lot of secret sauce in terms of the development work that's going on and we are very committed to giving back to the community. And so this is, as you can tell, a work in progress. And like I said, unfortunately the team couldn't be here with me today, but I would just say stay tuned to that GitHub and to a lot of our other online resources and perhaps you will see it. Thanks for that question. More questions? Hi, thank you for the presentations. My name is Yuval and I have a quick with regard, quick one for with regards to the testing. If I take it a little bit for the direction of 5G, what is the method with regards to different elements? I don't know what the deployment within dish, but in case that you're taking an open run deployment and not a G node B, so the validation of the layer two and layer one is very problematic to do it in a cut environment. So what is your solution for this? Meaning for the RU and the DU elements within 5G network, how do you make sure that everything, you have a full blown off flow work that starts with the five level to the layer three within your solution and not just one element or you take it into the G node B and then the question about acceleration, any kind of proprietary that needs to be part of this kind of deployment? Right now, yeah, right now we don't have a lot of 5G testing. So that's something that we're supposed to be concentrating on this year. Definitely anything, the way that we prioritize, we look at what the community wants and then we can implement that. So if there are things that you think should be higher priority than others, then we can definitely get on that. Thank you, we have another question over here. It's possible to ask question to any of the previous presenters, not only the last one. I will actually. Thank you, I have a question for the Swisscom guys and also the photo phone guys here in the room. Thank you for the presentations first of all. And both you stated that you need to connect to your backbone network. Do you have software defined networking in place and IPAM solutions which are API driven? Or is it all IPLDs I saw with the Swisscom which is steadily configured? So if I had the question right, you were asking about automation of the software defined networking and the IPAM and the kind of underlying infrastructure. Yeah, that was the topic for another talk that I think was on the wait list that I was gonna do. But one thing we're doing is very similar to what was described by Swisscom is using automation playbooks to control those automation layers. Cause most of those systems, the software defined networks, the IPAM solutions, they'll have APIs that can be used and controlled to get information, to put information, et cetera, et cetera. So we use a very similar approach that Swisscom described and try and build that out across all our markets to automate that layer. Yeah, maybe in regards to our pipeline, we are really now working on exactly getting also this more external configuration fully automated. So one example is to do all the firewall changes throughout the Swisscom network. So there we have basically already the tool which is tough in which provides such a capability. So it's really now on us to implement there the automation in the back to be able to dynamically based on the templates, we have to automatically order everything in the background. And for Yastian there we have the open daylight which is doing the cloud part, but also we are now working to do the whole transport capability. Also there, there is not something we can use out of the box, but we have an API we can start to use, but we need now to work on implementing there the logic behind we need to be able to deploy our telco workload and basically then send the configuration to the different nodes which need this in the end. So we are really working there to get really to a level where basically you just need to enter the parameters in this repository and then from there it gets pushed to wherever it's needed in the end. But this needs a lot of work that basically you are able to do this really declarative and intent based. Thank you. We got the time for one or two more questions. Anybody? Okay, if not, thank you very much once again. It's nice photo, I took it, I'll share it with you. Thanks again, give them a full round of applause.