 Hello. Good afternoon to all the audience here. And I'm very excited to be here. And I feel that all you are also excited to part of this Cube Day India. So here I am going to present this topic. Use Meridio to create network services for telecom use cases using secondary networking for Kubernetes. Little will talk about me. I'm Gaurav Bhatnagar. I'm playing the role of ceiling system manager as part of technology system management, business area, cloud services, business and operations support system. And you can see my LinkedIn profile here. We can always connect on the LinkedIn. And basically, I enjoy reading and playing tennis. I'm CK certified. I'm a AWS cloud practitioner also and have certifications from BrataStack as a team forum. So definitely we can connect here. So I have been involved with Kubernetes from last around seven or eight years, primarily in the telecom domain. So as part of Ericsson, we are basically in the journey of cloud native. And we are basically deploying our products on the Kubernetes platform. So this is the agenda that I want to cover. So what is Meridio and what is secondary networking for Kubernetes workloads, telco requirements for Kubernetes workloads, some telco use cases where Meridio can be useful. So secondary networking, so just starting with Meridio, what is Meridio itself? So it is an open source project. It provides various capabilities to attract the external traffic within Kubernetes via secondary networking. So it is available at the GitHub. And it is primarily written in the Guru Primary Alliance. And if you look into Meridio, basically it works on the Kubernetes 1.21. It requires Pyre, network service mesh, and Linux kernel. Basically, Pyre is just to give the mutual authentication between different workloads. So it provides a empty list kind of support there for various Meridio ports that are working and deploying on the Kubernetes. Network service mesh basically is a hybrid multi-cloud IP service mesh that, for example, it facilitates the multi-higher communication between L3 domain. So if you have your Kubernetes workload deployed across the clusters of VMware or across the clouds, you can use network service mesh to create connections between them. So basically, this is Meridio. And it is basically using these underlying softwares. So bulk of work or functionality that Meridio provides a build on the network service mesh. So what is secondary networking? So when I first heard this term, I was still confused. Because if you look into the documentation, it always talks about primary interface and all. So just to what it is all right, so we have the Kubernetes port with primary network interface, it's 0. So through this interface, all the communication is happening, the readiness probes are working on this, listening to the communication channels and all. So now if there are more network interfaces in your ports and these network interfaces can be used to talk to other networks, then we basically call these network attachments as secondary network interfaces. So anything other than primary is secondary network interface. So because when I first heard this term, I was a little confused. So I thought it would be good to catch it here. So again, coming to what is Meridio, right? So as you see, this is a Kubernetes cluster we have deployed here. And it is your user application that is running in a port. So user application is a container that is running in a user port. Now to use the Meridio, what it does, it provides a sidecar container called a Tapa Sidecar Container. It is called Target Access Point Ambassador. So it is based on the ambassador design pattern. So it provides an interface through which you can control what traffic that is coming to your secondary network interface. So it provides a GRPC API hosted as a Sidecar container. So your application is there as part of container itself in the user port. And the Meridio Sidecar container is providing you the GRPC interface to connect to the external world, right? Now there are more components to Meridio. So if you see, this Sidecar container in terms talks to a network service. Again, it's a port. And it basically provides two functionalities, a traffic classifier and network services. So again, Meridio operates on layer 3, layer 4 to provide distribution via, again, so-called secondary networking. So it holds the traffic separation on the default primary network within the cluster. So the primary network interface is not touched. It is the secondary interface through which all the user plan traffic is passing. And basically, there's a traffic classifier. You can use it to separate the user traffic into multiple groups. So I have an example for that. Maybe that will become more clear when we go through. So as we said, we use this term service attractor, right? So what Meridio does, basically, it is talking to the external gateways using routing protocols like BGP and all. And it can announce the virtual IP addresses to the external gateways. And through this week, it can attract the external traffic towards your Tapa sidecar container, right? So from the gateway itself, the data gateway, the traffic is coming to your network service. That is a port that is deployed in a Kubernetes cluster. And then through that, Tapa sidecar container, right? So user application has the logic to process the external traffic. While external traffic is reaches through your container in the user port through the GRPC API, right? So some of the Telecom requirements that we can basically look upon when we expect from the Kubernetes workload, right? So IT industry has been using Kubernetes workloads from quite some time. But Telco industry is catching up. And I'm also the part of that industry itself. But if you look into the Telecom perspective, we have some very specific requirements, like end-to-end traffic separation isolation. So basically, we want the traffic to be separated from the beginner hand to the actual port itself. Network address translation, there are many protocols in Telecom like SIP, STCP, that basically the network address translation is problematic there. Cluster-wide virtual IP as a source address for egress traffic for application ports. So we need a virtual IP or WIP that is same even for the egress traffic, right? So I have a slide for that. I think that will make much more sense. And high-performance user-plane traffic, right? For instance, we have the media servers that basically have to download or stream the media services or media traffic to the end customer. And there we need a high-performance, right? Dry networking of Kubernetes is not that sufficient for that. Support for non-IP protocols like Ethernet and all. So that is the price of the telco requirement that is basically affected from Kubernetes workloads, right? So what I try to cover, basically, try to capture some telecom use cases so that we can basically demonstrate or see what is the power of media, how, and which of the use cases where it can help, right? So one is SSM is a legal protocol, and it's still used in a number of telecom products. So how the legacy telecom products do IP validation while establishing connection between peers. So still the peers in SSM do the IP validation, right? Segregation of O&M and real-time traffic. So real-time traffic is the voice traffic that basically we are making voice calls. O&M traffic is the alarms and operational traffic. So as a telecom operator, they want to segregate this traffic into all together separate paths, right? So basically, you want to classify and steer traffic based on five tuples, basically the source and destination IP source and destination ports and protocol, right? So basically, using these five tuples, you want to segregate your traffic, right? So for instance, DNS protocol is there. It works on both TCP and UDP on port 53. So might be you want to handle UDP-based traffic and DNS TCP traffic separately, right? So this is an example of, again, first use case. We have a signaling protocol, SS7 communication happening. So our user pod and this can be a remote SS7 peer. So I can take, for instance, this user pod wants to do a location lookup through using this SS7 peer. This might be an HLR. So whenever the egress traffic originates, it takes the IP of the worker node, right? And when we configure in the remote SS7 peer, is the load balancer IP. So all the services or whatever services that I want to expose on the Kubernetes is through the load balancer only, right? So I have configured that load balancer IP at the remote SS7 peer. But when, suppose, my user pod wants to do some lookup, location inquiry, right? It's initiated traffic. And it initiates a query toward the remote SS7 peer. Now what happens is that this is the default behavior on the Kubernetes. Since user pod IPs are ephemeral, they don't exist or known to the outside world. So when it passes the boundary of the Kubernetes cluster, it is replaced by the worker node IP. Now this worker node IP is not recognized by the remote peer because we have configured the load balancer IP, right? And we don't know where the user pod might reside later on, right? It can be on the worker node A, then it can move to worker node B, and that way, right? So what this problem solved by Meridio is that, again, it has a concept of virtual IP. Or WIP, right? So rather than publishing the address of a load balancer, we publish or configure IP address of the WIP, right? Now, all the traffic that is passing through is through the Meridio pods. So the egress traffic, when it originates, basically user application invokes the GRPC API of the Tapa Sidecar Container, that is target access point ambassador, Sidecar. And then, basically, it passes through Meridio. And again, then it basically reaches a remote peer. So in this case, no NAT happens because the Meridio pods take care of that. And basically, your egress traffic has the same IP address that has been configured, right? So for the egress traffic as well as egress traffic, you have the same IP address, right? So that remote SS7 peer is able to establish a communication because it recognizes that IP, right? Now, there is a separate requirement in the telecom that we want to segregate O&M and real-time traffic, right? So I have some real-time traffic where voice calls happening. I have the O&M traffic where monitoring alarms and handling are happening. So again, we can use Meridio for this use case. For instance, in this use case, we have segregated this O&M traffic and the real-time traffic on two separate WIPs, that is WIP 1 and WIP 2, right? And this Meridio, what it does, it is a concept of trench. So trench, basically, again, it's a normally culture that is there used in Meridio. It's seen as an extension of the actual network. It is a trench that upholds the traffic separation inside each network, inside the realm of the cluster. So basically, trench, if I'm more going to it, it is a custom resource definition. But what it does, it gives you a functionality so that you can able to segregate your traffic, right? So for instance, here, what we have done, we have created two interfaces, NSM0, NSM1. And we have published WIP 1 and WIP 2. So using this, you can basically segregate your real-time traffic as well as O&M traffic on total different network paths. So for instance, one can be on the VLAN 100, other can be on VLAN 200, that way. Now, classification and steering of traffic is another important use case where Meridio can be very useful, right? So what I have taken a use case here, that suppose you want a requirement that you have a DNS traffic, and you want to handle DNS UDP traffic and TCP traffic separately, right? So here, what is in this diagram? What happens is, flow A takes care of the UDP part of the DNS, while flow B takes care of the TCP part. So flow A takes the UDP-based DNS traffic, which passes the conduit through stream 1 and into target A application, while flow B takes care of the TCP traffic, right? So let's go to the next slide. So starting point is that we have deployed our user part with a TAPA sidecar container. So we have the user application and the TAPA sidecar container as the user part. Now, we have also deployed the Meridio parts. So first, we created trench. So trench is, as we discussed, it's defined the extension of the external network into the Kubernetes cluster. So it's nothing but, again, a custom resource definition that is created, right? So when we create a trench, basically, we create a configuration within the Kubernetes in a conflict map, right? Now, we have created trench. So this trench can talk to external data center, right? Now, then we create a VIP. So VIP, basically, again, publishes a unique virtual IP that is known to the external data center. And these VIPs are basically user application is listening on these VIPs on the secondary network interface using the TAPA sidecar container. So basically, we have created a VIP here, and we have created a VIP with a trench. So we have created a VIP one. Similarly, VIP 2 can be created, and can be assigned an address, right? Again, conduit basically is, again, a custom resource definition there in Miradio. Basically, what it does, it provides as of now a stateless load balancer. So you have multiple pods, so it can load balance your traffic across that pod, right? So again, we create a conduit. We associate it with a trench, right? So now what we have done, basically, created a stream. Basically, streams a logical grouping of traffic flow. The stream points to the conduit where the traffic can pass. So it is an actual network entity in which you refer to in your target application, right? And so we basically create a stream. As in the example, ML, we associate with a conduit and trench, right? And now we create a flow, right? This flow basically what we are doing, this flow basically classifies your incoming traffic in the base of five tuples. That is, source address and destination address and the ports, right? And protocol, right? So basically, as part of this example, ML, we have created a flow, A, with that is associated with trench A with the stream one. And we also associated a VIP. And basically, it's accepting all the traffic. And the destination port is 53. And the protocol is UDP, right? Because we want to classify and define the use case itself, DNS traffic on the TCP and UDP, right? Now, what is basically, we have created a flow. But again, the traffic will come to the data center gateway. So we need to talk to the data center gateway. So there's another customer service definition in the media view that is called gateway. Basically, gateway is nothing but it basically it is a peer to the data center gateway node. And it basically mimics or basically have the configuration to talk to the data center gateway. So in this slide itself, we have created a gateway object, customer service definition that is associated with a trench A. And we basically give an address which gateway to talk to and what are the basically configuration or the specs through which it can talk to that gateway, right? Now, basically, this is the tractor part. So a tractor is the customer service definition in the media view that basically uses the gateway configuration to talk to the external gateway and extract the traffic from the external data center into the Kubernetes cluster. So it basically attracts and basically advertise this IP that is OK. This is a virtual IP that needs to and this IP needs to be this traffic, what's this IP, needs to be routed to this cluster. So it passes through the flow through this conduit stream and then to the target application, target application B. So using this, we basically can split the traffic, TCP and UDP traffic for the target A application and a target B application, right? So basically, you can use here an example of IPv4, but you can use a configuration for IPv6 also. So if in all, if I try to summarize, what are the key takeaway points that we want to summarize? So Billidio can be used to facilitate attraction and distribution of external traffic within Kubernetes. We have so-called secondary network with minimum changes to your application. So basically, your primary interface of Kubernetes is not touched. So all your readiness probes or liveness probes that our primary interface is not touched. So it is not expected to that you change your Kubernetes application to work with Billidio. But what you need to do, you need to deploy a STAPA sidecar container within your user port. And then you can use the GRPC API provided by the STAPA sidecar container to talk to the external world, right? So basically, it doesn't expect that you change your architecture or deployment model as such. It works with that only, right? It, again, next point, basically, provides a sidecar container using a GRPC API, right? So you can start-stop traffic toward the user port. So basically, using the GRPC API, you can start the traffic, stop the traffic, right? What is traffic that is coming from the external data center gateway towards your secondary networking interface, towards your user port? So using the STAPA sidecar container, using the API hosted there, you can basically start and stop the traffic, right? You have the control there. As a user application, your application is processing the data. So you have the control there on the traffic itself, how the traffic needs to come, how the traffic needs to be classified in that way. And the third point takeaway, I can say that, service address and web site are announced to the external data center. We have different kinds of routing protocols by a media front-end. So front-end, basically, is the component that is front-ending with the external data center. It basically hosts the attractor in the gateway components. So it is attracting the external traffic on that web itself. So it basically advertises that this is a virtual IP for which it wants to attract the traffic. And when the traffic lands on that Kubernetes cluster, it basically classifies the traffic based on your configuration, and it lands to the top-of-sidecar container and then consumes your user application, right? So again, it can solve different telco use cases which are quite tricky to solve using an existing primary network interface or Kubernetes workloads. So again, a lot many telco workloads or use cases can be solved here, right? Because primary interface, by default, what you get out of Kubernetes is a private interface. And there are a lot of use cases, like, for instance, we discussed that if you want to high-performance user plain traffic, right? Again, it's still tricky to solve through the primary interface. You need some second interface through which high-speed data that can come to your application itself, right? So these are the key takeaways. So I have captured here that some references that are quite good. For instance, one talk by Leon in this here, it is very good, so it gives you an internal working of how this midi-do is happening, right? It is very well explained there, right? So you can look into that stock itself to understand how midi-do is working, what is the role, and then some is playing there, right? So with this, thank you. Again, this is my Lincoln ID. You can always reach me on this Lincoln ID. And again, thanks for giving your patience listening and thanks for your time for coming for this talk. Thank you. Any questions? I'm here. Right, yes, that's all right. So my question is, if I want to expose multiple source IP address and each one for different, different destination IP address, then maximum how many source IP address it can retain for intercluster? Basically, you define a configuration that makes targets what you want to achieve, right? So you can configure multiple WIPs, right? So basically, it depends how much IP that you want to expose, right? Within your cluster, it's not an issue, but outside the cluster, outside the cluster, basically, when the issue is that when your egress traffic goes out, it takes the IP of your worker node, right? But you may not want that case to happen, right? You want to have your services to have a consistent IP even when your traffic goes out of the cluster itself, right? Yeah, so that IP, I don't want only one IP. See, if I want to talk destination one, I will use source one. If I want to talk destination two, I will use source two. And none of them will translate to worker node with this design. So how many source one, source two, up to source N? How many such N source IP, my port can have it? Virtual IP, can it have? You can add a number of virtual IP. As such, it depends on the application because these traffic that is coming on your, on these virtual IPs is consumed by your target application itself, right? So each IP represents a service itself, right? So there is not limit at such, right? But you can define a limit. For instance, you can define a max target itself that, OK, I want to only expose 100 target itself. But you can expose multiple VIPs. Even you can basically consume traffic of the multiple VIPs within your user cluster by the same port itself. OK, thank you. I have a small question in this context. And I'm just asking a very basic understanding. So this VIPs and all, do you need a multi-CNI to create this? How does it, because secondary interfaces are normally in telco world, they're always a multi-CNI. And then it creates this virtual Ethernet VTH0s and all. And then is it something in between? Or you can, and is it very similar to Envoy in primary? Is this meridio is very similar to Envoy? Is it correct understanding, actually, what? Correct, in a way that basically use the multis to create a secondary interface. But what it does, basically multis is a multiplexer, it basically creates a CNI or CNI. So it will help you create a secondary network interface. And then virtualize also. All of those use cases, what I was going to say. After creating a secondary interface, you want to attract traffic to that secondary interface. And basically you won't have services as published passing through. Because unless until you code in your application itself to listen to that secondary network interface, things will not happen. Where meridio help is, basically, it again uses an underlying NSM, right? A lot of wiring work is done by the NSM only. So it creates, basically, what multis is doing, it's done by this NSM itself. So it creates a secondary interface, adding your pod itself. And then basically meridio what it does, it basically using the attractors and gateways, attract the traffic, what the web app that you have published on that secondary interface. So NSM, the network service mesh, basically is what is doing that you will expect to multis. So but multis has a specific purpose, you create a secondary interface, and then you are there. But you have to basically develop an application or your logic itself to attract traffic to that. Your pod itself will not listen to that secondary network interface. Your all services will work on the primary network interface. Okay, so you're saying is that there's a network service mission on top of that, this architecture of meridio fridge? Right, right. There's no need of multis CNIs as such. As such, but there is a configuration where you can use multis also within the NSM also. Basically, multis, what is happening is basically is giving you flexibility or functionality to create secondary network interface. Absolutely. So NSM is also doing the same thing, right? Okay. So it's doing much more than that, right? Got it, got it. Thank you. So like in our environment, we are moving from service message to CELium to minimize that sidecar container context. And we were using multis in there, in user plane functions we have. So how like in here, Tapa is acting as a sidecar container. So is there any way that it can be handled without sidecar container? Like the kernel space, it can handle and there provision those. Okay, the present architecture, what we have is the meridio itself. It uses sidecar container because what it does, basically the sidecar container holds or basically give you a GRPC API interface where your user container within that user pod can query and basically get the data, right? So this is how it works, right? So basically the sidecar container is giving you a flexibility in the sense that your application is remained what it is, right? It only needs to call the GRPC API to get that external traffic, right? And definitely you need to do the configuration and all that for all this thing to working. But your application can work the same, right? You just need to add a sidecar container to your user pod and then call the API and then basically using the GRPC API you can consume that traffic itself, right? So you need to have a sidecar container, top of sidecar container to basically use this. Hi, so my question is how do we scale the pods here because is the VIP associated with each pod or is it at the service level? If it's at the pod level, then we might have to create the other resources, right? The range and the VIP CRs itself. Okay, it depends on your application how basically load intensive, how it is there, right? So Meridian, what it does, it is giving you a virtual IP where you are exposing your services, right? So same in the pod itself. So each pod will have a one sidecar container. That is the right. And number of pods that are there as part of the Meridian application itself. So it depends on how your application is basically looking, right? Or what are the throughput requirements of there, right? So that needs to be worked out, right, that way. So for each pod, we'll be having one trench defined or is it at the service level? So this, so again here, basically one user pod has one sidecar container and again these set of pods that are part of the Meridian pods are common, right? So these are, you can say this, this is part of something underline infrastructure that all pods there can use, right? So, but again, you might want to create a replica sets or replicas of the pod itself, right? So it depends on the throughput itself. But once deployed in your cluster itself, this is, the network service is common for all the pods, right? The top of sidecar container is specific to your user pod, but this, the network service is common. That's all. Thank you. Thank you. Again, thank you all for asking questions and coming here, giving your time. Thank you all.