 All right, good morning, everyone. I will have to do a confession. So this week at the OpenStack Summit Berlin, there were like 27 talks about edge compute. I actually counted them, and more than 30 on Kubernetes. So I tried to basically attract as much audience as I could by combining all the hot topics of the week. Unfortunately, I didn't manage to get a full room. I'm hoping that is because this is the last day of the summit. It's pretty early, 9 a.m. Anyway, thank you all for coming this morning. My name is Marc Rappoport. I'm a product manager in the contract business unit in Juniper. But this morning, I'm here to represent the TankStand fabric community. And we are going to talk about how to combine OpenStack, Kubernetes, and TankStand fabric to build an edge compute stack. Very briefly, who is familiar with TankStand fabric? Okay, I would say one third, maybe one fourth of the audience. How about maybe Contrail or OpenContrail? Oh, okay, I see that we still have some kind of branding recognition issue with TankStand fabric. Basically, TankStand fabric is a new name for OpenContrail. So let me give you a bit of history on what used to be Contrail, I mean OpenContrail that is now TankStand fabric. It's basically an SDN solution. And the objective is to provide connectivity and security for any workload in any kind of environment, whether we are talking about private cloud based on OpenStack, obviously, but also public cloud, whether it's Google, AWS, or Azure. It's not limited to IS environment, it provides the same set of connectivity and security services for containerized workloads, especially orchestrated through Kubernetes. And last but not least, TankStand fabric was recently extended also to cover bare metal workload management. As you can see, it's already a pretty mature product since it started five years ago. I would argue it's the most feature-rich commercial SDN solution today on the market that is based on an open source project. Back in the days, it started in 2013 as a startup called Contrail, which was acquired by Juniper. And when Juniper acquired Contrail at the time, they decided to open source it. And that was the beginning of OpenContrail. The original version was focused on the Telco Cloud NFV use case with a specific focus on how to integrate inside a Telco environment, leveraging all the standard BGP, MPLS VPN framework, and also building key capabilities like service chaining. So over time, the product was enhanced to cater with different orchestrators like VMware, providing support for Kubernetes, obviously. And this year, with the version five of the product, the scope was significantly enhanced to not only cover networking and connectivity, but also expand to providing an integrated layer of security and move beyond private data center to expand towards public cloud, like the three main public cloud players I just mentioned earlier, but also going beyond the pure software network virtualization layer by integrating also with the physical underlayer layer and providing the management of your data center fabric to natively extend the same set of services for bare metal servers. And last year when we decided to join the Linux foundation, that's when OpenContrail was renamed TankSendFabric. Hopefully, that clarifies for good the naming of the different projects. And so now there are multiple products actually based on TankSendFabric. The main one, I mean, being a Juniper employee is obviously Contrail, but there are two other companies who recently launched their own commercial product based on TankSendFabric. So we are currently hosted inside the LF networking next to a number of other open source SDN solutions like OpenDelight or ONOS or Datoplane projects like OVS or FIDO. What is unique about TankSendFabric is that it's the only SDN solution that combines both the Contrail plane and the Datoplane, since TankSendFabric includes a component called Virator and I'll go through it very quickly. Another thing to note is that we heavily contribute to a number of other projects that are also hosted inside the Linux foundation networking group. We are a member of Acreno and I'm going to talk a little bit about this. We are contributing a lot to DPDK and we also active members in ONAP and OPNV. So very briefly, as I mentioned earlier on one side, we have the Virator component which is a data plane which provides all the forwarding and security enforcement for the various workloads whether these are virtual machines, containers, bare metal servers, also potentially the same VMs or containers running in public cloud. TankSendFabric control plane itself includes a number of components, config, control analytics. I'll quickly go through them. And once you have this abstraction layer that is up and running, you can deliver a number of services across all these workloads, obviously the basic building block which is layer two and layer three connectivity through EVPN and IPVPN fabric, but also all the entire IPAM, IP addressing, DNS assignments of your workloads, cross, security, load balancing, service chaining all fully distributed on the Virator. There is obviously an analytics engine that is part of the solution that provides a visibility on every flow that is managed by TankSendFabric and there are obviously multiple ways to consume the product, APIs or UI and in the case of API, which is typically the main way to consume the product, that's the way all the orchestrators, whether it's obviously OpenStack or Kubernetes, ONAP, so on and so forth, this is the main way to consume TankSendFabric. So very briefly the architecture of the product. I mentioned earlier that we have this centralized control plane which is using actually BGP between the different control instances to provide HA and scaling. The control plane manages and program the Virator that runs inside the server through XMPP and it also orchestrates and configure your physical environment using a mix of all the standard networking protocols, BGP, MPLS, NetConf, EVP and the EXLAN, whether we are talking about the data center gateway or your top of rack switch. And what it provides is an abstraction layer where all your different workloads which are connected here, whether they are located on one server, the other server, whether they are behind the top of rack switch. On the left in the case of bare metal servers, you attach these workloads to virtual networks and all the blue ones, virtual machines or bare metal servers that are part of the same blue VN are natively connected between themselves, same for the red ones, and then you can define routing policies on how these virtual networks are connected between themselves and you can also define let's say service chaining saying I want all the traffic between the workloads that are part of the blue VN on the left to go through a specific firewall instance when they want to reach workloads that are connected to the red VN on the right. As I mentioned earlier, there are two main ways to consume the product, obviously there is a UI, but in a large production environment, typically the way to consume the product is through the standard APIs, whether these are the neutron or heat APIs in the case of OpenStack environment or using some of the native REST APIs of TankSend Fabric. TankSend Fabric has a pretty rich community of contributors, so we have a healthy mix of, I would say, one-third of vendors, networking players, all the main OpenStack distribution companies obviously are here, Red Hat, Canonical, Mirantis. And there is another third of committee members, which are mainly system integrators, software companies that build solutions around TankSend Fabric are integrated as part of a larger ecosystem or global solution. And the last third of the members are actual users, a lot of telco, as you can see, but not just telco, there are also a number of enterprise gaming companies, and some of these users are actual contributors who are providing and pushing code to the project. So if you want to have a quick feeling of the product, you can go to TankSend IO slash start, where you can deploy a sandbox in AWS of a TankSend Fabric cluster that is going to be orchestrated through Kubernetes. Now, just to obviously get a quick feeling, it's not meant to be used for any kind of real life environment. Having said that, we also have what we call a standalone, Ansible-based deployer to deploy TankSend Fabric. But typically the way users deploy the product is through the installers. We have done heavy work to be fully integrated with all the main lifecycle manager tools of the orchestration, HALM, where AT&T talked a lot about it during the last couple of days. They are investing heavily around HALM, airship, and all these set of tools that generate common life cycle management layer. Kola Ansible is another one. We've done extensive work to be fully integrated in terms of lifecycle with TripleO, which is the upstream life cycle manager of Red Hat OpenStack director, but also obviously Canonical and Mirantis. You can really pick whichever deployer solution you want based on what kind of orchestration you want to use alongside TankSend Fabric. As this talk is about edge compute, I just want to briefly mention that we recently joined the Ukraine project that is part of the LFN. That is the original blueprint that was published by AT&T, one of the key founders of Ekreno, where as you can see at the network control and data plane layers, there were a number of components that were considered to be in this blueprint. With TankSend Fabric, our ambition is to provide all these capabilities combined inside TankSend Fabric, whether it's about providing kernel-based forwarding, DPDK, SIOV, SmartNIC, which is not mentioned here, but which is also supported by TankSend Fabric. There is actually a very active group of SmartNIC vendors that are contributing to build a common set of offload APIs that can be shared across the SmartNIC vendors to basically accelerate the router data plane. And here are a few more pointers on how to get more information on TankSend Fabric if you want to get involved. OK, so let's talk about edge compute now. I'm not going to spend too much time on the requirements and use cases. There were a lot of presentation on this topic this week. I'm just here reusing the framework that was built by Acreino to identify the different use cases for edge compute. And just to mention that right now, TankSend Fabric in the context of edge computing is focusing heavily on the, I would say, telco network edge, not necessarily on the IoT or UCP use case. This is more the focus, I would say, right now in the short term on how to leverage this combination of OpenStack, Kubernetes, and TankSend Fabric for an edge compute deployment. So keeping in mind this use case, what are the key requirements for edge compute? Basically, the first one is having a centralized control plane. Why? Because in these remote locations where you don't have much resource, you cannot afford to deploy a full-blown control stack in each and every small location. Not only that, but that will also create a huge amount of control plane to manage if you have, let's say, thousands of tens of thousands of COs or base stations. If each of one has a full-blown control instance, you can imagine the pain it would create to manage them. Another key requirement in edge compute is being able to support containerized workload. Yes, NFV started heavily on virtual machines and virtualization, but now we see that more and more VNFs are being containerized. And a good reason to use that in the context of edge compute is that typically a container has a smaller footprint than a full-blown virtual machine. The third requirement, always in the same context of how do I deal with the fact that I have very limited resource is how to have a data plane, which in the case of Tencent's Fabric is V-Router, how to have a data plane which has a small footprint in terms of CPU and memory requirement. Fourth one is multi-tenancy, which is especially needed in the context of 5G for things like network slicing, how you can leverage the same physical infrastructure, partition it into multiple virtual instances that are dedicated to a set of different internal or external customers, whether we are talking about MVNO in the context of 5G or other types of, let's say, enterprise customers, enterprise VPN, for example. And last but not least, security, which is a very strong requirement, even more than, let's say, regular, typical, existing NFV. Because when we are talking about distribution in the case of edge compute, we are talking about small sites which do not necessarily have a lot of physical security. They can be easily or much more easily accessed than a large centralized data center. And also another key difference for the security that in some cases, the service provider that is deploying the service might not own the entire access network. They might need to lease capacity from some local, let's say, small sales provider. And so, as they are not using their own network, they need to include the traffic to protect it. All right, so let's talk about the first requirement, centralized control, and how we solve this requirement with Thanks.10 Fabric. So the main concept is basically to centralize all your control functions, obviously all the compute orchestration, whether it's OpenStack for VMs or Kubernetes for containers, and also all the control functions of Thanks.10 Fabric itself in your large regional data center. And then what you distribute and deploy in your remote COs is just all the data plane-related functions. Now, how does that work in practice? So bear with me, this is a bit of a busy slide here. As I mentioned earlier, what you have on the top right is inside your large regional data center, you have all your control cluster, typically a cluster of three instances for HA and scaling. In the case of Thanks.10 Fabric, you have this configuration control analytics cluster that manages both the local data center gateways using BGP peering that you see on the upper right, and XMPP protocol that I mentioned earlier that is used between Thanks.10 Fabric control plane and the VRouter data plane. What is new in this diagram that you don't see in a regular, let's say, data center-only environment is you have a number of extra instances of control plane, which are the control pop 1, 2 and 3 at the top. Basically, you have a number of virtual machines that you deploy in your large centralized regional data center that I use to manage all the routing and security control information for all these pops. They do it centrally, again, by building the same BGP sessions to the remote gateways that are deployed in the small pops and XMPP sessions towards the VRouter of the small compute nodes which are deployed remotely. One of the key values of this model is to allow really optimized routing while you don't have to send traffic between, let's say, two workloads between, let's say, in this example, the container that runs inside pop 1 needs to send traffic to the virtual machine that runs inside pop 2. Traffic is not going to go back and forth from pop 1 to your original data center and then back to pop 2. Since you have a full awareness of the routing topology of your NPRS-EPN network, you can optimize the traffic flow to go directly from one local site to the closest local site. Now, let's talk about how do we manage containerized workloads in TankSend Fabric. First of all, there are two main touch points between Kubernetes and TankSend Fabric. On the control plane side, we have the KubeManager which basically listens to the Kubernetes API servers for all the new events of pod being deployed on the specific compute nodes, let's say. And on the data plane side, the integration point with the KubeNet that runs in each compute node is through the standard CNI plug-in. And then the Virara replaces the KubeProxy. So these are the basic building blocks of the integration of TankSend Fabric in Kubernetes. And then each object that is specified in Kubernetes is mapped to an equivalent, namespace maps to a project, a pod maps to a VM in the context of TankSend Fabric. Service will map to an ECMP load balancer. I'll go into a bit more detail, both for service and ingress. Which is a layer 7 load balancer. Contrary to service which is a layer 4 load balancer. Ingress maps to an HAProxy instance. And the Kubernetes network policy is mapped to a security policy in TankSend Fabric. So just a quick clarification for the people who are not fully familiar with Kubernetes terminology. The difference between service and ingress. So service in Kubernetes is an abstraction to group a logical set of pods that deliver a service. Basically they are all used to provide HA and scaling. And what TankSend Fabric provides in the context of a service construct, it implements it by doing layer 4 based load balancing natively on the VR. So this is fully distributed. You don't need to send the traffic to some centralized processing points. On the other hand, ingress is typically an object that is visible externally. And that is used to do more granular type of load balancing all the way to layer 7. For example, to do URL based load balancing, sending one part of a website to another service, another part to a different service. And the way TankSend Fabric implements an ingress object is by spawning by default an HA proxy instance. Which will do this layer 7 load balancing. Having said that, this is fully configurable. And you can use the load balancer of your preference, AV load balancer or whichever other mainstream load balancer of the market can be used to basically implement ingress. Okay, this is a bit of a busy slide. There are two main models to have coexistence of virtual machines and containers, basically OpenStack and Kubernetes inside the same environment. So the first model that is described here is a model side-by-side model where you have your virtualized machines which are deployed by OpenStack. And you are running Kubernetes directly on top of bare metal. And so what you see here, you have your OpenStack domain that runs on one side all the control components, OpenStack, TankSendFabric. And on each compute, you have your VRouter, KVM that provides the virtualization layer for your VMs. And then NovaAgent basically talks to VRouter agents which then programs VRouter to implement all the forwarding and security policies. On the other hand, you have your Kubernetes master cluster and the Kube manager that listens to the API server. And in terms of running your actual pods, same thing you have VRouter. Here you see that the pods are directly connected to VRouter since they are running directly on top of bare metal. And inside each compute, the minions, you have your kubelet that talks to the CNI plugin which itself programs VRouter through VRouter agent. Now what we often see is the need to run Kubernetes not just on bare metal but inside virtual machine. And that's what's called the nested mode. And that brings another layer of complexity because by default what you would need would be two different layers of SDN which would also create two layers of encapsulation where the traffic that would exit from a pod would go through a first tunneling to get out of the VM. And then when it would reach the router that runs directly at the host, Linux operating system would get another layer of encapsulation. So not only would you have to manage like two separate planes of encapsulation but also two layers of tunneling. So the way we solve this challenge in TankStand Fabric in the nested mode is basically when your pods are running inside a virtual machine, what we are doing is just instantiating a VLAN ID to separate the traffic that comes from different pods inside the virtual machine. And that's only after that when the traffic reaches the router that we apply the regular layer of lookup, tunneling and forwarding outside of your compute server. And that's how you can have coexistence of virtual machines and pods running inside other virtual machines on the same compute server. That's a very busy style. I think I'm going to have to work on simplifying it at some point. Okay, one just quick comment on service chaining. Service chaining is a key requirement in NFV and there are many different scenarios that you typically have to deal with. Have a chain of multiple VNFs that you need to steer the traffic through between your different virtual networks. You typically also need to be much more granular than say I want to send all the traffic between these two VNs red and green through the same set of services but say I want my HTTP traffic to go through some kind of HTTP proxy optimization and the rest of the traffic to go through another service chain. The reality is you don't have one virtual machine that provides your VNF service. You're going to have a pool of VNFs that are grouped to provide logical scale out. And so you need to be able to balance across these pool of instances. And the new challenge that the containerization of VNF brings is how to provide service chaining for containers. And here the issue is that Kubernetes itself doesn't have a construct to identify interfaces. By default, a pod is single attached. And service chaining typically requires at least two interfaces, two logical interfaces, to identify the traffic that comes from your left virtual network and that needs to be sent to your right virtual network. So what we did was extending Kubernetes capabilities using CRD to be able to support more than one logical interface on a pod. Which will then allow us to reuse and leverage all the service chaining capabilities that were already available inside the product for a virtual machine and being able to extend them to containerized pod types of network functions. Okay, next requirement. They need to have a smaller footprint of the router. So originally when TankSendFabric was designed, it was meant to be deployed in large scale data centers with potentially hundreds, thousands of compute nodes, if not more, in service provider environments where you would have tens of thousands of virtual networks, hundreds of thousands if not millions of IP prefixes. And that's why the focus being on scalability, obviously VRR by default has a fairly big footprint in terms of CPU on memory. It's expected to be deployed on a high end server like typically 16 to 20 cores dual socket with a lot of memory and VRR itself can easily use 4gb of memory, can potentially use up to 8 cores if you try to get the highest possible foreign performance using DPDK where you have to fully dedicate your cores which are going to be used 100% CPU for VRR DPDK processing. So the need was to, and we are just at the beginning of this journey, to put VRR on a diet to deal with these environments where instead of having these super beefy Intel servers you have these much smaller compute servers in edge compute environments. And so the first step is to decrease the amount of memory and CPU needed. And in the future what we will also work on is supporting not just x86 but also ARM CPUs which are much more common on these smaller platforms that are used in edge compute environments. Okay, multi-tenancy, very briefly. What we did in Tangsten Fabric was extending the connectivity and isolation model way beyond what default Kubernetes provides which is basically the default model that any pod can communicate with any other pod. The point was to provide connectivity to developers and not having to deal with any kind of security or isolation. So the first simple isolation mode that is available with Tangsten Fabric is a namespace isolation which basically does how it's named. Isolating all the pods that are from different namespaces but inside one spaces all the pods can communicate together and that's probably the most common scenario deployment but we can go beyond that and even isolate different pods of the same namespace and there are some specific scenarios for that. Let's say the equivalent of what is a micro segmentation scenario where you want each and every single individual workload to be by default isolated from all the rest of the world if you have very stringent security requirements. As I mentioned at the beginning of the talk all the infrastructure of Tangsten Fabric is based on the IP, BGP, MPLS, VPN framework so the way traffic of different tenants is isolated is by leveraging typical VRF type of behavior where each tenant is connected to a separate routing and forwarding table whether it's a virtual machine or a pod that's the way each tenant is basically isolated and how you can provide some kind of network slicing behavior. Okay let me cover the last requirement as we see for this edge compute use case which is security. So one requirement related with security is basically the capability to define in a very concise way your security policies and being able to apply them across multiple orchestration environment. Let's say you define a set of security rules in OpenStack environment that would be security groups that are defined across the different virtual networks and then being able to reapply the exact set of security filters across your different environments and that's what Tangsten Fabric provides within an abstracted policy construct where you can define your domains, the type of deployment, the geographic location, you can use abstract tags, labels that could be for example very useful in the context of 5G network where you have all the 3GPP interfaces for people who are familiar with the mobile world there are a number of interfaces to deal with, GI, S1, X1 you can easily tag all these interfaces with these specific names and then by default all the endpoints that share the same tag will be connected between themselves and they are going to be isolated from all the other endpoints and that's just the way the network policies in the case of Kubernetes environments are being translated as firewall policies on Tangsten Fabric. So a Kubernetes label becomes a tag, same for namespace, a network policy maps to a firewall policy, I already mentioned about ingress and service how these are basically mapped in terms of Tangsten Fabric construct and last but not least the need to encrypt your traffic as I mentioned earlier in the case of H compute you have to deal with higher security issues the fact that you are deploying your compute nodes on sites that can be potentially owned by a third party that is not managed by the service provider or you might have to use some least capacity 5G itself come with some new security requirement and that's where being able to natively encrypt the traffic between your endpoints which is what Tangsten Fabric provides where you can define the very granular policies to say I want to encrypt traffic between these two workloads all of this is implemented natively inside the router and it also comes with a visualization engine which allows you to basically see in real time what is the flow matrix between your different endpoints obviously the first layer of aggregation is the virtual networks you see the traffic matrix between the different virtual networks and for each virtual network what are the endpoints that are sending and receiving traffic allowed between themselves and just a quick pointer if you want more information on the project the code how to participate so hopefully you've seen that we already have today a pretty solid base to leverage Tangsten Fabric to deploy an edge compute use case and we'll keep adding more and more capabilities over time as we identify more use cases thank you questions I have a question regarding the constraints on the network between your central data center and the data center or the places where you run your pods because in terms of edge computing we can have constraints on that and in my case in the banking area we have countries that are unreliable in terms of network or we have overseas location very far from western Europe for example right so if I understood correctly your question it's about what are the constraints in terms of latency between your control plane and VR data plane what we typically recommend is a 50 millisecond round trip time and that's relatively aligned with the constraint you would have also on the compute side with let's say OpenStack itself between your Nova compute your Nova agent and other components between your centralized control plane and what's running in the local remote compute node that's typically enough to cover scenarios where your edge compute nodes are in the same geographical region whatever it means in terms of tens hundreds of kilometers not clearly thousands of kilometers but that's just giving you a rough idea of what is possible to do in this context hello about security and the tags and so on this is also open source so all the security and tags part I think I understand where this question is coming from yes that's part of the tanks and fabric product what is followed in tanks and fabric is what is called the open core model all the key building blocks and infrastructure components are part of the tanks and fabric project in the case of Juniper what the commercial contrail product provides on top is a new UI that allows you to define your security policy in a more I would say user friendly fashion but all of these can be implemented in tanks and fabric using the API calls for the small compute you show those figures is is it possible we have a roadmap to go to virtual CPEs with the VRouter so use it as a solution to use that sorry could you repeat the question solution because you have IPsec now yes like running VRouter on a UCP type of platform yes that's definitely part of the plan it's not available as of today but that's part of the plan I would say probably something for next year and then lost on Arista and Cumulus networks they speak with EVPN or is it still with OVSDB so we we started with OVSDB to basically integrate data center fabric we are now moving away from this model that's something that was done two three years ago now we move to an EVPN VXLan model which is becoming the de facto standard to program a DC fabric and that's what can be used for third-party networking switches any other question thank you very much