 Okay, thanks for that right welcome everybody. So I'm Giles. This is Rasto And I'm gonna talk to a few slides giving an overview of BPP. So I guess it's been alluded to today I know earlier to about you know, BPF and XDP talk about vectors. Well, but I'll give you a sort of overview of that Then Rasto is going to go into contee BPP, which is a CNI plug-in that we built using BPP where I guess we're about to disagree with Chiara on And on the thing about using the kernels, connect containers, sorry guys, no We should keep it in user space But hey, a bit of controversy is always good. So, so what is BPP? So we set the universal data plane and really You know the history of this and sort of Genesis of this was it came out of Cisco It was initially like a sort of punk part software for that But then we realized it was pretty good and we should probably open source it So it's been open source through the Linux foundation So it's part of the LFN the next foundation networking sort of umbrella the key thing and again sort of if you remember earlier and You know with BPF and XDP this thing of oh if you want to extend it You have to build helpers and help us have to make it into the kernels. So, you know There's a there's an IP lookup. There isn't a bridge lookup yet And pretty key reason that we've done everything in user space is the fact that that gives us that feature velocity So if we want to put a new forwarding plane in we don't have to wait for Linus to bless it We can just push it straight in because it's user space the throughput, you know, that's that's the magic of vectors Which will come on to as well Initially in fact it ran on power PC But it now runs on arm and of course x86 which is way to see most of it the scope So in fact that the three bullets are nicely the opposite way up from the three on the picture So scope is the network IO. I mean typically we're using DPDK there though You know what opens to other things there's been other suggestions today again with things like XDP There are some native drivers. I think in VPP as well VPP itself really knows about the packet processing so about your actual forwarding But then of course the key is we go set it up. So you need some kind of a management layer You know as the control plane But we don't live in isolation So this is kind of just to give you a picture of the Linux Foundation networking and associated stuff and where The whole fight of VPP can sit so VPP itself when it got open-sourced we called it Fido fdio fast data You can't pronounce fdio. So so it's a dog So so we sit here in the data plane, you know, of course we sit on top of an operating system and hardware and Some of these other projects have been talked about already. So for example, I used to work on open daylight quite a bit That's been presented on us has as well. It's another SDN controller. I know I think Emma's talking about opnfv stuff after this On that strangely no one's here talking about on that. So I know if anyone here heard of on that Well, maybe that's why no one's talking about it. There's only like four people hands up So on that basically is is kind of a telco-ish thing It's sits on top of open daylight in terms of how it programs the network But it's very much about how you provision telco type services. So it's all unbelievably dull I'm glad not to be working on it anymore But I guess, you know in terms of buzz and the rooms the rooms quite quiet because we should have stuck the word kubernetes All over everything to get everybody in, you know, really kubernetes is where it's at And if you think about the stack now in container sense What you're really saying is that your your CNI plug-in really does this bit the network controller And kubernetes is really all about the orchestration and you know, how do you deploy? Pods onto different nodes and how do you figure out where to put things and Sort of, you know manage the lifecycle. So the projects we have in in Fido BPP itself Of course the core there's a bunch of different other projects around packet processing So things like you know open data plane P4 has been mentioned a few times already. I think There's even, you know, sort of information-centric networking and that kind of stuff. I Mention as I said, we typically run on top of DPK in terms of the network I owe and then the top layer of the management agent so You know honeycomb really is it's but about virtual machines It's really cut down version of open daylight so that you can manage Effect an instance of EPP running on a node But I guess the way I think about it is in that kind of VM world You're still probably thinking about things like netconf and yang and BGP You're very much in that kind of I am a host I am a router or whatever. So using those kind of more I Hate to use the word legacy those more like legacy approach to the configurations because really if you think about it as he Moves to containers like how often do you want to reconfigure a running container so multi-tenancy and networking traditionally We built IP VPNs right and the way we did multi-tenancy was we use VRF So we could have like a hundred VRS on a router multi-tenancy with containers more likely means spin up a hundred containers One per tenant and a reconfiguration probably means take this container blow it away and create a new one So our way of doing things is very different So what we've done to be more sort of cloud native in the container world is we built the go binding onto VPP and then what Praise and I've got a few more slides haven't I then Rasto is really going to go into Continue to pee which is a CNI the container networking so it pretty much sits on top of going to pee provides container networking We have a whole bunch of test stuff. I know that Ray and the absence sick matchech. We're going to present See sit so I guess Ray's going to be that on his own Some other interesting test stuff in there for example t-rex is like a high-performance packer generator So what does all this stuff mean with vectors? You know, how does it how does it actually work? So I'll skip through this because it's restates what I've already said Oh, yeah the wrong way Here we go So basically decompose packet processing into a graph of nodes and by nodes here We just mean little chunks of code that we kind of chain together So, you know classic you may be forwarding a packet what you're going to do You don't get a packet in you figure out from the driver that this is neat and a packet You then say oh actually we found out that it's IPv4 So there's like before graph knows we then do a lookup on it and then it might well be that we're forwarding it So we have to rewrite it put it out of a different interface that sort of thing So we break it down to the nose which kind of micro network functions And then the packet to move through this in vectors So again most people have talked about this already that the concept not so much the vectors with the concept that you know Don't have one interrupt a packet or don't pull and just pull one packet You know try and pull as many packets as you can at a time So typically TPTK or a polling mode if a load's high we might get back 2030 who knows a whole bunch of packets that were waiting for us So we process them as a vector and this is where the graph comes in with the graph nodes fit in the instruction cache So the goal is your graph nodes fits in L1 cache If you know anything about CPU cache hierarchies that's going to give you orders of magnitude more performance than going off to main memory The packets themselves are prefetched into the data cache The other thing to say is of course when you do things like lookups One of the challenges becomes well What if you have a million reeds in your reading table as you would in the internet and the the answer is of course that We can again paralyze all those lookups So it takes a while for memory to get back to us with the first lookup Next one on the next one rather than stalling waiting for that first lookup and and yeah, I think that what's interesting about this is I mean I'm Last my programs in assembler. It was like six six five oh two or something and you know, there was no cash There was none of this sort of intelligence But the people built this Yeah, exactly And people who came up with this stuff, you know really understand CPUs and modern CPUs and cache hierarchies And how all this stuff works and that's that's the genius of it and the sort of proof is in the pudding I mean again rail pretty talk about it But when we went from sort of generation end to generation n plus one of Intel CPU the performance Handily went up because it was taking advantage of it automatically And so in terms the actual packet program processing good with getting okay for time So we have the graph we mentioned so the individual sort of steps in a graph nodes I colored the input ones differently because this is really where it's coming in from your network driver or of course You know realistically of course one thing to say is that it's I would say this of course working for Cisco But it's probably fairly unlikely that people be deploying this in boxes with multiple network interfaces acting as routers You know, please don't do that. We want you to buy a route So more likely we're talking about something running on the host as v-switch and so quite a lot of packets Are going to be coming in through something like the host user rather than coming off in it So you have a vector of packets They come in They come from the input nodes And then as you see they're going through Step-by-step rather slowly so my useless animation and the point is at each point that first packet in the vectors Going to be warming up your L1 cache and all the other packets in the vector take advantage of that But of course we also in some cases are going to have split the vectors Imagine there's an art mixed in with a bunch of v6 packets You'll have to split it and in that case both the art itself will have to warm its own cache up Plug-ins, so you know as I mentioned the whole point here is that You should be able to add code quickly we can add it quickly, but so can you too and so how does that work? Well plug-ins really You just plug new nodes Instagram they can be run sort of separately from the main VPP code back so they can be you know effectively first-class citizens But you don't have to be merged in and go through all the c-sit stuff that people like rare magic do So yeah, you're building it yourself, but it To all intents and purposes is it has equal status for the rest of VPP But of course also you can have hardware plug-ins. So you know imagine you have a nick and Much to Luke's annoyance. This is an intelligent nick and this Nick, you know does whatever it might be IP sec off-load And you know we'd like to be able to take advantage of that Hardware supports it, you know, let's do it in hardware not in software, right? Yeah, so yeah as gels said The main topic of this presentation is actually the project legato which is Which is I would say a platform for building cloud native network functions based on VPP So before we go there, we may need to clarify what actually a cloud native network function is You know according CNC if I or cloud native means building the application as a set of microservices which are, you know packaged into containers and orchestrated by some orchestration software typically Kubernetes or or something else so With CNS basically we aim to do the same instead of deploying monolithic network functions We split the network functions into multiple loosely capped pieces Deployed as containers and Interconnected if possible with the fastest Links between them In case of VPP We put VPP into containers. We can use An interface called memif to interconnect the VPPs running in different containers on the same note and In which case the packets would go To share memory between the containers You know in case that that some of the ports are deployed on the different note Then it is a different story, but in that case we can leverage DPDK Etc and yeah to put VPP into containers. We basically need some some Management plane or control plane of of of this stuff and that is basically GATO That is the control plane part of building Coordinated network functions based on VPP Yeah, this is how it would like It would look like in case we deploy this Into production let's say we have a two hosts and we want to deploy some You know cloud native network function that we that is represented as a some chain of network functions Through which the traffic need needs to pass in order to to you know to do the overall network function that we need to achieve so in case of This you know you have you have different VPPs running on each note We usually use One of them as as the v-switch VPP and then we we deploy multiple CNFs on each note and you know we need to somehow consume the logical representation of the Service function and render that that into physical topology Which means that we need to have a management agent running in each of the containers Deployed on the notes and we need to have some entity which basically you know processes the logical representation Into configuration that will be programmed to each of the VPPs and Yeah, as I said legato is a development platform that allows you to do this it allows you to the first to do the Service function chaining as shown here part of it is a service function controller But even most important part is the base is the VPP agent that we deploy Together with VPP inside of each container So as we decided that we want to build Network functions like this We pretty much decided that we need to build a new management agent for VPP because you know Those that were available were not really good fit for the cloud native case. Yeah So in this case then legato is your control plane agent that talks to your cni provider Yeah, in this case legato actually has several components One of the components is the VPP agent that sits in each of the containers Next to the VPP and is able to program the VPP And then there is one more component As part of the legato project and that that is a sfc controller And that is the component that is able to process the logical representation of the chain Into the partial configurations that will need to go Into each VPP agent and Programmed to VPP. Yeah, it's pretty well same with concede VPP So when we when we hook in with kubernetes We use the VPP agent then we don't use the sfc controller for that Yeah, unless you're also running cns as well as kubernetes because the that's where the cni sort of hooks in with the VPP agent Yeah We'll get there. Uh, yeah, as I said, yeah, the VPP agent is a new Management agent for VPP Which was designed, you know with with For this cloud native use case We decided to use protobuf APIs As opposed to net com for rest comf which you can get from from honeycomb Pretty much the same API That is defined by protobuf messages can be used to program The the the VPP through any transport that we support So you can either choose rpc based Way of of putting The configuration to the agent In which case you would use the rest or g rpc or you can use the Datastore way of of putting the configuration to to EPP For this we support atcd ready's bold eb And we also do support message brokers for streaming notifications Uh, as as we were designing this we decided to use the goal language To build the new agent because we knew from from the very beginning that we will need to integrate with You know kubernetes and docker and that kind of stuff. So so that was pretty much a clear choice And you know since we need to have the agent packaged into docker container Uh Together with with vpp or to a pot together with with vpp It also has to be compact and small footprint, which would not be the case for the java based agents But for the go base agent, uh, it is actually You know just a static binary that that you put into container to get there with with vpp Uh, yeah, as you probably know if you want to program vpp, you need to use binary apis Those apis are quite low level They use like like numeric references to interfaces and You got byte ordering issues there and you use bit flex to to configure your stuff As part of the building the agent we we created the go vpp Project, which is part of the fdio And that is basically a goal and wrappers over binary apis So you have binary apis and we we produce the generator that is able to Generate the go structs that that basically reflect the binary apis and part of the go vpp we have Small infrared that that is able to you know Marshall and unmartial the go structs into binary and send to vpp and Communicate with vpp the vpp agent this The is part of the legato platform or project. So it is not part of the fdio It uses go vpp to talk to vpp And as I said, it provides more high level apis. So Uh, you know in case you you would need to configure vpp or binary apis you need to care about the About the versioning because each version of vpp may have different version of the binary api and that needs to exactly match Then you need to care about message ordering For instance, if you want to configure an interface and have a static route that is pointed towards the interface You first need to create the interface apply ip address and then create the static route So this is what vpp agent does For you you just put Everything what do you need to have configured on vpp to the vpp agent and it would automatically handle all dependencies between the configuration items It would handle some error states So it is able to to To do the retries on error or or or some auto healing. It is able to handle restarts et cetera It is also very modular. So if vpp is a set of plugins for for the You know forwarding then vpp agent is a set of plugins plugins for management So if you for instance develop a new new vpp feature, you would Build it into a vpp plugin. You can similarly write a new plugin for the vpp agent and just put it into the infra and vpp agents We package together into a docker container with vpp. So If you take any version of the vpp agent docker container, you always have the management agent with vpp Inside the the same container image. So you have no versioning issues This is this picture shows some architecture of the vpp agent as I said, uh, you know the center part is Is the plugins that are used to configure vpp through binary api Apart from vpp, we do have also linux plugin which can configure, you know, the the linux Interfaces and and starlight deck that which is needed In case you need to use this for containers And the bottom part is the apis that that we expose as I said, you would use the same Same protobuf api data model For any transport that we support Yeah, and so so this was basically the infrastructure part the ligato project So as I said the ligato provides the vpp agent Plus the service function chain controller, which can be used to to build the The providing chains the county vpp is actually actually an application of the vpp agent So we took the vpp agent and integrated it into kubernetes ecosystem Uh, basically, we just extend it, uh, the apis of of the vpp agent in the ligato project and you know, uh We we just edit kubernetes api on top of it and as the result of that County vpp is basically a cni plugin generic cni plugin Which can be used in any kubernetes cluster, uh, and, uh, if you deploy it you get vpp running on each note Providing the the connectivity to to the containers And at the same time it still exposes the same apis as the ligato vpp agent So if you want to you know, if you want to extend the default behavior of the cni plugin You just use those apis to you know, for instance create a new interface into into the pot Which would be used for some different purpose Uh, this, uh shows some architecture of of County vpp So with county vpp after you deploy it you you get a county vswitch running on each note in the kubernetes cluster Inside of the vswitch pot you have vpp and the vpp agent And, uh, of course you can you can use the apis to to somehow program the vpp But you automatically get, uh, the programming of the kubernetes pot's connectivity from kubelet through county cni And, uh, you know apart from from the cni plugin we have another, uh, Another connection to the kubernetes state of of the cluster Through our components county crd and ksr Which are basically just the reflectors of the kubernetes, uh, you know view on the cluster into our instance of atcd, which is basically Consumed consumed by each agent in the cluster This is how networking looks like if you used county vpp This is a on two note cluster or multi note cluster This is what you get by default. So as I said one vpp on each note connected to To, uh, data interface through dpdk interface And then we have, uh, one verve, uh, for pots We that this is basically just a preparation Later we we want to provide multi tenancy so you could have multiple verves per tenant And, uh, each verve could be, you know, each side of the pots could be separated from from the other tenance And then you have a bridge domain that is used to interconnect To form the overall network and interconnect between the nodes And you know each pot is by default connected to vpp through a tab interface Only because we need to support any generic application So either you have, you know, a web server or database running in the pot. It needs to work But of course, uh, there are options for for faster, uh, connections than than the tab interfaces And that would be on the next slide So what I've I've shown on the previous slide is basically this cloud verve Part of the picture. So if you deploy continue, uh, automatically with each pot Uh, you would get, uh, an interface connected to the cloud verve on con tv pp Running on the host And as I said, you can still use, uh, all the apis that we provide in the legato vpp agent to for instance create Faster data plan, uh, interface interconnects between the vswitch, uh, vpp and the vpp that might be running in the container Uh, and you can use sfc controller, which is part of the legato part to orchestrate by basically this this left part of of the vswitch picture Okay Well sfc controller is part of the legato project it's open source and, uh, the con tv pp is just, you know, this This part of the picture, but you can deploy it together on the same cluster So Currently the sfc has only its own api But there are other options. You can you can actually use Something else instead of sfc controller. For instance, there is a network service mesh concept And that would be another way of doing this this chains Okay, I think we are running Over time, so I'll just briefly go Uh through some load balancing stuff that we do with con tv pp With con tv pc and i uh, we have our own implementation of kube proxy, which is running completely in the user space within vpp This shows how how the programming of the kube proxy Looks like quite complex and complicated But the key message is that uh, you know, we do all the load balancing and netting on vpp So so you don't need to run kube proxy unless You want to have a pot that is connected to the host networking in which case you could still use kube proxy to to do the load balancing and netting This example shows some deployment with con tv pp. There is actually a mistake. I have two replicas of engines Defined here, but there are three replicas running in the actual cluster So I defined a replica set with with three web servers I covered that by a service of type load balancer which exposes basically two virtual IP addresses That are knotted to one of those backends In case of cluster ip that is an ip address that works within the cluster In case of external ip you need to have an external load balancer in this case We have metal build balancer that is configured here Running in l2 mode This would be the configuration of vpp that is rendered from that that type of of deployment So we have a one internet Gigabit ethernet interface that is the dpdk interface Used for connecting the node This one tab interface is actually the interface that goes to the top To the pot and those vx lan tunnels Are tunnels that go to the other nodes in the cluster This is the NAT rule For the service that I have just deployed on the previous slide So so you can see the both virtual IP addresses are knotted to local IP addresses of the ports And for the For the external ip one of the vpp's in the cluster acts as or responds to ARPs Using the proxy ARP So in case we had this kind of deployment to nodes external load balancer some router Then you know this vpp for instance instance would Responds to that ARP request for that virtual ip And all the traffic would be basically routed towards this node And together with traffic policy cluster. It can be still load balanced on vpp To the other node There is another mode of running these kinds of kinds of topologies and that is bgp based In which case, you know, both vpp's would would advertise those virtual ip to to the router And then you could have a proper load balancing So this is one of the applications of the legato vpp agent actually and Now jails I'll try and wrap it up quickly, but yeah, this is another application newer. I mean, I guess, yeah Think about history. Yeah, the first thing we did with the castle actually was a cloud-based cmts Which I think is this was announced last year But that was sort of internal proprietary And if you want to if you want to do cable broadband here's a cloud-based way of doing it The next thing was continued vpp which we open sourced And then we're now looking at other applications and here's an example, which is ip sec So vpp has ip sec today It has a full implementation with the forwarding plane to ip sec the vsp talent transport v4 v6 All the various authentication and encryption algorithms. I don't really understand On the control plane side, it has an iqv2 initiation and responder capability and that has been Interrupt tested with various other stacks, but the I guess what we're seeing though is that people are saying well vpp is more about data plane And these existing control planes like strong swan are very popular very well known And well understood and they have you know great sort of ha modes and that kind of stuff. So why don't we leverage those so Today typically we see deployments where people are rolling out strong swan in virtual machines And it'll just be a you know strong swan demon using the linux kernel for its forwarding You know programming it through net filter So the first step is to you know to plug vpp in to the vm and again, we can There's a couple ways we can handle that We've done things like intercepting net filter calls, but ultimately strong swan has some It's called vche it has its own kind of api for for hooking things in so we're integrating for that But really you know where we want to get to is these containerized deployments. So Having a separate containerized instance of that and then still having a v-switch in front of it and as Rasto was talking about with memmif so You know a single copy memory interface between a v-switch instance, which might be doing the excellent tunneling or whatever Let's talk to the outside world But then the vpp instance in the container that's doing ipsec So you told a very strong story about integrating kubernetes Yes With legato and then the local So in cunt even then legato the local agent So where the strong swan play in this is it is it are you building something like a 90 second clients or something? Yeah, this would be that kind of stuff you'd still have and I guess that's what we come onto You know eventually you want to integrate with kubernetes So using something like conTV pp The benefit of conTV pp, I guess is that we just run one v-switch for both the sort of ipsec data plane But also for the kubernetes kind of control and management plane But I guess the other thing to think about with the stuff we're doing memmifs I know there's a talk coming up on It's the next one, isn't it about multis and stuff like that And I guess that's what's interesting is these different approaches You know our approach is to say well, let's just have one interface that kubernetes knows about Use that for controller management plane and then we spin up these additional interfaces through the legato s of c controller Or in this case, we'll have a almost like a strong swan controller that's based on the s of c controller And those are kind of off on the side from kubernetes point of view And we use those for the for the network data planes But that's the kind of end goal is to have it. Yeah, so kubernetes can The kubernetes can take care of the placement piece of saying, okay, you know, how many you know run these as a record set Like how many instances of these do we want to run? Where do we run them if one of them dies spin up the new one? But then Have the set for a strong swan controller based on the s of c controller, which is programming These individual Vpp instances in the containers Rub and the shed one in v switch. So you might imagine, you know, you're accessing corporate vpn At massive scale the clients will understand coming in through a vxl tunnel Kind of like what rasto was showing before with vrs or bridge domains are per So multiple vrs. So one really for sort of southbound encrypted traffic Um, and then another one northbound for the unencrypted traffic Which is then heading off to your whatever your secure applications are behind this And the You know, I guess the goal is to be able to horizontally scale these instances of strong swan um, the challenge of course with the horizontal scaling becomes that you um If you get any kind of rehashing happens, so like one of the containers dies and everything gets rehashed Then you'll get a packet for which you don't have session state So you have to then have other ways of handling that. So for example another kv store on the back end where You know this container dies and this one gets their packet. It can look up. Oh, what's the state for that session? um, but I say strong swan itself has some pretty good ha so You know, there are there is stuff we can do where a certain number of failures can be handled simply through strong swans only And that's you know, a good reason for using it I think that was it. Yes. So any questions Other than from right They are not related Like small binary It doesn't Yeah, I think entails one was really like kind of a proof of concept of cni Whereas with con t we implemented full You know support for kubernetes services and policies I haven't talked about that, but policies are also implemented on vpp completely So we we program acls on on vpp On the interfaces facing to art spots So I think that that con t vpp is like You know more fully featured than than that one Okay, thanks I guess if there's more questions come and grab us after