 So today we're going to talk about control plane options for VPP and open stack and There was some discussion amongst ourselves about who exactly the audience is whether they're going to be like open-stack people that are curious about VPP or VPP people that are curious about open stack, but so we kind of took a best guess here and So we're going to actually start with With an introduction we're going to just talk a little bit about Neutron and open-stack exactly what it is, you know, I mean if that really bores you then that's okay If you fall asleep and the and then we're we're going to talk of various options for Networking for VPP and open stack and we'll start with networking VPP after that Which is what Frank is going to talk about and that is working and available now. It's it's more than prototype. It's in the D-release of opn fe and then we'll talk about Another option which is also done by the FDS project and D release of opn fe For using open daylight with a little sliver from group-based policy module for open daylight And then after that we'll talk about a couple of options for the future and among them is a An option for open daylight without the GP piece lever using net vert and genius and we'll hear more about that But so but let me just before I let the important people talk Let me just talk about a little bit about Neutron and on the advice of one of my colleagues I was saying well, you know people may need to know a little bit more about Neutron and how we got here So Neutron is the component and open stack that manages the networks for open stack and you Let me as a side note. I'm kind of like a low-level data plane kind of layer-to-guy So I'm not the greatest expert in what happens way up above my head and and Neutron So I may have a slightly different perspective. Hopefully what I say is accurate So it at first it was there was mainly only a few options in open stack Was it flat networks versus VLAN either static IP versus DHCP things were simple We just had to manage lots and lots of VMs and then came L2 VPNs L3 VPNs tunnels All kinds of configurations options and then a demands for SDN to more grant with more granularity manage What was happening on the data plane in the low-level options and there was a proliferation of Neutron variants and Implementations and there was a convergence or something. It's called model layer two Which kind of unifies the interface from Neutron's perspective and so all the things we're going to talk about now exists as either model layer to ML to drivers or variants and ODL underneath ML to so the idea is to deploy VPP and open stack and right now as I as I mentioned very briefly currently deployed an opiate is either networking VPP is available or ODL open daylight controller underneath Neutron with Gbp group-based policy and as a point point of As a POC is available ODL when network GbP and in the future And tomorrow there's a whole day on it. I think it's a whole day. We'll be talking about the the nirvana nirvana stack as well and at this point So and again this this diagram shows generally the different options and you can see here We whoops Where am I going am I going this way? Yeah, so you can see if I press that button I thought I got a laser instead of a laser. I got I changed so We we have open daylight networking ODL is on the left With that ML to plug in and then underneath that you could see the options on the left hand side of the screen for ODL and we'll be talking about both of those and on the right you see Networking VPP which is its own ML to a driver and that goes and talks to an agent that's in the compute node with VPP and now next We are going to talk about networking VPP VPP is is the first part of this presentation. It's really a basic integration of VPP with an open stack without ODL, but The before we go in the in the details of this presentation I really want to explain why we did that VPP is a very fast data plane for water That can be used to act to speed up east-west communications as well as north-east communications And we really wanted to have a project very easy to deploy simple to operate but that provides all the robust and production-grade features that are required to deploy NFV solutions as well as a large-scale cloud solutions Our goal with this project is ready to make a first-class citizen component in open stack and and to provide VPP as a de facto, you know good choice for for Virtual switching in open stack. So that's really the roots of of this project So the main principles wise were simplicity robustness and and scalability. So these are a bit of buzzwords, but what I really mean by that is by simplicity, we wanted to have a simple code and a simple architecture, right? And simple code means only few few unzobs of lines of Python and With no legacy things to operate something simple to simple to debug simple to write and to help also You know if people are interested in contributing having something simple is always better and in terms of communications everything is Is to have you know rest and JSON communications. We had some experience with RPC or slow model this kind of things and that really doesn't scale. So we decided to start with at CD, which is a very popular key value store in the industry very widely used in in the container space and That provides a lot of nice features such as a clustering and nature and blah blah blah so this is a really a key piece in in this architecture robustness means that any component in this solution can be Stopped and restart without stopping the service. So that's something also We we we did put in place. So Before I go in too much detail into more in more details in the in the technical architecture the features that are supported today are The minimum things that you need to operate a cloud. So we support the VLAN and VXLan GPE So these are the two kind of networks we support today And we don't necessarily plan to support many others because we don't see the requirement for that as of today In terms of port type, it's also very straightforward The host user interface not the not not the old virtario But the host user interface because that's simply super fast and we did a lot of we spend a lot of time Optimizing that at VPP level. So that's the de facto standard for VMs and VNF But and of course, we also have to support tap because you have some services in open stack such as DHCP which relies on on tap interface security is Something absolutely key in this project We believe we can do way better things that is what is currently done in the controller levels in security So what does that mean is we support things like a role-based access control? So if for some reasons a malicious users take control of a compute node, it doesn't take control of other fool the full thing so that's absolutely key and Beside malicious users that's that can be because of of simply bad operations So we want to to have Isolations between every compute nodes between the controller and the compute nodes So we have the we have that but it also means a lot of crypto to make sure that all the control plane operations are cycled So that's using regular TLS things and we are currently working on more advanced features such as Signing every message to make sure that there is no If dropping and there is no bad message in in the in the infrastructure That may be a bit overkill for small cloud But that is absolutely key if you want to operate serious NFV environment with so that may actually be a requirement for some regular regulatory compliance issues So that's something we are spending quite a bit of time on and in terms of a layer 3 networking so latest version of networking VPP was released last month and That was done In this that's so that is a 17.4 and in this version we support now Layer 3 networking which means a you know an alternative to the curator and so we support a SNAT We support floating IP. We support IPv4. We support IPv6. All these things are Supported in the latest version. So that's That's the current feature set few so that I will Skip so few words on on the architecture itself. So all the Communications are going through at CD. So when your throne wants to open a port, it will actually not directly Get connected to the compute node But it will first store the desired state in at CD in a key using a simple key value, right? and then all the compute nodes are Looking at what's happening on this key value store and when there is a new port The compute node there is an agent sitting on top of a VPP in the compute node That will be raised when a new entry is set and that will configure a port a network or whatever you want So the key point in this architecture is we provide the warranty that when Neutron is asking to create an object. It will be eventually be be created, right? And this is all asynchronous Communications because creating a view to the port can take, you know, a few seconds, right? So That can be subsequent but but that's not that's not my micro second. So What we do is we have a synchronous communication. So you first set the port in the at CD key value store then the compute node will read this value will create the port and will create another asynchronous communication using HTTPJs on our TLS and populate the at CD key value and then Neutron will be notified of that. So that's the overall schema So the other nice thing with that is we didn't have to care when we wrote the con and when I say we I'm really speaking about Jan Wells. We who is here and and What we did is we just had that we we heavily rely on at CD properties So meaning that to build a cluster. It's actually Super simple you just have to spawn several instances of at CD and that's it you do not care about it That's works really well that scales well and that provides all these goodness things. So so that's That's actually a very we can get all these nice properties security scalability robustness in Very few lines of Python. So that's really really cool In terms of robustness there is this rethink mechanism, which is absolutely key as well So this ink is state reconciliation So if you kill VPP or if you kill the agent or if you kill or if you try to kill It's hard to kill. But if you try to kill at CD itself You can you can retrieve the states and Reconciliate what are the desired state with with the operating states without stopping the service and that's absolutely key for operations H.I. deployment. I already talked about that Okay So this is the role-based access control as I told you before we have isolation So here you have four components at CD is one of them and you have compute not one compute not two and And the neutron server. So every component has rules for and and airback rules role-based access control. So Compute node can only read a subset of the keys and write another set of the keys same for compute node So you have no risk to have compute node one writing on compute node two entries and Neutron server will also have some limited rights, right? So it's very difficult if there is a problem with with code or malicious user You are you have all this nice isolation and all of that is a cyford Roadmap so next version will be 17.7 first version was 17.9 16.9 then we had the first 17.1 release which brings this security groups feature and in 17.7 we'll add some Remote group IDs which are really required for for security groups. We don't have that today and that will be supported in Next version. We also have some some Enhancement to do in VXLine GPE We will improve the layer 3 support because today we do not support VRRP and HA and those things and that's Of course mandatory for any NIV deployment and probably the most important thing in this slide is Testing testing testing up fully the code is not that huge. It's it's actually very very small We are speaking about like a few thousands of lines of code But but we need more testing. So here you consider this presentation as a call to community And if any of you is interested in testing or contributing you're more than welcome So that was the simple integration of VPP in OpenStack but some people may have more complex workflows and That's what Frank will describe you talk about two things Maybe more complex workloads, but also Latching on to what Jerome was just saying like In order to harm the thing you got to go test it test the test and then prior to testing it You need to go it deploy it and deploy it and to deploy it, which is where we are from here So if you're taking on and Want to go and run the thing for real? Well in an opnfv. We're not actually building stuff. We're Integrating things we're taking the various Components that we've been talking about OpenStack Fido and in certain cases also more sophisticated additional componentry if you need an SDN network controller We typically choose open daylight So that we can build an entire solution stack so that we not only have a forwarder and a network controller But we can also build life cycle management of the individual componentry on top so in an opnfv we Compose these stacks We integrate them we deploy them and we test them and we start over and We do this in an ongoing basis And I think by now opnfv has done more than three four thousand deployments of open stack as a solution stack over and over again We run tests against that and well We'll fail on a daily basis and we'll succeed on a daily basis and we improve on a daily basis One thing that we've done roundabout a year ago is we noticed that most of the solution stacks We're just based on vanilla OVS, and if you want to run NFV for real Well OVS gives you at solution stack level something like 60 50 60 thousand packets a second throughput. Is that good enough? Well, it's good enough for cloud, but it's definitely not good enough for NFV So we needed to go and change that Changing that meant we had to go and switch the forwarder for something faster better more flexible to go and build the feature set for NFV So we wanted something high performance highly scale. Well, we're talking at Fido mini summit guess what we've chosen We also wanted to be able to deal with a diversity of forwarders moving forward And that's where the SDN controller comes in and a more sophisticated feature set So that we're not really saying, okay We will build our entire network always out of VPP or always out of well, maybe OVS DPDK But we'll see use cases where these things combine Even more so where you start to combine virtual forwarders with physical routers because as soon as you have an NFV deployment with Kind of a say virtual ICP environment. It will be hooked up to a wide area router So how do you seamlessly configure that you need a controller that speaks multiple languages? Needs to speak to the router the physical router It needs to speak to Fido VPP and the likes so we had to have something there That can talk multiple domains multiple forwarding technologies. Hence you have the place for creating a stack that not only Is a direct integration between open-stack and VPP but also allows for That level of diversity and capability. So we were roping in Open daylight at that point what we've been setting out as part of the fast data stacks project in opnfv is to go and compose these stacks including networking VPP and Run them for real and that created us scenarios and well opnfv has a far fairly sophisticated notion to name scenarios so composition of solution stacks Without any SDN controller. That's the ML 2 scenario and Fido with high availability and open stack and without high availability and open stack Sometimes using the controller only for layer 2 networking sometimes using the controller to set up layer 2 and layer 3 networking So we've been composing these stacks and adding well VPP Fido as one of the key ingredients so The scenario list that we have an opnfv today is the direct integration between open-stack and VPP. That's ml2 We have open-stack just controlling layer 2 networking. That means east-west Is ODL controlled? North-south is just still using q-router So the Linux kernel and well in late may time for a late March time frame We also came up with a scenario that uses VPP for north-south in addition to east-west networking and That's what we've been releasing with a den of 1.0 scenario So all of that's been done as part of the fast data set net stay fast data stacks Project and these stacks are not only composed. They are deployed on a daily basis. They're deployed with a tool called apex And fun we'll talk about apex. I think at least for one slide and then later on In the afternoon in a greater level of detail that tool is based on triple-o With some minor modifications to make it more applicable and appealing to NFE deployments So what did we go and need to go do in order to make this step really happen? So Apparently it wasn't only about composing it was also about evolving the functionality because when we started off There was no ability an open daylight to speak VPP We had something that was available able to go and Well do basic netconf But we need to go and evolve it so that we were able to go and configure VPP at the same time when we were starting off VPP had very bare-bones capability from a VX land perspective And we started off more than a year ago remember right so we had to grow the capability set in VPP to be able to Offer the appropriate level of tunneling that we needed an open stack. We had to make opens open daylight speak The appropriate language southbound netconf yank towards VPP notes and in order to make that happen As Tom was saying earlier on we started off extending a project that is called group-based policy We've chosen group-based policy for the one and only reason that it only already had the capabilities to go and talk netconf Yang talk OBS at the same time and it had a generic abstraction that was independent from anything southbound and anything northbound because it had a Connectivity model that was very enhanced already so that we are not depending on one or the other forwarding technology That allowed us to integrate relatively quickly So we had to go and grow the feature set there and come up with another renderer that was just speaking VPP so to say Well, and once you had that we also got ahead had to go and to grow the capability set in opnfv So that we were able to test and integrate With the CI CD deployment pipeline in opnfv And that was the enhancements or the adaptations that we had to go do an apex to make the overall stack install So loads of moving pieces that we had to go and bring together in order to make these stacks compose and test on a daily basis how it works throughout the The the individual modules. Let me just go take you through a very simple example how you go and create a port In open stack and what that means from a data plane perspective. So if you're creating a new port The first thing is well new John the ML to ODL driver there will talk to new turn northbound and Then you have a first level of translation between Neutron language and its view of the world into generic Language that happens and group-based policy that is no longer well tied to the networking model that we have in in Open stack and open stack neutron. So that's a generic level The next level down is going from the generic Notion of a port back into a specific notion of what needs to happen on a particular switch And in our case, this is VPP now. So we're going from the general Neutron mapper into generic core a code and then from there on into the VPP renderer that allows us to speak generic or specific language towards VPP so that we can have a configuration commands go directly down to VPP those color configuration commands travel in In that conf so there is a Yang model exposed for every single VPP that talks to this Adapter that we have on top of VPP that exposes VPP's APIs Over a net conf Yang and that model or module is called well honeycomb. It's an agent that supplies a northbound net conf or rest conf interface And southbound is driving directly the the APIs of VPP The port configuration happens over the VPP renderer But in order to set up tunnels between the individual VPP's and eventually also other entities You need somebody that manages topology in the first incarnation of the project We've been using an ODL project called virtual bridge domain manager or VBD that configures the VX land tunnels It's not part of group-based policy It's an independent project and moving forward we will Even there use what ODL is really great at it's a very pluggable infrastructure So we will evolve from just static tunnel configuration Which is something that VBD supplies to us to use list so that we can set up tunnels dynamically and leverage yet another Componentry in in open daylight So we don't have to care about how tunnels are set up and maintain list will take care of the overall thing So and once these two things are coming together Well, you will have connectivity between individual ports on individual VM instances as they're distributed across So from a deployment perspective as I said we'll have we started off with a very simple deployment where Open daylight was just handling and VPP was just handling layer 2 forwarding and that meant out here You have this little entity called Q router Which is the Linux kernel that was kept in place day one Why we didn't want to overwhelm ourselves with the complexity so we used VPP just for east west forwarding and these models still exist You can can go and configure things so that we have individual bridge domains that are VPP supplied Integrated or connected by VX LAN You'll see the honeycomb instances to go and drive configuration From the central open daylight network controller that drives the connectivity here the only real Delta to what we released in late March time frame In the Danube release 1.0 of opnf v is Well, you see the Q router being gone So that means from this release onwards we use VPP for east-west and north-south OVS is entirely replaced off the map, which means we can go fast No matter whether you're communicating within the tenant or between tenants and that means fast means multi-million packets a second between individual and into Instances so going back to how we have been evolving the story We came out with the first Incarnation of a solution stack in September 2016 roughly six months after 5.0 launched And I think that was quite a success Because initially well, we were releasing with Fido and VPP were releasing an engine But people don't want an engine people want a car So they want the engine to be integrated into something that you can use as an end user In Colorado 3.0 it was the first incarnation where we released networking VPP as a solution stack So not that it had well a standalone thing and it runs in depth stack No, it ran for real on bare metal And we released it with Colorado 3.0 in December 2016 Denup 1.0 was the first time that we had layer 3 networking and layer 2 networking So east-west north-south all supplied by VPP and more recently in May early May Denup 2.0 was released where we started to also support HAA ODL clustering as well as open stack HAA componentry With the L3 scenario, so we've been evolving that solution stack And we will continue to evolve that solution stack and so the next obvious stack is to go full DVR With VPP and that will hopefully happen in a month's time And with this I just wanted to go and highlight one point that I made earlier on That's a tool that we are in the process of open sourcing as part of NFV Ben as part of OP NFV Which is called NFV bench. It's just a full stack simple way to measure the performance of a full stack and Well, we've been comparing Forwarding performance of a full stack deployment with OBS, which gives you something like 50 60 kilobits a second To what you can do in a full stack deployment not optimized not doing anything of tweaking If you do the whole thing with VPP and the outcome is well, it's relatively small Numbers up there, but the Delta is in a full stack deployment. We are without any optimizations. We were reaching something like Roughly 2.5 2.6 million 6 million packets a second with a full stack deployment of fast data stacks with VPP Compared to something like 60,000 packets a second with a vanilla OBS deployment Yes, this is almost an order of magnitude slower than if you do of well optimized focused VPP testing Where we can easily get to something like 18 million packets a second per core But hey, this is full stack deployment. This is not like testing the engine It's testing the car with the engine inside But it's still well 20x more than you would typically expect from well an earlier cloud focus stack And I just wanted to highlight that I think we really enabled NFV with what we're doing here One word on a packs. That was the wake-up call and you made it Yeah, so it's all around I think that making the whole thing real means you need to be able to deploy to a bare metal and Being able to deploy to bare metal is hard. It's really hard It's labor some it's cumbersome and you need something to take care of all the various kind of knobs that you need In order to tweak something that well is generic to run on a specific setup And I think that's exactly what apex has done and it's trust us really well throughout the project so far So yeah, so I just want to give a very brief Introduction to apex project. So apex is a opnfv project. That's a focus is on installing and deployment it assembles the upstream projects like, you know open stack open daylight and phytocomponents like VPP and Deploys them into a environment of some sort, right? It could be virtual. It could be bare metal In particular, you know, the virtual the virtual environment is very useful for development purposes, right of new features It allows you to use a single server It configures all the VMs for you and you know set up networking Especially we allow, you know, very flexible network configurations, right? You can do multiple network isolations in in your deployment all of the scenarios that Frank talked about and Jerome talked about are developed on this and We are running them in both in our CI environment with a virtual deployment And we actually run daily tests in the bare metal setup so that we can, you know, find issues and fix them very quickly, right? so you know that that is Really useful for us and also Once we where we integrate those new features into apex We turn the turn those features into upstream we go to you know the upstream project in this case It could be triple O neutron or others and and and go back to upstream. Yeah, so that you know the upstream project can improve right So I think that's that's all I want to say about apex We do have another session at 430 that goes a little bit more in detail about, you know What apex is and and how does it work and all that stuff? so we can talk about The future I suppose So just Because I'm holding it upside down Well, not if you're holding it upside down. Okay. So the first so But let there's a let's talk about some things that are happening in the future now We're almost out of time. So I'll try to be really brief. So There there's another project that's in POC level for ODL that bypasses the GPP sliver Well, you're using GPP as a as a just a Yang map a Yang mapping to to net comp and instead Does a more thorough Implementation using genius and ODL. This is what we're calling within ODL This is what we're calling a converge solution because it allows us to To to abstract completely within ODL whatever is happening on the data plane and the southbound that and the and the southbound interface can be like net com or open flow and OVsdb so you can see where we're going is that we for example, we can have we can deploy OVsdpdk with this model and We could actually deploy Vpp with a honeycomb agent with a net conf on the southbound and this is all within Within ODL and on the top. We're still using the networking ODL ML to for neutron And this is in POC now. So if you're interested in trying it out Let me know and I'll give you the contact information as to how to get it So again, this is a picture of both simultaneously Maybe in the future when ODL becomes multi-threaded and I think there's some work on that it can control multiple data planes at once and different Color so when we started this off that might be something that not everybody in the room knows Group-based policy and net bird were two projects that do not load at the same time and to open daylight Open daylight is modular. Yes But there are certain things that are mutually exclusive these two projects were mutually exclusive Why because they've been developed basically in two independent bubbles and that nobody ever thought about kind of why That turns out to be a problem because well group-based policy is very good at Adapting based on its generic abstraction to various types of forwarders. So we've done that with net conf Yang OVsdb open flow We've weren't really that much focused on implementing a variety of services like layer 3 BGP VPN for instance net word has exclusively bolted itself to Open flow and OVsdb, but spend ahead of time building out a rich set of services That sounds like oh, yeah, can we bring these two together? So Generic abstraction talk to multiple south bounds fine loads of services and then even make these two load at the same time That was the objective that well the Nirvana stack idea drove Which is why we started to combine that and what we're showing here take go go on one slide back is two different ways to go and integrate that where we started off is more on what we have on the right hand side where we are using the renderer capabilities and group-based policy to drive different southbound forwarders, so the complexity of Who you talk to southbound is kept with an open daylight. That's what the current proof of concept does We just wanted to go and showcase. Well, it's not only about engineering. It's also about social engineering. We started to work across these teams that formerly operated in independence There is another idea that Andrea Fredette just brought up a couple of weeks ago and that we've been debating since then quite intensely. We just say maybe we can simplify open daylight quite dramatically and Put the adaptation to what switch model you're talking to down Directly onto the device that runs the switch and have honeycomb as an agent Well apply for that adaptation so that you keep the flow-based logic and pull the flow-based logic now all the way down Into where it really belongs which is OBS as opposed to expose it all the way up to open daylight That is far more scalable from an approach perspective because here if you have kind of destination unknown It's no longer punted to the controller. It's kept locally here. It's operated here and So these things are are actually debated and if you have feedback from us for us Which model you like better? We've done a proof-of-concept now. It doesn't really mean that we're nailed to one or the other path It would be really helpful for for kind of getting an understanding what you see as the longer-term approach So it's really contrasting two options The right-hand side were more like what we have and running POC, but it's the left-hand side I think is viable as well So that tries to implement that Nirvana stack idea because Nirvana stack was about combining forces and Supplying additional levels of modularity at open daylight level. Why? Because we wanted to go faster as the industry Less competition in that space means More capability for people to innovate on top of open daylight or within an open daylight rather than us say competing in open daylight, sorry for kind of that was more color than I initially was planning for We have another one Yeah, we are you think you're done pretty much with Nirvana. I think I'm pretty much done with Nirvana Yeah, and then tomorrow that's I think that's towards the the POC thingy Okay, so We're almost out of time then so tomorrow. There's actually quite a bit more about Nirvana. So let's open this up to questions at this point Okay, by the way, if you want to see the code of what happens as kind of the first incarnation of Nirvana Oh, thank you. That's the slide is gone. There was a slide in there We can put the slide back in that had the various I think commits and whatever. Oh, yeah, where that went I didn't take it out. Anyway, it doesn't matter. It must have so I think that's the earlier flow diagram that I've shown you where You've seen the service traces of Nirvana and the group-based policy traces off of southbound adaptation By the way, so let me do a quick show of hands Who likes the integration directly on the v-switch level so that we put most of the complexity into honeycomb? Who that's quite so who would like to go and keep the adaptation at open daylight level? Oh That's a pretty solid statement. Thank you guys. I've never ever seen a room with such a Focused opinion on on one way or the other. Wow pretty cool. Thank you so much Made a move the move in that direction in the VPP project So honeycomb is an effect split in two and see if I can explain this well extemporaneously it does the honeycomb interface agent and the honeycomb infrastructure itself with the with the support for the North Brown netconf and the kind of a gutting of ODL and So those are two separate things. So honeycomb has been separated from VPP In that way, which so that kind of helps Facilitate this is are there any questions for any of us? Thank you