 Yeah, good morning. So I'm Frank Baudin. I'm a product manager for OpenStackNV at Red Hat. Morning everybody. My name is Uriel too with Intel. I work for the data center network solution group part of the data center group and what Frank and I have in store for you today is some introduction. Hopefully the light is actually in my eye so I can barely see you but introduction of FIDO, FDIO to the OpenStack community. Hopefully some of you have heard about it before and maybe some of you have not. So there is lots going on in that community. We are not going to be able to cover everything over here. We wanted to give you some of the key highlights, explain what that technology is all about and why and how it is relevant to the work that we are doing here in OpenStack. So we'll start with what is FIDO, what is the community, who's in that community, how that community works. Frank is going to go into some depth about the fundamentals of the technology that is used by VPP which is one of the projects under that bigger umbrella of FDIO. We'll give you some examples of a few different projects and few different approaches how this new data plane can integrate into components that supposedly are well known over here specifically OpenStack Neutron and Open Daylight and some options that are independent like Open Daylight and SFC directly and we'll talk about those options. Obviously in OpenStack community these days you cannot present anything without talking about containers. We are not going to break that tradition either and show you how this technology or this set of technologies I should say is relevant to the container discussions we are having. We have borrowed heavily from a large set of people who are working in this community and we want to thank them for some of the slides that we stole with pride. What is FDIO? I want to start with putting things in a little bit broader context before we dive in. Basically as you could see it's a high-speed programmable data plane that runs on the infrastructure that we use in order to fulfill what OpenStack is trying to do, what NFV is trying to do and so on, namely on on some servers. You could see at the bottom the fundamental layers or services and we'll touch on all of them what role they are playing. The bottom layer, DIO is really comprised of DPDK, the data plane developers kit. This is a technology that has been there for I believe about seven years or so and this group is also in transition. They are moving from an independent organization to a location to be advertised soon, most likely into the Linux foundation as well, but what DPDK provides here is a set of software libraries that are tightly coupled and matched to all the innovations and progression of the server architecture and therefore what you have is a set of primitives that would give you as close as possible to the capabilities of the underlying hardware technology. On top of that, the layer referred to on this slide as processing is the VPP and Frank is going to cover that in detail so I'm going to leave it for later, but basically it rides on top of the DPDK. It utilizes capabilities in the server that supports vector processing and it adds many other network related services and technologies as you'll see in the rest of this presentation. The other layer here, the management agent is a capability to control all of that locally or remotely and again we are going to show how all of that comes together. One option and we are going to go much deeper into it right now so I'm not going to take the time at this moment is to layer that such that this server resident data plane hooks up into an SDN controller like Open Daylight and Open Daylight has already known interfaces into OpenStack. Some innovation is happening on those interfaces in the context of FDIO and we'll talk about that, but that is putting things in a little bit of perspective. So the takeaway, these are the layers differentiate between what FIDO as a bigger umbrella gives you and what VPP is providing as a set of services as you could read from the slide, it's one of the key projects. What is the community all about? What are the key tenets? What are the key things that this community is trying to do? And maybe the four key elements that we want to touch on today you have these community members as you could see on the upper left side. Almost these days in the industry standard tiering of platinum and gold and silver for a community that was established just a few months ago. Very impressive acceptance by the industry and open source doesn't obviously go without a community and community is benefiting and the project benefits significantly from the right governance and for those of you who are familiar with the Open Daylight governance and I for one am with the Open Daylight community for a long time the governance structure that we have here really is from the good lessons that we all learned from the Open Daylight with a few announcements and maybe one of them that is worth mentioning I'll mention that on the other slides is the concept of sub projects that could individually progress and individually release and that is something that we did here intentionally to allow as large diversity of communities and projects come together under this umbrella. So the good governance goes hand in hand with the fast progression of the technology and the community and with enabling innovation and that is very important. The scope now you are familiar after the previous slide with these three layers and you will see that actually the community not only works on the data plane itself but also works with OpenStack with Open Daylight to name few other communities in order to make the data plane available to those orchestration solutions. Another element that this community brings to bear here which we believe is probably the first attempt to do something like that in the industry is to provide some sort of a CICD environment where if you with your project that you contributed to the community have a new feature that enhances functionality or performance there is a lab that is running in the background and enables continuous testing of that making sure that no new feature that has been introduced is going to cause any slowdown of the performance. So we want to make sure that we hold two flags as high as we can not only features and functionality but the performance aspects as well and that is the role of the CPL that was supposed to take me to the next slide and it does. So some of the projects that we have in the community starting again from the bottom we talked about that layer VPP which we are going to double click on next and Frank is going to cover that. Some of the projects and due to shortage of time here this morning we are only going to cover some of them. We will cover NSH, SFC that service function chaining technology and we'll show how it works in in this context. One I would not say one and only but it's an overlay network engine that is based on LISP technology and it is an attempt to show better scalability and ability to actually do overlays end to end supports multiple technologies we are not going to cover that in more details and you will have pointers here to the community wiki and you could get those informations. We also have here the concept of a sandbox which really allows the community the developers to play with new features that they want to add in a confined environment make sure they work and then they could launch that into the rest of the project and another piece that we are going to lightly touch on in this presentation this morning is the transport layer developer kit. This is an attempt to add the full functional tcp stack for those cases where such processing is required but all of the processing that we do in the context of fdio as it is based on dpdk is in the user space so when we have traffic either emerging from the server or ingress into the server that traffic is taken out of the kernel moved into the user space and therefore one would need to have a transport layer that transport layer for those use cases where it's required is going to be and that is the design goal to be much faster than the one in the kernel. On the right side we have a set of packaging options for popular distros and we have exercises like the trex and the continuous testing and integration that is part of the project and the last component here is the honeycomb. The honeycomb is that agent that resides on the server on top of the vpp layers that allows one to either manage that functionality on the server locally or hook it up remotely using netconf, resconf technologies in order to interact for instance with an open daylight. You have more text here on the governance are probably just going to hit a few key points here it's a fully open technology anyone could contribute you get to a commuter based on merits based on a recommendation of your friends. I mentioned earlier that we did something a little bit different here with the subprojects where subprojects got much higher level of autonomy than you normally find so anything that is defined as a subproject could also release independently and we also preserve that kind of coupling and decoupling of the board from the TSC so the board is focused on the business aspect and is not giving technical direction to the developers the technical community interacts with the board on business issues general direction but not more than that. You could see some of the rather impressive activity that is happening for a community that has just launched few months ago and we'll show you not only the functionality that is available today but actually also the fact that this technology is getting very very close to production and maturization on some aspects while there is we have many plans to do additional work and another project but provided some statistics for you so that you could relate to this and with this intro over let me hand it over to Frank to cover some of the fundamentals of the technology. Thank you. So what is VPP? So VPP is a protocol stack made of graph nodes so to take an example when we have let's say an IPv6 packet getting in we go into an internet input pass the header we that's an IPv6 so we go to the IPv6 input node then to the IPv6 lookup you find your exit route and then you go to the node to transmit the packet basically so far nothing during you now this is vector base so instead of going through the graph packet by packet you go with a bulk and 256 packet up to this is the value that is a default one today as a benefit you will always have processing packets by 256 in a node so your instruction cache is hot you don't have instruction cache miss and also all of the nodes are implemented in a way that you process the packet always 2 by 2 so I've got a first packet that's an IP header so I'm doing the instead of doing a lookup in the floor table I will prefetch based on the IP address the sale of the floor table lookup corresponding to the floor then I will prefetch for the second packet and while I'm back on the first packet uh the the sale entry of the routing table is in the packet is on the CPU cache so I don't have a cache miss by processing the packet 2 by 2 and I'm interleaving them so no data cache miss no instruction cache miss thanks to very clever prefetch and interleaving packets processing and a quick comparison with the ovs paradigm so we have a compiled graph when you have a you have a bulk of packets going through uh the graph node and exiting while with ovs you have a data pass and a control plane a slow pass so basically with ovs the packet gain in and if it's in the cache it gets out of the it gets out very fast if it's not in the cache meaning that's a new flow it gets in the control plane and go to a very very long pass compared to the cache and then you add an entry in the cache and then you go back so vpp has a kind of deterministic time of traverse all all packets are equal when you enter vpp while with ovs the difference is if you're a packet from you from a new flow you're going to be much slower another comparison with ovs is vpp is uh confless when you start vpp all the nodes have no configuration so an external agent has to push the configuration and i'll come back on that later on and finally vpp has no kernel implementation counterpart when you want to implement something in vpp you implement it once not twice you don't have to implement kernel and use the language you implement a node and you're done so it accelerates developments in term of portability so um vpp is portable on multiple architecture x86 ARM power pc and could be easily ported on other architecture it's ported on various operating system because it has an abstraction layer of a very operating system named celib that you have to implement for all operating system something very classical in term of nick supporter what you need is an so what is a driver a driver is an input node so what are the input nodes available today we have dpdk tune tap af packet net map and even legacy driver don't ask me why it's there i don't i don't think you will use them but anyway and last but not least the host user to interconnect vm to vpp also you have also a shared memory implementation named ssvm plus others that will come which is typically uh a faster shared memory that could be used between containers for vpp instances being in different containers i'll come back on that later on also vpp has some nodes which leverage hardware accelerator like for instance for ipsec so you go to your ipsec node and then you can offload some ipsec ciphering deciphering in hardware and you can deploy a vpp on the host bare metal as a v-switch as a router in vm in containers and a little trick from vpp is the critical for instance the ip lookup table nodes are compiled with various cpu option optimization so if your cpu is old an old version it will work but if you have the latest greatest avxv3 and you have an optimized lookup function it will dynamically use it you don't have to have a so one vpp version can work on older and latest greatest cpu so that's very practical in term of modularity flexibility so first it's easy to build up your graph node for your use case so you create your stack for your for your need now you can add plugins and the plugin is often a sub project in vpp so a plugin is a bunch of nodes uh that can rearrange the original graph uh that uh that can be built independently of vpp source tree and that can be added at runtime and of course that can extend the configuration api so for instance uh nshsfc is one of these plugins and once again all in user space so it's permit to build v switches, vrotor, anything processing packet so now we have nodes we have a nice framework but as a developer how do i debug this framework so vpp comes with a lot of embedded telemetry so not the open stack telemetry so for instance that's an example of counter so uh here i've got statistic per node and we can see per node how many calls how many vectors you know the vector that's a 256 packets how many clocks are spent in a given node and how many packets are so in this example i'm at 256 minimum i was quite loaded but not 100 percent so this is a very basic graph node but when you have something complex don't be afraid you have the tools to debug something very nice tcpdump so i want to tcpdump in my vpp graph node so what so this is uh this is a real screenshot like the previous one so you uh here i'm asking to get 10 packets on vpp and for and then i'm showing the trace this is something which is put in memory so it does not affect so much the performance so and i've got per node a lot of depending on the developer node a lot of or a little of traces and also time stamping per node so you can follow your packet in the graph node and identify a bottleneck if any so that's very handy vpp has a v switch v router so we are all here at open stack summit and the vpp first use case from my perspective as an open stack product manager at redhat is a v switch so how so first how neutron implement how the neutron implementation with vpp is based on bridges so when you create a tenant you create a kind of bridge object from a from a vpp perspective that will interconnect all of the ports it's very simple to understand if you want to implement a v router you will create a vrf which is a routing table that will you will interconnect with nodes uh the configuration has to be pushed and stored by an agent because as i said vpp is complex so you need to have an external agent so that's very modular so vpp is really modular by essence inside and also it does not want to do everything it just do packet processing fast uh one little note because that's my uh that's my favorite topic benchmarks so most of the benchmark in the wild today are either physical to physical either uh cross connecting ports with vm's which is fine which is a good step ahead but still not with open stack so uh just to say that most of the benchmark that you will see in the wild are not representative of an end-to-end use case with open stack it does not mean that the performance will be lower it just means that people take a shortcut to make the benchmark and we are working within opnv project to provide you end-to-end benchmark uh with vpp and with other v switches with open stack because that's easy to cross connect ports but now if you have a very complex topology uh of graph or something maybe the performance will collapse so we need to benchmark the end-to-end use case also just to say that vm's are connected via vhost user ports and today vpp has a specific vhost user implementation which is in fair competition with dpdk uh vhost user implementation and fido people and dpdk people are working together to take the best of both to finally come with one version so that will converge this is work in progress and uh this so last week we we were in in dpdk summit and we have seen a lot of people working to improve this performance because phost user today is a bottleneck for any v-switch including vpp so uh later on you can click on this link and get a full report about vpp performance and there is a thread on the mailing list about the performance of vpp comparison with ovs dpdk and nobody at fido or ovs dpdk is afraid of the comparison that's open source this is a fair comparison with real real arguments no marketing so today that's number that we've seen but once again i said this is work in progress we're working on the on the benchmarks vpp features so you can go on the website it will be soon updated i guess and uh so you have plenty of features more than you want more than you want the 16.09 bring you the latest dpdk the next version i guess with rebase by heart on the latest dpdk so vpp is always up to date regarding dpdk so far it brings some uacl which are uh useful for security group for neutron a lot of enhancements that i will just keep so i'll let you have a look by yourself and the the dual of the chrome from my perspective as a product manager that's their ci fido has a really great ci so name cc it basically uh the ci include performance test non-regression with these 15 tests so every time someone try to submit a patch if you have a regression in performance the patch does not pass the ci gating so that's really cool also for a given version of cc of sorry of vpp you have the test results available with for all of this configuration so you want you want the data you just get it get there also you can download cc to install it in your lab because that's all open source you can run the same test all is open no tricks regarding uh yeah sorry i've been i've been quick because we are kind of yeah kind of pushed by the time so uh from an open stack integration we have two paths first path is ml2 direct ml2 so there is an ml2 plugin which has been pushed so you have the link to the announcement so it's based on it cd and you have seen that in the fido community you have a calico which is also using hcd so i guess this is a coincidence there so uh basically you have an ml2 uh mechanical driver on the server which is pushing via hcd the configuration to the ml2 agent that uh store the configuration and in terms of features so we are not priority feature yet with neutral with ovs but we are closed so for instance today you can restart your vpp agent you can restart the ml2 driver and it's supported so i'm fast but uh basically it's connected to the qrotor the qd hcp and uh in term of distribution it's integrated in apex tripolo installer so if you take apex tripolo installer you can deploy the ml2 plugin end to end and it's also included in dev stack a next item in the in the ml2 roadmap is a security group to be fully implemented anti-spoofing tap as a service and for the version later vxlan support and integration with telemetry systems and with that i will hand on to uri so that's going to be a bullet train through the next few slides given time here uh the other option that that we have is the fast data stack uh with the fast data stack there is a different ml2 interaction with uh with the neutron um stack and you're utilizing the group-based policy stack all the way from um the top into uh open daylight in order to render whatever you need on on a server node give you um some more detailed example of uh of how it works so when neutron creates its uh uh post port command which really is about adding a new port um the following layers that are part of the open daylight implementation uh kick into uh into action uh the north bound which is uh the active layer over here is going to store that information in in the new uh neutron uh data store and uh group-based policy um is going to be on a listen mode and it's going to catch that information that the new port is being requested to be added and it's uh part of the nodes that it uh covers and it's going to be as an example a node that needs special configuration on the host like a vv host that's the interface that frank was uh showing uh earlier so uh the next step in the progression is to create a group-based policy uh endpoint and now we have um we add the policy say the policy would be um in this particular example something about let me create a vxl and tunnel between these two endpoints you'll see on the left side uh vpp one is that case where i have a virtual machine in the case that i have a virtual machine i need to take two types of actions one is the local configuration uh which in this particular case is about connecting a virtual machine using a vhost to uh the v switch that vpp um exposes and that's um that error but uh we also need to configure the quote unquote v switch in order to create the v tunnel and that is the other set of activities as you could see vpp2 as an example doesn't have a virtual machine so one of the steps is skipped um another uh example and again for shortage of time i'm not going to introduce this slide um at depth because we want to leave a few minutes for the container as well but this is um another technology the itf version uh nsh based where you have service functions as you could see over here the key idea is that i want to direct my traffic along a graph and i want to set my resources size them up in an efficient way so that only the traffic that needs to hit the resource really has to hit the resource plus adding the capability of metadata which is a standard way to allow those nodes to communicate with each other whatever information policy based classification based that you accumulate you could now uh progress through the chain uh with the rest of your traffic um this is what uh we we support uh over here and um the overall open daylight sfc architecture has been enhanced to include uh the vpp renderer that as you saw on the previous slide is that entity that uh that allows us to configure a vpp service function so that we create the right tie to the vswitch um and this is a fully compliant uh implementation in parentheses i would say that uh it would be really useful for us as an open stack community to take into account fido with this kind of services and functionality and implement against that at the moment we limit ourselves to a subset of the data planes that are available and therefore we are having difficulty releasing features as they almost the standard is finished and as open source we are not yet supporting that um this is an example of what frank was referring to earlier where you have the different nodes in the graph and you could see how uh the vxlan gpe combined with nsh processing is being plugged into the graph this is the way a project in integrates and inject itself into um the rest of the nodes that we have on the graph and we do not have time to go into the detail you could find um some of the information on the release on that pointer thank you ry so i'll be brief today's container so this is an example of two containers talking to each other via tcp on the same machine so you have two paths so uh the first container that's an application open a socket have a send system call go to the stack go to um a veth so if this is obvious it go backs between two vt v two of yes another v um another device and then back to the application with uh vpp today so we have a pair of veth that connect the dots because uh the advantage of vpp is being user learn for fast development but here you have no kernel implementation and you will see that on paper at least on slide these look longer but what are the numbers i don't have them they have not been published and also this is not a very targeted use case for vpp but people are working on that within vpp so this one i'll be brief basically so what you try to achieve is a send and receive to go via a five four and you completely bypass the tp cp stack but this assume that you will have a lot of tcp connection between two containers and i would say this is not very nv uh centric because nv is about packet coming from a cell phone going to the internet so you don't expect container to create packet on themself but this just to say that with vpp you can research and try to solve very complex problem and go for something very complex like this this could last for a complete presentation but now so we have on the right today and tomorrow so what i want to show is vpp is a future proof regarding container developments what we have today as i showed before is legacy container being interconnected to the v switch so between them and to the outside world via uh the host kernel via per vth port and an f packet socket now you can run dpdk based container on top of vhost user and you will say but you show me before vhost user is a bottleneck but i told you that the bottleneck will be uh okay we're going to optimize it drastically in the coming months not weeks and then if you want to go further say okay vhost user i want something even more fast so if you have vpp in the container and vpp out you can take any shortcut that you want apply any policy that you want because that's you just need to have the proper node in the container the proper node in the switch and you can either buy even bypass the switch if you want so just to say that there will be a lot of innovation in this field in order to have fast communication but as there is no free lunch if you want some multi-tenancy you will have copy but in some cases you can optimize okay that's the same tenant maybe i can so just to say that vpp is a is a good fit for this kind of development i'd be very quick on this one so today vnf developers are quite puzzled about should i go to vm to container mix a lot of vms today are big large vm that take consume a lot of memory coming from a legacy world but uh and people start to slide them into container into vm and i have to say i have no answer but what i can say is vpp can handle vm containers that's it and then that's an orchestration issue so whatever is the future vpp is there for the vnf so what we've seen is that on the production side vpp is quite mature despite being a very very recent projects because it has a lot of counter everywhere trace documentation training i encourage you to go to the fido uh wiki page you have a lot of session which has been recording a lot of hands-on exercise you have a very active community just look at the mailing list very open uh they have open weekly code they have a great ci gating ci based on cc that you can put in your lab it's still a very new project but as soon as it would be mature enough so what's mature enough we'll discuss in the community uh we're going to have an lts version of vpp so good for production and uh the goal of fido community and opnv community is to bring opnv rdo integration uh integrated by okata so people can go on poc with okata open stack version and evaluate vpp for real end to end in open stack and give number publish them we'll be happy to get them in opnv and opnv will publish numbers on the innovation pass uh if you want to to to work on the next big thing thanks to the portability the modularity the ease to add a new protocol the sandboxing project okay that's a very good vehicle for innovation and you already have a lot of cool stuff there including containers nsh lists whatever and now put just for okata so let's think about okata what do we need for okata so dpdk properly integrate so we need we need centers dpdk people to work together we need open stack open delite triple low integration to be in rdo so you can install rdo properly with in opnv and get your end to end use cases in opnv that's my last slide thanks for your attention do you have any questions probably have a few minutes here for questions if anyone has any questions or you're all overwhelmed with the innovation the pace of the technology hi i have one small question my name is gal you said i saw that you said that vpp support arm but you also mentioned that it relies on intel uh vectoring instructions so how that works together it's very simple it's portable so you just recompile it and uh you have the cd but you have an abstraction layer so the vector is okay you can use the vector instruction to optimize some things but it's different from the vector packet the vector packet is the table of packet if you want to see so it's not it's not tightened to intel it's worked well on intel okay thank you any other questions just to add to that response actually for those who are not aware dpdk for quite some time while its origins are clearly with intel is another open community supports multiple architectures as well yeah yeah there was another question here okay do you want to take the question is what's the advantage of vpp against other v switches so i would say look at my slides i guess that's a great framework that's a great community that's that's really for nv so that's my answer second one the load balancer is entering you have different maglev implementation ongoing for load balancer within vpp the result project so just have a look at the mailing list and you will see some activity there so for uh for the first question if i may i'd like to add few more points uh maybe two key points one is uh the direct attachment and reliance and taking advantage of um the tight coupling to the underlying forwarding that the platform can offer to you so that happens because vpp is a technology that has been there for uh long time its origins are 2002 dpdk is also a technology that is there for multiple years they are both mature and they are designed for the ground up so they are designed to take advantage of all the innovations that the cpu architecture can deliver and the server architecture as a whole so what you will see is that we you could think of it as we have a revolution of software as an example software evolves from our client server virtualization containers you have a revolution of hardware we take the hardware evolution and we make that to be as useful as as possible as frank showed earlier well uh you still the bottleneck may move as as as we progress on the technology but you are starting from the foundation you are relying on the best the platform can do as we clear all the bottlenecks that is supposedly going to emerge as a as a performance leader efficiency leader but there is another angle and the other angle is really looking at the problem more in a network centric fashion we gave you one example of of sfc over here there is a the whole notion that as a community you could add features that comes to play in two ways it comes to play first of all with the openness of the community you could walk in and you could add a project we are working for multiple years to add sfc to other data planes still it is not there so you have richness of all the network protocols there are other attempts in this community to get the network to offer a broader set of protocols that as an example nfv would would require this platform is designed to do that so so you could benefit from it and these are just just few a few examples the other the other maybe little note i would like to add is because things are in in the user mode you are not also dependent on the pace the kernel itself is moving there is a given take over there obviously sometimes you don't need the additional performance sometimes you would like to use the kernel but if you want to move faster if you have innovation you could take advantage of that so you want to do the full comparison any other question okay thank you guys hopefully that was useful