 Good afternoon folks. My name is Dilip. I work for Cisco Systems. In today's session, we are going to cover how is Neutron extension framework is designed and we will go into the details of how to add additional extensions using this framework. And also we will cover few use cases in terms of like some of the customer requirements, how we met using these extensions. With that introduction, I will hand over to Abhishek. Hi. Hi, I'm Abhishek Routh. I'm from Cisco Systems as well. Dilip has given you the agenda now. Let's just deep dive into it. I believe most of you guys who are here probably know about Neutron. I'm just going to touch upon what Neutron is. It's a networking service for OpenStack, which provides uniform northbound APIs which can be consumed by different clients like Horizon, the dashboard, or you can write your own CLIs, or you can have your own orchestration tools, scripts to manipulate these resources that are exposed by Neutron. Neutron is a pluggable architecture. You can have plugins. Different vendors can have their own plugins, which can enable the physical as well as virtual switches and networking devices. So with this, I'd like to go into the core resources that Neutron provides. The three basic core resources, the network, the subnet, and the port. A network is nothing but a virtual layer-to-domain. It can be realized in various technologies like VLAN, VXLAN, or it could be a flat network. Each of these networks have a pool of IP addresses which are called the subnets. They are a contiguous block of IP addresses where each of those IP addresses can be assigned to ports. These ports are nothing but the switch ports that we have, logical switch ports. They can be tied to OVM. It can be tied to a DHCP service or L3 agents. They carry the MAC address and the IP address. Having said that, when we started Neutron, these were the three resources that was enough to get the basic stuff going on. We wanted to make sure that Neutron is extensible. We can add more features to Neutron. That's why we made the architecture such that it's extensible, so that you can add new services, you can solve complex problems, and also various vendors. They have their own capabilities. These vendors can expose their capabilities and integrate into Neutron via extensions. That doesn't break your original API. It's not necessary that every plugin has to implement all the extensions that are available. That's why extensions are very useful. It helps in evolving the Neutron API. We started off with three resources, but slowly, gradually, we started adding more services, more features. Today, if you look at Neutron, it has over 25 core common extensions, and there are many other extensions which are vendor-specific. That also leads to the next phase of Neutron, which I think probably we will call it V3 or so. That's the versioning of Neutron, with the extension framework that we provided. We provided a breeding ground for these new common core extensions to evolve, and once they have become stabilized, we will absorb them into the core Neutron API. There's going to be a design talk around that versioning. For some of the examples like the common extensions are security groups, that's very vital for deployment. Extension is a way to make that happen. As I mentioned, based on what the intent of that extension is, it could be broadly classified into a vendor-specific extension or a common extension. A common extension is something that is common throughout for all the ... It will solve a common use case, while a vendor-specific extension is something that will help you enable your own vendor-specific capabilities into Neutron. Then there are two types of extensions, resource extension and an attribute extension. Neutron has three core resources, but you might want to do something which cannot be achieved with these three resources. You could introduce a brand new resource, and what are the different attributes that this brand new resource would define. If you want to do that, you will be doing the resource extension. If you want to use the existing core resource like the network sub-network code, and you think that the attribute ... Because it has limited set of attributes, but perhaps there's something that you would want to add into that core resource. You will use the attribute extension and enhance the core resource. Now we'll get a little technical as to how to write these extensions. These are the basic steps that you would do in order to expose your extensions while in Neutron. You will need to add a resource module. What this will basically do is that it will tell Neutron or the extension framework what the new resource looks like. You will need a database schema because now you want to store these attributes. You want to make them persistent. You will need migration scripts because Neutron releases every six months. You need to make sure that your tables are pre-loaded once you install Neutron. Once you have these database ready, you will need to access these resources. You will add a DB API layer and then you will enhance the existing Python Neutron client CLIs and your orchestration tools to consume these REST APIs which are exposed by Neutron. Based on the two different types of extensions, they could reside in either the core location or if it's a vendor-specific location, it will reside in the vendor repo. I'll talk a little bit more about the vendor repo later and talk about the core and vendor decomposition. Let's dive deep into each of those steps. What do you mean by resource module and why do we do this? The resource module is basically, as I said, it tells Neutron and the extension framework what the resource looks like. When the plugin is initialized and Neutron services started, it will look at all the extensions that are existing in the extensions path and it will try to load them. That is why the name of the extension and the class name should be the same as a requirement. Also, you need to implement some of the basic methods, the contract binding between the extension mechanism which will describe your extension in detail. For example, you will have a method like getResources or getDescription which will describe what this resource is all about. Once you are done with that, the extension manager takes care of loading it in Neutron and it will provide you with your URI, a REST API so that you can access these resources. Also, once you define these resources, as I said mentioned before, not every plugin implements every other extension so you can decide what your plugin needs to extend. That can be done by just adding the name of the extension in the supported extension aliases. Another thing as a side note is that if you have any helper modules which reside in the extensions directory and you don't want them to be loaded as extensions, you will start them with an underscore. That way Neutron knows that these are just helper modules. This is how a resource attribute map looks like and that's the fine difference between the resource extension and attribute extension. In this example, let's say you are extending a resource with a resource name. This guy is adding a new resource with ID and name while on the other hand if you want to extend an attribute you will take the existing core object and you're going to add the name of the extension or name of the attribute and then you can define the type, whether it's a string or a boolean. That's the example attribute map. The next thing was once you have the resource module ready, you need to persist them. So you will create tables and model classes which would store these attributes and then as I mentioned the database migration scripts needs to be written so that for upgrade or whenever you install Neutron all the tables are preloaded and this is where the migration scripts will reside. Also you'll need to then write database API to access these resources which are stored and to perform core operations. So let's say you have written an extension called security groups. So in that case if you want to retrieve a list of all the security groups from the database you will implement the get security groups method. I mean if you write a method called list security groups it won't work because the extension framework expects a certain naming convention to be followed. So the get extension name will retrieve a particular extension if you pass in the UUID or it will retrieve all a list of the resources. The create extension name would create, the update would update and the delete would be responsible to delete that extension. So these are the minimum set of DB APIs that you need to at least write. And the last step was to once everything is ready so Neutron will automatically load all these extensions and it will form a URI for your new extension and you can then access this URI via REST APIs. You could write your own scripts or you can enhance Horizon or you could add CLI's into Python Neutron Client. So if you look at the workflow the user will perform an API request for your extension. It would pretty much do the keystone authentication there and then in a way whether the extension is loaded or not if it's not loaded it's going to return a not found error code. If it is present then it will either let the extension process it or if it's an attribute extension it will let the core to process it and then a response will be generated. So as I previously mentioned that there was a vendor core decomposition that happened in Neutron. So as part of Kilo release we removed all the implementation and the code of vendor specific code out of Neutron tree and moved it into stack-forge repositories so that they're maintained by the vendors themselves. So how does this affect the extension framework? The extension framework is largely unaffected because it mostly, most of the code resides in the Neutron code today. The only thing that changes is that today if you want to write your own vendor specific extensions they will reside in your vendor repo. So you maintain it, you write unit tests for it and you will be responsible for that code. So having said that, lay the groundwork for what are the different use cases. We at Cisco encountered a few use cases and problems and how we solve that I'll invite Dilip to talk about that. Thank you. Thanks, Abhishek. So we went over the extension framework, the details. Now what I will do is I'll go into some of the cases, the examples where we need to address a problem and how we leverage this extension framework to address those issues in OpenStack environment. Just a quick recap, at a high level there are two types of extensions. One is a vendor specific extension. It's predominantly plugin specific and the implementation is in the plugin, most of the code work. The other one is it is the additional functionality that you would add to the existing Neutron capabilities. Some of the examples like we already went over that, security groups or DVR functionality or provider network kind of thing. But that is mostly a plugin independent. It's mostly like done in the common code and each individual plugin has to enable it if they want to realize it. But this example cases that we are going to cover today is predominantly going to be that vendor specific feature part. Though the implementation is somewhat very similar, the difference is where that code resides and how we realize that functionality. Except that most of them will be identical. As Abhishek mentioned, there is a discussion around whether this extension framework is the right way to handle the additional functionality. The reason being like the common code, there are three resources, 25 plus additional resources that's been added. The expectation was those features will be added initially as an experimentation. Once it becomes more mature and they expect it to become part of the code, that is right now it is not happening as fast as community expected. So there is a discussion about how to make it more standardized. But having said that the extension framework for vendor specific features will be largely unaffected and it will be the same model that we will be following. Having said that, let me explain like why are we looking at this functionality, right? The extensions. Before I go into that, let me explain like the current switch that we have and what this switch does. This is a Nexus 1000 v-switch from Cisco. This is a multi-hypervisor distributed virtual switch supported prior to KVM, ESX, Hyper-V platforms. And also it integrates with various orchestration tools. The key thing is this has a rich set of functionality because it leverages the same Nexus OS code base that is available across data center platforms from Cisco. Since all these features are available, the expectation is we provide the consistency across all these hypervisor environments. Some of the customers has dual environments. They will be using OpenStack and along with that they might have already has their existing virtual environments. And they have a specific operational model and the ask was how do I achieve the same operational model in the OpenStack environment so that from the network perspective it is transparent whether you are using OpenStack or using something else like virtual center or your own custom orchestration portal. That was one key requirement. The other key requirement is in a traditional OpenStack model largely when you deploy the VM, that is the time where you define the policies associate the policies. What it means is users are the ones who are controlling the policies. If you look at a typical data center environment that may not be the case all the time. Sometimes admins do want to have the control where they can define the policies at the same time users can deploy the VMs at will and leverage these policies or attach these policies. How do we satisfy these requirements? That was the major issue that we looked at when we started integrating with the OpenStack. Now let me go over a little bit into the details of what does admin define policies means. Typically like as I mentioned earlier the users could be defining the policies but there are certain policies for example compliance. It is best left to the administrators because admins know what exactly compliance means. It may be that if you have a HR web server you may want to monitor all the traffic from that. If you have those kind of requirements also how do I prevent DOS protection? How do I protect from the spoofing attacks? All those functionalities is best like the admins are the ones who knows what to do but at the same time every time when a user launches a VM you don't want to be coming on the way you need a way to define these policies and users can consume it. So this is exactly where the concept of port profiles came in. If you are using Nexus switches it's already there in the Nexus switches. So the idea here is you group the policies like in this example if you are using a web server profile you define a profile for that that defines all the policies needed for the web server and the same way for application servers or you can define these policies like you can even define by group whatever way you want HR web servers this is the policy. Now once these policies are defined as a user when I'm launching the VM I know that what this VM is this VM is meant for HR web service so I will associate the appropriate profile for that and once you associate that without worrying about what are all the policies underneath it those policies will be implemented by the network infrastructure. This is at a high level what admin defined policies is in a nutshell it is nothing but the port profile slash network templates. How do we get such a functionality into the infrastructure right? OpenStack doesn't have such a concept yet so what we did was we implemented an extension to the port called policy profile extension. There is a I mean we could have called it port profile extension but we decided to use a different name to avoid the confusion. The port profile itself is nothing but the entire capsule of the policies whereas the policy profile is the name of that port profile the model that we used is just like administrators would come and define the policies using whatever orchestration tools or preferred tools they use they can go ahead and define the policies. Once the policies are defined using this policy profile extension we were able to expose that port profiles into the open stack environment. Now the open stack environment has the list of all the port profiles that are available as policy profiles. In this example like I mentioned earlier we have four different port profiles exposed as policy profiles. Now when the user wants to launch the VM as part of the port create now he can specify what profile he needs to apply. He doesn't need to worry about what are all the policies underneath it. That is left to the administrator. That separation when this was one of the key differentiation like a lot of customers who are looking at uniformity across multiple environments open stack non open stack environments they were looking at this functionality and using the extension we were able to achieve that. This is a now let's go into little bit details into this extension. How did we realize this extension? Like Abhishek mentioned there are two types of extensions. One is the resource extension the other one is the attribute extension. The policy profile concept itself is new to the open stack. Since it is new to the open stack we need to define it as a resource. So we define first this was a resource and as earlier we went over that like what are all the steps we need to have DB schemas we need to have extension policies we need to have these are the places where it is defined once that resource is defined how do we use this resource that resource is available somewhere you have to use that resource that is where we did the attribute extension as part of the port create you would be attaching this policy profile so we did the extension attribute extension to the port create and there we specified the policy profile this is where that key construct comes in as part of the attribute maps the policy profile has two objects here one is a name and the other one is the ID the ID is the unique identifier of that resource you define the resource using this object now the second one is as part of the neutron port extension there in the attributes it's shown as we defined that there is a new extension called N1KV profile with that name you are able to now see that extension there will be REST APIs everything automatically generated even the neutron CLI we extended that where you can specify that that is what we showed earlier here let me go back and show that at the bottom of the list you can see that that hyphen hyphen N1KV colon profile with that additional changes that we made we were able to expose this this is the one extension we did and once we did this extension in fact like we were able to use this for many other purposes because we made it so generic now additional capabilities that we are looking at or the features that is been available and customers want to use it we were able to use this port profile extension I will go into one of such example this is a one classic example that people look at it today when you have provider networks and the VM shows up in the provider networks they can communicate with each other because they are all on the same segment you can use certain capabilities like security groups but they are very error prone or you have to build some orchestration systems on top of it especially if you are looking at non IP traffic it is very difficult so how do you achieve isolation one simple answer is well we can create multiple segments yes you can create multiple segments now you cannot use the same subnet well you can use the NAT that is another option yes we can use the NAT but that becomes too complex now you are adding more complexity one simple answer is things like private VLANs can be used these are certain things that people are already deployed and using it in other environments one example that came up is how do we use private VLANs in the open stack environment open stack by itself doesn't have private VLANs feature but the underline virtual switch has that capability how do we leverage that so we did a very simple thing as part of the port profile you have that command saying like private VLAN isolated once you have that capability now we know that port is say isolated port we can only communicate to the external world or any other resources like if you have your shared VMs meant for common services that could be accessed so we can use that there is also a slightly different variation of this isolation where group of VMs they can communicate with each other but not with other VMs we were able to use the same port profile extension capability or policy profile to achieve that the key next summary I want to highlight is the extensions that open stack provided it helped a lot to realize the functionality that the customers are looking at using this capabilities this one is a port extension now let's look at the other extension that we made this is on the network so one of the issues with today's open stack model at least which has certain constraints for certain deployments is it treats all the networks are identical take a VLAN network but the reality is what if I want to enable multicast on that VLAN network sure you can go ahead and enable IGMP everywhere but because your infrastructure may not be supporting that scale you may have VLANs but the infrastructure the physical hardware that is available may not be able to support IGMP and all 1000 VLANs now what do I do how do I enable IGMP for select set of VLANs that is a capability that it is not possible today with existing open stack there are ways like you can do a provider networks but the issue is for every VLAN you have to manually go and enable provider networks you want to have the same dynamism where dynamically users on demand basis can allocate and deallocate their resources if you have to achieve that the existing networking has a constraint and this became a real challenge in certain deployments the same thing goes for VXLAN I think in the other session previous session we were talking about VXLAN multicast mode I wouldn't go into the details there are different flavors VXLAN itself is evolving multicast is the original way of implementing the VXLAN when you have a multi destination traffic like broadcast you use a multicast group to send it again you have flavors like you can allocate a multicast group one per VXLAN segment or you can use a common shared one or there is a new mode which is completely multicast independent called unicast mode the underlying storage has the capability of supporting all of these models simultaneously they can coexist that's a key because it really depends on the workload requirements right if you have a production workload that has a lot of multi destination traffic or broadcast traffic you may want to use the multicast mode but you cannot enable multicast on all the VXLAN segments because the groups are so limited how do I what am I going to do I should be able to selectively use the segments with multicast for certain workloads and for rest of the workloads use the other methods how do I provide such a capability one simple answer that could be is take VXLAN type driver implement another type driver say VXLAN multicast shared mode type driver that could be one way to go it but if you take that approach that means you are if you look at the at the end it is multicast VXLAN it is not something different than VXLAN it is a flavor of the VXLAN for every flavor if you start defining a type that defeats the purpose of having a shared and product type drivers so how do we achieve this right so this is where again the extension mechanism became handy what we did was we did an extension to the network object and with that extension we were able to achieve the things like now you can define the profiles the similar way that we talked about policy profiles now you can define the profiles for the network which is the network properties that is the property of the entire network you can specify now okay here is a gold profile with multicast mode silver profile with multicast shared mode or you can have a branch profile with once you have that one and as part of the network create now you can go ahead and specify what kind of network you want so the user may be able to say that okay I know my workload requires a silver level treatment so I am going to use a silver level profile or based on the tenants the administrator itself can specify that if the tenant is a tier one tenant he only have access to the gold profiles all those options now is possible we were able to achieve that using this extension mechanism that's the two extensions that we want to cover today one is for the port to extend the port properties how did we do that and we were able to get all the features with just one extension we didn't try to extend it by having one separate extension for feature that is one thing that we achieved the other one is the network profile extension to do the network attach the properties to the network so in summary the key takeaway if you look at the extension mechanism that Neutron provides it's pretty flexible it's not that difficult to add extensions but it doesn't mean that we want to add extensions all the time you have to be at most careful because when you add extensions that means you will have different implementations on different platforms that is one thing you want to avoid from when you when you look at the open stack there you want to if you want to look at the main objective one of the things is have as much commonality as possible so that is something that we want to one needs to consider before adding an extension this is another factor what we did was to keep that gold we always keep these features as optional by default they are disabled that means the common functionality is only what is available everywhere only on need basis if the customer really is looking for such a functionality then they can enable it that's how you can have a balancing between what extensions can do without having too much divergence in this ecosystem that's what we have today if you have any questions feel free to ask us I'll take that up so a plugin is something that would realize the API call so let's say for in our case the Nexus 1000 is a virtual switch and we want our virtual switch to be integrated with OpenStack Neutron so all the Neutron APIs the request that we make we want to make sure that the same configuration is passed down to the Nexus 1000 V switch so the plugin will do that the plugin will take these calls and pass it down to your virtual switch or your physical switch the extensions on the other hand is just a something that would extend the current functionality of Neutron and the plugin will realize that as well so let me explain in summary what you have the functionality how to implement it because your infrastructure may have different ways of realizing it that's what plugin does whereas extensions is extending the functionality itself whether that functionality is additional attributes or additional resources once you extend the functionality the plugin should be capable of supporting it so you mentioned that after you've implemented your extension here then you need to update the Neutron CLI and add support into Horizon and everything so is there any automation of that at least for the CLI if you add an extension does it automatically show up in the Neutron CLI or do you have to go in and modify the CLI or you'll have to go and modify the CLI code because it seems like with the data model that you're expressing there that there should be some automation of that kind of thing it seems like if everybody is having to hack on the Neutron CLI that's a bit of a source of a problem so the vendor specific CLIs generally are the vendor will maintain them outside the tree it won't be a part of the core CLIs that come with OpenStrike Neutron so you will be modifying those CLIs as long as they're vendor specific but if you're bringing a common core extension in that case the CLI will be as part of the core CLI so if you add an attribute to the core a core attribute then that will automatically show up for attribute extensions you don't have to worry about the CLIs because the CLI what it will do is it will just pack these attributes and send that request over to the Neutron server so the Neutron will know based on what extensions are loaded it will look at the attribute map and see whether it's a valid attribute and it will be valid against that quick question maybe you covered it so excuse me if you did already but I think in the beginning you mentioned the vendor specific and the general ones what are these two use cases that you covered today are they just Cisco or this is vendor specific thank you alright thank you