 All right, welcome everyone My name is Carl Baldwin. I've been working on Neutron for about two years and I Play with basically everything L3 in Neutron and so I Notice this IPv6 work being done not by me, but by other very talented engineers in the community and I just couldn't ignore it anymore. So Started despite your best effort right despite my best efforts and So I started thinking about well, what what's What does IPv6 mean we can't we can take it seriously now and hope an open stack So I've been thinking about it over over kilo and and what we're gonna do for Liberty and so we got together and and started brainstorming and This is kind of what we've this talk is is about kind of what we've come up with Over the kilo release if I gave a talk here yesterday About the refactoring that that we've done in the L3 agent and some of that that I didn't talk about yesterday actually supports Supports IPv6 in Neutron and I'm gonna turn it over to Sean to talk about The refactoring that was done specifically for IPv6 So during the last development cycle I'd like to go through some of these slides and discuss some of the work that was done by others in the community If you're actually in the crowd and you did some of the work Why don't you go ahead and stand up and get recognized by some of your peers? So if Dane or Shreed are is anybody in the room? Okay So I'm briefly just gonna go through some of the things that they worked on that actually Accomplished all of the work during this release cycle so Dane for example did a change for the layer 3 agent In the internal data structures so that the routers would be able to support Multiple IP addresses on them the typical use case being an IPv4 address and then also an IPv6 Prefix or an IPv6 address It's a really it was a really difficult change To to get in because it was really deep in the guts of it and I it was really a great piece of work That really allowed Everything else to happen throughout the cycle So Shreed are did a lot of work on supporting IPv6 in highly available routers with the links to that and Then we also had I believe a new contributor Andrew Boyk who did a lot of fixes for neutron And some really important changes near the end of the cycle Actually going over into the nova code base and fixing some inconsistencies in the API and some of the behaviors that we're actually Causing issues at the gate and those were really critical fixes as well. So Thank you to everybody who's done this work Really the only piece that I was responsible for was at the end was enabling at the gate Which is used to test all of the patches that people propose to open stack is to enable all of the Infrastructure to use IPv6 For the networks that are built up and tested so that we can get closer to the ideal of testing Parity between IPv4 and IPv6 so what it happened is is I had proposed this patch and It actually triggered all of the bugs that the fine folks in the audience in these previous slides went and actually fixed It was a very difficult chicken and egg problem where we wanted to test at the gate But we had to fix existing bugs in the code in order to get this to actually work So it was it was really exciting work to work with everybody and I thank you again. I Think it's still you okay So what the big thing that's coming in this next release cycle is I believe the next step is to enable dual stack on the control plane Meaning that all of these services that currently Comprise the open stack cluster such as the identities the networking service everything else It should also have an IPv6 address that it's listening on so that we can just ensure that for the most part All of the components such as my sequel or whatever your database of choice is can also be ready for the future of perhaps Just an IPv6 only control plane so that you can free up some of your IPv4 addresses to be used for your tenants Or just to sort of stave off the IPv4 exhaustion just a little while longer Okay, and as I was so I I tried to review as many of the patches that that he linked to as I could fit in And I was reviewing them All great work now. We've got IPv6 working on networks, and we've got we've got it working with the routers I started thinking about well, what is What is routing out to the external world look like in with IPv6? and I started picturing it in my mind and I wanted to go over Well, why does why does routing work with IPv4 in an open stack? To the to the external world Well, we use floating IPs, and we use Nat two things that are that are not spoken a lot about in IPv6 If you if you haven't noticed the floating IP Nat kind of creates a barrier between the public and the private address spaces and We just we always use Nat right and You can you can pick your own addresses With IPv4 you kind of have to because it's it's all full and You've you've got to bring some private addresses use them on internal networks Completely isolated from everybody else and and then you've got to use Nat to get to get in and out So you bring your own address that's BYOA So what do we do with IPv6? Well, we don't have Nat We don't have floating IPs and we'll talk about those more later Our option right now is essentially to To put IPv6 on a flat provider network It works. It may fit it may fit your needs, but it doesn't allow you to use Neutrons richer L3 routing capabilities to create Networks and and do routing in between them So what do we do? How do we move to an IPv6 routed model? The the diagram looks essentially like the IPv4 one except now we've got we've got some external addresses and and We've got some internal addresses and if you take kilo right now and and you go create an IPv6 network You're asked for your addresses Well, where do you get the addresses from? So you might you might find some Some ULA addresses in there and you might find some some addresses that look like they're their public addresses They they look like, you know, 2601 14 that 413 64 that looks like it might be it might be routable to the to the outside world But the thing is is We let the user type that in how do we know where that came from? How do we know we can route it? Most likely we can't actually so How do we know if an address is routable? Well In this in this context, it's not really a function of the address. It's It's a function of the context. It's it's It's it's what you're actually able to route route to the external world and back in fact a ULA in a certain context may be routable and A global may not be so today in neutron the the neutron routers route everything If it doesn't know it belongs to an internal an internal network it routes it up to the external world So the trick is knowing what's going to come back How do we know if it routes back if if the return path comes back to us and And then how do we get it into that internal network? Well, we need to somehow set up the neutron router as the next hop for that for the router The the northbound router So how do we do that and I came up with a few thoughts And a couple of them were actually working on for Liberty prefix delegation Was pretty close for kilo But it didn't quite make it and it I Expect it'll be ready for for Liberty Dynamic routing is another option if we can get neutron to speak a routing protocol to the external world and Give it some hints. Hey, this is how you get back to me This this internal network. Oh, that's that's behind that router right there. That's your next hop There's a couple other these these are the ones of question marks proxy neighbor discovery Do we do we create a neutron router with static routes? Those are all possibilities But even if we do that Bring your own bring your own address still doesn't work Typically you you can't just route anything back that you want to you you're given a prefix of addresses that That come to your that come in to your world from the external world and so how do we solve that? Actually, this is just a little addition So to that point at least on the IPv6 side We have a way rather than the typical workflow on the IPv4 side with neutron is you talk to your network administrator who gives you a IPv4 sitter and then you actually with the neutron command line client will feed that address in with the Subnet create command and associate it with a network With the prefix delegation protocol in IPv6 instead what we can do is have neutron Fill out some of this information that we were requiring users of the API to provide to neutron itself so for example, we can have neutron requests from external systems prefixes that will then be routed into the neutron network and from a user side of things if you Aren't a network person to begin with the networking API can have a lot of complexity to it when a lot of people may just be like I want to have a connection to the internet and people can connect into me and Prefix delegation work. It would be one of those mechanisms that would realize that right, so I Think I already covered dynamic routing dynamic routing is is is not a mechanism for obviously for for doling out addresses, but it's a mechanism for Getting the next hop back to the back to the open-stack neutron router This is where I want to be sure Okay, so so how do we handle the be the bring your own address problem? We talked we talked about prefix delegation That's one possible solution to that. We actually prefix delegation gives the problem to an external system So when when you create your subnet instead of typing in your subnet details what you do is you give a keyword that says just just get it from prefix delegation and After some time that will be that will be filled in for you by the external system And you'll be able to see what it is with a with a neutron subnet show Another another possible solution to this is to use a new feature that that landed in in kilo called subnet allocation Subnet allocation is a way in neutron to define an address pool And allow neutron to manage that for you. I Saw Kyle and Mark do a presentation yesterday where they had a nice graphic for this. I wish I had it It shows you you create a pool. Let's say for example, you get a slash 56 That's yours and you want you want to give that out to to your tenants. Well as they As they create subnets instead of giving the subnet details like we're used to but subnet create we we instead passed the the uuid of subnet pool and optionally we Pass the size of the subnet we want with IPv6. It's it's pretty much always a slash 64 that you want There's really no reason to want anything bigger or smaller than that So that can be left off. So subnet create with this subnet pool It will it will assign you your prefects for your subnet and I wanted to mention that Ryan Tidwell and Zankfa Gao did a lot of work on implementing this and got it landed in kilo And they really deserve a lot of credit for getting that in there So that this is what it looks like We currently only have neutron command line support for it But we've got a neutron subnet create and I highlighted the important parts in red You'll notice. There's no subnet details on the command line. There's just a dash dash subnet pool I use demo subnet pool as the name prefix length and it and it creates it with With something allocated from From the pool and also the subnet pool Just going back to the slide the subnet pool ID is recorded with the subnet and that becomes important a bit later This shows that this was me playing around with dev stack. I actually I didn't want to do a live demo Because I thought that would take too time too much time and I I was sure it would crash and burn, but I was playing with dev stack and I You know, I created this subnet pool and I I allocated stuff to it and And I found that this was what I had to do to get my VMs on two different networks to talk to each other and this is this is basically just a IP route add to BRX and This this shows that there really is a hole right here and the hole is getting the route Getting things routed back into an internal network. And once I did that once I did this I was able to do this. I SSH to I've got VM one height highlighted here VM ones on on one network and from VM VM one I SSH to VM to which is on a completely different network and Just by adding those those routes with the next hop to the neutron router I was able to complete that that circuit so The subnet pool is recorded with with the subnet and this This is getting in this is Addressing the problem where right now we just route everything everywhere How do we know the difference between a subnet that That someone typed in at the command line and one that was obtained from from a pool that was that we can actually route and The answer is something that I'm looking to to do in Liberty called address scopes We take these these subnet pools And we group them into address scopes and then what we do with address scopes is We can tell the neutron routers to honor them to not route between address scopes and In the future we can also We can also use this to integrate with something like a BGP or an L3 VPN They use concepts like this to Isolate L3 traffic between Between data centers and between clouds and so the end goal for all of this is is to allow the user To just say I really don't care what addresses I have Just just give me an internet subnet and round it for me and that that's the end goal of this So here is the part where I give out a little bit of bad news Floating IPs is not currently on the roadmap for functionality between the IPv4 side of neutron and the IPv6 side of neutron and I wanted to sort of give everybody a couple Reasons why I believe this is and a little bit of history to it so that you all don't hunt me down and burn me at the stake afterwards So when I was at the QA summit, I had a really great conversation with Shondig and Dean Troyer And we sort of just got into a discussion about networking and Dean told us about how They had done Nova Network when they were at NASA So floating IPs is sort of a overloaded term that has two different concepts put together It's part of it is the elastic IP model from Amazon web services the idea that you can quickly Re-IP instances without any sort of interruption from the client side Usually when you create an instance you get a routable IPv4 address and then you can use the elastic IP functionality to apply that elastic IP overtop So what happened is that the original Nova install? It had a slash 26 for the entire cluster So IPv4 addresses that are external the external IP for the IPv4 addresses were an extremely scarce resource Especially for this computing environment where they were running simulations and stuff like that They couldn't just give out That all those addresses to every compute instance that was going to come up oops so we go into the part of Having the external inbound access and then the fault tolerance concepts They have now been currently mixed on the v4 side because there's a lot of use cases where if you have a node go down You would want to use an elastic I'm sorry a floating IP to reassign it to a node that is live And that would be one of the use cases that people are tempted to do on the v6 side especially with Slack types of subnet allocations Because the address is going to be based on the MAC address and unless you're preserving your ports in between Destructions of your instances Floating IPs would be your your quick thing that you would want to reach for to preserve that same sort of Parity between the v4 side and the v6 side But IPv6 addresses are not a scarce resource And we need to split out the two separate pieces of what a floating IP is The external IP address and external connectivity we solve by just giving you a globally routable IP address and Then it's in my mind if we want to talk about fault tolerance and other types of conveniences There are other techniques that we can use besides just creating a NAT where you know the Traffic for that floating IP address is intercepted and then rewritten and then given to the instance Perhaps through load balancing any cast you know, there's plenty of work that's been done in Trying to solve this problem And at this point, I believe it could be an advanced service It doesn't need to live within the neutron code base itself Just due to some of the complexity of the floating IP code And some of the bugs that have been exposed and then the fixes to them It's been very difficult Now this is just my opinion. I Think that if you come to me with a valid use case as to why you would need this It could I could change my mind on it It's really the IPv6 Subteam was driven by people in the community who said I need the following functionality Here's why and which forged a path forward. It wasn't it wasn't top-down in any sort of way So there's always the case that there really is some sort of use case that this is the only way they could have that happen but I think for the most part just the two concepts of Fault tolerance and then external connectivity We are at a place where we can separate those concepts out And it doesn't need to be done the way it was done on the v4 side, and I think at that point we're at the end of our slides, so Okay So we do have microphones up in the aisle. So if you do have a question, please line up at least one Yeah Also, this is a presentation thing. So be a little gentle if you have really intricate technical questions I usually try and defer them till afterward Prakash here from future why simple question Especially because IPv6 we have a suppose we have a slash 64 and I have a slash 56 and slash 48 let's say okay, and Since the outgoing routing is always slash 64 If you mentioned like if you get through DAVD that is Either it is allocated or obtained from external sources The IP now if the best practices says that allocation should be done by location or by usage within that 16 bits of subnets Will it impact anything in routing in L3 if they are at distributed locations? So Suppose I have Building one building two. I'm trying to have some 48 based on locations and some based on usage So how will it impact? Or will it impact or no impact for routing especially at L3? so I Think initially If if if your neutron deployment spans building one building two Then there will there will be really no distinction between the two If if you have a deployment in building one and a different deployment in building two Then Then you would split that off Split that the way that you want them allocated by location Did I understand the question so is that does that mean that still the preference will be based on location as a best practice? We should first In the specifics After the 48 when I come to another 16 the first eight I should allocate to location so that It's location of it. I think this is this is sort of a question that may be best taken offline. Okay. Thank you There could be some some thing you could create multiple pools You with the different prefixes that you have and maybe a lot further in the future, they'll be more Integrated support for this but Yeah, hi, this is Jonas on and with Nokia Excuse me. I actually wanted to comment on that you left out the floating eyepiece and I think that you guys did exactly the right thing I'm like looking at what you Looking at what you want and not looking at how you did it in before is the right thing to do here And basically you have other options in v6 that you don't have in v4 You have things like prefix delegation, which you didn't have before use those The other thing is also that you can renumber networks More easily there is help in v6 for that use those techniques to deal with those things that that When you really have to renumber so I think you guys did the right thing Just a kind of like I'm not sure I understood your complete problem with the Dynamic routing and stuff like that, but there's work going on in the itf called home net Which is not necessarily? Directly relevant to this but could have some hints on what we could do here is how to do kind of like Non-managed networks that are routable that do have multiple subnets and so on that use Automatic tools to do that and we might want to take a look at some of that work if it's applicable at all or not To open stack as well That's an interesting that's a very interesting suggestion. I definitely will follow up on that and to read in more Thank you. I'm happy to help. Thank you. That would be great. Thank you So the prefix is that you want to delegate have to come from somewhere. Yes, you didn't really talk about that Instead of anything like static configure you thinking apis to interface with an ipam or so I'm going to try and summarize work that was done by other people who are far more intelligent than me on it But yes, there is still the problem of Whatever so for example for prefix delegation. There is still somewhere Somebody is making a decision about what pool it is going to be delegated from So the easiest one to say is is that my residential internet service provider has a big prefix that they Delegate out to their actual customers in their houses. So yes, I don't There is still that problem. I guess Right and with with subnet pools We've moved that problem from from the individual tenant where they they have to get that out of band and type it in We've moved that to a subnet pool Which is still in the reference implementation is it's still manually entered, but it's it's entered by an admin And it's entered at at a larger level. There's a larger subnet from which you can carve So there so there's some enterprise best practices as much as you can call anything a best practice on something that We haven't really done very well yet And I would encourage you to look at some of the best practices for Non-cloud data centers and think how that would fit right I think we've tried to punt in some cases with the external IPAM work that you've had is that perhaps People who are deploying neutron they've already decided for their organization how they're going to do IP allocation and Neutron can just punt to that system. That's already been set up API's we know how good that is give us the API's back and forth. So right well on the prefix delegation side We're just using the PD protocol itself and the Dibbler client So in some cases is not really an API. It's the pro the lower level protocol itself, right? In the case of prefix delegation, we we push that completely to the external system There's also work being done for Liberty Called pluggable IPAM and we'll I'll be talking about that with With two two guys from info blocks tomorrow and with pluggable IPAM we can delegate IPAM to an external system As long as we have a driver for that external system, right API James cave Fortinet One of the things about IPv6 is there the address is you should have on your screen in a fairly unwieldy Especially when you're starting doing delegation just thought and and this kind of dovetails with the IPAM discussion as well Have you thought about maybe having a PTR record insertion at the same time for some of those zones? To automatically give you some some some ease of use for those particular users So I mean you create your delegate top-end delegation It creates a zone automatically because you'll give it a name and then all of a sudden you'll have sub delegation the The actual delegate networks actually automatically create, you know There are individual zones as well and so forth and so on so actually yes I I actually wrote a blueprint also on on integrating with external DNS systems and tying a zone with With a neutron network So that both both the a For a and and PTR records are managed automatically, right? So so for example Integrating with designate or another system so that we have that those things are also done for you as well Right. We're using one of that the designate sessions on Thursday I think it's at 130, but I can't promise that we're going to talk about that Between Nova neutron and and designate we're going to talk about doing that That's one of the problems with the adoption of the protocol itself is just people look at those addresses and they're like nope Right. I even simplified it. I tried to simplify them as much as possible for the slides Thank you very much. Yeah, thanks Can you talk a little bit about where the status is with accessing the meta metadata service via IPv6 There was a great Twitter conversation that I had with I'm totally blanking on the name, but I've Maintained the position that the metadata API comes from Amazon and We can't really make changes to the Amazon API because When they do IPv6 in their cloud, I'm sure they're probably going to address that issue of how does a instance access a metadata API on the v6 side and I think a problem for us would be as if we try and anticipate what they're going to do Do it and then they end up doing something different, but it should they should do a well-defined address so there there was part of the Twitter conversation where The person who said that indicated that they don't like that idea And I think they do have a point where it should be a well-established IPv4 address But yes, that would be that would be one way to solve that right Right, but I think there is still people that want that agree with that But then there's also people who disagree and I'm trying to see if maybe Amazon will make the decision for me so that it's Right. Well my cop out for that is as well Just use config drive because that's really the way that we should be doing it I think but yes, it's a difficult problem Yes Hey, Sean. Hey good to see you and thank you and also the IPv6 sub team for the great contribution I just want to clarify a couple of things since you mentioned both the previous delegation also the subnet pool, right? For the previous delegation, I just want to understand what they're really a Challenge here. Is it because we have to communicate with both? Neutron system and also the upstream router to make previous delegation work. Is that the main challenge for guys to kind of Slow down a little bit or or anything else? I think second my second question. I think you may already answer that. I assume there's no At least at this moment. There's no API for the user to provision any previous delegation through the Why is the new transystem? Is that right? so those those patches are still in flight, but My thinking on it is is that prefix delegation should just be a type of mechanism that is done on the Neutron back end to actually accomplish all this to make the API as general as possible To maybe have something where it's like just give me an internet address where you don't have to Have more neutron commands to learn just to do on the side But I see there's a lot of big pieces to this and I'm also not the one writing it So I don't want to Speak to something that I don't quite know as well But it's it's definitely something that we're gonna have to work on sure In summary, I believe the general trend is you want to keep the API at least that also together with the CLI syntax as Statics as possible and in the back end everything is actually Right the way the way we looked at doing that for pre-fixed delegation is to handle it like a subnet pool I see so the only API change is is already been done. It's it's dash dash subnet pool and then there's a He you'll put in either PD or prefix delegation He's he said the patch is still in flight the decision hasn't been made, but there will be a special string for I want a prefix delegated address sure and And coming back to the subnet pool since you you give us the example on the screen Is that something already available or Another thing I want to understand is what the main difference between this subnet pool and the previous delegation approach And in addition for the subnet pool Can submit pool really work in the provider network model or have to tie with the Larry III agent? That's some good questions Okay, the first question I heard was what's the main difference between prefix delegation and subnet pools The way I see it prefix delegation pushes everything to an external system, which may be exactly what you want, okay? subnet pools give neutron control over over a certain piece of the address space a part of it that we call a pool and Neutron will allocate with its internal ipan implementation Will allocate those addresses from from subnet so I think there's only a minute left if we want to I know There's at least one gentleman who has a question sure maybe we can give him a chance. Thank you So this Don't wait from China mobile So I I would like to comment previous comments about the differences between home networking and Cloud I think there's a that's a very big difference is they don't have no the balance So that's a bit different. We have to solve this problem for IPv6. Thank you. Thank you. I agree. I think that's it All right. Thank you very much. Thank you everyone