 Okay, good afternoon. My name is gamma night burger. I'm with wreck space and we are here to talk about load balancing That's basically the our introduction glass the 101 tomorrow. We have more advanced things going on Also in case you are here for the lunch. That's not a lunch here. It's It's open. It's load balancing. Here's my dear colleague. Oh, yeah, I'm Adam Harwell. I'm with go daddy I'm gonna be helping German out with this talk today and I just have to remember to face the mic And as on the slides you find the Michael Johnson. That's our Project technical lead. He couldn't be here today. He would love to be here and he wanted us to video him in but we said no He'll do it all by ourselves Okay, so then let's get started. Whatever you're going to talk today I want to talk about what is load balancing at all. What are some of the use cases for load balancing? I want to talk about how we do that in open stack And we talk about how you can do it in the back end what providers we have Talk a little about how you can set them up with a api with a cli We have a graphical use interface and then a bunch of other things you might not be familiar with like session persistence tls termination and l seven or layer seven l seven load balancing And so let's start off for people not familiar with load balancing. What is it all about? So let's imagine we have a user Who wants to get to open stack.org his sensors request It gets sent by the load balancer to a server server returns it And then he gets his website Now with the second user then this request and it goes to another server And means send back To the user so they both use the same dns address which went probably to the same ip address But then was serviced by two different servers And that's what a load balancer does. It basically balances the load of incoming web traffic between However many servers you want to put there. So there are four here, but in production environments people have hundreds so so, yeah this Is that so basically and then what are some use cases for the load balancers As I said the first one is you want to distribute a network load between many servers So a lot of requests coming in and then it's being distributed between many servers There's a one increased availability So a load balancer usually checks the what we call the back end servers and if one of them is not Responsive then it will take them out of rotation out of rotation means that subsequent requests by users Don't get sent to this server until it's been repaired and be Available again And so users don't see downtime because if you just do it that way and you always hit this broken down server users Keep getting their 500s or whatever So it also increases the ability availability because now you can have servers coming down and everything still works We will talk later about ssl offload and centralized certificate management So there's a little bit about that Another use case a lot of people have is protocol protocol conversion. Some people might want to Advertise the services as ipv6 But still under data center ipv4 so load balancer can take ipv6 traffic And convert it to ipv4 or the other way around they are really modern data center everything ipv6 But I still want to have an ipv4 address where people can come to them And then the load balancer can then translate ipv4 requests into ipv6. So it's very common because Then l7 policies you have a extra slide on that where we go more into details Just an example right now is so static image files Might be on a on different servers Then you have your application server those servers might have caches and things But you still want them on the same address so still going to say open stat.org slash images You don't want to say images dot open stat or when I do it that way and then And then the load balancer can redirect that to specific image servers and the application Things to specific application servers Another thing load balancers can do for you. They can abstract the physical network topology So basically all what people All what your users see is the address of the load balancer They don't really see how it looks behind it And that basically allows you to do upgrades or you can keep adding new Um application servers you service in the back end without anybody noticing can take old ones out And you can move things around as you see fit without Changing anything and keeping all your services up Yeah, there's actually a neat trick you can do with this I don't know if you have tried this But if you're introducing a new version of your software for example, you can actually just throw One node into your pool with the new version that way like every 10th or 15th requests hits that or you can wait it to be even lower Watch what's happening with that and if you notice that something's going wrong You can easily just pull that right back out of rotation But if it looks good, you can roll all the rest of your nodes in that way You know really minimize the impact of a possible bad upgrade. So we use that trick sometimes it works pretty well There's the same with a b testing if you want to do that just post stuff in and see how people react Okay, so that's the open stack model how we kind of broke that down So at the top of the things we have a load balancer, which basically this object keeps the VIP the IP under which you can access a load balancer This is our top level bar check So you can only have one of those ip's associated with load balancers. So to kind of Think about it. So the load balancer object holds the ip address of your service Then we have listeners they do the port. So like if you have an hdp traffic you put in a hdp listener at port 80 You might have a hdps there too, then you put in a listener at port 443 So basically listen us they specify the ports Under which the system should listen on this ip. We specified on the load balancer Now they listen now we have that end. They're listening to something requests come in they need to do something with them And they basically forward them to what we call a pool a pool is a collection of members with members You mean back end servers Which basically then do the do the application or serve the website or whatever needs to be served And we we kind of put them together in the pool And as you can see you can use the same pool for instance for two different listeners in the same load balancer The other thing we have is a health monitor So you can specify how the pool checks the health I said when a server goes down when it isn't responsive We take it out of rotation and a health monitor allows you to specify exactly When we decide on that so so there's for instance could be a symbol as just hitting the hdp page and if it's a 200 We are we are keeping it or it could be going to different port Doing internal things whatever to decide that this members unhealthy The other thing Which is you could have the same server At the same server as a member twice if you have a web server listening on different ports It's very common if you want something like kubernetes where they put all those pots on Very esoteric ports and so you can then add each of those ports As a member even if it's on the same ip in your back end And the computer will load balance that for you Okay The other thing um, so so we talked about the model the other thing when I talk about is How how it does actually happen in real life in open stack. How does the load balancing work? So when you create a load balancer you can specify a provider And that will basically implement will do the work of load balancing And the providers available are our octavia reference implementation So so what we are working on is octavia. That's the open source open stack Load balancer. It's operator grade and everything But there are also hardware vendors who sell load balancers and they have written drivers for our load balancing system There are a 10 brocade citrix net scaler F5 networks camp technologies redware and also vmware and sx So if you happen to have something like that in your basement you can continue using that with open stack Another thing we have is a legacy h-oproxy namespace driver Which we have planning on sort of deprecating because it's not really scalable highly available and whatever And so we don't really want To have that out there because people usually use load balancers to get their services highly available highly scalable And and if you have a load balancer, which can't do that that defeats the purpose of load balancing Okay, so how Can we set up things our load balancer? Well, we have an application programming interface So it's all like everything in open stack is all microservices rest base and so on so are we So we have a rest interface We can basically send some json to which specifies a load balancer and you see in the middle of the slide See how it says load balancer That does the load balancer can give you description. You have to give it a subnet where you want the whip beyond so and Can give you the address or not other if you don't give it the address it will allocate one itself, but if you Want to give it to that? That's good. You can give it a provider like in this case We are asking for an octavia load balancer And we always like to give octavia load balancers the name best load balancer If you like them the most I'm talking about The subnet and those things so our load balancers they So in open stack you can have subnets some people have public subnets some people with private subnets So you can put a load balancer for instance on a private subnet it gets an ip And then it would load balancer under this ip maybe want to expose it to the public You can then put a floating ip on it other people have public subnets you can put it on there You can keep your members still on some private subnet on some private tenant network And add them to it and the load balancers basically in the middle between your public subnet and your private subnet And has ports in both So you can be very flexible with that technology and a lot of And there are some people have provider networks for their public subnet So they have more performance we support that too. So it's really what works for you Maybe you don't want to even expose it to the public and just have it to load balancer databases Whatever you want to do. So there's all kinds of possibilities The other thing Our api supports is a single call grade. So so you can create everything pools listeners members, whatever you want in one call To make that much much quicker if you Need to set up them in a hurry The other way we do things is our load balancing command line interface like everything in open stack You also have a cli tool and We are we are having a python octavia client which will be coming in in pike right now We are still on the neutron client So if you are in a pre pike release like okada or even earlier You would have to use the neutron client, but the commands are almost the same And you can also create a load balancer with the command line So it's again the name best load balancer and this time we don't give a VIP address We just give the subnet and I said in that case the load balancer will we will allocate an ip for you based on the dhcp system might have in neutron and then Then when you run this command it will return to you some some statistics What's happening operating? So each load balancer starts with our operating system Static operating status offline because it takes some time to build it and provision it and you see the provisioning status pending create and then After some time it will switch to active One thing to keep in mind If a load balancer is in a pending create pending update Status, then you can't do any changes to it. So if you then would say, okay, I want to change the name I want to add a listener then we'll get back. You can't do that until it's in an active state again That's just For us to keep things Seek when and make things more logical because otherwise just queuing up stuff and yeah gets gets messy Okay And we also have a horizon dashboard and my colleague will show you more about that All right So, uh, we've actually a note about this. Um, I know In the past we've had some questions about, uh, our dashboard Because there was a I think we missed releasing a version there haven't actually been any changes to it since what uh, maybe Newton Yeah, I think I think Newton was the last actual change we had to the dashboard because the api has been stable since then So I think if you're if you notice that there's a version missing just install the version before that and it's the same thing Um, we we started doing releases. So so we started doing so basically people got confused and so we started releasing things But it didn't change the code. Yeah, we've we've done releases since then We hope to have some a few changes around this In the next Maybe queen's release, but for now what you should see in pike should be what we're showing you today. So Just gonna walk through really quickly creating a very basic load balancer. Um, so this ui is Made to capture the 95 use case There's some stuff that you'll notice here Maybe if you've used our api that's that's missing But the api will let you do a lot more complex stuff. So Just Let me just check when we designed the user experience for this ui What we had in mind was that somebody who has no idea about load balancers can get one very very quickly So when you they go through that will everything will be pre-populated It will be very easy for people to get a load balance. Of course, we had to cut down on complexity. So But people get if they have no idea to get to one very quickly So I'll go with yeah, and I'll have I have some more notes about that that I can explain after this So just walking through creating a very simple load balancer here So you would just go to the load the load balancer section of the ui click create load balancer We'll get a load balancer description up here. We'll put something in octavia Yeah, octavia rocks who I will select our subnet We're gonna go ahead and put this one on the private subnet And go to the next screen So now we would define our listener. So like I said, this is where we say what port and protocol we're actually listening for Um, so in this case, we're just gonna do hdp80 pretty standard We're gonna make a pool Round robin is probably good enough for us for this demo So we'll go ahead and define some members Here we'll just pull in a couple of our web servers that we've got in nova already Um Let's us select things really quickly. Uh, and then We will create a very basic monitor Just a little hdp monitor here that'll check for status code 200. So once we click create We'll go ahead and send the api requests Our load balancer will be in pending create status It'll be offline until it's fully created And then you can just you know check this page again until the load balancer is up and running So I don't think there's anything else to this demo Oh, here we go. We're refreshing it. Yep. And now it's online and active So You can from here you can do some of the more complicated stuff like add more listeners to it, etc The initial create workflow just has the one listener because it's just trying to get things up and running quickly So Do uh some quick notes about session persistence. Uh, if you are not familiar with session persistence um, basically It allows us to keep requests Going to the same server for the same user in case you have like some sort of state that you're maintaining inside your application so Without session persistence. What happens when you go to a website like open stack.org is your request will go in It'll go through the load balancer be sent to a server return to you And then if you make the same request again as the same user or let's say go to the news Site it might send you to a second server Because it doesn't have any sort of state tracking built in So if we enable session persistence, there's a few methods to do this but Http cookie is a really easy one. So if we enable session persistence, it will Be somewhat intelligent about things. It'll send the user to a server that it picks for the first request And then for subsequent requests, it will attempt to send the user to the same server obviously if that server goes down it will Redirect them to another one, but if it can it will send the user to the same server The other interesting thing that we can do with load balancing in open stack is we can do tls termination at the load balancer So this makes the load balancer actually decrypt the request When it goes to the before it goes to the back end server and then re-encrypt When it's returning the response to the user So this has the the benefit of all of your back end servers don't have to handle all the heavy lifting of decrypting your tls requests All of that is handled by the load balancer And if your load balancers have special like hardware to do that you can decrypt way more requests that way Very quickly It also simplifies certificate management because certs only need to be installed and updated on the load balancer You don't have to maintain all of your certificates on every app server in your fleet And they're stored they can be stored in a secure location. So octavia Uses and neutron albass uses the barbican project to store our certificates So the user will actually put their certificates And their private key in barbican, which uses a secure storage mechanism And then just pass the id reference to the load balancing so we can go fetch it from there So when we when we handle it, it's all over hdps We throw it away when we're done with it And on the load balancing servers It's kept in memory or In the case of octavia or stored However, hopefully securely The third party vendor does That it also allows for some advanced load balancing of tls requests like header inspection and injection government There's an important thing so so when we had on the case where we didn't encrypt hdp requests on the load balancer and hdp protocol allows a client to specify a timeout And then when you go and the client specifies a timeout, which is longer than the timeout You have set globally on the load balancer then the load balancer will prematurely cut a connection And it will be very bad experience on the client So when you do hdps termination on the load balancer, it can pass the hdp headers See that somebody requests a longer timeout and then implement that on the load balancer And make sure that doesn't cut a connection prematurely So that's it's usually best practice to kind of terminate a load balancer So it can do the timeouts and read all those other things which are in the header and do something smart about it But if you don't do that be mindful that you Might end up with prematurely Disconnected things if you don't choose your timeouts wisely on the load balancer Yeah, definitely that I know also it's useful to be able to inject headers like It exported for exported ports So your application knows where the request is coming from because otherwise that gets masked in this process The only other thing I would note about this is we don't do Re-encryption to the back end yet So what that means and we'll see in this little demo is that when your request Comes into the load balancer It's encrypted but then when it goes to the back end member it's decrypted So this is fine if you're on like a private tenant network If you're confident of your local network security But if you're not It would be good to Push for us to implement re-encryption because I know we're looking to do that. So We're also I don't know how many of you guys are upstream contributors But we're always looking for for help on some of this stuff. So Yeah, let's see. So the last thing I want to talk about I think is Layer seven load balancing. So if you're familiar with the OSI model Layer seven is just the application layer. Mostly most of our load balancing happens at layer four But you can do some interesting things at the application layer. So We have two objects that are relevant for this in open stack We have the policy which is describing what you want to do And then the rule which describes when you want to do it So an l7 policy could be like I want to reject all traffic or I want to redirect traffic to this url Or I want to redirect traffic to this pool of members And then the rule could be like if the host name is abc or if the url path is slash images or if the file type is exe Or any of these things starts with ends with contains So for example, I could say reject all traffic where the file name ends with exe Just I know I don't serve executable files. Let's not take a chance. Let's just block possible malware attacks You can also do things like uh redirect to url is useful If you wanted to redirect all your traffic from htp to htps So you can actually put in a policy that says redirect to url htps colon slash slash urip And do that always And it are on I think path slash is how we do that It's that captures every request and that way if a user goes into your website Unsecure, it'll automatically redirect them to the secure site So just an example of what things look like in this case So I know we had our our load balancer here And listeners and pools and you've seen that before German showed you that part What we have now is on our Our listener port 443. We have an l7 policy And I guess we're saying redirect to a different pool And then we have some rule we didn't define what this rule is but What this is going to do is if it matches the rule It's going to instead of forwarding traffic to The default pool there the the one with our two members It will forward traffic to an additional pool with this one member so Just in case you have like two apps running on under the same ip And they're like separated by different addresses You could run them both on the same load balancer and just split them off with this rule so Like I said earlier, we love when people get involved Whether that be contributing code or just giving us comments because we love to hear what users actually are looking for We don't like to operate like In behind the scenes in a inside of a black box or something. We want to hear what you guys have to say so if you have Suggestions if you have Bugs that you've run into by all means Come report those to us. Talk to us. Give us feedback. Yeah. Yeah for bugs Launchpad slash octavia Is just file bugs there. We really do like to see stuff show up there because If we if we don't fix stuff, you know, it It's just as hard on us when we have to track this stuff down by hand because we're going to run into it eventually hopefully But any reports you can give us are are great And yeah, definitely get involved. We have weekly meetings on irc We're always around on our irc channel Yeah, i'm rm work. This is ex-german And our fearless leader is johnson. Yes michael johnson The ptl is johnson. He wasn't able again to make it today, but he's on there as well So thank you very much for coming Questions. Yeah, do you have any questions because we think that's probably the most valuable thing here is if we can Get your questions answered I'll run around in the microphone. Oh, is there cool? Awesome. Yeah, we are recording this session So it would be very good. Awesome. Don't have to run around like I did last time. No As far as I know, it's still possible to use lb AA as As was the agent only so I would like to know if this deprecation of the h.a proxy namespace driver Is it for octavia only or for both projects? so Octavia right now is In the process of adding the Existing drivers under it. So there's another talk actually. Did we put the slide on? Okay, we didn't do to talk. Okay, we don't have the other There's another talk where we're going to talk very in-depth about that tomorrow About exactly what's going on with octavia versus neutron lbass right now neutron lbass Not octavia has then the namespace driver running under it octavia does not yet Octavia will as of I think queens is our target Support third-party drivers under it, but for now, that's all still under neutron And we have and we understand a lot of people are still using the namespace driver So there's so so there's a plan to maybe move it in its own repositories So the community can take care of it because as we said we Yeah, we don't think people really need a non scalable non ha Driver, but it's very useful for testing. So the thing about porting it over. Yeah Yeah, I know you can yeah, and we're actually using ha proxy in our ha solution, which is octavia It's just the namespace driver itself is not a great implementation of that So this is this was my second question because I was asking what's wanting to ask Which features depend on octavia and which are with Yeah, just the agent but now wait till tomorrow Yeah, sure Any other questions? Okay, I guess then Thank you everybody for coming. Please get involved and Yeah, thanks