 Cool stuff. How's it going everyone? My name is Aaron Rosen. I had a few people who helped me out setting up this lab Eric Lopez and Janet you So basically what we're going to do today is we're going to build a multi-tier application on top of OpenStack And we're going to be using neutron with all the open source components and we have that deployed for you guys today and we're going to allow you guys to get access to a lab environment that's running on our NSX cloud and VMware and on top of that we have all the these OpenStack components So the way that you access the lab is if you go to this URL right here at the bottom of the screen That'll take you to I'll leave it there for a sec That'll take you to this web page I'll put it back in a second and then this top link here It says go to this URL to get access to the lab if you paste that in here and fill this information out this lab access code is It's right here. It's OpenStack Atlanta 14 and when you hit submit it'll give you a IP address and password to access the lab and there are two different ways to access the lab the first way is to SSH directly into the instance Using the IP address and password and the second way is if you put it in your browser There's a no VNC Console that runs for it So for instance in mine You have to put a HTTPS before it It'll prompt you for the password I don't have to look up what the password is that I'm using and then it'll allow allow you to use the no VNC Desktop as well if you don't have an SSH client So during the lab if anyone has any questions I have two helpers here that should be able to help you guys out if you just raise your hand We can come over and help you guys out So does any is anyone confused or stuck at this point on how to get access to the lab? Cool, so there's this input box That asks you for an access code and the access code is on that codepad thing OSS Atlanta 14 I think is what it is The SSH user is Nacera Cool, so I'll go ahead and get started NIC IRA I'll leave it right here for a second so you guys can see that All lowercase the password is when you hit submit it should be displayed to you on the next prompt The password randomly generated Yeah, it'll give it to you an email, but it should also be on the page when you hit submit Cool Yeah, so if you want to use so there are two ways to access the lab The first way is you could just SSH directly into the instance if you have a SSH client Or you could use your browser and you have to put HTTPS First and then put the IP address and then it'll prompt you for the password pasted in The username to SSH into is Nacera NIC IRA Yeah, I'm not sure what What's up? Yeah, the username is Nacera, so if you're gonna SSH into it you would just do something like that SSH Yeah, it's only command line so you don't need the GUI. Yeah, it doesn't matter I would just use like the SSH just using a shell if you have it Sure, it's right here at the bottom of the screen Oh Cool, is everyone like able to SSH in? All right, so before we get started I'm just gonna give you a little bit of an overview of what we're gonna try and do in this lab So we're gonna go ahead and deploy a multi-terror application on top of open stack So what this is gonna consist of is we're gonna leverage security groups in order to control which things are able to communicate with each other So in the application that we're gonna deploy we're gonna have two web servers And we're going to use a security group on those to only allow TCP port 80 into them so only allow HTTP traffic We also have this jump host and what we're gonna use this host for is In order to get into your web servers This host is gonna have a public IP address So you're gonna be able to SSH into the jump host from that host You're gonna be able to SSH into the web servers so this allows us to prevent having to put the web servers directly on the WAN and then we're gonna create a load balancer and Create a pool of nodes and put the web servers in that so the load balancer will balance the requests across the servers after we're done with that we're gonna go ahead and Use the firewall as a server stuff just to show how you can Use firewall rules on the router in order to have a little bit more enforcement Besides having enforcement on the nodes directly So this is the lab topology that we have deployed And a little bit. I'll show you what it looks like under the hood Running on top of NSX, but this is just a high-level overview. So there are two compute nodes and the compute nodes are actually in different subnets and The instances are are able to have L2 connections between each other because we have we're using overlays Which allows us to not worry about what the physical network looks like So even though they don't have real IP or real L2 connectivity with the overlays we're allowed We're able to simulate that on the compute nodes. There are two pieces of software running There's a L2 agent that's responsible for programming OBS the flows in it and setting up the tunnels between the nodes And there's also Nova compute, which is responsible for spinning up the KVM instances There's also a network node. The network node sits on the WAN of network that has the DATP agent Which is responsible for handing out IP addresses to the instance. This is the L2 agent in order to Connect all the L2 networks the L3 agent which is going to be used to do natting and floating IPs It also has the load balancer agent and the metadata agent and there's also a third Instance that is running that has all the OpenStack API endpoints Cool Right the L2 agent is the OBS agent and in this in your setup You're actually using that ML2 plugin from from IceHouse There is tunneling so the compute nodes are in two different Subnets, so we're able to create instances on each compute node and have L2 connectivity between them And it's using a GRE tunneling to accomplish that cool, so if you go to that bit.ly link there's some instructions that are That are laid out and The steps that we're going to take so the first thing we're going to do is we're going to source this openrc file And all this does is it put it sets some environment variables in your In your bash environment in order for in order for you to be able to issue commands one sec Do you know what the root password is for this? Sorry about that one sec. Oh, I know what the problem is cool, so we're going to go ahead and source that a demo RC file and if you look in that file all it does is it sets some bash variables up and which Contains the credentials of the the user and also has the API endpoint So the first thing we're going to go ahead and do is we're going to go ahead and create a network So what this does is this just creates a L2 broadcast Zone so that just creates basically a L2 segment like a switch and After doing that we're going to create a subnet so basically what that does is it adds L3 onto the network so whenever you create ports on the network It's going to go ahead and allocate IP addresses for you and there's also a DHCP agent that's going to Assign these IP addresses when instances get created on the network Okay, so when you SSH into the machine You come in through the VMWare interconnect onto this jump post That and I think I'm not sure which box you're actually on but it's on a I think it's actually on the API controller node You come on to this box here, but once you source the file Yeah, it's a separate VM So it's actually a separate VM. Yeah that you source the file that has connectivity to all of these networks as well So after you create the network after you do neutron net create private net that creates The L2 broadcast down then we're going to attach a subnet to it So I'm going to go ahead and follow along and do the same things Cool. So the next step is we're going to go ahead and create a router and After we create a router we're going to uplink this router to the public network So if you so this basically allows things attach the router to go in and out to the internet And we've already gone through and created this public network for you Because this is a step that the cloud administrator would need to do Because we need to tell opens back the public IP or just ranges that are attached to the land So so the first step is we're going to do neutron router create and then we're going to do neutron router gateway set To attach it to a network called public So right now if I do neutron Netlist You'll see that I have two networks I have a public network that that was already created for you and then a private network, which we just created So I just created a router now. I'm going to go ahead and uplink the router to the public network So the next step after doing that is we need to uplink this network that we created to the router as well So this will allow ports that are created on this network to be able to flow to the router and then to the public internet and by default when you create routers they are Natted so you're not going to have a public IP address So you won't be able to come in from the internet into your instance unless you associate a floating IP address So I'm going to go ahead and do that as well Cool So the next step is we're going to go ahead and create a security group and what a security group is It's basically just a container of rules. So we're going to create a security group called jump hosts which we're going to use to assign to this one host and Inside the security group. We're going to create a we're going to have a rule that allows that TCP port 22 into it So this will allow traffic For directed towards SSH to go into the instance Yeah, jump host is just a random name that I chose. I Would probably use the same name here just to make it easier because it refers to jump host all over Excuse me. Oh font. Sure Right, so the previous command so previously we had a router and then we uplinked the router to the public network And then after that we uplinked the network that we had created called private to the router So these are all the steps. We create a router. We do neutron Right, so you you could specify like the IP address that you want on the private side to attach But by default it takes the dot one But if you wanted to use like any other IP address you could specify that when you attach the network to the router Is that what you're asking? so on This side this guy's gonna have an IP address of ten dot zero to zero dot one and on This side this guy's gonna have an IP address out of the public IP pool That isn't exposed to as a tenant because this is gonna be an added IP address Yeah, by default if you don't specify it it's gonna be dot one But you could specify it if you wanted to um Does it check if dots one is in use? It probably does that it might not there could potentially be like a corner case or a bug around there, but Actually We can check Yeah, so the way it works is that IP allocation pool starts at dot two and ends it That two five four so one less than the broadcast address so it ensures that it's outside of that allocation range Cool, so we've just created this security group called jump host now. We're gonna put a couple rules in here We're gonna allow ICMP traffic and then TCP port 22. I'm just allowing ICMP traffic just like Just so it's easy to ping and like help debug things to see that things are kind of working so this Allows us ICMP ingress to the instance and then this is going to allow TCP port 22 Into the jump host security profile so the next step that we're gonna do is we're gonna go ahead and boot a VM and We specify the image that we're gonna use and we're just using this a serous image. That's just a really small VM form factor. We're specifying a we want a flavor of one and that's basically the size of the VM and We're specifying the security groups that we want to attach to the instance so in order to figure out that The images that are available you could do nova image list And that shows you there's a serous image and this other image two images here and if you do Nova flavor list That'll explain to you a little bit about the flavors that we're selecting So we're just using flavor one because we want to boot a small instance That's gonna have one desk one CPU and two hundred fifty 512 megs of RAM Excuse me So things that we're doing here with command lines are exactly could be done from the horizon Right, these exact same steps you could do through the horizon everything could be done through the horizon So the horizon interface can like hide a little bit of the complexities of doing this Like when you create an instance you I mean it just shows you like an UI form factor The flavors and everything yep the security group that we're not touch. Yep. Those are all like checkboxes and horizon in order Yeah, so I went ahead and boot the instance So the next thing you can do is if you do Nova less the shows the state of the instance So it's in spawning state right now so if you keep doing Nova less this will go ahead and like refresh and see what What the instance is doing? So this is running nested nested in the cloud. So things are a little bit slow to like kick off So it should come active within 30 seconds or so when you add the interface to the public network And then you add the interface or you attach the router to the public network And then you add the interface to your private subnet How do you go in and list out that? Router to see which IPs that it grabbed when you when you created it sure so there's a new Chon I think it's called router interface list Neutron router port list So this shows The router ports that are attached to it. Sorry. What was your question? How do you know which ports are attached to the router? Cool. Yeah, so this neutron router port list or shows the ports that are attached to the router So you're able to uplink multiple subnets to the router as well We can do that after we walk through this if you want to do that And when we issue this command you'll see that there are multiple networks Connected to the router and this would show it to you. Is everyone to about this step of booting the VM? Booting the jump host instance Cool. So at this point, hopefully this instance should be up So it's active So the next thing that we're going to do is we're going to go ahead and associate a floating IP address But this instance so this will allow us to get into the instance. So right now If we ping this IP address, this is just an internal IP address that is behind the router So we were not able to access it directly so in order to do that it needs to have a floating IP address so we can route into it Yeah, it should eventually it should eventually work So one of the reasons why the spawning is a little bit slower in ice house is One of the things that we do is we start the instance and pause state and we wait for the network port to be wired And as soon as it's wired the L2 agent will set the port status is active in neutron And then neutron will send the event to Nova and then Nova will unpause the instance So this avoids a race condition of the instance coming up and the networking not being ready Because some of the client scripts inside of the instances will try and DHCP and the some of the scripts only tried to DHCP twice and then give up after like 30 seconds or so So this helps us to avoid that type of race condition. This all happens automatically behind the scenes If you had to do we use the mic, sorry You said you're posting it until all the VMs come up, right? So how do we do that? I know you and you're done it So can you share that one more time? Sorry, I didn't you said you pause because the if everybody asked for Sure. So what I was saying is the instance has started in pause state Automatically so Nova will create the instance and start it in pause state with the tap interfaces wired to the bridge Oh, how do you pause it? So this happens like these are internal details and like the compute node it automatically does this So it does this through libvert It does it like starts instance automatically for you and pause state when the networking is ready for the instance Then Nova compute will unpause the instance for you But you could pause the instance through an API call to do that. You could do Nova pause Cool. So is everyone's instance active or we still spawning is there a way to we passed the mic back, sorry Is there a way to access horizon horizon? Yes, you could access HTTPS just the IP slash Dashboard slash admin right I believe if you try that doesn't work I think it's Eric. Do you know what the IP address for horizon is inside of the instance? Or just local host What oh, you mean within the VM itself, but not external. Oh, no, you can't access horizon externally Now you have to access it through the instance got it. Yeah, that like port 80 isn't forwarded into the instance But I would probably follow along with this because you might get off on the workflow So spawning okay, I should eventually get active. We'll get another minute Cool, so I'm gonna go ahead and Continue in doing the next steps because I have to boot two more instances and it'll probably take a while for those to Come up so what I'm gonna do is I'm gonna go ahead and associate a Floating IP with this instance so in order to do that I need to figure out the port ID that's attached to this instance So one of one of the ways you can do that is you could do neutron port list to see the ports So this displays all the ports that are available to my tenant. So this is the port ID that I need There's the stop one is the gateway address and this stop three is the DHCP agent that creates a port on the network in order to provide DHCP for you so So as soon as there's a a port created on the network, that's Not a as soon as there's another port created on the network then the DATP agent creates a port and it just does that just to save resources But so now I'm gonna associate a floating IP address to this port here No, you only care about the ID. So one thing you could do is you could do neutron port list That's the ID So this will go ahead and find the port that you're looking for If you do dash CID, that's just the ID and you're gonna pass that to a neutron Floating IP create dash dash port ID that and then public which is the public network Yeah, that will only work once Instances active right Yeah, if it's still in spawning state you'll have to wait so once that's done I can do NovaList and we'll see that that floating IP address is a displayed right here and And we should be able to ping that and SSH to it as well Dot 2 is the one I grabbed when you do Neutron floating IP create it'll show you this output here and it'll have the floating IP address and If you like cleared the screen out you could also do a neutron floating IP list and this display is the internal IP the Floating IP and the port ID that it's mapped up to you So right The port ID is the current corresponding to the dot 2 IP address So you need to do something like Like this in order to get the ID that you need I'm not sure if you could pass in a name so there is a name you could pass in a name on the port if you wanted and then And then pass that in I'm not sure if the Python Neutron client supports that But we could definitely extend it so that if there's a name there I could do a search and then do it to simplify things for the user Everyone doing alright, or are we still out of spawning state or some people still are still spawning Still spawning cool. What's it sure? So What this demo is using is it's just using the reference implementation for load balancing which just uses ha proxy But there are also a number of vendors that have driver mechanisms in the tree. I think Yeah, Radware f5 There are a few others Will you repeat that one more time when you would create the security group? Yeah, these are the drivers that are in the tree right now net-scaler, Radware H a proxy and brain What's that? Yeah, the the VM has one IP address It has an internal private IP address and then it also has a floating IP address Which is inserted into the router which performs NAT So this allows you to like move that IP address between multiple VMs if you wanted to Yeah, the instance will only see this 10.0.0.2 IP address So for example now we can SSH into this instance the username is siras and I think there's already a The key okay, it's not and the password is cubs when and it should be in this dock here somewhere Right here Cubs when with the smiley face so So once you get in the instance if you do if convict you'll see that there's only He only sees this internal IP address of 10.0.0.2 and We accessed it and we got into him using the floating IP address right, so it works on that machine because That's like up to the router so it has a route into that Right, so we could change the IP address We could boot another VM and then move this floating IP address from one VM to another VM So it's helpful if you want to deploy like an updated version of your software And when you want to switch switch it over so traffic goes to the other place you could just Reassociate the floating IP with a different port Right, so you would make the API call to neutron and then in the back end it would go ahead and make this happen So we're using the ML 2 implementation with the L3 agent So what happens is when you make this API call puts a message on the RPC bus and the L3 agent will then go ahead and Like we change the state of things in himself in order to reflect this change Pretty close to instantaneous Yeah, you shouldn't be able to ping the dot to IP Because that's the internal IP. So you need to create the floating IP Are you pinging the 170 to? What's up cool, so are some people up to this step? Yes, okay I'm gonna keep moving on just because just so we can get all the way through it and we can retract back and Catch people up when we get to the end of it So as you can see I SSH into that floating IP address which gets me into this instance that has a only one neck with the private IP address So the next thing that we're gonna do is we're gonna create another security group Called web and in this security group this security group is going to be mapped to our web servers So we're gonna go ahead and do this and create the group And the next thing we're gonna do is we're gonna create two rules This first rule here allows a TCP port 80 into the instance and the second rule here Has a self-referential rule. So what this does is it says it allows a TCP port 22 into it from members Who are a member of the jump host so this allows us to continuously add more web servers? And they'll automatically be able to be accessed from members of the jump host group So it allows us not to have to like make any manual configurations when we add more host Exactly in the network the VM has 10 the VM has 10 dot 0 dot 0 dot 2 IP address, right? That's the IP address of the VM. So who exactly in network beside that VM knows about that IP So that happens the router. Yeah, the router has that NAT rules that makes that happen But is that is that it? That's yeah, that's it So like when an IP address comes into the router He says oh the destination IP is is this floating IP address So then it goes ahead and does a D NAT to send the destination to 10 dot 0 dot 2 So even if there are VMs on the same subnet even for them Same same thing right that still has to go through the router So if VMs communicate on the same subnet, they'll could just go directly to them Okay, so they don't need that obviously it won't go through that NAT function, right? It won't because it's just gonna go directly to it I think in this case you don't have any just one VM. Yeah, this is just one VM at this point separate subnet So I can show you that once we launch the web servers. Okay, so that what's that? Why does it only show one interface? Because there's only one interface like when we launch the instance So when we launch the instance, there's only one network available So it only creates an interface on that network. So there's only one interface the floating IP isn't actually an interface It's just a NAT rule that goes ahead and translates that floating IP to the internal IP And the internal IP to the floating IP when it goes in and out Cool, so I'm gonna go ahead and boot these two web servers because this could potentially take a while So if I do NovaList, we'll see that I have the jump post. That's an active state Here's its floating IP and then we have two more instances that are coming up web server one and web server two Yeah, I would go ahead and spawn spawn those as well. Did your instance ever go in active state or it did? Yeah, I already I just did those steps So if you want to like follow ahead you can if you just follow through that codepad link and just follow the exact steps You can go ahead of me if you want or catch up if you're if you're behind So we can see web server two is active and he got a dot five IP address and Web server one is still in spawning state The IP addressing and stuff so the way that it happens is you you're telling Nova to boot an instance And if you don't pass in any networks by default It'll go ahead and create a port on the first on the network that it has so there's only one network So it'll go ahead and create a port on that network for you so So the way that it works is it gets to a compute node the compute node goes ahead and talks to neutron and creates a port of Neutron neutron returns an IP address to Nova then an IP like the port information and then Nova will then boot the Instance using the MAC address that neutron told it and then the DATP agent will go ahead and Or the VM will come up do a DATP request the DATP agent knows the Mac and nobody's using that Mac and he'll get the IP Right it's all being managed above he just gets the IP through DTP which IP are you paying? Okay, the floating IP address Okay, so one one thing that you can do in order to like help debug like why things aren't working Is if you do Nova console log and then type in the Like say the instance I like jump post This will go ahead and display What's going on in the instance because instance writes to a serial port and that data is there so you can see Like that instance it instance booted up He did a DATP request and hear the routes and my jump. I see is this epi the command is Nova console log Then you pass in the host name So it could be possible then since it's still like booting and it hasn't gotten to the DATP state yet So in my setup, we've got the two web servers up I'm just gonna go ahead and keep our people up to this point at all Could I get a show of hands of people who are at this point? Okay? Looks like there are a good number of people so I'm gonna keep pushing on and we can circle back So what we're gonna go ahead and do is we're gonna SSH into these web servers and we're just gonna start up a little dummy web Server, we're just gonna use a net cat just to return back web server one on web server two So when someone does a get request against it, it's just gonna return return that so So this step is a little bit tricky So what you'll have to do is you'll have to jump into the you'll have to SSH into the jump post So you'll do SSH seros at and then the floating IP you have so now we're inside of the The jump post and the next thing we're gonna do is we're gonna SSH into web server one and web server two So you can see web server one has an IP address of dot four web server two is an IP address of dot five So we'll do SSH ten dot zero dot zero dot four We'll type in the password again cubs win with the smiley face and Then we'll go ahead and paste this line right here Cool So after you do that you'll type exit and now you're back on the jump post again It's a little bit confusing because the terminal prompt doesn't pay put what coaster on so once you're back on the jump post Now we're gonna SSH the second web server web server two So if I scroll up here web server two is ten dot zero dot zero dot five. So if I SSH ten dot zero dot zero dot five Enter the same password again cubs win the smiley face and then I'm gonna go ahead and use this second command I accidentally caught that beginning of the paste So now when I type exit I'm back on the jump post so at this point I can use curl and curl to either of these Hosts and it should respond return the host name. So if I do curl ten dot zero dot zero dot four say web server one Five web server two. So the next step we're gonna do is we're gonna go ahead and deploy a load balancer And create a VIP and it's gonna load balance between these two when the VIP is accessed Give a question Okay, so I'm gonna go ahead and do those steps So the first thing we're gonna do is we're gonna create a load balancer pool So what this is responsible for is it's just kind of like the security group concept as well. It's just a container So we're gonna create this container and then we're gonna add our Our web servers to it as members So if you do a nova list again This displays the IP addresses Of the hosts, so we'll go ahead and do this and we'll enter the IP addresses of Web server one and web server two so ten dot zero dot zero dot four and Ten dot zero zero five So the next step is we need to create a health monitor But the health monitor does is it's gonna continuously probe the members of the pool to check to see if they're still alive So the load balancer allows us to have some kind of hive allows us to provide high availability in a sort of way So you can like add more nodes to it if the nodes die the load balancer will automatically take them out of the pool And won't serve traffic to them so it allows things to crash and and have things keep up running Sure, so the way that it works is when you create this health monitor right here You can see the type of check that it's using. I believe you can do ICMP as well to like ping it But this is just gonna do a get request using HTTP against it For three seconds and it will allow three timeouts so after nine seconds It'll if the host isn't responding to a get request. It'll pull it out of the pool. So this way When you go ahead and request against the VIP You won't get like an error. It'll go to a host. That's actually known to be alive We're not gonna use a floating IP for the web server because We want to create a load balancer which is gonna get a VIP and then we're gonna associate that VIP with a floating IP Because we want to have some way to load balance between the web servers So this way you can access one IP address and it'll automatically write you to one of the web servers So this goes ahead and creates the health monitor and after creating the health monitor We need to associate that with the pool that you created So in order to do that you need to copy the ID of the health monitor here and Issue this command So right now I have a load balancer that has two members of it and then a health a health check That's associated with the pool So the next step is I'm gonna go ahead and create a VIP So basically I'm gonna tell it the protocol so HTTP and port 80 and it's gonna go ahead and create a port and When the IP address on this port is accessed it's gonna automatically like round robin between Between the members of the pool and the way that that works is we selected the like the load balancer method It was wrong robin before but you could pick a different algorithm if you wanted to so I'll go ahead and do this and you can see it return this IP address 10.0.0.6 and It's the same case before with internal IP addresses I'm not able to like access that IP address from the outside world until I associate a public IP address with that and One of the reasons for that is you could potentially have an application that wants to be load balancing internally and Not actually be on the public Internet if there's if you had an application that wanted to load balance Like something else as well Yeah, or a database. Yeah, that's a good good example Cool, so after doing that if we want to allow this This VIP to be accessed on the public Internet We're gonna need to associate a floating IP address with it So in order to do that we're gonna do the same command that we did before Neutron floating IP create and we'll tell it the port ID. So you can see the port ID is already here So that'll go ahead and return this to us So when this IP address is accessed 172.161.131 it's gonna go ahead and Convert that to 10.0.0.6 and forward that to the load balancer The load balancer is gonna receive the request and then route it to one of the pool members Right, so the question was if you're using a different like vendors driver Is it gonna be different? So one of the big things that we Tried to do is we tried to have a generic API that all the vendors can come behind So this way if you're using one vendors back-end implementation a client can go ahead and write code against the neutron APIs And doesn't matter what vendor is in the back-end. So this allows it to be like very portable. So there are some cases where company might have some special feature That isn't part of the API and the way that we handle that is we have extensions the API to allow That vendor to expose that type of functionality Okay, so at this point we have the floating IP associated with the VIP So now what I should be able to do is I should be able to curl against this And if everything's working right to return web server one and then web server two So you can see it's automatically load balancing for me So the next step that we're gonna do is we're gonna create a firewall and we're basically good Cool, so at this point the load balancer is a round robinning these requests So we're gonna go ahead and create a firewall to do enforcement at the router. So for example, we don't need to allow like Like other miscellaneous traffic to reach all the way to the VIP If we don't want it to so we can stop it at the router instead of allowing it to get all the way to the instance So the firewall API stuff is still a little bit experimental. It's still missing a zone edge. So it's kind of a It kind of maps to a tenant. So when a tenant creates a firewall, it doesn't actually map to a router It's kind of a global a global thing right now So one of the things that we would like to do is scope that to a location as well But that isn't done yet. So right now when you create a firewall, it's gonna be mapped globally It's all your instances and all your networks that you own So the first thing that we're gonna do is we're gonna create a policy and This is just creating a default policy that we're gonna add rules to After we create the policy, we're gonna create a firewall and then map it to this default policy So after you do that, you'll no longer be able to curl it to these IP addresses because they're By default the firewall blocks all traffic. So we're gonna go ahead and create a rule to allow HTTP traffic And we're gonna go ahead and insert it into the the firewall policy So this creates a firewall rule that says if the protocol is TCP and the destination port is 80 Allow access to it or allow access and we called this rule allow HTTP So the next step is to go ahead and insert that into the firewall policy So we're gonna go ahead and into it this rule into the firewall policy and after doing that We should be able to curl against it and see things that are working So one of the nice things that the load balancer allows you to do is say we one of the instances experience is a failure So if I do a novel list and delete one of the web servers say I delete web server one The load balancer is gonna be doing these health checks against it So after it sees that web server one is no longer responding It's gonna go ahead and stop sending traffic to it. So when I run this command It's gonna just restore on web server to because he's the only pool member that it's active So the instance has been deleted. So probably by now the health checks have timed out So if I do this you can see it's always returning a web server to No, I typed I hit backspace so the app you was messed up. It doesn't load balance for you, right? so Did you SSH to both hosts web server one and then run that while loop and then web server to Are you able to? Okay I would like SSH into the jump post and then check if you can curl to those individually first like debug that. Oh, that works Okay, so if you So if you do a neutron LP member list It should show What does it show here? So you can see that there are two members of the pool and this one's marked as an active because I deleted the instance Cool. Well, this kind of concludes like the steps that or what I was gonna demo But if there's anything you guys want to see I can go ahead and like demonstrate it up here If there's like additional things are curious about Can I show you the public network stuff? Oh How the public network was created? Sure, so the way the public network was created is you needed to be an admin user and the way that it was created with something like Neutron net create So that's the command that was run Before the cloud admin ran this like as part of the installation that basically says this network is a public network That's a map to the floating IPs. That's mapped to IP pool No, there's nothing additional. That's really special. The L3 agent is basically gonna I mean pick up messages when ports are created out of that and one thing I wanted to show you guys is so we I'd all of this is running on top of NSX, which is the product that VMware develops for our network Virtualization solution. So Neutron is a public pluggable framework There's a multiple vendors that have plugins and the tree in order to implement like networking functionalities. So for example, when you log into NSX this is a this is just a UI interface that makes API calls in order to Communicate with the controllers in order just to display information So we have controllers So we as you can see we have five controllers here running and what the controllers do is they go ahead and program all the OBS nodes on all of the hypervisors. So you can see here We have 188 hypervisors Registered those are machines that VMs are going to be booted on We have nine gateways what the gateways are Those are nodes that allow those are nodes that connect to the public internet with the floating IPs connect you So in the open-source implementation the L3 agent is the gateway, which is used But in in our implementation, we have these things called gateway services We create a gateway service and then you put machines in it and then it provides high availability between For your L3. So if one of the gateways dies The traffic will continue to flow and it'll fall over the NAT tables for you So you can't give two floating IP addresses to the same internal IP because it wouldn't know what to do with that So as I was showing you before In the lab topology here, we have several networks that are created and a router uplinked to it So I went through here and I found One of the networks Or one of the labs that were deployed. This is just the tenant ID and we can see here all the networks That's accessible accessible for the tenant. Here's all the the ports That are in the lab so you can see everything's green. This one's down, but it's marked down on purpose in order to prevent like access into VMware infrastructure and here Here are the ports that are attached to routers. So one of the cool things here, so I'm gonna go ahead and click on this Lab management network. So this is just this management network here that all of the hosts are connected to So you can see over here. It said there are six ports on it and then here there are There are six different Machines on that or there are five, but one of them is the DHTB port. Oh, sorry One of them is the patch port up to the router So this shows all of the ports that are on that network and one of the things you can do is you can click on one of the ports here And they'll go ahead and like show you additional information about it Here's the MAC address that's on the port It's going to go ahead and look up the transport node. So this is on server 490 and One of the things that can be challenging when you're debugging these virtual networks is when things aren't pinging Understanding where things are even in in the data center since now It things can be anywhere like the L2 is able to span anywhere So one of the tools that we provide is this port connection tool So if you find there's a connectivity problem, you can select the ports here and click go What happens is the controller is going to insert packets Into the data plane in order to figure out where things aren't flowing and what the problem is So I went ahead and did that And it's going to go ahead and show what the topology looks like There's one thing that I forgot to mention is that this service node concept So what the service nodes do is the service nodes are kind of an offloading mechanism for broadcast So if you have a VM that's doing a ton of broadcasting The hypervisor has to Duplicate those port duplicate this packets across all of the port so that can be kind of CPU intensive So in order so that the instance doesn't feel any of that pain because the hypervisor is doing all that duplication It goes ahead and sends the packets to one of the service nodes and the service nodes jobs are just for Like doing replicate packet replication for you So those guys go ahead and duplicate the packets for you so that the instance doesn't experience any kind of effect if there's a VM on it just doing a ton of broadcasting Yeah, so that's where multicast traffic would would go as well if someone's using multicast it would traverse through the service node You could also have the hypervisors Do the replication for you, but if you add the service nodes, it just provides a way to offload that so in this case We have two instances one's on server 490 the other ones on server 477 and They have an IP tunnel directly through them the broadcast will go We'll go through the service nodes and you can see there are two service nodes here for high availability and Any other traffic that's traversing the router is going to go to one of these two gateways also for high availability Right, yeah, that was the next step after you create the network and market dash dash router External equals true you need to create a subnet on top of it And the reason why we didn't have you guys do that step is you need to know the IP addresses that map into the infrastructure That's so that's something that someone has to do ahead of time So if you were using a public cloud that's using open stack and neutron that would be something they would already be created for you Can you run two L2 agents? on the same box Yeah So in in the NSX solution there are no L2 agents so it's that it's an agent It's kind of the plug-in specific thing So however the plug-in decides to implement it but in the ML to plug in there's only one L2 agent that runs Because that's the only one that's needed. He handles all the tunnel stuff for you. Oh, share the workload Usually it's really not a lot of work just to Like program a few flows it just runs a couple commands to set up the flows and sets up like tunnels after it does that It basically just its idle so there wouldn't I don't think there'd be much benefit for us to scale that out We could also make it multi-threaded or something like that if we really needed to So the question was if you're using NSX and open stack together with the hypervisors ESX or KVM This is multi hypervisor. So you could be using ESX KVM Zen Docker anything any hypervisor more or less that's That has like Linux tap support really Do I have the URL to the slides? Unfortunately, it doesn't allow me to share it, but I'll go ahead and set post the link on Twitter after the presentation So you can find it there if you want My username is Aaron O'Rozan upset my cousin So when you select the IPs for your channel network, you can pick any IP address that you want But it's probably best to use like RFC 1918 space so it doesn't overlap with like public things that can be routed So like for instance if I created a subnet with 8.8.8.8.8 Then I wouldn't really be able to go to Google's DNS stuff. Is that your question? Right you pick anything you want so one of the coolest things is if we have multiple tenants They could choose to use the same IP address and that's one of the nice things that Neutron allows you to do so like one of the Like big use cases is say you have like a physical deployment of a lot of machines Say you have like a mySQL server that's configured to talk to this other server And everything's already configured with specific IP addresses You just suck those images up push them to the cloud choose the exact same IP addresses and everything will Continuously work so you don't have to read IP anything The floating IP addresses. Yeah, that that would be Because the only reason the floating IPs would be a problem is because it's actually routed space So technically you could have multiple public networks with the same IP space And at VMR we actually have a few different public networks We have one that's a Kolo interconnect that connects into the Kolo network And then another network that's a VM or interconnect that connects into the VM or infrastructure So if you have an instance on the VM or infrastructure that needs to access some VM or specific thing that can't be done from the Kolo You'd have to connect to that network And if you wanted to have a public IP address you would connect to the Kolo network and in theory You could have those IP addresses overlapping if you wanted to so as in the guest is doing tagging So right now everything in Neutron is untagged so like OBS is gonna strip all the VLAN tags for you But what you could do It's not actually an OBS limitation. That's just Like Neutron is just creating networks that are untagged But one of the things that's there is you can create a private provider network So say I have some infrastructure that's using VLANs I can go ahead and create a network that's mapped to a specific VLAN and then create VMs on that And then the VM will still be sending on tagged packets, but then OBS will tag the traffic for you Right if you wanted to have an instance on you would need multiple interfaces in order to access Different VLANs What why do you need different images? Okay, so Sorry, so why do you need to rebake them like there's configuration that you need I mean, why would you need to because there's different Okay, so like when you launch an instance that you can specify the networks that you want to use there and another possibility is you can leverage cloud in it or Config drive to basically bootstrap the instance you could have extra data in it So they want to control the tagging inside of their instance. So if they if they send traffic up one One interface that'll use one specific tech is that logic built into the application itself Okay, so if it's built into the application as I was gonna say it could just use like standard like routing And then it would just automatically happen in the guest for you, but if it's baked into the application then unfortunately It's not able to do that today So where is the load balancer and the health monitor services running? Where is it running? Yeah, sure. So that's just running on one of the Beginning here. It's just running on the network node. That's running the load balancer for you so one of the things that The open-source plugins provide as it allows you to scale things out a little bit And the way that that works is you could run multiple instances of the load balancer across multiple nodes Then when you create a pool it would be mapped to one of the load balancer agents the same with the L3 agent So if you create multiple routers if you want to scale out your L3 network You'd run multiple L3 agents that connect to the public network and then they'll get scheduled to two different ones So it provides you a little bit of a scaling out and is this like predetermined I have to make all that decision up front or can I say now I want to install or run another L3 agent another L2 agent Like say like you're running for a while, and then you want to expand. Yeah, you just go ahead and install more It's added up at a law and actually register connect to the rabbit bus and then it'll show up Okay, but is there a way to see them in the horizon? I'm not sure about in the horizon interface, but there's a what there's a I don't know if the horizon interface exposes this But as an admin you can just do a neutron agent list and the list all the agents that you have running So one of the things is is you can actually scale up the DHCP agent as well So when you do neutron agent list, it'll show you all the various agents You can map networks to various agents and you can also provide high availability With that sort of but it to be honest. It has some problems But in theory, okay, thanks does kind of work What's running right now in this lab is ice-has So if you do the curl command does it no longer respond Hmm. Oh No, I haven't it sounds like there could be maybe a bug that you're hitting. I haven't experienced that that's that Sorry. Oh Sorry If you kill netcat, I think if you kill netcat There's actually a outer loop There's a while loop that runs because when the request comes in it eats that netcap thing and terminates the while loop We'll spawn up another one. So Yeah, if you just like re in yeah, if you restart the instance that would that would kill it as well Right the L-bass agent is going to be yeah, it's actually the HA proxy instance that's running on the L-bass agent Does it So when you hit So you hit the VIP with tell that Did you tell that on port 80? Okay, so this is so I think this is an implementation detail of Okay, so it might be an implementation detail of HA proxy I'm not sure like what guarantee like the API doesn't really make any specific guarantees about behavior Which should but like each vendor decides to implement things differently But it's possible that there's a way to do that and we're not passing something to HA proxy or there's probably a way to do it Yeah, it'd be HA proxy. That's running in this Yeah, so ESX and KVM can coexist together and we actually have ESX and KVM Running in the clouds to some of your instances on ESX some could be on KVM ESX and KVM together Today you would need NSX for that you could actually you could do this Well in the next release of Juno, we're looking to integrate Virtual center with neutron directly so you could create networks in VC and so you could create a network in VC That's mapped to a VLAN and then also have that VLAN spans a KVM host as well So like here you can see we have 39 ESX ESX iNodes and 149 Ubuntu servers running KVM Cool. Thanks. I think we can keep this active for a while I'll have to ask Eric and see when How long this is gonna be active? But one of the things that we're planning on doing is if you want to do this lab at any time later I think we'll put in an email address or some contact info on that URL And you should be able just to email that and someone will give you a request code And we'll go ahead and spin up the lab for you and like let it run for a week or so and then it'll automatically tear down But I think we can leave these running for a while. I don't think we need to delete them I believe that there's a if you go to there's a VM or training lab site But I think you can take a lab that allows you to play with NSX Yeah, there's You want to see the slides Okay, yeah, I'll post them later. I need to upload it to Dropbox or something like that Cool